The Elektor Forum will close. See also this link. From Friday March 15st it is no longer possible to log in to the forum. However, the content of the forum will remain visible until the end of March. As of April 1st the forum will definitely go off the air.

HTTP server does not like IE ?

The topic on number crunching

Postby ponedelnik » Fri Aug 26, 2011 12:00 am

The compiled web in the Digibutler software includes the page "variables.htm", which reloads itself every second or so. I observed that with Internet Explorer (IE) this process would stop after some unpredictable time, whereas with Firefox it would usually survive for days.
After a stop, the server would still be alife and accept a new request.
So, how could it tell IE from Firefox ?
The only difference I could think of was the contents of the request the user agent (a technical name for the navigator) was sending to the server, a collection of data called a header.

A header is made of a number of lines, each with the syntax :

data_name: data_value

except for the first and the last lines.
The last line is blank and is used to terminate the header.
The first line specifies the access method, the name of the requested file and the protocol :

GET /variables.htm HTTP/1.1

That is the only line that's useful to our little server.

A typical header sent by IE is as following :

GET /variables.htm HTTP/1.1 Accept: image/jpeg, application/x-ms-application, image/gif, application/xaml+xml, image/pjpeg, application/x-ms-xbap, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword,
Accept-Language: fr-FR User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.507
27; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0;.NET4.0C) Accept-Encoding: gzip, deflate
Host: 192.168.1.59
Connection: Keep-Alive

The size of this one is 474 bytes, and it is probably not the largest !

Now let's look within module "freescale_http.c" at function "freescale_http_read_header(…)", where the received header is read with the code :

length = m_recv( freescale_http_sessions[session].socket, (char *)buffer, RECV_BUFFER_SIZE );

The size of the receive buffer is defined within module "freescale_http_server.h" :

//*****************************************// Receive buffer, DO NOT CHANGE!!!//*****************************************#define RECV_BUFFER_SIZE0x100

This is 256 bytes, much smaller than the received 474 bytes.

This means that the TCP/IP stack will keep the rest and return it on the next call of m_receiv().
If this call is assumed to return a new header, well process garbage, which we may detect as such or not (in the best case we can't satisfy an invalid request and return a "page not found" error, effectively creating the condition we started with), therefore we should discard the data in excess by calling m_receiv() repeatedly until the end-of-header line is found.
But m_receiv() has got to copy the data to some buffer. Of course, we could allocate a buffer from the stack, but this would increase the usage of the stack, not the best solution.
If we accept to limit the size of the file name (including ?forms) to a manageable 160 characters, the first header line would never exceed 192 bytes, so we could use the last 25% of the receive buffer to throw away the garbage.

The modified function "freescale_http_read_header(…)" would be as following :


void freescale_http_read_header( int session, char *buffer ){  HEADER_TYPESi;  intlength, j, k;if(session>MAX_NUMBER_OF_SESSIONS){  freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;  return;}  if( freescale_http_sessions[session].keep_alive  )    freescale_http_sessions[session].keep_alive--;  // PARAM1: M_SOCK socket,  // PARAM2: char * buffer  // PARAM3: unsigned length  //  // RETURNS: number of bytes actually read, or -1 if error.    // int m_recv(M_SOCK so, char * buf, unsigned buflen)// commented out - HM 23-aug-11  length = m_recv( freescale_http_sessions[session].socket, buffer, RECV_BUFFER_SIZE );      if( length < 0 )  {//    if( !freescale_http_sessions[session].keep_alive )      freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;    return;  }// ******************************************************************************// Discard header data in excess//As the headers supplied by the user agents tend to inflate, they have overgrown//the size of the receive buffer. Only the first header line is of interest to us,//the rest can safely be discarded.//A scratch zone at the end of the receive buffer is used to store the header data//in excess. It is overwritten by calls to m_recv until the returned data is//shorter than expected or is terminated by a blank line (double CR/LF).//Note : it is assumed that the useful data does never overflow into the//scratch zone. This limits the size of the scratch zone, which should be large//to limit processing overhead.#define EXCESS (RECV_BUFFER_SIZE/4)// size of scratch zone at end of buffer  if (length >= RECV_BUFFER_SIZE)// if smaller, there is no excess data    do {//The following test takes care of the possibility that the returned data fills the//the scratch zone exactly, meaning there is no more data to receive.      if ((buffer[RECV_BUFFER_SIZE-4] == 13) &&        (buffer[RECV_BUFFER_SIZE-3] == 10) &&        (buffer[RECV_BUFFER_SIZE-2] == 13) &&        (buffer[RECV_BUFFER_SIZE-1] == 10)) break;    } while (EXCESS == m_recv( freescale_http_sessions[session].socket,        &buffer[RECV_BUFFER_SIZE-EXCESS], EXCESS));// ******************************************************************************  // Scan through the buffer looking for the method  for( i=NO_HEADER_FOUND; header_table.header_string[0]; i++ )  {    if(i > 3)/* empty or invalid header */    {        freescale_http_sessions[session].state = EMG_HTTP_STATE_CLOSE;        return;    }    for( j=0, k=0; j < 8; j++ )// the name of the method must be within the 7 first chars    {      if( buffer[j] == header_table.header_string[k] )        k++;              if( header_table.header_string[k] == 0 )      {#if HTTP_VERBOSE>3        printf( "\n6 header[%d] = %s", session, &header_table.header_string[0] );#endif        freescale_http_sessions[session].headertype = header_table.header_type; //       header = freescale_http_sessions[session].headertype;#if 0        // Scan to /        for( ; j
ponedelnik
 
Posts: 10
Joined: Thu Jan 02, 2014 10:44 am

Postby shorty3 » Sun Sep 04, 2011 12:00 am

Hello,

I tried your code modification of freescale_http.c, but this leads to an error as in my version the struct headertable is the following:

const HEADER_STRUCTURE header_table[] ={...};

whereas your code tries to access

header_table.header_string[0]  


This leads to the following error message:

Error : not a struct/union/class
freescale_http.c line 611 for( i=NO_HEADER_FOUND; header_table.header_string[0]; i++ )

Maybe I missed another code modification?


regards

Werner
shorty3
 
Posts: 7
Joined: Thu Jan 02, 2014 10:44 am

Postby ponedelnik » Tue Sep 06, 2011 12:00 am

The definition of HEADER_STRUCTURE is as following in freescale_http_server.h :

//*********************************************// Max size of HTTP command, GET, POST, ...//*********************************************#define HEADER_TYPE_SIZE4typedef struct{unsigned charheader_string[HEADER_TYPE_SIZE+1];HEADER_TYPESheader_type;} HEADER_STRUCTURE;
The header_table is an array of these structures, therefore, any
access to it must provide a session index:

if ( buffer[j] == header_table[session].header_string[k] )
which does not appear in your quotation.

Hope this helps

Helmut

PS. The text formatter of this forum seems to play strange tricks, (it just silently discards a letter i enclosed in brackets ?)
You may look at my "port to Windows" message for a link to my sources, in case there are other errors of the same kind.
ponedelnik
 
Posts: 10
Joined: Thu Jan 02, 2014 10:44 am

Postby shorty3 » Wed Sep 07, 2011 12:00 am

Thank you, I will test it...

I already assumed that this might be due to the formatter (I also had to edit my reply several times as it was e.g. not possible to post the entire header structure without strange interaction by the formatter)

...not very useful in a forum, where the posting of source code snippets is important....

regards

Werner
shorty3
 
Posts: 7
Joined: Thu Jan 02, 2014 10:44 am

Postby shorty3 » Wed Sep 14, 2011 12:00 am

Hello,

I tried the suggested modification but without success.

Using the code modification as proposed I could not reach the server at all (neither with firefox nor with IE). After uncommenting the line
if (!freescale_http_session[session].keep_alive) 

(as in the original code) I was at least prompted with the authentication window, when the web-page was requested.
However even after entering the correct authentication values no web page was shown.
Might it be that the for-loop
for (j=0 ,k=0 ;j < 8 ;j++)

causes the problems in case of a zero length of the received header (length = 0)?
In the original code the loop would be bypassed whereas in the modified code the buffer content - which is not valid as no bytes have been received - would be compared to the header-table.

Thank you very much for your help and regards

Werner
shorty3
 
Posts: 7
Joined: Thu Jan 02, 2014 10:44 am

Postby ponedelnik » Wed Sep 14, 2011 12:00 am

What was it you did as modifications ?
When I wrote :

if ( buffer[j] == header_table[session].header_string[k] )

it was an example, not the actual code because of the formatter problem :

[session] stands here for i enclosed in brackets

May be I should have made this plainly clear.

Personally, I have no interest in authentication and passwords for a little thing I don't expect to be accessed by anybody else, so I disabled these (all the validation or pw manipulation functions are patched to return with a success code). I could well use the freed memory space for more useful application code.

Helmut
ponedelnik
 
Posts: 10
Joined: Thu Jan 02, 2014 10:44 am

Postby shorty3 » Thu Sep 15, 2011 12:00 am

maybe we had a little misunderstanding here because I was unclear....
with code modification I just meant YOUR modified freescale_http_read_header() function (no problem....!!))

The problem arises indeed if authentication is used, because the subsequent calls of m_recv() in case of an
excess header seem to prevent the correct extraction of the authentication string which is part of the header.

However, when I tested the "original" function yesterday I found out that also using Firefox EVERY call of m_recv() in
freescale_http_read_header() leads to an excess of the receive buffer. Nevertheless, even using authentication,
which extends the header at its end does not lead to problems. At least from that it seems that a header length exceeding the recv_buffer_size is not really a problem in the original freescale_http_read_header() function - or do I overlook something?

regards
Werner
shorty3
 
Posts: 7
Joined: Thu Jan 02, 2014 10:44 am

Postby ponedelnik » Thu Sep 15, 2011 12:00 am

My understanding (which may be incorrect) of the original code is that when the TCP/IP stack receives a header frame, it stores it away and delivers to the user just as much of it that's wanted by the byte count. If something remains, it is returned on the next read request 'which may not exhaust the remaining data. This subsequent data may be interpreted as a new header and discarded as invalid or not, if by chance it contains some expected data, like a valid method (which search I limited to the 8 first chars of the buffer).
I suppose that theTCP/IP stack itself could experience an overflow, which I did not investigate.

Helmut
ponedelnik
 
Posts: 10
Joined: Thu Jan 02, 2014 10:44 am


Return to Microcontrollers & Embedded

Who is online

Users browsing this forum: No registered users and 1 guest