recv() loop adding last "chunk" twice to buffer - html

Im new to C and very new to network programming. I wrote some code that would read an HTML file chunk by chunk. Everything works great until the last chunk which gets read in twice and Im not inertly sure why.
char request[] = "GET /~kkredo/file.html HTTP/1.0\r\n\r\n";
char response[10000];
int bytes, p_count = 0;
int res_code = 1;
send(sock, request, sizeof(request), 0);
while(res_code != 0) {
res_code = recv(sock, &response, chunk, 0);
printf("%s",response);
bytes += res_code;
}
That's the code for the loop, it prints out each chunk until the last chunk which prints twice. Ive put an If statement to double check what the loop is checking, and even though res_code = 0, it still prints. I think it has something to do with the char buffer but Im not sure.
<p> that's it for this page </p>
</body>
</html>
</body>
</html>
That would be an example of the duplication. The second html and body tags are duplicated and dependent on if the chunk size is larger, then it would duplicate more of the document. Thanks for the help.

you call
printf("%s",response);
without regarding res_code, thus if recv() doesn't receive anymore data and returns 0, you print whatever still is in response. Furthermore: response isn't necessarily NUL terminated, so your printf() invokes UB. And, as a third point, your code doesn't handle that recv() might return -1.
So you should change yout loop to:
while(res_code > 0) {
res_code = recv(sock, response, chunk, 0); // '&' isn't necessary here
if(res_code > 0) {
printf("%.*s",res_code,response);
bytes += res_code;
}
}

Related

How to convert large UTF-8 encoded char* string to CStringW (UTF-16)?

I have a problem with converting a UTF-8 encoded string to a UTF-16 encoded CStringW.
Here is my source code:
CStringW ConvertUTF8ToUTF16( __in const CHAR * pszTextUTF8 )
{
_wsetlocale( LC_ALL, L"Korean" );
if ( (pszTextUTF8 == NULL) || (*pszTextUTF8 == '\0') )
{
return L"";
}
const size_t cchUTF8Max = INT_MAX - 1;
size_t cchUTF8;
HRESULT hr = ::StringCbLengthA( pszTextUTF8, cchUTF8Max, &cchUTF8 );
if ( FAILED( hr ) )
{
AtlThrow( hr );
}
++cchUTF8;
int cbUTF8 = static_cast<int>( cchUTF8 );
int cchUTF16 = ::MultiByteToWideChar(
CP_UTF8,
MB_ERR_INVALID_CHARS,
pszTextUTF8,
-1,
NULL,
0
);
CString strUTF16;
strUTF16.GetBufferSetLength(cbUTF8);
WCHAR * pszUTF16 = new WCHAR[cchUTF16];
int result = ::MultiByteToWideChar(
CP_UTF8,
0,
pszTextUTF8,
cbUTF8,
pszUTF16,
cchUTF16
);
ATLASSERT( result != 0 );
if ( result == 0 )
{
AtlThrowLastWin32();
}
strUTF16.Format(_T("%s"), pszUTF16);
return strUTF16;
}
pszTextUTF8 is htm file's content in UTF-8.
When htm file's volume is less than 500kb, this code works well.
but, when converting over 500kb htm file, (ex 648KB htm file that I have.)
pszUTF16 has all content of file, but strUTF16 is not. (about half)
I guess File open is not wrong.
In strUTF16 m_pszData has all content how to I get that?
strUTF16.Getbuffer(); dosen't work.
The code in the question is stock full of bugs, somewhere in the order of 1 bug per 1-2 lines of code.
Here is a short summary:
_wsetlocale( LC_ALL, L"Korean" );
Changing a global setting in a conversion function is unexpected, and will break code calling that. It's not even necessary either; you aren't using the locale for the encoding conversion.
HRESULT hr = ::StringCbLengthA( pszTextUTF8, cchUTF8Max, &cchUTF8 );
This is passing the wrong cchUTF8Max value (according to the documentation), and counts the number of bytes (vs. the number of characters, i.e. code units). Besides all that, you do not even need to know the number of code units, as you never use it (well, you are, but that is just another bug).
int cbUTF8 = static_cast<int>( cchUTF8 );
While that fixes the prefix (count of bytes as opposed to count of characters), it won't save you from using it later on for something that has an unrelated value.
strUTF16.GetBufferSetLength(cbUTF8);
This resizes the string object that should eventually hold the UTF-16 encoded characters. But it doesn't use the correct number of characters (the previous call to MultiByteToWideChar would have provided that value), but rather chooses a completely unrelated value: The number of bytes in the UTF-8 encoded source string.
But it doesn't just stop there, that line of code also throws away the pointer to the internal buffer, that was ready to be written to. Failure to call ReleaseBuffer is only a natural consequence, since you decided against reading the documentation.
WCHAR * pszUTF16 = new WCHAR[cchUTF16];
While not a bug in itself, it needlessly allocates another buffer (this time passing the correct size). You already allocated a buffer of the required size (albeit wrong) in the previous call to GetBufferSetLength. Just use that, that's what the member function is for.
strUTF16.Format(_T("%s"), pszUTF16);
That is probably the anti-pattern associated with the printf family of functions. It is the convoluted way to write CopyChars (or Append).
Now that that's cleared up, here is the correct way to write that function (or at least one way to do it):
CStringW ConvertUTF8ToUTF16( __in const CHAR * pszTextUTF8 ) {
// Allocate return value immediately, so that (N)RVO can be applied
CStringW strUTF16;
if ( (pszTextUTF8 == NULL) || (*pszTextUTF8 == '\0') ) {
return strUTF16;
}
// Calculate the required destination buffer size
int cchUTF16 = ::MultiByteToWideChar( CP_UTF8,
MB_ERR_INVALID_CHARS,
pszTextUTF8,
-1,
nullptr,
0 );
// Perform error checking
if ( cchUTF16 == 0 ) {
throw std::runtime_error( "MultiByteToWideChar failed." );
}
// Resize the output string size and use the pointer to the internal buffer
wchar_t* const pszUTF16 = strUTF16.GetBufferSetLength( cchUTF16 );
// Perform conversion (return value ignored, since we just checked for success)
::MultiByteToWideChar( CP_UTF8,
MB_ERR_INVALID_CHARS, // Use identical flags
pszTextUTF8,
-1,
pszUTF16,
cchUTF16 );
// Perform required cleanup
strUTF16.ReleaseBuffer();
// Return converted string
return strUTF16;
}

AS3 How to Send Multiple Messages in writeUTF?

So with TCP in AS3, I'm trying to write strings over to the server and then to the clients, but it appears to only be able to read one at a time. This is not good because I'm sending messages based on keys being pressed, and the client has an action that needs to be taken place based on what key is pressed. Since multiple keys can be pressed at a time, obviously it does not work correctly.
Client example:
if (keys[p1_jump_key])
{
p1_up = "down";
sock.writeUTF("p1_up_down");
sock.flush();
}
else
{
p1_up = "up";
sock.writeUTF("p1_up_up");
sock.flush();
}
if (keys[p1_crouch_key])
{
p1_down = "down";
sock.writeUTF("p1_down_down");
sock.flush();
}
else
{
p1_down = "up";
sock.writeUTF("p1_down_up");
sock.flush();
}
And then here is the server:
function socketDataHandler(event:ProgressEvent):void{
var socket:Socket = event.target as Socket;
var message:String = socket.readUTF();
for each (var socket:Socket in clientSockets)
{
socket.writeUTF(message);
socket.flush();
}}
And finally, here is the recieving client (I have a method that allows the server to differentiate between the two):
if(msg=="p1_down_down"){
p1_down="down";
}
if(msg=="p1_down_up"){
p1_down="up";
}
if(msg=="p1_up_down"){
p1_down="down";
}
if(msg=="p1_up_up"){
p1_down="up";
}
Now many of you already see the issue, as when the down key is up, it sends the message "p1_down_up". When the up key is up, it sends the message "p1_up_up". Both messages are sending at once when neither of them are being pressed. The receiving client is, I suppose, just getting one of the signals, or perhaps neither of them. How do I make MULTIPLE signals get wrote and read over the server? I tried using an array but you can't write those apparently. Thank you.
For anyone else who comes across this like I did trying to figure out how to accomplish this, I have to say that the other solution supplied is highly inefficient, overly complicated and too hard to read. Instead of adding a null byte to the end of your message and pushing packets into arrays, you can simply have the server send the length of the entire message to determine whether the message is in multiple packets, and to make sure you read multiple writes in the correct order.
Here is a working client receive function. I send data as compressed Objects converted to ByteArrays. Objects can store any type of data which make them a good candidate for transferring complex information from server to client.
var messageBytes: ByteArray = new ByteArray;
var messageLength: int = 0;
var readBuffer = false;
function receiveTCP(e: ProgressEvent): void {
while (e.target.bytesAvailable >= 4) {//Loop to see if there is data in the socket that is available to read.
if (messageLength == 0) {
messageLength = e.target.readUnsignedInt();//read the first 4 bytes of data to get the size of the message and store it as our message length.
readBuffer = true;//set the readBuffer to true. While this is true, the function wont be reading any more data from the server until the current message is finished being read.
}
if (messageLength <= e.target.bytesAvailable) {//check to see if our message is finished being read.
if (readBuffer == true) {//Make sure another message isn't being read. Without this check, another incoming message from the server will overwrite the current one from being executed.
e.target.readBytes(messageBytes, 0, messageLength);//reads the byte array into our messageBytes ByteArray declared at the top.
messageBytes.uncompress();//decompresses our message
var message: Object = new Object();
message = messageBytes.readObject();//Writes our ByteArray into an object again.
receiveMessage(message);//send our message to another function for the data to be executed.
messageLength = 0;//reset our messageLength for the next message.
readBuffer = false;//reset our buffer so that the client can read the next message being received, or the next message in line.
} else {
receiveTCP(e);//if there is another message in wait and the previous one isn't finished reading, execute the same function with the same event data again. Eventually the buffer will open and the message will be read.
}
} else {
break;//If the message is split up into multiple packets, this will reset the loop to continue reading the message until all the packets are accounted for.
}
}
}
This will prevent Clogging, allow you to receive multiple messages simultaneously and put together messages that were delivered in multiple packets.
Admittedly, this took a while to figure out. If you need the serverside code for sending the message, let me know.
The TCP protocol consists of a bi-directional stream of bytes, and as such message boundaries are not preserved. Due to this, if you are sending messages too often, you can end up reading from the buffer two messages instead of one.
Thus, some of the messages you are sending might be being processed together, which fails the comparison test on the receiving end.
To fix this, you can append a delimiter at the end of each message. This way, you'll know where every message you sent begins and ends, allowing you to split and process them separately. This also means you don't need to flush after every write to the buffer.
For instance, you could use a null byte (\0) as delimiter.
if(keys[p1_jump_key])
{
p1_up = "down";
sock.writeUTF("p1_up_down\0");
}
...
sock.flush();
And on the receiving end, we trim any trailing delimiters and then split the message into the smaller packets.
EDIT: Fixing the issue you described gives rise to another one: your messages might be being split over two buffer reads. I've edited my code to reflect a solution to this, by creating a temporary variable where partial messages are stored, to be prepended to the other part(s) of the message in the following reads.
// First we check to see if there are any 'parts' of a message that we've stored temporarily from the last read. If so, prepend them to the message string, which must be its continuation
if(temp != "")
{
msg = temp + msg;
temp = "";
}
// Now we begin processing our packet
var splitPackets:Array = msg.split("\0");
var packets:Array = new Array();
// Deal with incomplete message reads
if(splitPackets[splitPackets.length - 1] != "")
{
// If the last item in our array isn't empty, it means out message stream ended in a partial message. Hence, we have to store this value temporarily, and append it on the next read of the stream
temp = splitPackets[splitPackets.length - 1];
}
// If the last item in our array is empty, it means our message stream ends in a full message
for(var i = 0; i < splitPackets.length - 1; i++)
{
packets[i] = splitPackets[i];
}
for(var i = 0; i < packets.length; i++)
{
if(packets[i] == "p1_down_down")
{
// Process each packet individually here
...
// If you have already processed the packet, move on to the next
continue;
}
// Processing code for other packets
...
}
Hope this helps!

CGI wont display variables through HTML in c (Eclipse)

I have used a fifo pipe to read in some data (weather data) into a char variable. The console will display this variable correctly. However, when I try to display it through HTML on the CGI page, it simply does not display. Code below -
int main(void) {
int fd;
char *myfifo = "pressure.txt";
char buff[BUFFER];
long fTemp;
//open and read message
fd = open(myfifo, O_RDONLY);
read(fd, buff, BUFFER);
printf("Received: %s\n", buff);
close(fd);
printf("Content-type: text/html\n\n");
puts("<HTML>");
puts("<BODY>");
printf("Data is: %s", buff);
puts("</BODY>");
puts("</HTML>");
return EXIT_SUCCESS;
}
As you can see in the console is displays correctly -
Received: 2014-08-13 16:54:57
25.0 DegC, 1018.7 mBar
Content-type: text/html
<HTML>
<BODY>
Data is 2014-08-13 16:54:57
25.0 DegC, 1018.7 mBar
</BODY>
</HTML>
logout
But on the CGI webpage it does not display the weather data, but it does display "data is".
Two important things when writing a CGI program:
the program will be run by the webserver, which is normally
started as a different user (the 'www' user for example).
it's possible that the program is started from within another
directory, which can cause different behaviour if you don't
specify the full path of a file you want to open.
Since both these things can cause problems, it can be helpful
to add some debug information. Of course, it's always a good idea
to check return values of functions you use.
To make it easier to display debug or error messages, I'd first
move the following code up, so that all output that comes after
it will be rendered by the browser:
printf("Content-type: text/html\r\n\r\n");
puts("<HTML>");
puts("<BODY>");
It may be useful to know what the webserver uses as the directory
from which the program is started. The getcwd
call can help here. Let's use a buffer of size BUFFER to store
the result in, and check if it worked:
char curpath[BUFFER];
if (getcwd(curpath, BUFFER) == NULL)
printf("Can't get current path: %s<BR>\n", strerror(errno));
else
printf("Current path is: %s<BR>\n", curpath);
The getcwd function returns NULL in case of an error, and sets the value
of errno to a number which indicates what went wrong. To convert this
value to something readable, the strerror
function is used. For example, if BUFFER was not large enough to be
able to store the path, you'll see something like
Can't get current path: Numerical result out of range
The open call returns a negative number
if it didn't work, and sets errno again. So, to check if this worked:
fd = open(myfifo, O_RDONLY);
if (fd < 0)
printf("Can't open file: %s<BR>\n", strerror(errno));
In case the file can be found, but the webserver does not have permission
to open it, you'll see
Can't open file: Permission denied
If the program is started from another directory than you think, and
it's unable to locate the file, you would get:
Can't open file: No such file or directory
Adding such debug info should make it more clear what's going on, and more
importantly, what's going wrong.
To make sure the actual data is read without problems as well, the return
value of the read function should be
checked and appropriate actions should be taken. If read fails,
a negative number is returned. To handle this:
numread = read(fd, buff, BUFFER);
if (numread < 0)
printf("Error reading from file: %s<BR>\n", strerror(errno));
Another value indicates success, and returns the number of bytes that were
read. If really BUFFER bytes were read, it's not at all certain that the
last byte in buff is a 0, which is needed for printf to know when the
string ended. To make sure it is in fact null-terminated, the last byte in
buff is set to 0:
if (numread == BUFFER)
buff[BUFFER-1] = 0;
Note that this actually overwrites one of the bytes that were read in this
case.
If fewer bytes were read, it's still not certain that the last byte that was
read was a 0, but now we can place our own 0 after the bytes that were read
so none of them are overwritten:
else
buff[numread] = 0;
To make everything work, you may need the following additional include files:
#include <unistd.h>
#include <string.h>
#include <errno.h>
The complete code of what I described is shown below:
int main(void)
{
int fd, numread;
char *myfifo = "pressure.txt";
char buff[BUFFER];
char curpath[BUFFER];
long fTemp;
// Let's make sure all text output (even error/debug messages)
// will be visible in the web page
printf("Content-type: text/html\r\n\r\n");
puts("<HTML>");
puts("<BODY>");
// Some debug info: print the current path
if (getcwd(curpath, BUFFER) == NULL)
printf("Can't get current path: %s<BR>\n", strerror(errno));
else
printf("Current path is: %s<BR>\n", curpath);
// Open the file
fd = open(myfifo, O_RDONLY);
if (fd < 0)
{
// An error occurs, let's see what it is
printf("Can't open file: %s<BR>\n", strerror(errno));
}
else
{
// Try to read 'BUFFER' bytes from the file
numread = read(fd, buff, BUFFER);
if (numread < 0)
{
printf("Error reading from file: %s<BR>\n", strerror(errno));
}
else
{
if (numread == BUFFER)
{
// Make sure the last byte in 'buff' is 0, so that the
// string is null-terminated
buff[BUFFER-1] = 0;
}
else
{
// Fewer bytes were read, make sure a 0 is placed after
// them
buff[numread] = 0;
}
printf("Data is: %s<BR>\n", buff);
}
close(fd);
}
puts("</BODY>");
puts("</HTML>");
return EXIT_SUCCESS;
}

RangeError: Error #2006: The supplied index is out of bounds AS3

I've loved using this site for little tips on code, and I try to solve all the errors I can by myself. However, this one has had me stumped for days. I just can't crack it.
RangeError: Error #2006: The supplied index is out of bounds.
at flash.text::TextField/setTextFormat()
at BibleProgram_fla::MainTimeline/checkAgainstBible()
at BibleProgram_fla::MainTimeline/compileInputString()
at BibleProgram_fla::MainTimeline/spaceBuild()
function spaceBuild(event:Event):void //This program runs every frame
{
compileInputString();
}
function compileInputString():void
{
inputVerse = inputText.text; // takes text from the input field
inputVerse = inputVerse.toLowerCase();
inputVerse = inputVerse.replace(rexWhiteSpace, ""); //Removes spaces and line breaks
inputVerse = inputVerse.replace(rexPunc, ""); // Removes punctuation
inputVerse = addSpaces(inputVerse); //adds spaces back in to match the BibleVerse
inputVerse = addCaps(inputVerse); //adds capitalization to match the BibleVerse
checkAgainstBible();
}
function checkAgainstBible()
{
outputText.text = inputVerse; // sets output text to be formatted to show which letters are wrong
for(var n:Number = 0; n < inputText.length; n++)
{
var specLetter:String = inputVerse.charAt(n);
if(specLetter != bibleVerse.charAt(n))
{
outputText.setTextFormat(red, n); // sets all of the wrong letters to red
}
}
}
Whenever I run the program and type a string longer than the BibleVerse, it returns the error, but I cannot figure out how to fix it.
I hope I provided enough information for you to help me. If you need more code or something, please ask!
Thanks in advance!!
Well, you would get that error if n is greater than the number of characters in the outputText when it sets its format color to red, and it looks like your outputText's characters are extended or shortened when you make it equal to your inputVerse because inputVerse had all the regex operations that i can't seen done to it. So most likely these operations are shortening the characters and so outputText.text is shorter than it should be and when it loops over the inputText.length, when it gets to the end of the outputText, n goes past its character length and so you get that error (That is what the error is - you are attempting to access something that is not there). So the way i see it is (using example made up strings);
// Pseudo code...
inputVerse=inputText.text; // (lets say its "Thee ")
// inputVerse and inputText.text now both have 5 characters
inputVerse=lotsOfOperations(inputVerse);
// inputVerse now only has 4 characters (got rid of the " " at the end)
outputText.text=inputVerse;
// outputText.text now has the new 4 character
for(var n:Number = 0; n < inputText.length; n++)
// loops through inputText.length (so it loops 5 times)
outputText.setTextFormat(red, n);
// if n=4 (since n starts at 0) from the inputText.length, then when it access
//outputText.setTextFormat(red,n) it is accessing a character of outputText.text
//that is at the end and not there. outputText.text is too short for the loop.
So, your problem is that your operations to inputVerse are making it too short to compare to the other strings, I don't know your other code so I can't say whats wrong, but this is why you are getting the error. Please comment if you have any questions or to notify me of what I am missing.

AS3 / AIR readObject() from socket - How do you check all data has been received?

If you write a simple object to a socket:
var o:Object = new Object();
o.type = e.type;
o.params = e.params;
_socket.writeObject(o);
_socket.flush();
Then on the client do you simply use:
private function onData(e:ProgressEvent):void
{
var o:Object = _clientSocket.readObject();
}
Or do you have to implement some way of checking all of the data has been received recieved before calling .readObject()
There's 2 approaches:
If you're confident that your object will fit into one packet, you can do something like:
var fromServer:ByteArray = new ByteArray;
while( socket.bytesAvailable )
socket.readBytes( fromServer );
fromServer.position = 0;
var myObj:* = fromServer.readObject();
If you have the possibility of having multiple packet messages, then a common usage is to prepend the message with the length of the message. Something like (pseudo code):
var fromServer:ByteArray = new ByteArray();
var msgLen:int = 0;
while ( socket.bytesAvailable > 0 )
{
// if we don't have a message length, read it from the stream
if ( msgLen == 0 )
msgLen = socket.readInt();
// if our message is too big for one push
var toRead:int = ( msgLen > socket.bytesAvailable ) ? socket.bytesAvailable : msgLen;
msgLen -= toRead; // msgLen will now be 0 if it's a full message
// read the number of bytes that we want.
// fromServer.length will be 0 if it's a new message, or if we're adding more
// to a previous message, it'll be appended to the end
socket.readBytes( fromServer, fromServer.length, toRead );
// if we still have some message to come, just break
if ( msgLen != 0 )
break;
// it's a full message, create your object, then clear fromServer
}
Having your socket able to read like this will mean that multiple packet messages will be read properly as well as the fact that you won't miss any messages where 2 small messages are sent almost simultaneously (as the first message will treat it all as one message, thereby missing the second one)
Rule # 1 when dealing with TCP: it is an octet stream transfer protocol. You may never ever assume anything about how many octets (8 bit long values, commonly called bytes) you get in one go, always write code that can deal with any amount, both too few and too many. There is no gurantee that the write will not be split into multiple reads. There is also no gurantee that a single read will be from a single write.
The way I handled it was to make a call back that the server tell the client that the null bit was received.
The null bit is appended to the end of the data string you are sending to the server.
String.fromCharCode(0)
Also in your case you are doing
_socket.writeObject(o);
You should be sending a string not an object.
So Like this.
_socket.writeUTFBytes( 'Hellow World" + String.fromCharCode(0) );
NOTE *************
And one thing that most first time socket creators over look is the
fact that the first request to from the client to the server over the
port that the socket is connected on is a request for the
crossdomainpolicy.xml
If you only wish to send Objects, the simplest solution is if you send an int(size) before every object. Its not important to send the exact size, you can send a bit less. In my case, I've sent a bitmapdata, and the width and height of the object. obviously the bitmapdata's size is so big, its okay if you send only that, and ignore the rest.
var toRead=0;
protected function onSocketData(event:ProgressEvent):void{
if(toRead==0) {
toRead=socket.readInt()
}
if(socket.bytesAvailable>toRead){
var recieved:Object=socket.readObject()
/*do stuff with recieved object*/
toRead=0
}