We use AS3 Event:ProcessEvent.SOCKET_DATA to listen for socket data.
So this is my AS3 code for socket data handle.
private function packetHandler( e:ProgressEvent ):void
{
while( m_socket.bytesAvailable && m_socket.bytesAvailable >= pLen )
{
//pLen means the packet length
//pLen init is zero
if( pLen == 0 )
{
//PACKET_LEN stands for the solid length of one packet
//PACKET_LEN = HEAD_LEN + 4
//the 4 means an unsigned int which means the packet content length
if( m_socket.bytesAvailable > PACKET_LEN )
{
m_socket.readBytes( headByteBuffer, 0, HEAD_LEN );
headByteBuffer.clear();
pLen = m_socket.readUnsignedInt() + 4;
}
else
{
break;
}
}
//recieved a whole packet now handle it
else
{
var newPacket:ByteArray = new ByteArray();
newPacket.endian = Endian.LITTLE_ENDIAN;
m_socket.readBytes( newPacket, 0, pLen );
parasMsg( newPacket, pLen-4 );
pLen = 0;
}
}
}
A whole packet can be described in this picture:
My Problem is: When there has one incomplete packet received in Flash and triggered the handle.
But the left part of the packet will never trigger the handle and it seems like that the left part of the packet has lost!!!
I used a capture tool, find that the tcp packet is ok, but why the left part doesn't trigger the event again?
You can get more debug information below. Thank you!
This is my log:
byteava means bytesAvailable of m_socket
==>sendPacket: {"rangeID":"1","uid":"145962","serviceType":"copyscene","cmd":"CopySceneMoveAsk","pathPoint":[{"col":7,"row":6},{"col":7,"row":5},{"col":7,"row":4},{"col":7,"row":3},{"col":6,"row":3}],"sn":"79","smallPathPoint":[[22,19],[22,18],[22,17],[22,16],[22,15],[22,14],[22,13],[21,13],[21,12],[21,11],[20,11],[20,10]]}, bytesLoaded = 463
ProgressEvent Triggered!0 socket byteava = 373 evt loaded:373 evt total:0 evt:[ProgressEvent type="socketData" bubbles=false cancelable=false eventPhase=2 bytesLoaded=373 bytesTotal=0]
Find a packet from socket, pLen=288 socket byteava = 276
ProgressEvent Triggered!288 socket byteava = 441 evt loaded:165 evt total:0 evt:[ProgressEvent type="socketData" bubbles=false cancelable=false eventPhase=2 bytesLoaded=165 bytesTotal=0]
Start to Read a packet to buffer, pLen=288 socket byteava = 441
whole packet content: Readed a packet to buffer, pLen=288 socket byteava = 153
Server packet content byte buffer ava:288 len:288 pos: 0
Server Paras Data : data len: 284 data content: {"cmd":"CopySceneMoveNotify","gtcmd":"108","layer":"1","pathPoint":[{"col":7,"row":6},{"col":7,"row":5},{"col":7,"row":4},{"col":7,"row":3},{"col":6,"row":3}],"smallPathPoint":[[22,19],[22,18],[22,17],[22,16],[22,15],[22,14],[22,13],[21,13],[21,12],[21,11],[20,11],[20,10]HTTP/1.1 200
_[20,10]HTTP/1.1 200_ This is what went wrong!! The incomplete packet cat with another packet's header.
Here is the capture of the TCP connections:
Hope you can vote it up so that I can put my pictures of this question on!
My English is not very good, hope you can understand what I mean.
Thank you!
The Socket's event flash.events.ProgressEvent.SOCKET_DATA will fire when you receive the data at this point you can get the received bytes ( check .bytesAvailable ). When the msg is split into multiple packages you will receive event for each packet.
In your case maybe the pLen have wrong value when check m_socket.bytesAvailable >= pLen.
I assume you send the msg size in the begging of the message ( in this case you can check if the whole msg is received ). In this case you must have a class member (ByteArray ) as buffer that holds a received bytes. When new data come you must copy the new bytes to this member and than check if you receive whole msg. If buffer contains whole msg than remove the msg from it.
In general your event handler must looks like this:
protected function onSocketData( pEvt: Event ): void
{
try
{
if ( 0 < pEvt.target.bytesAvailable )
{
var byteStream: ByteArray = new ByteArray();
pEvt.target.readBytes( byteStream, 0, Socket( pEvt.target ).bytesAvailable );
// Append readed data to your buffer
do
{
//Check if you have enough bytes to read whole msg and execute it
//do..while because maybe it can be more than one msg in buffer
}
while ( null != msgContent );
}
}
catch ( exc )
{
}
}
Problem should caused by the packet's solid header.
Below is the 93 bytes solid header of a packet.
private static const HTTP_RESPONSE_CONTENT : String = "HTTP/1.1 200 OK \r\n"
+ "Connection: keep-alive \r\n"
+ "Content-Length: 280 \r\n"
+ "Content-Type: text/html \r\n\r\n";
This header will be in every packet's header, which AS3 could treat it to a http and might cut the flow with Content-Length: 280. So the left part of the 280 bytes will never trigger the SOCKET_DATA event.
When I remove this header, it's ok now.
Related
I am using http.get() to get a JSON from an API I am using, but it's not getting the data. I have the suspicion that this JSON is too big for the NodeMCU. I only need the information in the subpart "stats:". Is it possible to only http.get() that part of the JSON?
EDIT:
This is my code
function getstats()
http.get("https://api.aeon-pool.com/v1/stats_address?address=WmsGUrXTR7sgKmHEqRNLgPLndWKSvjFXcd4soHnaxVjY3aBWW4kncTrRcBJJgUkeGwcHfzuZABk6XK6qAp8VmSci2AyGHcUit", nil, function(code, pool)
if (code < 0) then
print("can't get stats")
else
h = cjson.decode(pool)
hashrate = h[1]["hashrate"]
print(hashrate)
dofile('update_display.lua')
end
end)
end
I also have another function getting data from another api above getstats()
function getaeonrate()
http.get("https://api.coinmarketcap.com/v1/ticker/aeon/?convert=EUR", nil, function(code, dataaeon)
if (code < 0) then
print("can't get aeon")
else
-- Decode JSON data
m = cjson.decode(dataaeon)
-- Extract AEON/EUR price from decoded JSON
aeonrate = string.format("%f", m[1]["price_eur"]);
aeonchange = "24h " .. m[1]["percent_change_24h"] .. "% 7d " .. m[1]["percent_change_7d"] .. "%"
dofile('update_display.lua')
end
end)
end
But now the weird thing is, when I want to access 'pool' from getstats() I get the json data from getaeonrate(). So "hashrate" isn't even in the json because I am getting the json from another function.
I tried making a new project only with getstats() and that doesn't work at all I always get errors like this
HTTP client: Disconnected with error: -9
HTTP client: Connection timeout
HTTP client: Connection timeout
Yesterday I thought that the response was too big from api.aeon-pool.com, I if you look at the json in your webbrowser you can see that the top entry is 'stats:' and I only need that, none of the other stuff. So If the request is to big It would be nice to only http.get() that part of the json, hence my original question. At the moment I am not even sure what is not working correctly, I read that the nodemcu firmware generally had problems with http.get() and that it didn't work correctly for a long time, but getting data from api.coinmarketcap.com works fine in the original project.
The problems with the HTTP module are with near certainty related to https://github.com/nodemcu/nodemcu-firmware/issues/1707 (SSL and HTTP are problematic).
Therefore, I tried with the more bare-bone TLS module on the current master branch. This means you need to manually parse the HTTP response including all headers looking for the JSON content. Besides, you seem to be on an older NodeMCU version as you're still using CJSON - I used SJSON below:
Current NodeMCU master branch
function getstats()
buffer = nil
counter = 0
local srv = tls.createConnection()
srv:on("receive", function(sck, payload)
print("[stats] received data, " .. string.len(payload))
if buffer == nil then
buffer = payload
else
buffer = buffer .. payload
end
counter = counter + 1
-- not getting HTTP content-length header back -> poor man's checking for complete response
if counter == 2 then
print("[stats] done, processing payload")
local beginJsonString = buffer:find("{")
local jsonString = buffer:sub(beginJsonString)
local hashrate = sjson.decode(jsonString)["stats"]["hashrate"]
print("[stats] hashrate from aeon-pool.com: " .. hashrate)
end
end)
srv:on("connection", function(sck, c)
sck:send("GET /v1/stats_address?address=WmsGUrXTR7sgKmHEqRNLgPLndWKSvjFXcd4soHnaxVjY3aBWW4kncTrRcBJJgUkeGwcHfzuZABk6XK6qAp8VmSci2AyGHcUit HTTP/1.1\r\nHost: api.aeon-pool.com\r\nConnection: close\r\nAccept: */*\r\n\r\n")
end)
srv:connect(443, "api.aeon-pool.com")
end
Note that the receive event is fired for every network frame: https://nodemcu.readthedocs.io/en/latest/en/modules/net/#netsocketon
NodeMCU fails to establish a connection to api.coinmarketcap.com due to a TLS handshake failure. Not sure why that is. Otherwise your getaeonrate() could be implemented likewise.
Frozen 1.5.4 branch
With the old branch the net module can connect to coinmarketcap.com.
function getaeonrate()
local srv = net.createConnection(net.TCP, 1)
srv:on("receive", function(sck, payload)
print("[aeon rate] received data, " .. string.len(payload))
local beginJsonString = payload:find("%[")
local jsonString = payload:sub(beginJsonString)
local json = cjson.decode(jsonString)
local aeonrate = string.format("%f", json[1]["price_eur"]);
local aeonchange = "24h " .. json[1]["percent_change_24h"] .. "% 7d " .. json[1]["percent_change_7d"] .. "%"
print("[aeon rate] aeonrate from coinmarketcap.com: " .. aeonrate)
print("[aeon rate] aeonchange from coinmarketcap.com: " .. aeonchange)
end)
srv:on("connection", function(sck, c)
sck:send("GET /v1/ticker/aeon/?convert=EUR HTTP/1.1\r\nHost: api.coinmarketcap.com\r\nConnection: close\r\nAccept: */*\r\n\r\n")
end)
srv:connect(443, "api.coinmarketcap.com")
end
Conclusion
The HTTP module and TLS seem a no-go for your APIs due to a bug in the firmware (1707).
The net/TLS module of the current master branch manages to connect to api.aeon-pool.com but not to api.coinmarketcap.com.
With the old and frozen 1.5.4 branch it's exactly the other way around.
There may (also) be issues with cipher suits that don't match between the firmware and the API provider(s).
-> :( no fun like that
How should the client's credentials packet look when communicating with a MySql server?
I am working on using a microcontroller to communicate with a MySql server on AWS.
Once I open a TCP socket to the server I receive the server's greeting packet. However, I am having trouble with the client's credentials packet.
When I send the packet I created in the code below, I do not receive a response from the server.
As a side note, I do receive the error "Got packets out of order" when I set the packet sequence number to 0 instead of 1. Any ideas?
Server's greeting:
4e0000000a352e362e33372d6c6f67006a3300005871507a363d6e3400ffff0802007fc01500000000000000000000542d2364524c65392d752659006d7973716c5f6e61746976655f70617373776f726400
Server's Error message:
21000001ff8404233038533031476f74207061636b657473206f7574206f66206f72646572
char buf[76];
memset(buf, 0, sizeof(buf));
buf[0] = 0x4C;
buf[1] = 0x00;
buf[2] = 0x00;
buf[3] = 0x01;
buf[4] = 0x08 | 0x04;
buf[5] = 0x80 | 0x02 | 0x01;
buf[6] = 0x00;
buf[7] = 0x00;
buf[8] = 0xB8;
buf[9] = 0x0B;
buf[10] = 0x00;
buf[11] = 0x00;
buf[12] = 0x35;
char username[] = "username";
size_t usernameLength = sizeof(username);
strncpy(&buf[36],username,usernameLength-1);
buf[48] = 0x00;
char password[] = "password";
size_t passwordLength = sizeof(password);
unsigned char hash[SHA_DIGEST_LENGTH];
SHA1(password, passwordLength-1, hash);
buf[49] = 0x14;
memcpy(&buf[50],&hash[0],20);
strncpy(&buf[70],"dbname",sizeof("dbname")-1);
buf[75] = 0x00;
I understand that this was not an appropriate place to ask the question I had. However, after posting, I did find a very helpful repository on Github here: https://github.com/ChuckBell/MySQL_Connector_Arduino
The repository contains all of the source code necessary to write a client which connects to a MySQL database. Most importantly, it shows the proper way to hash the password.
I hope this helps other people since there is very little documentation of MySQL provided by Oracle unless you license the software, regular clients don't provide source code, and observing packet traffic provides limited insight into how the handshake process works.
Aim:- Writing a kernel module,when inserted does the following things.
1.if a user wants to open a website named "abcde.in" in any browser ,he should get "google.co.in" webpage, in place of "abcde.in" web page.
Normally accessing google.in i get:-
Observation..(in http header)
302 Moved
The document has moved
HREF=http://www.google.co.in
:-) next i send an ack And i got the google homepage:-
With Modification:-
Steps i have taken
a. edited /etc/hosts
(google.in IP ) abcde.in
So that it gets ip of "google.in"'s for "abcde.in"
b.Next inserting the following piece of module in nf_hook POST_ROUTING implementation
Observation..
i am able to correctly modify and wireshark shows that in reply i am able to get 302 Moved
But i am sending Tcp retransmissions GET / HTTP/1.1. again and again.
I feel i am missing the ack which i need to send as in original case.
why am i unable to send an ack instead of that why i am retransmitting tcp packets to "GET / HTTP/1.1"
if(skb){
liph=(struct iphdr *)skb_network_header(skb);
if(liph->protocol == 6) {
ltcph = (struct tcphdr *)skb_transport_header(skb);
data=skb->data+(liph->ihl*4) + (ltcph->doff*4);
datalen=skb->len - ((liph->ihl*4)+(ltcph->doff*4));
replace_n=strstr(data,"abcde.in");
if(replace_n) {
temp = kmalloc(512 * sizeof(char), GFP_KERNEL);
memcpy(temp, data, datalen);
replace_n=strstr(temp,"abcde.in");
replace_size=strlen("google.in");
site_diff=replace_size - strlen("abcde.in");
memmove(replace_n+strlen("abcde.in")+site_diff,replace_n+strlen("abcde.in"),strlen(replace_n)-strlen("abcde.in"));
memcpy(replace_n,"google.in",strlen("google.in"));
skb_put(skb,site_diff);
memcpy(data,temp,datalen+site_diff);
liph->tot_len=htons(datalen+site_diff+20+(ltcph->doff*4)); /*modifing necessary fields*/
liph->check=0;
liph->check=ip_fast_csum((unsigned char *)liph,liph->ihl);
int ipl=liph->ihl * 4;
int ihl=ntohs(liph->tot_len);
ltcph->check = 0;
skb->csum = csum_partial(ltcph, ihl - ipl, 0);
ltcph->check = tcp_v4_check(ihl - ipl,liph->saddr, liph->daddr, skb->csum);
skb->ip_summed = CHECKSUM_NONE;
Retransmission tck packet wireshark capture
I'm trying to modify a custom web server app to work with HTML5 video.
It serves a HTML5 page with a basic <video> tag and then it needs to handle the requests for actual content.
The only way I could get it to work so far is to load the entire video file into the memory and then send it back in a single response. It's not a practical option. I want to serve it piece by piece: send back, say, 100 kb, and wait for the browser to request more.
I see a request with the following headers:
http_version = 1.1
request_method = GET
Host = ###.###.###.###:##
User-Agent = Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20100101 Firefox/16.0
Accept = video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
Accept-Language = en-US,en;q=0.5
Connection = keep-alive
Range = bytes=0-
I tried to send back a partial content response:
HTTP/1.1 206 Partial content
Content-Type: video/mp4
Content-Range: bytes 0-99999 / 232725251
Content-Length: 100000
I get a few more GET requests, as follows
Cache-Control = no-cache
Connection = Keep-Alive
Pragma = getIfoFileURI.dlna.org
Accept = */*
User-Agent = NSPlayer/12.00.7601.17514 WMFSDK/12.00.7601.17514
GetContentFeatures.DLNA.ORG = 1
Host = ###.###.###.###:##
(with no indication that the browser wants any specific part of the file.) No matter what I send back to the browser, the video does not play.
As stated above, the same video will play correctly if I try to send the entire 230 MB file at once in the same HTTP packet.
Is there any way to get this all working nicely through partial content requests? I'm using Firefox for testing purposes, but it needs to work with all browsers eventually.
I know this is an old question, but if it helps you can try the following "Model" that we use in our code base.
class Model_DownloadableFile {
private $full_path;
function __construct($full_path) {
$this->full_path = $full_path;
}
public function get_full_path() {
return $this->full_path;
}
// Function borrowed from (been cleaned up and modified slightly): http://stackoverflow.com/questions/157318/resumable-downloads-when-using-php-to-send-the-file/4451376#4451376
// Allows for resuming paused downloads etc
public function download_file_in_browser() {
// Avoid sending unexpected errors to the client - we should be serving a file,
// we don't want to corrupt the data we send
#error_reporting(0);
// Make sure the files exists, otherwise we are wasting our time
if (!file_exists($this->full_path)) {
header('HTTP/1.1 404 Not Found');
exit;
}
// Get the 'Range' header if one was sent
if (isset($_SERVER['HTTP_RANGE'])) {
$range = $_SERVER['HTTP_RANGE']; // IIS/Some Apache versions
} else if ($apache = apache_request_headers()) { // Try Apache again
$headers = array();
foreach ($apache as $header => $val) {
$headers[strtolower($header)] = $val;
}
if (isset($headers['range'])) {
$range = $headers['range'];
} else {
$range = false; // We can't get the header/there isn't one set
}
} else {
$range = false; // We can't get the header/there isn't one set
}
// Get the data range requested (if any)
$filesize = filesize($this->full_path);
$length = $filesize;
if ($range) {
$partial = true;
list($param, $range) = explode('=', $range);
if (strtolower(trim($param)) != 'bytes') { // Bad request - range unit is not 'bytes'
header("HTTP/1.1 400 Invalid Request");
exit;
}
$range = explode(',', $range);
$range = explode('-', $range[0]); // We only deal with the first requested range
if (count($range) != 2) { // Bad request - 'bytes' parameter is not valid
header("HTTP/1.1 400 Invalid Request");
exit;
}
if ($range[0] === '') { // First number missing, return last $range[1] bytes
$end = $filesize - 1;
$start = $end - intval($range[0]);
} else if ($range[1] === '') { // Second number missing, return from byte $range[0] to end
$start = intval($range[0]);
$end = $filesize - 1;
} else { // Both numbers present, return specific range
$start = intval($range[0]);
$end = intval($range[1]);
if ($end >= $filesize || (!$start && (!$end || $end == ($filesize - 1)))) {
$partial = false;
} // Invalid range/whole file specified, return whole file
}
$length = $end - $start + 1;
} else {
$partial = false; // No range requested
}
// Determine the content type
$finfo = finfo_open(FILEINFO_MIME_TYPE);
$contenttype = finfo_file($finfo, $this->full_path);
finfo_close($finfo);
// Send standard headers
header("Content-Type: $contenttype");
header("Content-Length: $length");
header('Content-Disposition: attachment; filename="' . basename($this->full_path) . '"');
header('Accept-Ranges: bytes');
// if requested, send extra headers and part of file...
if ($partial) {
header('HTTP/1.1 206 Partial Content');
header("Content-Range: bytes $start-$end/$filesize");
if (!$fp = fopen($this->full_path, 'r')) { // Error out if we can't read the file
header("HTTP/1.1 500 Internal Server Error");
exit;
}
if ($start) {
fseek($fp, $start);
}
while ($length) { // Read in blocks of 8KB so we don't chew up memory on the server
$read = ($length > 8192) ? 8192 : $length;
$length -= $read;
print(fread($fp, $read));
}
fclose($fp);
} else {
readfile($this->full_path); // ...otherwise just send the whole file
}
// Exit here to avoid accidentally sending extra content on the end of the file
exit;
}
}
You then use it like this:
(new Model_DownloadableFile('FULL/PATH/TO/FILE'))->download_file_in_browser();
It will deal with sending part of the file or the full file etc and works well for us in this and lots of other situations. Hope it helps.
I want partial range requests, because I'll be doing realtime transcoding, I can't have the file completely transcoded and available upon request.
For response which you don't know the full body content yet (you can't guess the Content-Length, live encoding), use chunk encoding:
HTTP/1.1 200 OK
Content-Type: video/mp4
Transfer-Encoding: chunked
Trailer: Expires
1E; 1st chunk
...binary....data...chunk1..my
24; 2nd chunk
video..binary....data....chunk2..con
22; 3rd chunk
tent...binary....data....chunk3..a
2A; 4th chunk
nd...binary......data......chunk4...etc...
0
Expires: Wed, 21 Oct 2015 07:28:00 GMT
Each chunk is send when it's available: when few frames are encoded or when the output buffer is full, 100kB are generated, etc.
22; 3rd chunk
tent...binary....data....chunk3..a
Where 22 give the chunk byte length in hexa (0x22 = 34 bytes),
; 3rd chunk is extra chunk infos (optional) and tent...binary....data....chunk3..a is the content of the chunk.
Then, when the encoding is finished and all chunks are sent, end by:
0
Expires: Wed, 21 Oct 2015 07:28:00 GMT
Where 0 means there not more chunks, followed by zero or more trailer (allowed header fields) defined in the header (Trailer: Expires and Expires: Wed, 21 Oct 2015 07:28:00 GMT are not required) to provide checksums or digital signatures, etc.
Here is the equivalent of the server's response if the file was already generated (no live encoding):
HTTP/1.1 200 OK
Content-Type: video/mp4
Content-Length: 142
Expires: Wed, 21 Oct 2015 07:28:00 GMT
...binary....data...chunk1..myvideo..binary....data....chunk2..content...binary....data....chunk3..and...binary......data......chunk4...etc...
For more information: Chunked transfer encoding — Wikipedia, Trailer - HTTP | MDN
I'm trying to set up a websocket server in node.js but having problems. I found a bit code here on stackoverflow and heres the servercode I have now:
var net = require("net"), crypto = require("crypto"), users = [];
net.createServer(function(socket) {
this.name = "Anonymous";
users.push(socket);
socket.on('data', function(buffer) {
if(buffer.toString('utf-8').substring(0, 14) === "GET / HTTP/1.1") {
this.securyPattern = /Sec-WebSocket-Key: (.*)/g;
this.key = this.securyPattern.exec(buffer);
this.magic = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11";
this.sha1 = crypto.createHash("sha1");
this.sha1.update(this.key[1] + this.magic);
this.accept = this.sha1.digest("base64");
socket.write("HTTP/1.1 101 Switching Protocols\r\nUpgrade: WebSocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: " + this.accept + "\r\n\r\n");
} else {
console.log(buffer);
console.log(buffer.toString('utf-8'));
}
});
socket.on('end', function() {
users.splice(users.indexOf(socket), 1);
});
}).listen(1337);
Everything works fine as it connects, and users.length is updated when that happens and when someone disconnects.
The problem is that I dont know how to read messages except the header (which is plain text), so the lines that I have to print the buffer and buffer.toString('utf-8') only prints something binary different all the time, example for the word "hello":
<Buffer 81 85 71 dc c1 02 19 b9 ad 6e 1e>
??q??☻↓??n▲
<Buffer 81 85 8e 8f 0f a2 e6 ea 63 ce e1>
????☼???c??
I'm sending this "hello" with Chrome 16 using:
myWebSocket.send("hello"); where myWebSocket is the WebSocket object.
So how do I read and write messages to the socket with this?
Note that after the handshake, the data is framed with 2 or more header bytes at the beginning of each frame. Also, note that payload sent from the client (browser) to the server is masked using a simple 4-byte running XOR mask.
The framing definition is defined in section 5 of the spec
Instead of implementing your own WebSocket server in Node you might consider using a higher level abstraction like Socket.IO.
Here's my code on handling that buffer:
socket.ondata = function(src,start,end) {
src = src.slice(start,end);
var maskKeys = [src[2],src[3],src[4],src[5]];
var dest = new Array();
for(var i=0;i<src.length-6;i++){
var mKey = maskKeys[i%4];
dest[i] = mKey ^ src[6+i];
}
console.log(new Buffer(dest).toString());
}
Found from here: http://songpengfei.iteye.com/blog/1178310