I'm used to sending a request to an endpoint with a method and whatever other options I need attached, e.g. axios({ method: `get`, url: `https://foo.bar/baz&q=123`}).
However, I came across this API https://vndb.org/d11 with docs specifying neither the method nor the url (aside from host, which I supposed isn't enough?) where I'm supposed to send my request, so how am I supposed to get any data out of it? Is it possible to access it via Postman or some such?
Host api.vndb.org
Port (plain tcp) 19534 ('VN')
Port (TLS) 19535
Sample request message looks like:
login {
"protocol" : 1,
"username" : "ayo"
}
0x04
0x04 - but this doesn't even look like JSON, so how do I send it across?
Looking at the VNDB API documentation it says:
all communication between the client and the server is done using one
TCP connection. This connection stays alive until it is explicitely
closed by either the client or the server.
so it's not like a normal REST API where you GET, POST etc. You open a connection and leave it open. It's session based, hence the initial login command and the 0x04 byte at the end of the JSON is the
End Of Transmission
character. So it works like a two way radio. You say "login, over" where the "over" is the 0x04 byte. That tells the server a command is waiting for it to process. I presume without the 0x04 character the command won't be interpreted. A bit like typing a command and not hitting the return key.
As for how to use the API, you could study this NodeJS VNDB client.
Related
I am writing an air application in which some keystrokes are sent that activate functions.
Now, since this means that the application shall not be iconized (otherwise the shortcuts would not work), I want to understand if it is possible to send commands (variables) using http query strings.
I have an external device that is capable to send the requests for example: http://localhost/?var=1.
I tried opening a socket but it receives 0 length data when I send something.
Any other suggestion is of course welcome.
Thank you
Protocol: A standard to define a method of exchanging data over a network.
If a browser wants to communicate with a server, it has to create an HTTP request and send that HTTP request to the server to convey its request of resources and options. The server receives the request and process it and do the needful and create an HTTP response to send to the browser. The browser has to follow the HTTP specification in creating the HTTP request. The server also has to follow the HTTP specification in creating the HTTP response. This is how the communication between the browser and the server happens in a standard way to avoid conflicts by following the HTTP protocol.
Json Wire Protocol: A client has an object that has to be sent to a server. The client converts this object into a JSON object and sends it to the server. The server parses the JSON object and converts it back to object for use. The server converts the response object into a JSON object and sends it back to the client. The client then converts the JSON object to object for use.
Why the later is called as Json Wire Protocol?
You are pretty correct both about Protocol and JsonWireProtocol. At this point it is worth to mention that, earlier all implementations of WebDriver that communicated with the browser, or a RemoteWebDriver server shall use a common wire protocol. This wire protocol defines a RESTful web service using JSON over HTTP.
JSON Wire Protocol is an abstract specification of how automation behavior like clicking or typing or whatever you actually want to do with your automation script is mapped to selenium or appium or HTTP requests and response. The protocol will assume that the WebDriver API has been "flattened", but there is an expectation that client implementations will take a more Object-Oriented approach, as demonstrated in the existing Java API. The wire protocol is implemented in request/response pairs of "commands" and "responses".
What is JSON Wire protocol?
JSON (JavaScript Object Notation) is a lightweight format for data exchange between client and server. Applications use JSON objects to send and receive data between each other in the web world. JSON data structure is industry standard and can be used for sending and receiving data as Key & Value pair. Some people say its a very nice alternative for XML. We can save JSON files as .json extension.
How JSON looks like?
A simple json file looks like below and there are many online editors which can be used to edit and verify JSON structure.
{
"Student":{
"FirstName":"Pawan",
"LastName":"Garia",
"IdNumber":"12345",
"City" : "New Delhi",
"EmailID" : "email#gmail.com" }
}
Why JSON Wire Protocol was used in first place?
To implement a client-server architecture which can give us the following benefits.
You write test in any programming language.
You can perform or run test on cloud services like SauceLabs, BrowserStack or Selenium Grid setup.
You are not bound to run test only on the local machine.
Different Drivers(FirefoxDriver, ChromeDriver) can be crated for browsers and separate implementation by using the same standards.
So client-server implementation requires a standard set of the specification beforehand so that Server and Client should be in sync with each other in term of what is coming and going on request and response. It’s something like a language of communication with each other. So we need some common specification to solve this kind of requirement and the solution was HTTP.
Why HTTP is the solution?
HTTP is a standard for web and can be a good base for the specification. Every programming language has a good HTTP libraries which can be used for creating client and server for request and response calls.
How JSON Wire protocol worked with HTTP?
HTTP request and response are generally made of GET and POST requests which is out of scope for this discussion.
Current status
From Selenium perspective, JSON Wire Protocol is obsolete now and the WebDriver W3C Living Document is the new implementation.
WebDriver Communication
The WebDriver protocol is organised into commands. Each HTTP request with a method and template defined in the specification represents a single command and hence each command produces a single HTTP response. In response to a command, the remote end will run a series of actions known as remote end steps. These provide the sequences of actions that a remote end takes when it receives a particular command.
Command Processing
The remote end is an HTTP server reading requests from the client and writing responses typically over a TCP socket. In the specification the communication is modeled as the data transmission between a particular local end and remote end with a connection to which the remote end may write bytes and read bytes. The exact details of how this connection works and how it is established is a bigger topic and out of scope for this question. After a connection has been established, the remote end must read bytes from the connection until a complete HTTP request can be constructed from the data. If it is not possible to construct a complete HTTP request, the remote end must either close the connection, return an HTTP response with status code 500, or return an error with error code unknown error.
Outro
Difference between JsonWireProtocol mechanisms and the new standards in W3C Living Document when using Selenium
What I am tried to do is to have a lambda function proccess emails forwarded by mailgun.
So far, I have setup mailgun's route so it will forward emails to a AWS api gateway, then the api gateway triggers a lambda function.
The problem comes when I try to process the mail, instead of getting a pretty Json that I am expecting inside the lambda's event.body, I m getting raw post form data like
--cff4e6b3-a3a4-4131-bb8d-90a73f1b4c36\r\nContent-Disposition: form-data; name=\"Content-Type\"\r\n\r\nmultipart/mixed; boundary=\"001a1140216cee404d05440c49e7\"\r\n--cff4e6b3-a3a4-4131-bb8d-90a73f1b4c36\r\nContent-Disposition: form-data; name=\"Date\"\r\n\r\nTue, 20 Dec 2016 13:40:53 +1300\r\n--cff4e6b3-a3a4-4131-bb8d-90a73f1b4c36\r\nContent ......
My question is, what should I do to get the JSON version of the forwarded emails in lambda?
Not sure if you ever came to a solution, but I have this working with the following settings.
Setup your API Gateway method to use "Use Lambda Proxy integration"
In your lambda (I use node.js) use busboy to work through the multi-part submission from mailgun. (use this post for help with busboy Busboy help)
Make sure that any code you are going to execute after all busboy is complete is executed in the 'finish' portion of the busboy code.
This suggests that your mailgun route is misconfigured and ends with a MIME request:
When you specify a URL of your application as a route destination through a forward() action, Mailgun will perform an HTTP POST request into it using one of two following formats:
Fully parsed: Mailgun will parse the message, transcode it into UTF-8 encoding, process attachments, and attempt to separate quoted parts from the actual message. This is the preferred option.
Raw MIME: message is posted as-is. In this case you are responsible for parsing MIME. To receive raw MIME messages, the destination URL must end with mime
From Receiving Messages via HTTP through a forward() action
I am working on a TCP-based proxy that must first do a REQ/REPLY handshake in json on a given connection. Because JSON is a self-delimiting protocol I reach for Go's json.Decoder to pull off this work which does the job nicely.
Here are the steps I take:
Dial a connection to a remote server
Write a single json request to a remote server (REQ)
Read a single json reply from the same remote server (completing the proxy handshake REPLY)
Upon a valid json handshake, pass the client connection onto another part of the code which will (going forward) switch to a text based protocol from this point on.
The problem is, when json.Decoder reads data into its internal buffer it can potentially read more data than it needs in which case the json.Decoder has a Buffered() method which gives back an io.Reader with the remainder of the data.
This data (available in the Buffered() method) is now the text-based protocol data which needs to get read from the connection after the json hand-shake did its work. But if I pass the connection forward as is without considering the left over buffer, the connection gets into a locked state because it is waiting to read this data which never comes. The code that deals with the text-based protocol expects a net.Conn going forward and once I pass the connection forward (after the json handshake has been made) the code utilizing the connection understands how to speak the text-based protocol at this point on. So there should be a clear boundary of work.
My question is what is the ideal way to solve this issue so I can still take advantage of the json.Decoder, but ensure that when I pass the connection to a different part of the code in my proxy I know the start of the data for the text-based protocol will still be readable. I somehow need to take the remaining data in the json.Decoder's Buffered() method and put that back in front of the connection so it can be properly read going forward.
Any insight is much appreciated.
You can try
type ConnWithBuffIncluded struct{ //Implement net.Conn so can be passed through pipeline
net.Conn
json.Decoder
}
func (x ConnWithBuffIncluded) Read(p []byte) (n int, err error){ //Will Read both sources
return io.MultiReader(x.Decoder.Buffered(), x.Conn).Read(p)
}
I am implementing a Jetty Websocket servlet.
When the server receives a new connection, I want to send a message that will be read by websocket's onopen function. I want this message to be sent only during the open and not using the regular connection.SendMessage() function. Is it possible to do that? and how?
Don't forget the query string. It's valid in WebSocket url.
new Websocket('ws://yoursite.com/path?a=1&b=2&c=3')
Then you can easily parse this url on server side to retrieve the data.
There is no support for this in the protocol but you could fudge something yourself.
When your server completes a handshake, store the initial message you want to deliver to a client.
In your client's onopen function, send a "read initial message" request.
In your server, check that this client hasn't read its initial message; respond with the message; set a flag saying that the initial message has been sent.
Your client and server are both now free to send other messages.