I have an application which uploads a large file in the XML format and sometimes a zip file. Now I want to have that file transferred to other application via REST API. I am thinking to pass the binary data in to json response.
I have the following questions for my approach.
Is sending binary in json the best approach/practice to do it?
Will this be PUT scenario as receiver application doesn't know about new uploaded file?
If that makes it easier for the second service to consume it, I see no problem with it. You can send it on any format you want, as long as it's accepted and you're setting the Content-Type and Accept headers properly.
You use a PUT only when you're sending a complete replacement of the resource at the target URI. If you know the final URI for that and if a GET to the same URI right after the PUT will retrieve as response the same body you just submitted, it makes sense to use PUT, otherwise, use a POST.
Related
In my karate tests i need to write response id's to txt files (or any other file format such as JSON), was wondering if it has any capability to do this, I haven't seen otherwise in the documentation. In the case of no, is there a simple JavaScript function to do so?
Try the karate.write(value, filename) API but we don't encourage it. Also the file will be written only to the current "build" directory which will be target for Maven projects / stand-alone JAR.
value can be any data-type, and Karate will write the bytes (or plain-text) out. There is no built-in support for any other format.
Here is an example.
EDIT: for others coming across this answer in the future the right thing to do is:
don't write files in the first place, you never need to do this, and this question is typically asked by inexperienced folks who for some reason think that the only way to "save" a response before validation is to write it to a file. No, please don't waste your time - and please just match against the response. You can save it (or parts of it) to variables while you make other HTTP requests. And do not write your tests so that scenarios (or features) depend on other scenarios, this is a very bad practice. Also note that by default, Karate will dump all HTTP requests and responses in the log file (typically in target/karate.log) and also in the HTML report.
see if karate.write() works for you as per this answer
write a custom Java (or JS function that uses the JVM) to do what you want using Java interop
Also note that you can use karate.toCsv() to convert JSON into CSV if needed.
My justification for writing to a file is a different one. I am using karate explicitly to implement a mock. I want to expose an endpoint wherein the upstream system will send some basic data through json payload using POST/PUT method and karate will construct the subsequent payload file and stores it the specific folder, and this newly created payload file will be exposed through another GET call.
Is there any limit on sending JSON data to a rest API?I am building an app where I have used a rest API to send an email. Here I want to send app logs to API,but before going through that I want to know about the limitation on send data to rest API.
Secondly which one is better option file or send JSON data to rest API?
In ASP.NET there's by default a 4MB limit of the size of a request. This can be adjusted using the maxRequestLength attribute on the httpRuntime element:
<httpRuntime targetFramework="4.5.2" maxRequestLength="1048576" />
In this example we set the maximum request size to be 1GB.
Secondly which one is better option file or send JSON data to rest API?
If you are planning to send large request payloads I would recommend you using the multipart/form-data content type for the request instead of JSON. This would allow you to directly send the raw bytes in the request payload. If you use JSON then you would need to encode those raw bytes to something like base64 which would make the request even larger. To even further optimize network traffic the client could gzip the raw bytes before sending them over the wire and then unzip the stream on the server.
You may also find the following article useful in setting up this file upload on the server side.
I have a situation where I have to write a api to create a resource and amongst datafields that I need to accept is a string that is basically contents of a html file. As I see it I have a choice between structuring the entire thing as a json object where this field is a string field with urlencoded html string , and having the Content Type as multipart/form-data where each of the fields and the html string (UTF-8 encoded) is a part in the message.
Not using json is something I am not comfortable with as I feel violating the REST standards in not structuring the content of the entity I am about to create thus there is a loss of information for the consumers as they can't tell immediately looking at my api definition about what data to feed to it. But practically multipart/form-data handles stuff like html file content better and more efficient as I will not have to urlencode it and can control the char-encoding also.
What will be a better approach in current context and upholding RESTful principles ? Also are there other trade-offs i should be aware of ? what about parsing a json with a huge string field (~ 200 Kb)embedded?
EDIT :- I was reading some similar questions on SO and one approach that stood out was the 2-step approach of making a first call with metadata to create the entity and then upload the file as an UPDATE process to the created entity wherein we use multipart/form-data. In that context, I guess , what I am asking is how sound is an approach where I send both metadata and the file in a single api call as multipart data , where each metadata field is actually a part in the multipart message as is the file.
The canonical way to upload files to REST API is using the multipart/form-data. As W3 recommendation guide says:
The content type "multipart/form-data" should be used for submitting
forms that contain files, non-ASCII data, and binary data.
Multipart/form-data has advantages over base64 to represent binary data. Is sticked to REST/Http philosophy, and simplify the develop of API clients.
Returning values from Forms: multipart/form-data
W3 Recommendation guide
The good practice is to use multipart/form-data whenever files are uploaded to the server along with database fields. Do not send a base64 JSON string as the request to your Rest API as it might corrupt the file or degrade the performance of your application.
As far as documenting multipart/form-data Rest API for your consumers is concerned you have to force your API consumers to use the same form fields which you have predefined in your web service.
Returning Values from Forms: multipart/form-data
I started using FormData objects everywhere on the client-side, in lieu of regular form input fields, for dynamic REST posts. FormData is presented in a positive light in various tutorials, so I went with it.
However, down the line, this caused me problems when decoding the form data into my Go structs. FormData objects are sent as "multipart/form-data" (regardless of files being sent) and I believe my decoder in Go didn't convert the raw data back to string form. Eventually my SQL queries were throwing panics, as hex data was being sent in where strings should have been.
So with some adjustment, I could use FormData however I've decided to revert to the simple universal recommendation: Use "multipart/form-data" only for special cases like when sending files. Otherwise, just use regular "application/x-www-form-urlencoded".
I'm learning how to use a third party API called Wunderground and I don't know how to request, receive, and use their results which is in a Json format.
If you see their website a developer can sign up for free and receives an API KEY. You can then request weather data in the following URL format.. http://api.wunderground.com/api/KEY/FEATURE/[FEATURE…]/[SETTING…]/q/QUERY.FORMAT
So I have tried it in my web browser by typing some parameters and I received a very long Json file with the correct information (I checked). Problem is I don't have the slightest idea of how to create a variable which can make this request, and even if I was able to do that I don't know where should I receive the file and how to get only the results I want (in this case current weather).
You have to use Titanium.Network.HTTPClient to make request.
For code examples related to Json parsing you can use:
Appcelerator: Using JSON to Build a Twitter Client
HTTPClient()
I am trying to send a generated PDF file (Apache FOP) to the client. I know this can be done by writing the array to the response stream and by setting the correct content type, length and so on in the servlet. My problem is that the whole app was built based on the idea that it will only receive/send JSON. In the servlet's service() method, I have this:
response.setContentType("application/json");
reqBroker.process(request, response);
RequestBroker is the class who processes the JSON (jackson processor), everything is generic and I cannot change it. On top of this, I have to receive the JSON from the request correctly, to access the data and generate my pdf. So those two lines are necessary. But when I send the response, I need to have another content type so that the pdf is displayed correctly in the browser.
So far, I am able to send the byte array as part of the JSON, but then I don't know how to display the array as PDF on the client (if smth like this is even possible).
I would like some suggestions on how can I send my pdf and set the right header, without messing with the JSON. Thanks.
JSON and byte arrays don't mix.
Instead, you should create an <iframe> and point it to a URL that returns a raw PDF.
Take a look here:How to send pdf in json, it lists couple of approaches that you can consider. The easiest way is to convert the binary data into string by using Base64 compression. In C#, this would mean a call to Convert.FromBase64String. However this has space overhead as Base64 compression means around +33% more memory. If you can get away with it, this is the least complicated solution. in case additional size is an issue you can think about zipping it up.