Incorrect order of *Provider - json

I use resteasy 2.3.6.Final
And met strange issue with JSON parsing.
I send to server the JSON like:
{
"prop1": "val1",
"prop2": "val2",
"#Class": "come.home.CommonDTO"
}
And have the response:
javax.xml.bind.UnmarshalException - with linked exception: [com.sun.istack.internal.SAXParseException2; columnNumber: 0; unexpected element (uri:"", local:"prop1"). Expected elements are <{}commonDTO>]
So seems that that Resteasy doesn't see JSON properties.
The most strange thing, that it works it my local PC. Bur is not working at remote tesing server.
I was debuged the code and find out, that for parsing JSON, resteasy uploads the list of MessageBodyReader, like:
class: ResteasyProviderFactory
method: getMessageBodyReader
List<ResteasyProviderFactory.SortedKey> readers =
this.messageBodyReaders.getPossible(mediaType, type);
And I have different order of those MessageBodyReaders.
In my local PC is
org.jboss.resteasy.plugins.providers.jackson.ResteasyJacksonProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JsonCollectionProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JsonMapProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonXmlRootElementProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonXmlSeeAlsoProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonXmlTypeProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonElementProvider
org.jboss.resteasy.plugins.providers.DataSourceProvider
org.jboss.resteasy.plugins.providers.StringTextStar
org.jboss.resteasy.plugins.providers.InputStreamProvider
org.jboss.resteasy.plugins.providers.ByteArrayProvider
org.jboss.resteasy.plugins.providers.FileProvider
But in remote server the order is different:
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonXmlRootElementProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JsonCollectionProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JsonMapProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonXmlSeeAlsoProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonXmlTypeProvider
org.jboss.resteasy.plugins.providers.jaxb.json.JettisonElementProvider
org.jboss.resteasy.plugins.providers.jackson.ResteasyJacksonProvider
org.jboss.resteasy.plugins.providers.DataSourceProvider
org.jboss.resteasy.plugins.providers.StringTextStar
org.jboss.resteasy.plugins.providers.InputStreamProvider
org.jboss.resteasy.plugins.providers.ByteArrayProvider
org.jboss.resteasy.plugins.providers.FileProvider
Somehow this JettisonXmlRootElementProvider comes at fist position and breaks the JSON parsing.
I came deeper and foud that Providers are filling in org.jboss.resteasy.core.MediaTypeMap.
They are uploading and sorting:
Collections.sort(matches, new MediaTypeMap.TypedEntryComparator(type));
cached = convert(matches);
if (useCache) {
this.classCache.put(cacheEntry, cached);
}
But! sorting works correctly, and seems like the registration of Providers is incorrect:
class: RegisterBuiltin
method: registerProviders
Enumeration en = Thread.currentThread().getContextClassLoader().getResources("META-
INF/services/" + Providers.class.getName());
And the order of this resources is different in different environments!
So that's the case.
But I have no idea why this order is different...
PS.
In all enviromnents openjdk version "1.8.0_265"

Related

How to stop Angular 6 from sanitizing json payload

I recently upgraded my angular project from version 4.2 to version 6. Everything works great, except I noticed that angular is now removing functions from my json payload when making a post request(this may be happening for other http options also but haven't checked them). Is there a way to disable this or a work around besides json.stringify?
Here's the json payload that I'm trying to post:
{
name: "ghq_employeesTotal",
fieldMapping: ['item1'],
translation: Ć’unction(val){ return val.toLowerCase()}
}
Here's the payload that I see being sent in the network tab:
{
name: "ghq_employeesTotal",
fieldMapping: ['item1']
}
As you can see it's completely removing my translation property.
Here's my function sending the post request:
public publishConfig(config): Observable<any>{
return this.http.post<any>(this.serviceURL + 'publish', config);
}
Ć’unction(val){ return val.toLowerCase()} is not a valid JSON data type.
JSON only supports arrays, objects and primitive values.

Angular resource 404 Not Found

I've read other posts that have similar 404 errors, my problem is that I can correctly query the JSON data, but can't save without getting this error.
I'm using Angular's $resource to interact with a JSON endpoint. I have the resource object returning from a factory as follows:
app.factory('Product', function($resource) {
return $resource('api/products.json', { id: '#id' });
});
My JSON is valid and I can successfully use resource's query() method to return the objects inside of my directive, like this:
var item = Product.query().$promise.then(function(promise) {
console.log(promise) // successfully returns JSON objects
});
However, when I try to save an item that I've updated, using the save() method, I get a 404 Not Found error.
This is the error that I get:
http://localhost:3000/api/products.json/12-34 404 (Not Found)
I know that my file path is correct, because I can return the items to update the view. Why am I getting this error and how can I save an item?
Here is my data structure:
[
{
"id": "12-34",
"name": "Greece",
"path": "/images/athens.png",
"description": ""
},
...
]
By default the $save method use the POST verb, you will need to figure out which HTTP verbs are accepted by your server en order to make an update, most modern api servers accept PATCH or PUT requests for updating data rather than POST.
Then configure your $resource instance to use the proper verb like this :
app.factory('Product', function($resource) {
return $resource('api/products.json', { id: '#id' }, {'update': { method:'PUT' }});
});
check $resource docs for more info.
NOTE: $resource is meant to connect a frontend with a backend server supporting RESTful protocol, unless you are using one to receive data & save it into a file rather than a db.
Otherwise if you are only working with frontend solution where you need to implement $resource and have no server for the moment, then use a fake one, there is many great solutions out there like deployd.
You probably don't implement POST method for urls like /api/products.json/12-34. POST method is requested from angular for saving a new resource. So you need to update your server side application to support it and do the actual saving.
app.factory('Product', function($resource) {
return $resource('api/products.json/:id', { id: '#id' });
});
Try adding "/:id" at the end of the URL string.

DART lang reading JSON file saved in the client, i.e. without using a server

I'm trying to read data from JSON file, using the blow code:
void makeRequest(Event e){
var path='json/config.json';
var httpRequest= new HttpRequest();
httpRequest
..open('GET', path)
..onLoadEnd.listen((e)=>requestComplete(httpRequest))
..send('');
}
this worked very well when the app run as http:// .../ index.html, but gave the below error when trying to open it as file:///.../index.html
Exception: NetworkError: Failed to load 'file:///D:/DartApp/web/json/config.json'. main.dart:53makeRequest main.dart:53<anonymous closure>
Is there another way, other than httpRequest that can read JSON file from client side!
I understand I've 3 options, 2 of them only can use HttPRequest, which are:
saving the file of the server, and reading it from the server => can use HttpRequesit
saving the file on the server, and reading it from the client => can use HttpRequesit
saving the file on the client, and reading it from the client itself => CAN NOT use HTTPRequest
I'm searching for the way to do the 3rd option, which is like making off-line Android App using webview, or making off-line Chrome packaged app, i.e I do not want to use a server at all. thanks
thanks
If all you need is the data in the json file, you can just include that data in your .dart files (as a Map variable/constant, for example).
Map config = {
"displayName": "My Display Name",
"anotherProperty": 42,
"complexProperty": {
"value_1": "actual value",
"value_2": "another value"
}
};
If you need the actual json, you can put in a String. Something like:
const configJson = '''
{ "displayName": "My Display Name",
"anotherProperty": 42,
"complexProperty": {
"value_1": "actual value",
"value_2": "another value"
}
}
''';
The json data can be in a separate .dart file, which can be included as part of the same library (through part of ...), or imported (import 'package:mypackage/json.dart';).
If you're looking for something that you can change and the changes are persisted, you're going to need to use some sort of offline storage, which can be web storage if you're running in a browser. You can use the approach above to define inital config data, store it in web storage, and from then on read and edit it from there.
[Previous answer below, before original question was edited.]
Sorry, read "client side", thought "server side". My mistake.
If by "client side" you mean "running in a browser", and you're trying to access a json file which is on the server, then no, there isn't any other way, other than an http request. In fact, that's the only way to read any file on the server, not just json ones. (Well, I guess you could open a WebSocket and stream the content, but that doesn't seem to be a solution you're looking for.)
[Old solution below, before my mistake (server vs client) was pointed out.]
Try:
// THIS DOESN'T WORK IN A BROWSER ENVIRONMENT (aka client side)
import 'dart:io';
import 'dart:convert';
// ...
new File('json/config.json')
.readAsString()
.then((fileContents) => json.decode(fileContents))
.then((jsonData) {
// do whatever you want with the data
});
This poor example works fine in the chrome dev editor dart web app example.
Using HttpRequest.getString works fine with filename and path.
Chris has a good write for json web service stuff at
https://www.dartlang.org/articles/json-web-service/
import 'dart:html';
import 'dart:convert';
void main() {
HttpRequest.getString('json/config.json').then((myjson) {
Map data = JSON.decode(myjson);
var version = data["version"];
var element = new DivElement();
element.text = "version = $version";
document.body.children.add(element);
});
}

Gson Syntax Exception causing Retrofit Error

I am in the process of connecting to a custom RESTful API using Retrofit. I have testing communicating with the device/API via Curl and with some test Java code running on a non-Android system (mac os x). The curl and java commands return the expected response.
However, using retrofit, I am unsure if I am using an inadequate configuration in my RestAdapter (i.e. RestAdapter.Builder()) to talk to this device.
The error I am getting appears to be that the entire json contents is not returned and only the first 47 bytes or so. I expect this is a behavior of the device I'm talking to, and am curious if this means I need to implement an Asynchronous callback as described in the Retrofit API docs. Before I do this I wanted to get feedback from some who have more experience with Retrofit.
The error is as follows (sanitized for public consumption):
12-06 08:50:52.962 28267-1735/com.mycompany.project D/Retrofit? [ 12-06 08:50:52.972 28267: 1735 D/Retrofit ]
{"OBJECT1":{"#Version":1,"OBJECTARRAY1":[
12-06 08:50:52.972 28267-1735/com.mycompany.project D/Retrofit? <--- END HTTP (46-byte body)
12-06 08:50:52.982 28267-1735/com.mycompany.project W/System.err retrofit.RetrofitError: retrofit.converter.ConversionException: com.google.gson.JsonSyntaxException: java.io.EOFException: End of input at line 1 column 47
I am setting up the RestAdapter as such:
RestAdapter restAdapter = new RestAdapter.Builder()
.setServer(serverUrl)
.setClient(new OkClient(RestUtils.getHttpClient(3000, 3000, username, password)))
.setRequestInterceptor(new RequestInterceptor() {
#Override
public void intercept(RequestFacade requestFacade) {
requestFacade.addHeader("Accept", "application/json");
requestFacade.addHeader("Client-Id", "12345");
}
})
.setLogLevel(RestAdapter.LogLevel.FULL)
.build();
service = restAdapter.create(OBJECT1.class);
Any insight into this problem would be greatly appreciated. I understand this error is related to the java.io.EOFException, but have been unable to verify the contents I am getting returned from the device, except for the output shown above in the error. I am heavily leaning towards the async vs. sync being the issue, but am open to any recommendations.
Cheers!

Is there any standard for JSON API response format?

Do standards or best practices exist for structuring JSON responses from an API? Obviously, every application's data is different, so that much I'm not concerned with, but rather the "response boilerplate", if you will. An example of what I mean:
Successful request:
{
"success": true,
"payload": {
/* Application-specific data would go here. */
}
}
Failed request:
{
"success": false,
"payload": {
/* Application-specific data would go here. */
},
"error": {
"code": 123,
"message": "An error occurred!"
}
}
Yes there are a couple of standards (albeit some liberties on the definition of standard) that have emerged:
JSON API - JSON API covers creating and updating resources as well, not just responses.
JSend - Simple and probably what you are already doing.
OData JSON Protocol - Very complicated.
HAL - Like OData but aiming to be HATEOAS like.
There are also JSON API description formats:
Swagger
JSON Schema (used by swagger but you could use it stand alone)
WADL in JSON
RAML
HAL because HATEOAS in theory is self describing.
Google JSON guide
Success response return data
{
"data": {
"id": 1001,
"name": "Wing"
}
}
Error response return error
{
"error": {
"code": 404,
"message": "ID not found"
}
}
and if your client is JS, you can use if ("error" in response) {} to check if there is an error.
I guess a defacto standard has not really emerged (and may never).
But regardless, here is my take:
Successful request:
{
"status": "success",
"data": {
/* Application-specific data would go here. */
},
"message": null /* Or optional success message */
}
Failed request:
{
"status": "error",
"data": null, /* or optional error payload */
"message": "Error xyz has occurred"
}
Advantage: Same top-level elements in both success and error cases
Disadvantage: No error code, but if you want, you can either change the status to be a (success or failure) code, -or- you can add another top-level item named "code".
Assuming you question is about REST webservices design and more precisely concerning success/error.
I think there are 3 different types of design.
Use only HTTP Status code to indicate if there was an error and try to limit yourself to the standard ones (usually it should suffice).
Pros: It is a standard independent of your api.
Cons: Less information on what really happened.
Use HTTP Status + json body (even if it is an error). Define a uniform structure for errors (ex: code, message, reason, type, etc) and use it for errors, if it is a success then just return the expected json response.
Pros: Still standard as you use the existing HTTP status codes and you return a json describing the error (you provide more information on what happened).
Cons: The output json will vary depending if it is a error or success.
Forget the http status (ex: always status 200), always use json and add at the root of the response a boolean responseValid and a error object (code,message,etc) that will be populated if it is an error otherwise the other fields (success) are populated.
Pros: The client deals only with the body of the response that is a json string and ignores the status(?).
Cons: The less standard.
It's up to you to choose :)
Depending on the API I would choose 2 or 3 (I prefer 2 for json rest apis).
Another thing I have experienced in designing REST Api is the importance of documentation for each resource (url): the parameters, the body, the response, the headers etc + examples.
I would also recommend you to use jersey (jax-rs implementation) + genson (java/json databinding library).
You only have to drop genson + jersey in your classpath and json is automatically supported.
EDIT:
Solution 2 is the hardest to implement but the advantage is that you can nicely handle exceptions and not only business errors, initial effort is more important but you win on the long term.
Solution 3 is the easy to implement on both, server side and client but it's not so nice as you will have to encapsulate the objects you want to return in a response object containing also the responseValid + error.
The RFC 7807: Problem Details for HTTP APIs is at the moment the closest thing we have to an official standard.
Following is the json format instagram is using
{
"meta": {
"error_type": "OAuthException",
"code": 400,
"error_message": "..."
}
"data": {
...
},
"pagination": {
"next_url": "...",
"next_max_id": "13872296"
}
}
I will not be as arrogant to claim that this is a standard so I will use the "I prefer" form.
I prefer terse response (when requesting a list of /articles I want a JSON array of articles).
In my designs I use HTTP for status report, a 200 returns just the payload.
400 returns a message of what was wrong with request:
{"message" : "Missing parameter: 'param'"}
Return 404 if the model/controler/URI doesn't exist
If there was error with processing on my side, I return 501 with a message:
{"message" : "Could not connect to data store."}
From what I've seen quite a few REST-ish frameworks tend to be along these lines.
Rationale:
JSON is supposed to be a payload format, it's not a session protocol. The whole idea of verbose session-ish payloads comes from the XML/SOAP world and various misguided choices that created those bloated designs. After we realized all of it was a massive headache, the whole point of REST/JSON was to KISS it, and adhere to HTTP. I don't think that there is anything remotely standard in either JSend and especially not with the more verbose among them. XHR will react to HTTP response, if you use jQuery for your AJAX (like most do) you can use try/catch and done()/fail() callbacks to capture errors. I can't see how encapsulating status reports in JSON is any more useful than that.
For what it's worth I do this differently. A successful call just has the JSON objects. I don't need a higher level JSON object that contains a success field indicating true and a payload field that has the JSON object. I just return the appropriate JSON object with a 200 or whatever is appropriate in the 200 range for the HTTP status in the header.
However, if there is an error (something in the 400 family) I return a well-formed JSON error object. For example, if the client is POSTing a User with an email address and phone number and one of these is malformed (i.e. I cannot insert it into my underlying database) I will return something like this:
{
"description" : "Validation Failed"
"errors" : [ {
"field" : "phoneNumber",
"message" : "Invalid phone number."
} ],
}
Important bits here are that the "field" property must match the JSON field exactly that could not be validated. This allows clients to know exactly what went wrong with their request. Also, "message" is in the locale of the request. If both the "emailAddress" and "phoneNumber" were invalid then the "errors" array would contain entries for both. A 409 (Conflict) JSON response body might look like this:
{
"description" : "Already Exists"
"errors" : [ {
"field" : "phoneNumber",
"message" : "Phone number already exists for another user."
} ],
}
With the HTTP status code and this JSON the client has all they need to respond to errors in a deterministic way and it does not create a new error standard that tries to complete replace HTTP status codes. Note, these only happen for the range of 400 errors. For anything in the 200 range I can just return whatever is appropriate. For me it is often a HAL-like JSON object but that doesn't really matter here.
The one thing I thought about adding was a numeric error code either in the the "errors" array entries or the root of the JSON object itself. But so far we haven't needed it.
Their is no agreement on the rest api response formats of big software giants - Google, Facebook, Twitter, Amazon and others, though many links have been provided in the answers above, where some people have tried to standardize the response format.
As needs of the API's can differ it is very difficult to get everyone on board and agree to some format. If you have millions of users using your API, why would you change your response format?
Following is my take on the response format inspired by Google, Twitter, Amazon and some posts on internet:
https://github.com/adnan-kamili/rest-api-response-format
Swagger file:
https://github.com/adnan-kamili/swagger-sample-template
The point of JSON is that it is completely dynamic and flexible. Bend it to whatever whim you would like, because it's just a set of serialized JavaScript objects and arrays, rooted in a single node.
What the type of the rootnode is is up to you, what it contains is up to you, whether you send metadata along with the response is up to you, whether you set the mime-type to application/json or leave it as text/plain is up to you (as long as you know how to handle the edge cases).
Build a lightweight schema that you like.
Personally, I've found that analytics-tracking and mp3/ogg serving and image-gallery serving and text-messaging and network-packets for online gaming, and blog-posts and blog-comments all have very different requirements in terms of what is sent and what is received and how they should be consumed.
So the last thing I'd want, when doing all of that, is to try to make each one conform to the same boilerplate standard, which is based on XML2.0 or somesuch.
That said, there's a lot to be said for using schemas which make sense to you and are well thought out.
Just read some API responses, note what you like, criticize what you don't, write those criticisms down and understand why they rub you the wrong way, and then think about how to apply what you learned to what you need.
JSON-RPC 2.0 defines a standard request and response format, and is a breath of fresh air after working with REST APIs.
The basic framework suggested looks fine, but the error object as defined is too limited. One often cannot use a single value to express the problem, and instead a chain of problems and causes is needed.
I did a little research and found that the most common format for returning error (exceptions) is a structure of this form:
{
"success": false,
"error": {
"code": "400",
"message": "main error message here",
"target": "approx what the error came from",
"details": [
{
"code": "23-098a",
"message": "Disk drive has frozen up again. It needs to be replaced",
"target": "not sure what the target is"
}
],
"innererror": {
"trace": [ ... ],
"context": [ ... ]
}
}
}
This is the format proposed by the OASIS data standard OASIS OData and seems to be the most standard option out there, however there does not seem to be high adoption rates of any standard at this point. This format is consistent with the JSON-RPC specification.
You can find the complete open source library that implements this at: Mendocino JSON Utilities. This library supports the JSON Objects as well as the exceptions.
The details are discussed in my blog post on Error Handling in JSON REST API
For those coming later, in addition to the accepted answer that includes HAL, JSend, and JSON API, I would add a few other specifications worth looking into:
JSON-LD, which is a W3C Recommendation and specifies how to build interoperable Web Services in JSON
Ion Hypermedia Type for REST, which claims itself as a "a simple and intuitive JSON-based hypermedia type for REST"
There is no lawbreaking or outlaw standard other than common sense. If we abstract this like two people talking, the standard is the best way they can accurately understand each other in minimum words in minimum time. In our case, 'minimum words' is optimizing bandwidth for transport efficiency and 'accurately understand' is the structure for parser efficiency; which ultimately ends up with the less the data, and the common the structure; so that it can go through a pin hole and can be parsed through a common scope (at least initially).
Almost in every cases suggested, I see separate responses for 'Success' and 'Error' scenario, which is kind of ambiguity to me. If responses are different in these two cases, then why do we really need to put a 'Success' flag there? Is it not obvious that the absence of 'Error' is a 'Success'? Is it possible to have a response where 'Success' is TRUE with an 'Error' set? Or the way, 'Success' is FALSE with no 'Error' set? Just one flag is not enough? I would prefer to have the 'Error' flag only, because I believe there will be less 'Error' than 'Success'.
Also, should we really make the 'Error' a flag? What about if I want to respond with multiple validation errors? So, I find it more efficient to have an 'Error' node with each error as child to that node; where an empty (counts to zero) 'Error' node would denote a 'Success'.
I used to follow this standard, was pretty good, easy, and clean on the client layer.
Normally, the HTTP status 200, so that's a standard check which I use at the top. and I normally use the following JSON
I also use a template for the API's
dynamic response;
try {
// query and what not.
response.payload = new {
data = new {
pagination = new Pagination(),
customer = new Customer(),
notifications = 5
}
}
// again something here if we get here success has to be true
// I follow an exit first strategy, instead of building a pyramid
// of doom.
response.success = true;
}
catch(Exception exception){
response.success = false;
response.message = exception.GetStackTrace();
_logger.Fatal(exception, this.GetFacadeName())
}
return response;
{
"success": boolean,
"message": "some message",
"payload": {
"data" : []
"message": ""
... // put whatever you want to here.
}
}
on the client layer I would use the following:
if(response.code != 200) {
// woops something went wrong.
return;
}
if(!response.success){
console.debug ( response.message );
return;
}
// if we are here then success has to be true.
if(response.payload) {
....
}
notice how I break early avoiding the pyramid of doom.
I use this structure for REST APIs:
{
"success": false,
"response": {
"data": [],
"pagination": {}
},
"errors": [
{
"code": 500,
"message": "server 500 Error"
}
]
}
A bit late but here is my take on HTTP error responses, I send the code, (via status), the generic message, and details (if I want to provide details for a specific endpoint, some are self explanatory so no need for details but it can be custom message or even a full stack trace depending on use case). For success it's a similar format, code, message and any data in the data property.
ExpressJS response examples:
// Error
res
.status(422)
.json({
error: {
message: 'missing parameters',
details: `missing ${missingParam}`,
}
});
// or
res
.status(422)
.json({
error: {
message: 'missing parameters',
details: 'expected: {prop1, prop2, prop3',
}
});
// Success
res
.status(200)
.json({
message: 'password updated',
data: {member: { username }}, // [] ...
});
Best Response for web apis that can easily understand by mobile developers.
This is for "Success" Response
{
"code":"1",
"msg":"Successfull Transaction",
"value":"",
"data":{
"EmployeeName":"Admin",
"EmployeeID":1
}
}
This is for "Error" Response
{
"code": "4",
"msg": "Invalid Username and Password",
"value": "",
"data": {}
}