JSON posting, am i pushing JSON too far? - json

Im just wondering if I am pushing JSON too far? and if anyone has hit this before?
I have a xml file:
<?xml version="1.0" encoding="UTF-8"?>
<customermodel:Customer xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:customermodel="http://customermodel" xmlns:personal="http://customermodel/personal" id="1" age="1" name="Joe">
<bankAccounts xsi:type="customermodel:BankAccount" accountNo="10" bankName="HSBC" testBoolean="true" testDate="2006-10-23" testDateTime="2006-10-23T22:15:01+08:00" testDecimal="20.2" testTime="22:15:01+08:00">
<count>0</count>
<bankAddressLine>HSBC</bankAddressLine>
<bankAddressLine>London</bankAddressLine>
<bankAddressLine>31 florence</bankAddressLine>
<bankAddressLine>Swindon</bankAddressLine>
</bankAccounts>
</customermodel:Customer>
Which contains elements and attributes....
Which when i convert to JSON gives me:
{"customermodel:Customer":{"id":"1","name":"Joe","age":"1","xmlns:xsi":"http://www.w3.org/2001/XMLSchema-instance","bankAccounts":{"testDate":"2006-10-23","testDecimal":"20.2","count":"0","testDateTime":"2006-10-23T22:15:01+08:00","bankAddressLine":["HSBC","London","31 florence","Swindon"],"testBoolean":"true","bankName":"HSBC","accountNo":"10","xsi:type":"customermodel:BankAccount","testTime":"22:15:01+08:00"},"xmlns:personal":"http://customermodel/personal","xmlns:customermodel":"http://customermodel"}}
So then i send this too the client.. which coverts to a js object (or whatever) edits some values (the elements) and then sends it back to the server.
So i get the JSON string, and convert this back into XML:
<customermodel:Customer>
<id>1</id>
<age>1</age>
<name>Joe</name>
<xmlns:xsi>http://www.w3.org/2001/XMLSchema-instance</xmlns:xsi>
<bankAccounts>
<testDate>2006-10-23</testDate>
<testDecimal>20.2</testDecimal>
<testDateTime>2006-10-23T22:15:01+08:00</testDateTime>
<count>0</count>
<bankAddressLine>HSBC</bankAddressLine>
<bankAddressLine>London</bankAddressLine>
<bankAddressLine>31 florence</bankAddressLine>
<bankAddressLine>Swindon</bankAddressLine>
<accountNo>10</accountNo>
<bankName>HSBC</bankName>
<testBoolean>true</testBoolean>
<xsi:type>customermodel:BankAccount</xsi:type>
<testTime>22:15:01+08:00</testTime>
</bankAccounts>
<xmlns:personal>http://customermodel/personal</xmlns:personal>
<xmlns:customermodel>http://customermodel</xmlns:customermodel>
</customermodel:Customer>
And there is the problem, is doesn't seem to know the difference between elements/attributes so i can not check against a XSD to check this is now valid?
Is there a solution to this?
I cannot be the first to hit this problem?

JSON does not make sense as an XML encoding, no. If you want to be working with and manipulating XML, then work with and manipulate XML.
JSON is for when you need something that's lighter weight, easier to parse, and easier to write and read. It has a fairly simple structure, that is neither better nor worse than XML, just different. It has lists, associations, strings, and numbers, while XML has nested elements, attributes, and entities. While you could encode each one in the other precisely, you have to ask yourself why you're doing that; if you want JSON use JSON, and if you want XML use XML.

JsonML provides a well thought out standard mapping from XML<->JSON. If you use it, you'll get the benefit of ease-of-manipulation you're looking for on the client with no loss of fidelity in elements/attributes.

I wouldn't encode the xml schema information in the json string-- that seems a little backwards. If you're going to send them JSON, they shouldn't have any inkling that this is anything but JSON. The extra xml will serve to confuse and make the your interface look "leaky".
You might even consider just using xml and avoid the additional layer of abstraction. JSON makes the most sense when you know at least one party is actually using javascript. If this isn't the case it'll still work as well as any other transport format. But if you already have an xml representation it's a little excessive.
On the other hand, if your customer is really using javascript it will make it easier for them to use the data. The only concern is the return trip, and once it's in JSON who do you trust more to do the conversion back to xml correctly? You're probably better qualified for that, since it's your schema.

For this to work you would need to build additional logic/data into your serialize/unserialize methods - probably create something like "attributes" and "data" to hold the different parts:
{"customermodel:Customer":
{
"attributes": {"xmlns:xsi":"...", "xmlns:customermodel":"..."},
"data":
{
"bankAccounts":
{
"attributes": { ... }
"data" :
{
"count":0,
"bankAddressLine":"..."
}
}
}
}

Related

Generic JSON to XML transformation by templating

I'm trying to devise a service to convert a generic JSON data representation into XML data representation.
The first idea that came to my mind (and that I found on the internet) takes advantage of the Go templating utilities.
If I have a JSON data representation like the following:
{
"user": {
"name": "Luca",
"surname": "Rossi"
}
}
I could devise a template like the following:
<xml>
<user name="{{.user.name}}" surname="{{.user.surname}}" />
</xml>
to generate:
<xml>
<user name="Luca" surname="Rossi" />
</xml>
The problem is: Go requires the definition of a struct which declares how to marshal and unmarshal a JSON data representation; at the same time, however, I'd like to provide the template to generate the XML as a service configuration available at runtime.
The question is: "is it possible"?
I know (thanks tothis question) that I can take do something like this:
var anyJson map[string]interface{}
json.Unmarshal(bytes, &anyJson)
The problem comes when I have to access values: I'd be asked to do a type assertion, like
anyJson["id"].(string)
Now, I might be able to know the type of anyJson["id"] by means of JSON schema, for example, but I don't know if I can do a parametric type assertion, something like
anyJson["id"].(typeForIDFromJSONSchema)
When you unmarshal into map[string]interface{}, every nested JSON object will also be map[string]interface{}. So type assertions of the contained elements to string may typically work, but not to any struct type - the unmarshaller would always be unaware of your structs.
So the two options I suggest are
to do it 'the hard way' using type switches and type assertions - this is workable and fast, but not all that nice; or
to use a different tool such as jsoniter or gjson - these might be a little slower but they do allow you to traverse arbitrary JSON graphs
I have used both GJson and Jsoniter. GJson works by reading byte by byte through the input, using buffering to keep its speed up, providing an API that allows inspection of the current element and assertions to convert the values.
Jsoniter looks to me like a cleaner implementation along the lines of successful parsers in Java, but I haven't used it for parsing in this manner yet. (It can also be used simply as a fast drop-in replacement for the standard Go encoding/json.) I suggest you focus on using its Iterator and its WhatIsNext method.

Are you able to subtract in JSON?

Here is my JSON code.
{
"user_email": "{User.Email}",
"activity_date": "{Lead.LastAction.Date}",
"record_id": "{Lead.Id}-{Lead.LastAction.Date}",
"action_type": "{Lead.LastAction}",
"milestone": "{Lead.Milestone}",
"date_added": "{Lead.Date}"
}
Is it possible to add calculations in the code?
For example, can I add a line where the date_added is subtracted from activity_date?
No: JSON is a way to transport JS Objects.
You can do that while you format the JSON in your native language ( for example in PHP or JS serverside), basically creating the JSON object with the result of the calculation.
In JSON just by itself you cannot do that, it's just a data format, it's totally passive, like a text file. (If you happen to use JSONP, then the story would be a bit different, it might be possible, but using JSONP to do such things would step into area of 'hack/exploit' and it probably should not be used in that way:) )
However, I see you are using not only JSON - there is some extra markup like {User.Email}. This is totally outside JSON spec so clearly you are using some form text-templating engine. These can be quite intelligent at times. Check that path, see which one you are using, see what are its features, maybe you can write a custom function or expression to do that subtraction for you. Maybe, just maybe, it's as easy as
"inactivity_period": "{Lead.LastAction.Date - Lead.Date}"
or
"inactivity_period": "{myFunctionThatIWrote(Lead.LastAction.Date, Lead.Date)}"
but that all depends on the templating engine.

Generate XML from XSD repeatable elements

Whats the tool to generate xml from xsd and the generated xml should contain more than one entries for the repeatable elements? I tried out tools that are available on eclipse and some online tools like xml-generator, but none of these work. They all generate only one entry for the repeatable elements.
Note: I want to convert the generated xml to json, but the xml-json convertor treats the repeatable elements in the xml as an array only if it has more than one entry.
Generating XMLs from XSD can be quite challenging, if only because what people expect to see may not be possible to be captured by an XSD.
QTAssistant (I am associated with it) has quite extensive features when it comes to sample XML creation.
The simplest (and dumbest) one (available by right clicking on the element in the graphical XSD visualizer) is still able to create two elements if the associated maxOccurs is greater than one.
However, the XML may be off: just because one may have named a field dateTime, it doesn't mean the generated text node will be a valid date time value, if the schema defined it as a string. The tool will also only create one of the choices (if your schema uses xsd:choice), etc.
QTAssistant can make use of additional metadata which gives the user ultimate control over the generated samples. It can even create thousands of XMLs by doing combinations captured using metadata items. (You should contact us on the support site if you're interested in these scenarios).
Regarding XML to JSON conversion, QTAssistant can also correctly convert XMLs to JSON for repeating fields. Given this XML:
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<!-- Sample XML generated by QTAssistant (http://www.paschidev.com) -->
<fundamo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:common="http://www.myorg.com/xsd/gen_fin">
<response>
<common:code>code1</common:code>
<common:description>description1</common:description>
</response>
<transaction>
<transactionRef>transactionRef1</transactionRef>
<dateTime>dateTime1</dateTime>
<userName>userName1</userName>
</transaction>
</fundamo>
The corresponding JSON is:
{
"response": {
"code": "code1",
"description": "description1"
},
"transaction": [
{
"transactionRef": "transactionRef1",
"dateTime": "dateTime1",
"userName": "userName1"
}
]
}
You may notice that transaction is an array, even if XML has only one of those elements. This conversion works for valid XMLs, as long as you have a defined XSD for all its content. For the past 2+ years we've been calling it "XSD-aware JSON conversion". It is also possible to define casing conversion strategies (e.g. change upper case to lower case since XML elements tend to be upper case, while JSON "people" prefer them lower case).
If you're in for a free tool or to write your own, I am sure you can use the free evaluation as a source of inspiration to address only the specific features you're interested in.

Specifiy type when converting from XML to JSON in MarkLogic

Using MarkLogic 8, I'm using a custom XML to JSON conversion for json:transform-to-json, and I've got it working just about right except the conversion is outputting numbers as strings.
Is there a way to specify that the value of a particular element should be a number value, not a string?
I don't see anything in the doc for json:config, but just in case there's something I've missed, or if you have a neat post-processing trick, I'd love to hear about how to solve this problem.
You can do that by defining an XML Schema for the non-string type elements. Just make sure it is available in the context (by loading it into xdmp:schemas-database()), and that it is recognized (your XML needs to have a namespace that matches the XML Schema, and you might wanna use import schema)..
HTH!

Should JSON null be preserved?

I tried to convert the JSON data
{
"a": {
"b": null
}
}
to XML using an online converter. The response was
<a>
<b />
</a>
Converting this back to JSON using the same converter gave me
{
"a": {
}
}
This made me wonder – if you're explicitly given a null value, are you required to preserve it when dealing with JSON? I'm fairly sure that the XML <a><b /></a> is not equivalent to <a></a>, and especially not <a /> (which happens to be what I get when I continue with the same exercise).
In other words, if I'm handed JSON of unknown origin and am supposed to hand it over to an unknown recepient, am I required to preserve the nulls or can I safely remove them? Conversely, can I rely on my nulls to end up in the same way I outputted them when delivered by third-party software?
Here's a similar question: Should JSON include null values – However, the question there is whether the code should output nulls if you define the format yourself, not what you should do if you don't know anything about the original format.
EDIT – Clarification: The way I asked the question was bad and apparently caused confusion. To rephrase it: I do understand that XML and JSON are different formats and are able to carry different kinds of (meta)data. I do know that null is a valid value, as defined by RFC4627. I do understand that there are different ways to convert between XML and JSON since the formats don't have a 1-to-1 relationship. I do understand that the converter I found might be buggy. However, the fact that the same converter didn't provide the same conversion in both directions (no information was lost when converting from "b": null to <b /> and a similar translation in the opposite direction would have been possible) made me wonder something that I couldn't find an answer to despite attempts:
Is it legal, according to the JSON standard, to treat {"a":{"b":null}} and {"a":{}} as one and the same object when transferring them on behalf of other software?
Note that I'm here assuming that it's legal to add or remove whitespace as I see fit (e.g. pretty-printing, which is okay according to RFC4627), and even to rearrange the name/value pairs in a collection (again according to RFC4627). I simply don't know if a null must be preserved in the same way as significant data, or can be dropped in the same way as insignificant whitespace.
Yes, null is a separate value in JSON, and is distinct from not having an attribute, obviously. Also, you can see this question about nulls in XML. The thing to conclude here isn't that there is something wrong with JSON or XML, but simply that the tools you use aren't coded to handle these cases.
One of the problems in converting JSON to XML is that if you try and make the conversion lossless, you end up with somewhat "unnatural" XML, whereas if you try to create the most natural XML representation, it ends up losing information. That's why there are lots of different converters that all do it in slightly different ways. Choose the one that meets your requirements.