Generate java classes from a JSON schema - json

I would like to generate JAVA classes from a given JSON Schema draft 4 version
I evaluated couple of tools and jsonschema2pojo was found to be useful. But it supports json schema draft-3 version only(although json schema draft 4 is in their roadmap).
Can anyone suggest me a tool or a way to generate java classes from a json schema (compliant to json schema draft4)
?
Thanks in advance.

You might try cog, a general purpose code generator written in Ruby. I put a simple project on github called json2java which demonstrates how cog might be used to generate Java classes from json data.
Not sure exactly what you want to do, but here is what I assumed. The json data would look something like this
{
"classname": "Sample",
"methods": [
{
"name": "foo",
"rtype": "void",
"params": [
{
"name": "arg1",
"type": "int"
}
]
},
{
"name": "bar",
"rtype": "int",
"params": []
}
]
}
And the corresponding Java class would look something like this
public class Sample {
void foo(int arg1) {
// keep: foo {
// While the interface in this example is generated,
// the method bodies are preserved between multiple invocations
// of the generator.
// It doesn't have to be done this way, the method bodies can be
// generated aswell, all depends on what your json data encodes
// keep: }
}
int bar() {
// keep: bar {
return 1;
// keep: }
}
}
If you want to try cog, install it like this gem install cog, and run generators like this cog gen. Check out the cog homepage for documentation.

Related

Does config.GetSection not work in Azure Functions? And what is the recommended alternative?

Azure Function with a complex (List of objects) configuration type is working locally (with that complex type in local.settings.json) but fails to read / create list of objects in Azure (with that complex type in Azure Function configuration settings). I'm looking for the recommended / optimal way to support that across both platforms / methods of access.
This works great in my local.settings.json where I use the configuration builder and pull data out like
var myList = config.GetSection("ConfigurationList").Get<List<MyType>>();
however this doesn't seem to work in Azure Functions?? Now I think that is because in local.settings.json it is a json file and looks like
"ConfigurationList" : [ { "Name": "A", "Value": 2 }, { "Name": "B", "Value": 3 }]
while in Azure Functions it is a setting "ConfigurationList" with the value
[ { "Name": "A", "Value": 2 }, { "Name": "B", "Value": 3 }]
(so there isn't really a "section" in Azure Functions?)
It seems like the "easy" solution to this is to just change the .json to be a quoted string and deserialize the string (and then it would work the same in both places); but that doesn't seem like it would be the "best" (or "recommended" solution)
i.e. something like
"ConfigurationList" : "[ { \"Name\": \"A\", \"Value\": 2 }, { \"Name\": \"B\", \"Value\": 3 }]"
var myList = (List<MyType>)JsonConvert.DeserializeObject(config["ConfigurationList"], typeof(List<MyType>));
Which isn't the worst; but makes the json a bit "not as nice" and doesn't "flow" across the two platforms ... if it is what I have to do, fine; but hoping for a more standard approach / recommendation
As I metioned in the comment, on local you can process local.settings.json as a json file, but when on azure, the value in configuration settings is environment variable. There is no section, it just string.
Please notice that only string values are allowed, and that anything nested will break. Learn how to use nest settings on azure web app(azure functon is based on azure app service sandbox, so it is the same.):
https://learn.microsoft.com/en-us/archive/blogs/waws/asp-net-core-settings-for-azure-app-service
For example, if this is the json structure:
{
"Parent": {
"ChildOne": "C1 from secrets.json",
"ChildTwo": "C2 from secrets.json"
}
}
Then in web app, you should save it like this:
(source: windows.net)
Not sure if you are looking something like this , it seems a list but if it is a simple JObject like
"ConfigurationList" : {
"Name": "A",
"Value": 2
}
Then you can declare ConfigurationList:Name , ConfigurationList:Value in the configuration settings of function app

ASP.NET Core 3 - Serilog how to configure Serilog.Sinks.Map in appsettings.json file?

I came across the Serilog.Sinks.Map addon today which will solve my challenge with routing specific log events to a specific sink interface. In my environment, I am writing to a log file as well as using the SQL interface. I only want certain logs to be written to the SQL Server though.
Reading the instructions on GitHub by the author, I can only see an example for implementing the LoggerConfiguration through C# in the Program.CS, but I am using the appsettings.json file and unsure what to change from the provided example to the required json format.
Example given by Serilog on GitHub:
Log.Logger = new LoggerConfiguration()
.WriteTo.Map("Name", "Other", (name, wt) => wt.File($"./logs/log-{name}.txt"))
.CreateLogger();
My current configuration: Note I haven't implemented the Sinks.Map in my code yet.
Program.CS File:
public static void Main(string[] args)
{
// Build a configuration system with the route of the app settings.json file.
// this is becuase we dont yet have dependancy injection available, that comes later.
var configuration = new ConfigurationBuilder()
.AddJsonFile("appsettings.json")
.Build();
Log.Logger = new LoggerConfiguration()
.ReadFrom.Configuration(configuration)
.CreateLogger();
var host = CreateHostBuilder(args).Build();
}
And here is my appsettings.json file. I want to be able configure sink name 'MSSqlServer' as the special route, then use the standard file appender sink for all the other general logging.
"AllowedHosts": "*",
"Serilog": {
"Using": [],
"MinumumLevel": {
"Default": "Information",
"Override": {
"Microsoft": "Warning",
"System": "Warning"
}
},
"Enrich": [ "FromLogContext", "WithMachineName", "WithProcessId", "WithThreadId" ],
"WriteTo": [
{ "Name": "Console" },
{
"Name": "File",
"Args": {
//"path": "C:\\NetCoreLogs\\log.txt", // Example path to Windows Drive.
"path": ".\\Logs\\logs.txt",
//"rollingInterval": "Day", // Not currently in use.
"rollOnFileSizeLimit": true,
//"retainedFileCountLimit": null, // Not currently in use.
"fileSizeLimitBytes": 10000000,
"outputTemplate": "{Timestamp:dd-MM-yyyy HH:mm:ss.fff G} {Message}{NewLine:1}{Exception:1}"
// *Template Notes*
// Timestamp 'G' means UTC Time
}
},
{
"Name": "MSSqlServer",
"Args": {
"connectionString": "DefaultConnection",
"schemaName": "EventLogging",
"tableName": "Logs",
"autoCreateSqlTable": true,
"restrictedToMinimumLevel": "Information",
"batchPostingLimit": 1000,
"period": "0.00:00:30"
}
}
//{
// "Name": "File",
// "Args": {
// "path": "C:\\NetCoreLogs\\log.json",
// "formatter": "Serilog.Formatting.Json.JsonFormatter, Serilog"
// }
//}
]
}
Lastly if i could squeeze in another quick question on the topic, when using the SQL sink interface, how do manage the automatic purging/deletion of the oldest events i.e. DB should only store max 1,000,000 events then automatically write over the oldest event first, thanks in advance
I believe it is currently impossible to configure the standard Map call in json, since it relies on a few types that have no serialization support right now, like Action<T1, T2>. I created an issue to discuss this in the repository itself:
Unable to configure default Map call in json? #22
However, there is a way to still get some functionality out of it in Json, by creating a custom extension method. In your particular case, it would be something like this:
public static class SerilogSinkConfigurationExtensions
{
public static LoggerConfiguration MapToFile(
this LoggerSinkConfiguration loggerSinkConfiguration,
string keyPropertyName,
string pathFormat,
string defaultKey)
{
return loggerSinkConfiguration.Map(
keyPropertyName,
defaultKey,
(key, config) => config.File(string.Format(pathFormat, key));
}
}
Then, on your json file, add a section like this:
"WriteTo": [
...
{
"Name": "MapToFile",
"Args": {
"KeyPropertyName": "Name",
"DefaultKey": "Other",
"PathFormat": "./logs/log-{0}.txt"
}
}
]
To have these customizations work properly, Serilog needs to understand that your assembly has these kinds of extensions, to load them during the parsing stage. As per the documentation, you either need to have these extensions on a *.Serilog.* assembly, or add the Using clause on the json:
// Assuming the extension method is inside the "Company.Domain.MyProject" dll
"Using": [ "Company.Domain.MyProject" ]
More information on these constraints here:
https://github.com/serilog/serilog-settings-configuration#using-section-and-auto-discovery-of-configuration-assemblies

Loading JsonObject model in manifest.json - sapui5

I'm trying to set a data model inside my manifest.json within my webapp.
Im using sapui5 and I'm quite new to it.
the resource I'm getting from my api is a jsonObject but somehow the model is not initiated properly. I checked the model with console.log() and it's empty.
when I do the same with an jsonArray it is working.
I should mention that I use a mockserver.js
Here is the code I'm using.
manifest.json:
"sap.app": {
...
"dataSources": {
"invoiceRemote": {
"uri": "https://services.odata.org/V2/Northwind/Northwind.svc/",
"type": "OData",
"settings": {
"odataVersion": "2.0",
"localUri": "localService/metadata.xml"
}
}
}
}
...
"sap.ui5": {
...
"models": {
"i18n": {
"type": "sap.ui.model.resource.ResourceModel",
"settings": {
"bundleName": "MyInboxUI5.i18n.i18n"
}
},
"invoice": {
"dataSource": "invoiceRemote"
}
...
and with JsonObject I mean a .json of this style:
{
"field1": value1,
"field2": value2,
"field3": [
{
"field4": value4,
"field5": value5
},
{
"field6": value6,
"field7": value7
} ]
}
(that's the one not working)
and with JsonArray I meant
[
{
"field4": value4,
"field5": value5
},
{
"field6": value6,
"field7": value7
}
]
(This one is working)
To check my model I used the simple console.log()
Component.js (part of it)
init: function() {
console.log(this.getModel("invoice"));
UIComponent.prototype.init.apply(this, arguments);
this.getRouter().initialize();
}
I did not post the mockserver.js or metadata.xml because I'm not sure it's that relevant and they take a lot of space.
So does anyone know if there is a way to load the model of a JsonObject inside the manifest.json?
I'm aware that there are other possibilities to load the model that do work, but I'm only interestet in that specific case.
Without having additional information about what you actually try to achieve it's hard to point you into the right direction.
The important information is that you are using an ODataModel + a mockserver. Thanks to the mockserver you can easily mock your data for the entities of your OData service - actually you can even mock much more...
Basically, the mock data files need to contain flat lists. In other words, you have always an array of flat objects. The mockserver gets the data (i.e. entities by id) from exactly from these files. The mockserver can only find the files if they have the correct name (see walkthrough tutorial for details). As a rule of thumb "1 file contains data for one entity/entityset".
There is no way to model JsonObjects inside the manifest. What you can do is mocking your mockserver (i.e. by reading json files manually), that works perfectly (the explored app has some examples). However, don't forget we are talking about OData!
Hint: your data looks like a tree, so I guess you want to model a tree structure. If you check the explored app there are a few examples for OData Tree binding and there I'm using the mockserver as well. Maybe that helps...

Node.js SOAP client parameter formatting

I'm having trouble properly formatting one particular soap parameter using the node-soap module for node.js as a client, to a 3rd-party SOAP service.
The client.describe() for this method says this particular input should be in the shape of:
params: { 'param[]': {} }
I have tried a bunch of different JSON notations to try to fit my data to that shape.
Examples of formats that do NOT work:
"params": { "param": [ {"myParameterName": "myParameterValue"} ] }
"params": [ "param": { "name": "myParameterName", "_": "myParameterValue"} ]
"params": { "param" : [ {"name": "myParameterName", "_": "myParameterValue"} ] }
"params": { "param[]": {"myParameterName": "myParameterValue" } }
"params": { "param[myParameterName]": {"_": "myParameterValue" } }
I must be overlooking something, and I suspect I'm going to feel like Captain Obvious when some nice person points out what I'm doing wrong.
Here is what DOES work, using other soap clients, and how they handle the "named parameter with a value"
soapUI for this method successfully accepts this particular input via XML in the shape of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>
Also, using PHP, I can successfully make the call by creating a stdClass of arrays like so:
$parms = new stdClass;
$parms->param = array(
array(
"name"=>"myParameterName","_"=>"myParameterValue"
)
);
and then eventually passing
'params' => $parms
to the PHP soap client
Many thanks!
To get a better look at what XML was being generated by node-soap, I added a console.log(message) statement to the node_modules/soap/lib/client.js after the object-to-XML encoding. I then began experimenting with various JSON structures to figure out empirically how they were mapping to XML structures.
I found a JSON structure for node-soap to generate the XML in my 3rd-party's required named-parameter-with-value format. I was completely unaware of the "$value" special keyword. Looks like this may have been added in the 0.4.6 release from mid-June 2014. See the change history
"params": [
{
"param": {
"attributes": {
"name": "myParameterName"
},
$value: "myParameterValue"
}
}
]
(note the outer array, which gives me the luxury of specifying multiple "param" entries, which is sometimes needed by this particular 3rd-party API)
generates this XML:
<tns:params>
<tns:param name="myParameterName">myParameterValue</tns:param>
</tns:params>
which perfectly matches the structure in soapUI (which I already knew worked) of:
<ns:params>
<ns:param name="myParameterName">myParameterValue</ns:param>
</ns:params>

XSLT equivalent for JSON

Is there an XSLT equivalent for JSON? Something to allow me to do transformations on JSON like XSLT does to XML.
XSLT equivalents for JSON - a list of candidates (tools and specs)
Tools
1. XSLT
You can use XSLT for JSON with the aim of fn:json-to-xml.
This section describes facilities allowing JSON data to be processed using XSLT.
2. jq
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
There are install packages for different OS.
3. jj
JJ is a command line utility that provides a fast and simple way to retrieve or update values from JSON documents. It's powered by GJSON and SJSON under the hood.
4. fx
Command-line JSON processing tool
Don't need to learn new syntax
Plain JavaScript
Formatting and highlighting
Standalone binary
5. CsvCruncher
Command-line tabular data SQL-based processing tool
Don't need to learn new syntax
Suitable when JSON contains a large array with similar items
Supports multiple documents to be processed as multiple SQL tables
Written in Kotlin, runs on Java
Also available as a Java library in Maven Central repository
6. jl
jl ("JSON lambda") is a tiny functional language for querying and manipulating JSON.
7. JOLT
JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document.
8. gron
Make JSON greppable!
gron transforms JSON into discrete assignments to make it easier to grep for what you want and see the absolute 'path' to it. It eases the exploration of APIs that return large blobs of JSON but have terrible documentation.
9. json-e
JSON-e is a data-structure parameterization system for embedding context in JSON objects.
The central idea is to treat a data structure as a "template" and transform it, using another data structure as context, to produce an output data structure.
10. JSLT
JSLT is a complete query and transformation language for JSON. The language design is inspired by jq, XPath, and XQuery.
11. JSONata
JSONata is a lightweight query and transformation language for JSON data. Inspired by the 'location path' semantics of XPath 3.1, it allows sophisticated queries to be expressed in a compact and intuitive notation.
12. JSONPath Plus
Analyse, transform, and selectively extract data from JSON documents (and JavaScript objects).
jsonpath-plus expands on the original specification to add some additional operators and makes explicit some behaviors the original did not spell out.
13. json-transforms Last Commit Dec 1, 2017
Provides a recursive, pattern-matching approach to transforming JSON data. Transformations are defined as a set of rules which match the structure of a JSON object. When a match occurs, the rule emits the transformed data, optionally recursing to transform child objects.
14. json Last commit Jun 23, 2018
json is a fast CLI tool for working with JSON. It is a single-file node.js script with no external deps (other than node.js itself).
15. jsawk Last commit Mar 4, 2015
Jsawk is like awk, but for JSON. You work with an array of JSON objects read from stdin, filter them using JavaScript to produce a results array that is printed to stdout.
16. yate Last Commit Mar 13, 2017
Tests can be used as docu https://github.com/pasaran/yate/tree/master/tests
17. jsonpath-object-transform Last Commit Jan 18, 2017
Pulls data from an object literal using JSONPath and generate a new objects based on a template.
18. Stapling Last Commit Sep 16, 2013
Stapling is a JavaScript library that enables XSLT formatting for JSON objects.
Instead of using a JavaScript templating engine and text/html templates, Stapling gives you the opportunity to use XSLT templates - loaded asynchronously with Ajax and then cached client side - to parse your JSON datasources.
Specs:
JsonPointer
JSON Pointer defines a string syntax for identifying a specific value
within a JavaScript Object Notation (JSON) document.
JsonPath
JSONPath expressions always refer to a JSON structure in the same way as XPath expression are used in combination with an XML document
JSPath
JSPath for JSON is like XPath for XML."
JSONiq
The main source of inspiration behind JSONiq is XQuery, which has been proven so far a successful and productive query language for semi-structured data
JMESPATH
JMESPath is a query language for JSON. The JMESPath language is described in an ABNF grammar with a complete specification.
Interesting idea. Some searching on Google produced a few pages of interest, including:
an outline of how such a "jsonT" tool might be implemented, and some downloads
some discussion of that implementation
a company which may have implemented something suitable
Hope this helps.
Try JOLT. It is a JSON to JSON transformation library written in Java.
It was created specifically because we did not want to play the "JSON -> XML -> XSLT -> XML -> JSON" game, and using a template for any sufficiently complex transform is unmaintainable.
XSLT supports JSON as seen at http://www.w3.org/TR/xslt-30/#json
XML uses angular brackets for delimiter tokens, JSON uses braces, square brackets, ... I. e. XML's fewer token recognition comparisons means it's optimized for declarative transformation, whereas more comparisons, being like switch statement, for speed reasons assume speculative branch prediction that imperative code in scripting languages is useful for. As direct consequence, for different mixes of semi-structured data, you may want to benchmark XSLT and javascript engines' performance as part of responsive pages. For negligible data payload, transformations might work just as well with JSON without XML serialization. W3's decision ought to be based on better analysis.
jq - lightweight and flexible command-line JSON processor
It's not template-based like XSLT, but more concise. e.g. to extract name and address fields into an array: [.name, .address]
The tutorial walks through an example of transforming Twitter's JSON API (and the manual has many examples).
I recently found a tool that I love for styling JSON: https://github.com/twigkit/tempo. Very easy tool to use--in my opinion, it is much easier to work with than XSLT--no need for XPATH queries.
Have a look at jsonpath-object-transform
I wrote my own small library around this, recently, which tries to stay as close to
5.1 Processing Model (XSLT REC)
https://www.w3.org/TR/xslt#section-Processing-Model
as is possible (as I could anyway), in a few lines of JavaScript code.
Here are a few not-completely-trivial examples of use...
1. JSON-to-some-markup:
Fiddle: https://jsfiddle.net/YSharpLanguage/kj9pk8oz/10
(inspired by D.1 Document Example (XSLT REC) https://www.w3.org/TR/xslt#section-Document-Example)
where this:
var D1document = {
type: "document", title: [ "Document Title" ],
"": [
{ type: "chapter", title: [ "Chapter Title" ],
"": [
{ type: "section", title: [ "Section Title" ],
"": [
{ type: "para", "": [ "This is a test." ] },
{ type: "note", "": [ "This is a note." ] }
] },
{ type: "section", title: [ "Another Section Title" ],
"": [
{ type: "para", "": [ "This is ", { emph: "another" }, " test." ] },
{ type: "note", "": [ "This is another note." ] }
] }
] }
] };
var D1toHTML = { $: [
[ [ function(node) { return node.type === "document"; } ],
function(root) {
return "<html>\r\n\
<head>\r\n\
<title>\r\n\
{title}\r\n".of(root) + "\
</title>\r\n\
</head>\r\n\
<body>\r\n\
{*}".of(root[""].through(this)) + "\
</body>\r\n\
</html>";
}
],
[ [ function(node) { return node.type === "chapter"; } ],
function(chapter) {
return " <h2>{title}</h2>\r\n".of(chapter) + "{*}".of(chapter[""].through(this));
}
],
[ [ function(node) { return node.type === "section"; } ],
function(section) {
return " <h3>{title}</h3>\r\n".of(section) + "{*}".of(section[""].through(this));
}
],
[ [ function(node) { return node.type === "para"; } ],
function(para) {
return " <p>{*}</p>\r\n".of(para[""].through(this));
}
],
[ [ function(node) { return node.type === "note"; } ],
function(note) {
return ' <p class="note"><b>NOTE: </b>{*}</p>\r\n'.of(note[""].through(this));
}
],
[ [ function(node) { return node.emph; } ],
function(emph) {
return "<em>{emph}</em>".of(emph);
}
]
] };
console.log(D1document.through(D1toHTML));
... gives:
<html>
<head>
<title>
Document Title
</title>
</head>
<body>
<h2>Chapter Title</h2>
<h3>Section Title</h3>
<p>This is a test.</p>
<p class="note"><b>NOTE: </b>This is a note.</p>
<h3>Another Section Title</h3>
<p>This is <em>another</em> test.</p>
<p class="note"><b>NOTE: </b>This is another note.</p>
</body>
</html>
and
2. JSON-to-JSON:
Fiddle: https://jsfiddle.net/YSharpLanguage/ppfmmu15/10
where this:
// (A "Company" is just an object with a "Team")
function Company(obj) {
return obj.team && Team(obj.team);
}
// (A "Team" is just a non-empty array that contains at least one "Member")
function Team(obj) {
return ({ }.toString.call(obj) === "[object Array]") &&
obj.length &&
obj.find(function(item) { return Member(item); });
}
// (A "Member" must have first and last names, and a gender)
function Member(obj) {
return obj.first && obj.last && obj.sex;
}
function Dude(obj) {
return Member(obj) && (obj.sex === "Male");
}
function Girl(obj) {
return Member(obj) && (obj.sex === "Female");
}
var data = { team: [
{ first: "John", last: "Smith", sex: "Male" },
{ first: "Vaio", last: "Sony" },
{ first: "Anna", last: "Smith", sex: "Female" },
{ first: "Peter", last: "Olsen", sex: "Male" }
] };
var TO_SOMETHING_ELSE = { $: [
[ [ Company ],
function(company) {
return { some_virtual_dom: {
the_dudes: { ul: company.team.select(Dude).through(this) },
the_grrls: { ul: company.team.select(Girl).through(this) }
} }
} ],
[ [ Member ],
function(member) {
return { li: "{first} {last} ({sex})".of(member) };
} ]
] };
console.log(JSON.stringify(data.through(TO_SOMETHING_ELSE), null, 4));
... gives:
{
"some_virtual_dom": {
"the_dudes": {
"ul": [
{
"li": "John Smith (Male)"
},
{
"li": "Peter Olsen (Male)"
}
]
},
"the_grrls": {
"ul": [
{
"li": "Anna Smith (Female)"
}
]
}
}
}
3. XSLT vs. JavaScript:
A JavaScript equivalent of...
XSLT 3.0 REC Section 14.4 Example: Grouping Nodes based on Common Values
(at: http://jsfiddle.net/YSharpLanguage/8bqcd0ey/1)
Cf. https://www.w3.org/TR/xslt-30/#grouping-examples
where...
var cities = [
{ name: "Milano", country: "Italia", pop: 5 },
{ name: "Paris", country: "France", pop: 7 },
{ name: "München", country: "Deutschland", pop: 4 },
{ name: "Lyon", country: "France", pop: 2 },
{ name: "Venezia", country: "Italia", pop: 1 }
];
/*
Cf.
XSLT 3.0 REC Section 14.4
Example: Grouping Nodes based on Common Values
https://www.w3.org/TR/xslt-30/#grouping-examples
*/
var output = "<table>\r\n\
<tr>\r\n\
<th>Position</th>\r\n\
<th>Country</th>\r\n\
<th>City List</th>\r\n\
<th>Population</th>\r\n\
</tr>{*}\r\n\
</table>".of
(
cities.select().groupBy("country")(function(byCountry, index) {
var country = byCountry[0],
cities = byCountry[1].select().orderBy("name");
return "\r\n\
<tr>\r\n\
<td>{position}</td>\r\n\
<td>{country}</td>\r\n\
<td>{cities}</td>\r\n\
<td>{population}</td>\r\n\
</tr>".
of({ position: index + 1, country: country,
cities: cities.map(function(city) { return city.name; }).join(", "),
population: cities.reduce(function(sum, city) { return sum += city.pop; }, 0)
});
})
);
... gives:
<table>
<tr>
<th>Position</th>
<th>Country</th>
<th>City List</th>
<th>Population</th>
</tr>
<tr>
<td>1</td>
<td>Italia</td>
<td>Milano, Venezia</td>
<td>6</td>
</tr>
<tr>
<td>2</td>
<td>France</td>
<td>Lyon, Paris</td>
<td>9</td>
</tr>
<tr>
<td>3</td>
<td>Deutschland</td>
<td>München</td>
<td>4</td>
</tr>
</table>
4. JSONiq vs. JavaScript:
A JavaScript equivalent of...
JSONiq Use Cases Section 1.1.2. Grouping Queries for JSON
(at: https://jsfiddle.net/YSharpLanguage/hvo24hmk/3)
Cf. http://jsoniq.org/docs/JSONiq-usecases/html-single/index.html#jsongrouping
where...
/*
1.1.2. Grouping Queries for JSON
http://jsoniq.org/docs/JSONiq-usecases/html-single/index.html#jsongrouping
*/
var sales = [
{ "product" : "broiler", "store number" : 1, "quantity" : 20 },
{ "product" : "toaster", "store number" : 2, "quantity" : 100 },
{ "product" : "toaster", "store number" : 2, "quantity" : 50 },
{ "product" : "toaster", "store number" : 3, "quantity" : 50 },
{ "product" : "blender", "store number" : 3, "quantity" : 100 },
{ "product" : "blender", "store number" : 3, "quantity" : 150 },
{ "product" : "socks", "store number" : 1, "quantity" : 500 },
{ "product" : "socks", "store number" : 2, "quantity" : 10 },
{ "product" : "shirt", "store number" : 3, "quantity" : 10 }
];
var products = [
{ "name" : "broiler", "category" : "kitchen", "price" : 100, "cost" : 70 },
{ "name" : "toaster", "category" : "kitchen", "price" : 30, "cost" : 10 },
{ "name" : "blender", "category" : "kitchen", "price" : 50, "cost" : 25 },
{ "name" : "socks", "category" : "clothes", "price" : 5, "cost" : 2 },
{ "name" : "shirt", "category" : "clothes", "price" : 10, "cost" : 3 }
];
var stores = [
{ "store number" : 1, "state" : "CA" },
{ "store number" : 2, "state" : "CA" },
{ "store number" : 3, "state" : "MA" },
{ "store number" : 4, "state" : "MA" }
];
var nestedGroupingAndAggregate = stores.select().orderBy("state").groupBy("state")
( function(byState) {
var state = byState[0],
stateStores = byState[1];
byState = { };
return (
(
byState[state] =
products.select().orderBy("category").groupBy("category")
( function(byCategory) {
var category = byCategory[0],
categoryProducts = byCategory[1],
categorySales = sales.filter(function(sale) {
return stateStores.find(function(store) { return sale["store number"] === store["store number"]; }) &&
categoryProducts.find(function(product) { return sale.product === product.name; });
});
byCategory = { };
return (
(
byCategory[category] =
categorySales.select().orderBy("product").groupBy("product")
( function(byProduct) {
var soldProduct = byProduct[0],
soldQuantities = byProduct[1];
byProduct = { };
return (
(
byProduct[soldProduct] =
soldQuantities.reduce(function(sum, sale) { return sum += sale.quantity; }, 0)
),
byProduct
);
} ) // byProduct()
),
byCategory
);
} ) // byCategory()
),
byState
);
} ); // byState()
... gives:
[
{
"CA": [
{
"clothes": [
{
"socks": 510
}
]
},
{
"kitchen": [
{
"broiler": 20
},
{
"toaster": 150
}
]
}
]
},
{
"MA": [
{
"clothes": [
{
"shirt": 10
}
]
},
{
"kitchen": [
{
"blender": 250
},
{
"toaster": 50
}
]
}
]
}
]
It is also useful to overcome the limitations of JSONPath wrt. querying against the ancestor axis, as raised by this SO question (and certainly others).
E.g., how to get the discount of a grocery item knowing its brand id, in
{
"prods": [
{
"info": {
"rate": 85
},
"grocery": [
{
"brand": "C",
"brand_id": "984"
},
{
"brand": "D",
"brand_id": "254"
}
],
"discount": "15"
},
{
"info": {
"rate": 100
},
"grocery": [
{
"brand": "A",
"brand_id": "983"
},
{
"brand": "B",
"brand_id": "253"
}
],
"discount": "20"
}
]
}
?
A possible solution is:
var products = {
"prods": [
{
"info": {
"rate": 85
},
"grocery": [
{
"brand": "C",
"brand_id": "984"
},
{
"brand": "D",
"brand_id": "254"
}
],
"discount": "15"
},
{
"info": {
"rate": 100
},
"grocery": [
{
"brand": "A",
"brand_id": "983"
},
{
"brand": "B",
"brand_id": "253"
}
],
"discount": "20"
}
]
};
function GroceryItem(obj) {
return (typeof obj.brand === "string") && (typeof obj.brand_id === "string");
}
// last parameter set to "true", to grab all the "GroceryItem" instances
// at any depth:
var itemsAndDiscounts = [ products ].nodeset(GroceryItem, true).
map(
function(node) {
var item = node.value, // node.value: the current "GroceryItem" (aka "$.prods[*].grocery[*]")
discount = node.parent. // node.parent: the array of "GroceryItem" (aka "$.prods[*].grocery")
parent. // node.parent.parent: the product (aka "$.prods[*]")
discount; // node.parent.parent.discount: the product discount
// finally, project into an easy-to-filter form:
return { id: item.brand_id, discount: discount };
}
),
discountOfItem983;
discountOfItem983 = itemsAndDiscounts.
filter
(
function(mapped) {
return mapped.id === "983";
}
)
[0].discount;
console.log("Discount of #983: " + discountOfItem983);
... which gives:
Discount of #983: 20
'HTH,
To say lack of tools suggest lack of need is just begging the question. The same could be applied to support for X or Y in Linux (Why bother developing quality drivers and/or games for such a minority OS? And why pay attention to an OS that big game and hardware companies don't develop for?). Probably the people who would need to use XSLT and JSON end up using a somewhat trivial workaround: Transforming JSON into XML. But that's not the optimal solution, is it?
When you have a native JSON format and you want to edit it "wysywyg" in the browser, XSLT would be a more than adequate solution for the problem. Doing that with traditional javascript programming can become a pain in the arse.
In fact, I have implemented a "stone-age" approach to XSLT, using substring parsing to interpret some basic commands for javascript, like calling a template, process children, etc. Certainly implementing a transformation engine with a JSON object is much easier than implementing a full-fledged XML parser to parse the XSLT. Problem is, that to use XML templates to transform a JSON object you need to parse the XML of the templates.
To tranform a JSON object with XML (or HTML, or text or whatever) you need to think carefully about the syntax and what special characters you need to use to identify the transformation commands. Otherwise you'll end up having to design a parser for your own custom templating language. Having walked through that path, I can tell you that it's not pretty.
Update (Nov 12, 2010): After a couple of weeks working on my parser, I've been able to optimize it. Templates are parsed beforehand and commands are stored as JSON objects. Transformation rules are also JSON objects, while the template code is a mix of HTML and a homebrew syntax similar to shell code. I've been able to transform a complex JSON document into HTML to make a document editor. The code is around 1K lines for the editor (it's for a private project so I can't share it) and around 990 lines for the JSON transformation code (includes iteration commands, simple comparisons, template calling, variable saving and evaluation). I plan to release it under a MIT license. Drop me a mail if you want to get involved.
As yet another new answer to an old question, I'd suggest a look at DefiantJS. It's not an XSLT equivalent for JSON, it is XSLT for JSON. The "Templating" section of the documentation includes this example:
<!-- Defiant template -->
<script type="defiant/xsl-template">
<xsl:template name="books_template">
<xsl:for-each select="//movie">
<xsl:value-of select="title"/><br/>
</xsl:for-each>
</xsl:template>
</script>
<script type="text/javascript">
var data = {
"movie": [
{"title": "The Usual Suspects"},
{"title": "Pulp Fiction"},
{"title": "Independence Day"}
]
},
htm = Defiant.render('books_template', data);
console.log(htm);
// The Usual Suspects<br>
// Pulp Fiction<br>
// Independence Day<br>
There is now! I recently created a library, json-transforms, exactly for this purpose:
https://github.com/ColinEberhardt/json-transforms
It uses a combination of JSPath, a DSL modelled on XPath, and a recursive pattern matching approach, inspired directly by XSLT.
Here's a quick example. Given the following JSON object:
const json = {
"automobiles": [
{ "maker": "Nissan", "model": "Teana", "year": 2011 },
{ "maker": "Honda", "model": "Jazz", "year": 2010 },
{ "maker": "Honda", "model": "Civic", "year": 2007 },
{ "maker": "Toyota", "model": "Yaris", "year": 2008 },
{ "maker": "Honda", "model": "Accord", "year": 2011 }
]
};
Here's a transformation:
const jsont = require('json-transforms');
const rules = [
jsont.pathRule(
'.automobiles{.maker === "Honda"}', d => ({
Honda: d.runner()
})
),
jsont.pathRule(
'.{.maker}', d => ({
model: d.match.model,
year: d.match.year
})
),
jsont.identity
];
const transformed = jsont.transform(json, rules);
Which output the following:
{
"Honda": [
{ "model": "Jazz", "year": 2010 },
{ "model": "Civic", "year": 2007 },
{ "model": "Accord", "year": 2011 }
]
}
This transform is composed of three rules. The first matches any automobile which is made by Honda, emitting an object with a Honda property, then recursively matching. The second rule matches any object with a maker property, outputting the model and year properties. The final is the identity transform that recursively matches.
I've been really tired of the enormous amount of JavaScript templating engines out there, and all their inline HTML-templates, different markup styles, etc., and decided to build a small library that enables XSLT formatting for JSON data structures. Not rocket science in any way -- it's just JSON parsed to XML and then formatted with a XSLT document. It's fast too, not as fast as JavaScript template engines in Chrome, but in most other browsers it's at least as fast as the JS engine alternative for larger data structures.
I am using Camel route umarshal(xmljson) -> to(xlst) -> marshal(xmljson). Efficient enough (though not 100% perfect), but simple, if you are already using Camel.
JSONiq is such a standard and Zorba an open-source C++ implementation. JSONiq can also be seen as XQuery with adding JSON as a native data type.
JSLT is very close to a JSON equivalent of XSLT. It's a transform language where you write the fixed part of the output in JSON syntax, then insert expressions to compute the values you want to insert in the template.
An example:
{
"time": round(parse-time(.published, "yyyy-MM-dd'T'HH:mm:ssX") * 1000),
"device_manufacturer": .device.manufacturer,
"device_model": .device.model,
"language": .device.acceptLanguage
}
It's implemented in Java on top of Jackson.
it is very possible to convert JSON using XSLT: you need JSON2SAX deserializer and SAX2JSON serializer.
Sample code in Java:
http://www.gerixsoft.com/blog/json/xslt4json
Yate (https://github.com/pasaran/yate) is specifically designed after XSLT, features JPath (a natural XPath equivalent for JS), compiles to JavaScript and has quite a history of production use. It’s practically undocumented, but reading through samples and tests should be enough.
Why don't you converts JSON to XML using Mr. Data Coverter , tranform it using XSLT and then change it back to JSON using the same.
For a working doodle/proof of concept of an approach to utilize pure JavaScript along with the familiar and declarative pattern behind XSLT's matching expressions and recursive templates, see https://gist.github.com/brettz9/0e661b3093764f496e36
(A similar approach might be taken for JSON.)
Note that the demo also relies on JavaScript 1.8 expression closures for convenience in expressing templates in Firefox (at least until the ES6 short form for methods may be implemented).
Disclaimer: This is my own code.
I wrote a dom adapter for my jackson based json processing framework long time ago. It uses the nu.xom library. The resulting dom tree works with the java xpath and xslt facilities. I made some implementation choices that are pretty straightforward. For example the root node is always called "root", arrays go into an ol node with li sub elements (like in html), and everything else is just a sub node with a primitive value or another object node.
JsonXmlConverter.java
Usage:
JsonObject sampleJson = sampleJson();
org.w3c.dom.Document domNode = JsonXmlConverter.getW3cDocument(sampleJson, "root");
One approach not yet given is to use a parser generator to create a parser in XSLT which parses JSON and produces an XML output.
One option that gets mentioned a lot at the XML conferences is the ReX parser generator (http://www.bottlecaps.de/rex/) - although totally undocumented on the site, recipes are available on searching.
It may be possible to use XSLT with JSON.
Verson 3 of XPath(3.1) XSLT(3.0) and XQuery(3.1) supports JSON in some way.
This seems to be available in the commercial version of Saxon, and might at some point be included in the HE version.
https://www.saxonica.com/html/documentation/functions/fn/parse-json.html
-
What I would expect from an alternative solution:
I would want to be able input JSON to fetch a matching set of data, and output JSON or TEXT.
Access arbitrary properties and evaluate the values
Support for conditional logic
I would want the transformation scripts to be external from the tool, text based, and preferably a commonly used language.
Potential alternative?
I wonder if SQL could be a suitable alternative.
https://learn.microsoft.com/en-us/sql/relational-databases/json/json-data-sql-server
It would be nice if the alternative tool could handle JSON and XML
https://learn.microsoft.com/en-us/sql/relational-databases/xml/openxml-sql-server
I have not yet tried to convert the XSLT scripts I use to SQL, or fully evaluated this option yet, but I hope to look into it more soon. Just some thoughts so far.
CsvCruncher
Despite it's name does not suggest it, CsvCruncher can be efficiently used to read tabular* JSON data and process them using SQL.
* Tabular means that some part of the JSON tree is regular, i.e. one to many entries of same or similar structure.
CsvCruncher turns this part of the tree into a SQL table, and then you have a full SQL syntax to query it.
You may also load multiple JSON trees, and then have multiple tables to play with.
You may also store the SQL result as a JSON (or CSV) and then process it again.
This has served me way better than really complicated XSLT approach (despite it's a lot of fun when you really know it deeply).
Disclaimer: I am the developer of CsvCruncher.
Not too sure there is need for this, and to me lack of tools suggests lack of need. JSON is best processed as objects (the way it's done in JS anyway), and you typically use language of the objects itself to do transformations (Java for Java objects created from JSON, same for Perl, Python, Perl, c#, PHP and so on). Just with normal assignments (or set, get), looping and so on.
I mean, XSLT is just another language, and one reason it is needed is that XML is not an object notation and thus objects of programming languages are not exact fits (impedance between hierarchic xml model and objects/structs).