I need to create AVRO file but for that I need 2 things:
1) JSON
2) Avro Schema
From these 2 requirements - I have JSON:
{"web-app": {
"servlet": [
{
"servlet-name": "cofaxCDS",
"servlet-class": "org.cofax.cds.CDSServlet",
"init-param": {
"configGlossary:installationAt": "Philadelphia, PA",
"configGlossary:adminEmail": "ksm#pobox.com",
"configGlossary:poweredBy": "Cofax",
"configGlossary:poweredByIcon": "/images/cofax.gif",
"configGlossary:staticPath": "/content/static",
"templateProcessorClass": "org.cofax.WysiwygTemplate",
"templateLoaderClass": "org.cofax.FilesTemplateLoader",
"templatePath": "templates",
"templateOverridePath": "",
"defaultListTemplate": "listTemplate.htm",
"defaultFileTemplate": "articleTemplate.htm",
"useJSP": false,
"jspListTemplate": "listTemplate.jsp",
"jspFileTemplate": "articleTemplate.jsp",
"cachePackageTagsTrack": 200,
"cachePackageTagsStore": 200,
"cachePackageTagsRefresh": 60,
"cacheTemplatesTrack": 100,
"cacheTemplatesStore": 50,
"cacheTemplatesRefresh": 15,
"cachePagesTrack": 200,
"cachePagesStore": 100,
"cachePagesRefresh": 10,
"cachePagesDirtyRead": 10,
"searchEngineListTemplate": "forSearchEnginesList.htm",
"searchEngineFileTemplate": "forSearchEngines.htm",
"searchEngineRobotsDb": "WEB-INF/robots.db",
"useDataStore": true,
"dataStoreClass": "org.cofax.SqlDataStore",
"redirectionClass": "org.cofax.SqlRedirection",
"dataStoreName": "cofax",
"dataStoreDriver": "com.microsoft.jdbc.sqlserver.SQLServerDriver",
"dataStoreUrl": "jdbc:microsoft:sqlserver://LOCALHOST:1433;DatabaseName=goon",
"dataStoreUser": "sa",
"dataStorePassword": "dataStoreTestQuery",
"dataStoreTestQuery": "SET NOCOUNT ON;select test='test';",
"dataStoreLogFile": "/usr/local/tomcat/logs/datastore.log",
"dataStoreInitConns": 10,
"dataStoreMaxConns": 100,
"dataStoreConnUsageLimit": 100,
"dataStoreLogLevel": "debug",
"maxUrlLength": 500}},
{
"servlet-name": "cofaxEmail",
"servlet-class": "org.cofax.cds.EmailServlet",
"init-param": {
"mailHost": "mail1",
"mailHostOverride": "mail2"}},
{
"servlet-name": "cofaxAdmin",
"servlet-class": "org.cofax.cds.AdminServlet"},
{
"servlet-name": "fileServlet",
"servlet-class": "org.cofax.cds.FileServlet"},
{
"servlet-name": "cofaxTools",
"servlet-class": "org.cofax.cms.CofaxToolsServlet",
"init-param": {
"templatePath": "toolstemplates/",
"log": 1,
"logLocation": "/usr/local/tomcat/logs/CofaxTools.log",
"logMaxSize": "",
"dataLog": 1,
"dataLogLocation": "/usr/local/tomcat/logs/dataLog.log",
"dataLogMaxSize": "",
"removePageCache": "/content/admin/remove?cache=pages&id=",
"removeTemplateCache": "/content/admin/remove?cache=templates&id=",
"fileTransferFolder": "/usr/local/tomcat/webapps/content/fileTransferFolder",
"lookInContext": 1,
"adminGroupID": 4,
"betaServer": true}}],
"servlet-mapping": {
"cofaxCDS": "/",
"cofaxEmail": "/cofaxutil/aemail/*",
"cofaxAdmin": "/admin/*",
"fileServlet": "/static/*",
"cofaxTools": "/tools/*"},
"taglib": {
"taglib-uri": "cofax.tld",
"taglib-location": "/WEB-INF/tlds/cofax.tld"}}}
But how to create AVRO Schema based on it?
Looking for programatic way to do that since will have many schemas and can not create Avro Schema manually every time.
I checked 'avro-tools-1.8.1.jar' but that can not create Avro Schema from JSON directly.
Looking for a Jar or Python code that can create JSON -> Avro schema.
It is ok if Data Types are not perfect (Strings, Integers and Floats are good enough for start).
This one works cool with a simple copy and paste of avro schema.
https://toolslick.com/generation/metadata/avro-schema-from-json
you can use Kite SDK util to infer avro schema from a json input.
https://github.com/kite-sdk/kite/blob/master/kite-data/kite-data-core/src/main/java/org/kitesdk/data/spi/JsonUtil.java#L539
Example:
String json = "{\n" +
" \"id\": 1,\n" +
" \"name\": \"A green door\",\n" +
" \"price\": 12.50,\n" +
" \"tags\": [\"home\", \"green\"]\n" +
"}\n"
;
String avroSchema = JsonUtil.inferSchema(JsonUtil.parse(json), "myschema").toString();
System.out.println(avroSchema);
Result:
{
"type":"record",
"name":"myschema",
"fields":[
{
"name":"id",
"type":"int",
"doc":"Type inferred from '1'"
},
{
"name":"name",
"type":"string",
"doc":"Type inferred from '\"A green door\"'"
},
{
"name":"price",
"type":"double",
"doc":"Type inferred from '12.5'"
},
{
"name":"tags",
"type":{
"type":"array",
"items":"string"
},
"doc":"Type inferred from '[\"home\",\"green\"]'"
}
]
}
You can find the maven dependency here
Give this one a shot. http://www.dataedu.ca/avro
It basically infers the Avro schema that accepts the JSON.
You can even give it a JSON array. What it would do is generating an Avro schema that is compatible with all the JSON documents in your array.
There are other tools that you can verify the result.
If you want to avoid creating a dedicated AVRO schema for every JSON format, you can use rec-avro package.
It allows you to take any python data structure, including parsed XML or JSON and store it in Avro without a need for a dedicated schema.
I tested it for python 3.
You can install it as pip3 install rec-avro or see the code and docs at https://github.com/bmizhen/rec-avro
I gave a json to avro example code here: https://stackoverflow.com/a/55444481/6654219
Is there an XSLT equivalent for JSON? Something to allow me to do transformations on JSON like XSLT does to XML.
XSLT equivalents for JSON - a list of candidates (tools and specs)
Tools
1. XSLT
You can use XSLT for JSON with the aim of fn:json-to-xml.
This section describes facilities allowing JSON data to be processed using XSLT.
2. jq
jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.
There are install packages for different OS.
3. jj
JJ is a command line utility that provides a fast and simple way to retrieve or update values from JSON documents. It's powered by GJSON and SJSON under the hood.
4. fx
Command-line JSON processing tool
Don't need to learn new syntax
Plain JavaScript
Formatting and highlighting
Standalone binary
5. CsvCruncher
Command-line tabular data SQL-based processing tool
Don't need to learn new syntax
Suitable when JSON contains a large array with similar items
Supports multiple documents to be processed as multiple SQL tables
Written in Kotlin, runs on Java
Also available as a Java library in Maven Central repository
6. jl
jl ("JSON lambda") is a tiny functional language for querying and manipulating JSON.
7. JOLT
JSON to JSON transformation library written in Java where the "specification" for the transform is itself a JSON document.
8. gron
Make JSON greppable!
gron transforms JSON into discrete assignments to make it easier to grep for what you want and see the absolute 'path' to it. It eases the exploration of APIs that return large blobs of JSON but have terrible documentation.
9. json-e
JSON-e is a data-structure parameterization system for embedding context in JSON objects.
The central idea is to treat a data structure as a "template" and transform it, using another data structure as context, to produce an output data structure.
10. JSLT
JSLT is a complete query and transformation language for JSON. The language design is inspired by jq, XPath, and XQuery.
11. JSONata
JSONata is a lightweight query and transformation language for JSON data. Inspired by the 'location path' semantics of XPath 3.1, it allows sophisticated queries to be expressed in a compact and intuitive notation.
12. JSONPath Plus
Analyse, transform, and selectively extract data from JSON documents (and JavaScript objects).
jsonpath-plus expands on the original specification to add some additional operators and makes explicit some behaviors the original did not spell out.
13. json-transforms Last Commit Dec 1, 2017
Provides a recursive, pattern-matching approach to transforming JSON data. Transformations are defined as a set of rules which match the structure of a JSON object. When a match occurs, the rule emits the transformed data, optionally recursing to transform child objects.
14. json Last commit Jun 23, 2018
json is a fast CLI tool for working with JSON. It is a single-file node.js script with no external deps (other than node.js itself).
15. jsawk Last commit Mar 4, 2015
Jsawk is like awk, but for JSON. You work with an array of JSON objects read from stdin, filter them using JavaScript to produce a results array that is printed to stdout.
16. yate Last Commit Mar 13, 2017
Tests can be used as docu https://github.com/pasaran/yate/tree/master/tests
17. jsonpath-object-transform Last Commit Jan 18, 2017
Pulls data from an object literal using JSONPath and generate a new objects based on a template.
18. Stapling Last Commit Sep 16, 2013
Stapling is a JavaScript library that enables XSLT formatting for JSON objects.
Instead of using a JavaScript templating engine and text/html templates, Stapling gives you the opportunity to use XSLT templates - loaded asynchronously with Ajax and then cached client side - to parse your JSON datasources.
Specs:
JsonPointer
JSON Pointer defines a string syntax for identifying a specific value
within a JavaScript Object Notation (JSON) document.
JsonPath
JSONPath expressions always refer to a JSON structure in the same way as XPath expression are used in combination with an XML document
JSPath
JSPath for JSON is like XPath for XML."
JSONiq
The main source of inspiration behind JSONiq is XQuery, which has been proven so far a successful and productive query language for semi-structured data
JMESPATH
JMESPath is a query language for JSON. The JMESPath language is described in an ABNF grammar with a complete specification.
Interesting idea. Some searching on Google produced a few pages of interest, including:
an outline of how such a "jsonT" tool might be implemented, and some downloads
some discussion of that implementation
a company which may have implemented something suitable
Hope this helps.
Try JOLT. It is a JSON to JSON transformation library written in Java.
It was created specifically because we did not want to play the "JSON -> XML -> XSLT -> XML -> JSON" game, and using a template for any sufficiently complex transform is unmaintainable.
XSLT supports JSON as seen at http://www.w3.org/TR/xslt-30/#json
XML uses angular brackets for delimiter tokens, JSON uses braces, square brackets, ... I. e. XML's fewer token recognition comparisons means it's optimized for declarative transformation, whereas more comparisons, being like switch statement, for speed reasons assume speculative branch prediction that imperative code in scripting languages is useful for. As direct consequence, for different mixes of semi-structured data, you may want to benchmark XSLT and javascript engines' performance as part of responsive pages. For negligible data payload, transformations might work just as well with JSON without XML serialization. W3's decision ought to be based on better analysis.
jq - lightweight and flexible command-line JSON processor
It's not template-based like XSLT, but more concise. e.g. to extract name and address fields into an array: [.name, .address]
The tutorial walks through an example of transforming Twitter's JSON API (and the manual has many examples).
I recently found a tool that I love for styling JSON: https://github.com/twigkit/tempo. Very easy tool to use--in my opinion, it is much easier to work with than XSLT--no need for XPATH queries.
Have a look at jsonpath-object-transform
I wrote my own small library around this, recently, which tries to stay as close to
5.1 Processing Model (XSLT REC)
https://www.w3.org/TR/xslt#section-Processing-Model
as is possible (as I could anyway), in a few lines of JavaScript code.
Here are a few not-completely-trivial examples of use...
1. JSON-to-some-markup:
Fiddle: https://jsfiddle.net/YSharpLanguage/kj9pk8oz/10
(inspired by D.1 Document Example (XSLT REC) https://www.w3.org/TR/xslt#section-Document-Example)
where this:
var D1document = {
type: "document", title: [ "Document Title" ],
"": [
{ type: "chapter", title: [ "Chapter Title" ],
"": [
{ type: "section", title: [ "Section Title" ],
"": [
{ type: "para", "": [ "This is a test." ] },
{ type: "note", "": [ "This is a note." ] }
] },
{ type: "section", title: [ "Another Section Title" ],
"": [
{ type: "para", "": [ "This is ", { emph: "another" }, " test." ] },
{ type: "note", "": [ "This is another note." ] }
] }
] }
] };
var D1toHTML = { $: [
[ [ function(node) { return node.type === "document"; } ],
function(root) {
return "<html>\r\n\
<head>\r\n\
<title>\r\n\
{title}\r\n".of(root) + "\
</title>\r\n\
</head>\r\n\
<body>\r\n\
{*}".of(root[""].through(this)) + "\
</body>\r\n\
</html>";
}
],
[ [ function(node) { return node.type === "chapter"; } ],
function(chapter) {
return " <h2>{title}</h2>\r\n".of(chapter) + "{*}".of(chapter[""].through(this));
}
],
[ [ function(node) { return node.type === "section"; } ],
function(section) {
return " <h3>{title}</h3>\r\n".of(section) + "{*}".of(section[""].through(this));
}
],
[ [ function(node) { return node.type === "para"; } ],
function(para) {
return " <p>{*}</p>\r\n".of(para[""].through(this));
}
],
[ [ function(node) { return node.type === "note"; } ],
function(note) {
return ' <p class="note"><b>NOTE: </b>{*}</p>\r\n'.of(note[""].through(this));
}
],
[ [ function(node) { return node.emph; } ],
function(emph) {
return "<em>{emph}</em>".of(emph);
}
]
] };
console.log(D1document.through(D1toHTML));
... gives:
<html>
<head>
<title>
Document Title
</title>
</head>
<body>
<h2>Chapter Title</h2>
<h3>Section Title</h3>
<p>This is a test.</p>
<p class="note"><b>NOTE: </b>This is a note.</p>
<h3>Another Section Title</h3>
<p>This is <em>another</em> test.</p>
<p class="note"><b>NOTE: </b>This is another note.</p>
</body>
</html>
and
2. JSON-to-JSON:
Fiddle: https://jsfiddle.net/YSharpLanguage/ppfmmu15/10
where this:
// (A "Company" is just an object with a "Team")
function Company(obj) {
return obj.team && Team(obj.team);
}
// (A "Team" is just a non-empty array that contains at least one "Member")
function Team(obj) {
return ({ }.toString.call(obj) === "[object Array]") &&
obj.length &&
obj.find(function(item) { return Member(item); });
}
// (A "Member" must have first and last names, and a gender)
function Member(obj) {
return obj.first && obj.last && obj.sex;
}
function Dude(obj) {
return Member(obj) && (obj.sex === "Male");
}
function Girl(obj) {
return Member(obj) && (obj.sex === "Female");
}
var data = { team: [
{ first: "John", last: "Smith", sex: "Male" },
{ first: "Vaio", last: "Sony" },
{ first: "Anna", last: "Smith", sex: "Female" },
{ first: "Peter", last: "Olsen", sex: "Male" }
] };
var TO_SOMETHING_ELSE = { $: [
[ [ Company ],
function(company) {
return { some_virtual_dom: {
the_dudes: { ul: company.team.select(Dude).through(this) },
the_grrls: { ul: company.team.select(Girl).through(this) }
} }
} ],
[ [ Member ],
function(member) {
return { li: "{first} {last} ({sex})".of(member) };
} ]
] };
console.log(JSON.stringify(data.through(TO_SOMETHING_ELSE), null, 4));
... gives:
{
"some_virtual_dom": {
"the_dudes": {
"ul": [
{
"li": "John Smith (Male)"
},
{
"li": "Peter Olsen (Male)"
}
]
},
"the_grrls": {
"ul": [
{
"li": "Anna Smith (Female)"
}
]
}
}
}
3. XSLT vs. JavaScript:
A JavaScript equivalent of...
XSLT 3.0 REC Section 14.4 Example: Grouping Nodes based on Common Values
(at: http://jsfiddle.net/YSharpLanguage/8bqcd0ey/1)
Cf. https://www.w3.org/TR/xslt-30/#grouping-examples
where...
var cities = [
{ name: "Milano", country: "Italia", pop: 5 },
{ name: "Paris", country: "France", pop: 7 },
{ name: "München", country: "Deutschland", pop: 4 },
{ name: "Lyon", country: "France", pop: 2 },
{ name: "Venezia", country: "Italia", pop: 1 }
];
/*
Cf.
XSLT 3.0 REC Section 14.4
Example: Grouping Nodes based on Common Values
https://www.w3.org/TR/xslt-30/#grouping-examples
*/
var output = "<table>\r\n\
<tr>\r\n\
<th>Position</th>\r\n\
<th>Country</th>\r\n\
<th>City List</th>\r\n\
<th>Population</th>\r\n\
</tr>{*}\r\n\
</table>".of
(
cities.select().groupBy("country")(function(byCountry, index) {
var country = byCountry[0],
cities = byCountry[1].select().orderBy("name");
return "\r\n\
<tr>\r\n\
<td>{position}</td>\r\n\
<td>{country}</td>\r\n\
<td>{cities}</td>\r\n\
<td>{population}</td>\r\n\
</tr>".
of({ position: index + 1, country: country,
cities: cities.map(function(city) { return city.name; }).join(", "),
population: cities.reduce(function(sum, city) { return sum += city.pop; }, 0)
});
})
);
... gives:
<table>
<tr>
<th>Position</th>
<th>Country</th>
<th>City List</th>
<th>Population</th>
</tr>
<tr>
<td>1</td>
<td>Italia</td>
<td>Milano, Venezia</td>
<td>6</td>
</tr>
<tr>
<td>2</td>
<td>France</td>
<td>Lyon, Paris</td>
<td>9</td>
</tr>
<tr>
<td>3</td>
<td>Deutschland</td>
<td>München</td>
<td>4</td>
</tr>
</table>
4. JSONiq vs. JavaScript:
A JavaScript equivalent of...
JSONiq Use Cases Section 1.1.2. Grouping Queries for JSON
(at: https://jsfiddle.net/YSharpLanguage/hvo24hmk/3)
Cf. http://jsoniq.org/docs/JSONiq-usecases/html-single/index.html#jsongrouping
where...
/*
1.1.2. Grouping Queries for JSON
http://jsoniq.org/docs/JSONiq-usecases/html-single/index.html#jsongrouping
*/
var sales = [
{ "product" : "broiler", "store number" : 1, "quantity" : 20 },
{ "product" : "toaster", "store number" : 2, "quantity" : 100 },
{ "product" : "toaster", "store number" : 2, "quantity" : 50 },
{ "product" : "toaster", "store number" : 3, "quantity" : 50 },
{ "product" : "blender", "store number" : 3, "quantity" : 100 },
{ "product" : "blender", "store number" : 3, "quantity" : 150 },
{ "product" : "socks", "store number" : 1, "quantity" : 500 },
{ "product" : "socks", "store number" : 2, "quantity" : 10 },
{ "product" : "shirt", "store number" : 3, "quantity" : 10 }
];
var products = [
{ "name" : "broiler", "category" : "kitchen", "price" : 100, "cost" : 70 },
{ "name" : "toaster", "category" : "kitchen", "price" : 30, "cost" : 10 },
{ "name" : "blender", "category" : "kitchen", "price" : 50, "cost" : 25 },
{ "name" : "socks", "category" : "clothes", "price" : 5, "cost" : 2 },
{ "name" : "shirt", "category" : "clothes", "price" : 10, "cost" : 3 }
];
var stores = [
{ "store number" : 1, "state" : "CA" },
{ "store number" : 2, "state" : "CA" },
{ "store number" : 3, "state" : "MA" },
{ "store number" : 4, "state" : "MA" }
];
var nestedGroupingAndAggregate = stores.select().orderBy("state").groupBy("state")
( function(byState) {
var state = byState[0],
stateStores = byState[1];
byState = { };
return (
(
byState[state] =
products.select().orderBy("category").groupBy("category")
( function(byCategory) {
var category = byCategory[0],
categoryProducts = byCategory[1],
categorySales = sales.filter(function(sale) {
return stateStores.find(function(store) { return sale["store number"] === store["store number"]; }) &&
categoryProducts.find(function(product) { return sale.product === product.name; });
});
byCategory = { };
return (
(
byCategory[category] =
categorySales.select().orderBy("product").groupBy("product")
( function(byProduct) {
var soldProduct = byProduct[0],
soldQuantities = byProduct[1];
byProduct = { };
return (
(
byProduct[soldProduct] =
soldQuantities.reduce(function(sum, sale) { return sum += sale.quantity; }, 0)
),
byProduct
);
} ) // byProduct()
),
byCategory
);
} ) // byCategory()
),
byState
);
} ); // byState()
... gives:
[
{
"CA": [
{
"clothes": [
{
"socks": 510
}
]
},
{
"kitchen": [
{
"broiler": 20
},
{
"toaster": 150
}
]
}
]
},
{
"MA": [
{
"clothes": [
{
"shirt": 10
}
]
},
{
"kitchen": [
{
"blender": 250
},
{
"toaster": 50
}
]
}
]
}
]
It is also useful to overcome the limitations of JSONPath wrt. querying against the ancestor axis, as raised by this SO question (and certainly others).
E.g., how to get the discount of a grocery item knowing its brand id, in
{
"prods": [
{
"info": {
"rate": 85
},
"grocery": [
{
"brand": "C",
"brand_id": "984"
},
{
"brand": "D",
"brand_id": "254"
}
],
"discount": "15"
},
{
"info": {
"rate": 100
},
"grocery": [
{
"brand": "A",
"brand_id": "983"
},
{
"brand": "B",
"brand_id": "253"
}
],
"discount": "20"
}
]
}
?
A possible solution is:
var products = {
"prods": [
{
"info": {
"rate": 85
},
"grocery": [
{
"brand": "C",
"brand_id": "984"
},
{
"brand": "D",
"brand_id": "254"
}
],
"discount": "15"
},
{
"info": {
"rate": 100
},
"grocery": [
{
"brand": "A",
"brand_id": "983"
},
{
"brand": "B",
"brand_id": "253"
}
],
"discount": "20"
}
]
};
function GroceryItem(obj) {
return (typeof obj.brand === "string") && (typeof obj.brand_id === "string");
}
// last parameter set to "true", to grab all the "GroceryItem" instances
// at any depth:
var itemsAndDiscounts = [ products ].nodeset(GroceryItem, true).
map(
function(node) {
var item = node.value, // node.value: the current "GroceryItem" (aka "$.prods[*].grocery[*]")
discount = node.parent. // node.parent: the array of "GroceryItem" (aka "$.prods[*].grocery")
parent. // node.parent.parent: the product (aka "$.prods[*]")
discount; // node.parent.parent.discount: the product discount
// finally, project into an easy-to-filter form:
return { id: item.brand_id, discount: discount };
}
),
discountOfItem983;
discountOfItem983 = itemsAndDiscounts.
filter
(
function(mapped) {
return mapped.id === "983";
}
)
[0].discount;
console.log("Discount of #983: " + discountOfItem983);
... which gives:
Discount of #983: 20
'HTH,
To say lack of tools suggest lack of need is just begging the question. The same could be applied to support for X or Y in Linux (Why bother developing quality drivers and/or games for such a minority OS? And why pay attention to an OS that big game and hardware companies don't develop for?). Probably the people who would need to use XSLT and JSON end up using a somewhat trivial workaround: Transforming JSON into XML. But that's not the optimal solution, is it?
When you have a native JSON format and you want to edit it "wysywyg" in the browser, XSLT would be a more than adequate solution for the problem. Doing that with traditional javascript programming can become a pain in the arse.
In fact, I have implemented a "stone-age" approach to XSLT, using substring parsing to interpret some basic commands for javascript, like calling a template, process children, etc. Certainly implementing a transformation engine with a JSON object is much easier than implementing a full-fledged XML parser to parse the XSLT. Problem is, that to use XML templates to transform a JSON object you need to parse the XML of the templates.
To tranform a JSON object with XML (or HTML, or text or whatever) you need to think carefully about the syntax and what special characters you need to use to identify the transformation commands. Otherwise you'll end up having to design a parser for your own custom templating language. Having walked through that path, I can tell you that it's not pretty.
Update (Nov 12, 2010): After a couple of weeks working on my parser, I've been able to optimize it. Templates are parsed beforehand and commands are stored as JSON objects. Transformation rules are also JSON objects, while the template code is a mix of HTML and a homebrew syntax similar to shell code. I've been able to transform a complex JSON document into HTML to make a document editor. The code is around 1K lines for the editor (it's for a private project so I can't share it) and around 990 lines for the JSON transformation code (includes iteration commands, simple comparisons, template calling, variable saving and evaluation). I plan to release it under a MIT license. Drop me a mail if you want to get involved.
As yet another new answer to an old question, I'd suggest a look at DefiantJS. It's not an XSLT equivalent for JSON, it is XSLT for JSON. The "Templating" section of the documentation includes this example:
<!-- Defiant template -->
<script type="defiant/xsl-template">
<xsl:template name="books_template">
<xsl:for-each select="//movie">
<xsl:value-of select="title"/><br/>
</xsl:for-each>
</xsl:template>
</script>
<script type="text/javascript">
var data = {
"movie": [
{"title": "The Usual Suspects"},
{"title": "Pulp Fiction"},
{"title": "Independence Day"}
]
},
htm = Defiant.render('books_template', data);
console.log(htm);
// The Usual Suspects<br>
// Pulp Fiction<br>
// Independence Day<br>
There is now! I recently created a library, json-transforms, exactly for this purpose:
https://github.com/ColinEberhardt/json-transforms
It uses a combination of JSPath, a DSL modelled on XPath, and a recursive pattern matching approach, inspired directly by XSLT.
Here's a quick example. Given the following JSON object:
const json = {
"automobiles": [
{ "maker": "Nissan", "model": "Teana", "year": 2011 },
{ "maker": "Honda", "model": "Jazz", "year": 2010 },
{ "maker": "Honda", "model": "Civic", "year": 2007 },
{ "maker": "Toyota", "model": "Yaris", "year": 2008 },
{ "maker": "Honda", "model": "Accord", "year": 2011 }
]
};
Here's a transformation:
const jsont = require('json-transforms');
const rules = [
jsont.pathRule(
'.automobiles{.maker === "Honda"}', d => ({
Honda: d.runner()
})
),
jsont.pathRule(
'.{.maker}', d => ({
model: d.match.model,
year: d.match.year
})
),
jsont.identity
];
const transformed = jsont.transform(json, rules);
Which output the following:
{
"Honda": [
{ "model": "Jazz", "year": 2010 },
{ "model": "Civic", "year": 2007 },
{ "model": "Accord", "year": 2011 }
]
}
This transform is composed of three rules. The first matches any automobile which is made by Honda, emitting an object with a Honda property, then recursively matching. The second rule matches any object with a maker property, outputting the model and year properties. The final is the identity transform that recursively matches.
I've been really tired of the enormous amount of JavaScript templating engines out there, and all their inline HTML-templates, different markup styles, etc., and decided to build a small library that enables XSLT formatting for JSON data structures. Not rocket science in any way -- it's just JSON parsed to XML and then formatted with a XSLT document. It's fast too, not as fast as JavaScript template engines in Chrome, but in most other browsers it's at least as fast as the JS engine alternative for larger data structures.
I am using Camel route umarshal(xmljson) -> to(xlst) -> marshal(xmljson). Efficient enough (though not 100% perfect), but simple, if you are already using Camel.
JSONiq is such a standard and Zorba an open-source C++ implementation. JSONiq can also be seen as XQuery with adding JSON as a native data type.
JSLT is very close to a JSON equivalent of XSLT. It's a transform language where you write the fixed part of the output in JSON syntax, then insert expressions to compute the values you want to insert in the template.
An example:
{
"time": round(parse-time(.published, "yyyy-MM-dd'T'HH:mm:ssX") * 1000),
"device_manufacturer": .device.manufacturer,
"device_model": .device.model,
"language": .device.acceptLanguage
}
It's implemented in Java on top of Jackson.
it is very possible to convert JSON using XSLT: you need JSON2SAX deserializer and SAX2JSON serializer.
Sample code in Java:
http://www.gerixsoft.com/blog/json/xslt4json
Yate (https://github.com/pasaran/yate) is specifically designed after XSLT, features JPath (a natural XPath equivalent for JS), compiles to JavaScript and has quite a history of production use. It’s practically undocumented, but reading through samples and tests should be enough.
Why don't you converts JSON to XML using Mr. Data Coverter , tranform it using XSLT and then change it back to JSON using the same.
For a working doodle/proof of concept of an approach to utilize pure JavaScript along with the familiar and declarative pattern behind XSLT's matching expressions and recursive templates, see https://gist.github.com/brettz9/0e661b3093764f496e36
(A similar approach might be taken for JSON.)
Note that the demo also relies on JavaScript 1.8 expression closures for convenience in expressing templates in Firefox (at least until the ES6 short form for methods may be implemented).
Disclaimer: This is my own code.
I wrote a dom adapter for my jackson based json processing framework long time ago. It uses the nu.xom library. The resulting dom tree works with the java xpath and xslt facilities. I made some implementation choices that are pretty straightforward. For example the root node is always called "root", arrays go into an ol node with li sub elements (like in html), and everything else is just a sub node with a primitive value or another object node.
JsonXmlConverter.java
Usage:
JsonObject sampleJson = sampleJson();
org.w3c.dom.Document domNode = JsonXmlConverter.getW3cDocument(sampleJson, "root");
One approach not yet given is to use a parser generator to create a parser in XSLT which parses JSON and produces an XML output.
One option that gets mentioned a lot at the XML conferences is the ReX parser generator (http://www.bottlecaps.de/rex/) - although totally undocumented on the site, recipes are available on searching.
It may be possible to use XSLT with JSON.
Verson 3 of XPath(3.1) XSLT(3.0) and XQuery(3.1) supports JSON in some way.
This seems to be available in the commercial version of Saxon, and might at some point be included in the HE version.
https://www.saxonica.com/html/documentation/functions/fn/parse-json.html
-
What I would expect from an alternative solution:
I would want to be able input JSON to fetch a matching set of data, and output JSON or TEXT.
Access arbitrary properties and evaluate the values
Support for conditional logic
I would want the transformation scripts to be external from the tool, text based, and preferably a commonly used language.
Potential alternative?
I wonder if SQL could be a suitable alternative.
https://learn.microsoft.com/en-us/sql/relational-databases/json/json-data-sql-server
It would be nice if the alternative tool could handle JSON and XML
https://learn.microsoft.com/en-us/sql/relational-databases/xml/openxml-sql-server
I have not yet tried to convert the XSLT scripts I use to SQL, or fully evaluated this option yet, but I hope to look into it more soon. Just some thoughts so far.
CsvCruncher
Despite it's name does not suggest it, CsvCruncher can be efficiently used to read tabular* JSON data and process them using SQL.
* Tabular means that some part of the JSON tree is regular, i.e. one to many entries of same or similar structure.
CsvCruncher turns this part of the tree into a SQL table, and then you have a full SQL syntax to query it.
You may also load multiple JSON trees, and then have multiple tables to play with.
You may also store the SQL result as a JSON (or CSV) and then process it again.
This has served me way better than really complicated XSLT approach (despite it's a lot of fun when you really know it deeply).
Disclaimer: I am the developer of CsvCruncher.
Not too sure there is need for this, and to me lack of tools suggests lack of need. JSON is best processed as objects (the way it's done in JS anyway), and you typically use language of the objects itself to do transformations (Java for Java objects created from JSON, same for Perl, Python, Perl, c#, PHP and so on). Just with normal assignments (or set, get), looping and so on.
I mean, XSLT is just another language, and one reason it is needed is that XML is not an object notation and thus objects of programming languages are not exact fits (impedance between hierarchic xml model and objects/structs).