Create Nested JSON data using U-SQL Json Outputter - json

Question

I have to output the table data into a nested json (make the address, state, city columns as children object for Address) something like below,
[
{
"name": "Country",
"size": 0,
"children": [
{
"name": "America",
"size": 0,
"children": [
{
"name": "SouthAmerica",
"size": 2,
"children": []
}
]
}
]
}
]
But by default JSON outputter is only creating normal json file like below,
[
{
"name": "Europe",
"size": 1,
}
]
How can I create a nested json using U-sql custom outputter? Suggest me some samples.
Thanks in Advance!

Answer 1

The sample JSON outputter that is provided on the U-SQL GitHub page does not support nested output. You have to write your own nested outputter I am afraid.
One of the complications will be, that you will need to be able to keep nesting correlations intact and decide if you need to support sibling nestings (e.g., A contains B and C at the same non-leaf level) or only single path nestings (e.g., A contains B which in turn contains C).
You have a couple of options to write such an outputter, for inspiration, I would look at SQL Server's FOR XML capabilities:
If you only want single path nesting, look at the FOR XML AUTO mode semantics on how to decompose a rowset into nesting levels. You would probably need to pass in parameters into the Outputter that identify how column maps to levels, to mimick the AUTO mode's lineage heuristic.
If you want sibling support, you can either look at FOR XML EXPLICIT's model: Users will have to write a universal table generating SQL query that then can be transformed in a streaming fashion by the outputter, or
You can generating some hierarchy using SQL.MAP and SQL.ARRAY typed columns and then write a custom outputter that produces the nesting.
You can write JSONifier functions that compose smaller JSON documents that then can be nested as strings containing JSON fragments and build up the nesting with several SELECTs (a bit like FOR XML PATH in SQL Server, but probably not easily done at the rowset level).
Alternatively, produce the flat JSON and find a post processing tool to reshape the JSON into the structure you need.
I would currently look into first trying approach #3 (with SQL.MAP and SQL.ARRAY).