JSONiq - how do you convert an array to a sequence? - json

Using the JSONiq to JavaScript implementation of JSONiq, say I have an array
let $a := [1,2,3]
I'd like to get the elements as a sequence, but all of these return the array itself -
return $a()
return $a[]
return members($a)
What is the correct way to extract the members of the array?
My ultimate goal is to convert objects in an array to strings, like so -
let $updates := [
{"address": "%Q0.1", "keys": ["OUT2", "output.2"], "value": 0},
{"address": "%Q0.7", "keys": ["OUT8", "output.8"], "value": 1}
]
for $update in $updates()
return "<timestamp>|address|" || $update.address
in order to convert an array of JSON objects to a set of strings like <timestamp>|address|%Q0.7, etc
Edit: Using Zorba the $a() syntax seems to work okay - is it an issue with the node jsoniq parser?
e.g.
jsoniq version "1.0";
let $updates := [
{"address": "%Q0.1", "keys": ["OUT2", "output.2"], "value": 0},
{"address": "%Q0.7", "keys": ["OUT8", "output.8"], "value": 1}
]
for $update in $updates()
return current-dateTime() || "|address|" || $update.address
returns
2021-02-19T23:10:13.434273Z|address|%Q0.1 2021-02-19T23:10:13.434273Z|address|%Q0.7

In the core JSONiq syntax, an array is turned into a sequence (i.e., its members are extracted) with an empty pair or square brackets, like so:
$array[]
Example:
[1, 2, 3, 4][]
returns the sequence:
(1, 2, 3, 4)
This means that the query would be:
let $updates := [
{"address": "%Q0.1", "keys": ["OUT2", "output.2"], "value": 0},
{"address": "%Q0.7", "keys": ["OUT8", "output.8"], "value": 1}
]
for $update in $updates[]
return "<timestamp>|address|" || $update.address
The function-call-like notation with an empty pair of parenthesis dates back to JSONiq's early days, as it was primarily designed as an extension to XQuery and maps and arrays were navigated with function calls ($object("foo"), $array(), $array(2)). As JSONiq started having its own life, though, more user-friendly and intuitive syntax for JSON navigation was introduced:
$array[[1]]
for array member lookup given a position
$object.foo
for object lookup given a key and
$array[]
for array unboxing.
While the JSONiq extension to XQuery still exists for scenarios in which users need both JSON and XML support (and is supported by Zorba 3.0, IBM Websphere, etc), the core JSONiq syntax is the main one for all engines that specifically support JSON, like Rumble.
Some engines (including Zorba 3.0) support both the core JSONiq syntax and the JSONiq extension to XQuery, and you can pick the one you want with a language version declaration:
jsoniq version "1.0";
[1, 2, 3, 4][]
vs.
xquery version "3.0";
[1, 2, 3, 4]()
Zorba is relatively lenient and will probably even accept both () and [] in its core JSONiq implementation.
(Warning: Zorba 2.9 doesn't support the latest core JSONiq syntax, in particular the try.zorba.io page still runs on Zorba 2.9. You need to download Zorba 3.0 and run it locally if you want to use it).
A final note: JSON navigation works in parallel, on sequences of arrays and objects, too:
(
{"foo":1},
{"foo":2},
{"foo":3},
{"foo":4}
).foo
returns
(1, 2, 3, 4)
while
(
[1, 2],
[3, 4, 5],
[6, 7, 8]
)[]
returns
(1, 2, 3, 4, 5, 6, 7, 8)
This makes it very easy and compact to navigate large sequences:
$collection.foo[].bar[[1]].foobar[].foo

Related

How to run JSONiq from JSON with try.zorba.io

I need to write a JSONiq expression that lists only the name of the products that cost at least 3. This is my JSON file which i had typed in the XQuery section:
{ "supermarket_visit":{
"date":"08032019",
"bought":[
"item",{
"type":"confectionary",
"item_name":"Kit_Kat",
"number": 3,
"individual_price": 3.5
},
"item",{
"type":"drinks",
"item_name":"Coca_Cola",
"number": 2,
"individual_price": 3
},
"item",{
"type":"fruits",
"item_name":"apples",
"number": "some"
}
], 
"next_visit":[
"item",{
"type":"stationary",
"item_name":"A4_paper",
"number": 1
},
"item",{
"type":"stationary",
"item_name":"pen",
"number": 2
}
]
}
}
and this is my JSONiq Xquery JSONiq command, which i dont really know where to type in try.zorba.io:
let $x := find("supermarket_visit")
for $x in $supermarket.bought let $i := $x.item
where $i.individual_price <=3
return $i.item_name
I am getting many errors in try.zorba.io and im really new to JSONiq and JSON. Is something wrong with my JSON or JSONiq part?
The following selection works for me at the site you linked to:
jsoniq version "1.0";
{ "supermarket_visit":{
"date":"08032019",
"bought":[
"item",{
"type":"confectionary",
"item_name":"Kit_Kat",
"number": 3,
"individual_price": 3.5
},
"item",{
"type":"drinks",
"item_name":"Coca_Cola",
"number": 2,
"individual_price": 3
},
"item",{
"type":"fruits",
"item_name":"apples",
"number": "some"
}
],
"next_visit":[
"item",{
"type":"stationary",
"item_name":"A4_paper",
"number": 1
},
"item",{
"type":"stationary",
"item_name":"pen",
"number": 2
}
]
}
}.supermarket_visit.bought()[$$ instance of object and $$.individual_price le 3].item_name
The original query can be slightly modified to (in order to keep a FLWOR expression):
jsoniq version "1.0";
let $document := { (: put the document here :) }
for $x in $document.supermarket_visit.bought[]
where $x instance of object and $x.individual_price le 3
return $x.item_name
Note that try.zorba.io is an older version of Zorba (2.9) that does not implement the latest, stable JSONiq version. This is why () must be used instead of [] on this specific page. If you download the latest version of Zorba, the above query should work.
Also, the original document provided in the question is not well-formed JSON, because it contains a special em space character (Unicode 2003) on the line above "next_visit". This character must be removed for this JSON to be parsed successfully.

Ruby output numbers to 2 decimal places

I'm having trouble serializing my ruby object to json, more specifically the format of the numbers.
I have written an rspec test to illustrate my issue more precisely.
expected = '{ "foo": 1.00, "bar": 4.50, "abc": 0.00, "xyz": 1.23 }'
it 'serializes as expected' do
my_hash = { "foo": 1, "bar": 4.5, "abc": 0, "xyz": 1.23}
expect(my_to_json_method(my_hash)).to eq expected
end
This is the case that I am having trouble with. I can use the sprintf but how do I get the string output as shown in the above example?
First of all, you should not use floats to represent monetary values. So instead, let's use a more appropriate type: (there's also the Ruby Money gem)
require 'bigdecimal'
my_hash = {
foo: BigDecimal.new('1.00'),
bar: BigDecimal.new('4.50'),
abc: BigDecimal.new('0.00'),
xyz: BigDecimal.new('1.23')
}
There are several options to represent monetary values. All of the following JSON strings are valid according to the JSON specification and all require special treatment upon parsing. It's up to you to choose the most appropriate.
Note: I'm implementing a custom to_json method to convert the BigDecimal instances to JSON using Ruby's default JSON library. This is just for demonstration purposes, you should generally not patch core (or stdlib) classes.
1. Numbers with fixed precision
This is what you asked for. Note that many JSON libraries will parse these numbers as floating point values by default.
class BigDecimal
def to_json(*)
'%.2f' % self
end
end
puts my_hash.to_json
Output:
{"foo":1.00,"bar":4.50,"abc":0.00,"xyz":1.23}
2. Numbers as strings
This will work across all JSON libraries, but storing numbers as strings doesn't look quite right to me.
class BigDecimal
def to_json(*)
'"%.2f"' % self
end
end
puts my_hash.to_json
Output:
{"foo":"1.00","bar":"4.50","abc":"0.00","xyz":"1.23"}
3. Numbers as integers
Instead of representing monetary values as fractional numbers, you simply output the cents as whole numbers. This is what I usually do.
class BigDecimal
def to_json(*)
(self * 100).to_i.to_s
end
end
puts my_hash.to_json
Output:
{"foo":100,"bar":450,"abc":0,"xyz":123}
User Sprintf
sprintf('%.2f', 5.5)
And simply interpolate into your JSON as an ERB template.
You can use, sprintf and can take as many decimal points as you needed by mentioning %.(number)f.
Eg: For two decimals, %.2f
Here is a real implementation,
2.2.2 :019 > test = { "foo": (sprintf "%.2f","1.11"), "bar": (sprintf "%.2f","4.55"), "abc": (sprintf "%.2f","0.2") }
=> {:foo=>"1.11", :bar=>"4.55", :abc=>"0.20"}
Here is the reference
puts '{' << my_hash.map { |k, v| %Q|"#{k}": #{"%.2f" % v}| }.join(', ') << '}'
#⇒ {"foo": 1.00, "bar": 4.50, "abc": 0.00, "xyz": 1.23}

Uniform conversion of numeric values to JSON

I'm trying to convert a data.table object to JSON. The columns, which have numeric values, should be converted to JSON values in an "uniform" way, i.e. all values should contain the decimal part, even if it's ".0". What I have is:
library(RJSONIO)
test <- data.table(V1 = c(1.0, 2.0, 4.5, 3.0))
cat(toJSON(test))
{
"V1": [ 1, 2, 4.5, 3 ]
}
However, what I'm trying to do is getting the following output:
{
"V1": [ 1.0, 2.0, 4.5, 3.0 ]
}
I've tried with other libraries such as rjson and I'm getting the same result. I can't seem to find any option that would control this. My last option would be to manually process the output the JSON string, adding the required ".0"s at the end, but I'm wondering if there is a better option. Any help would be greatly appreciated.
The jsonlite has an always_decimal option for this:
> test <- data.table(V1 = c(1.0, 2.0, 4.5, 3.0))
> test
V1
1: 1.0
2: 2.0
3: 4.5
4: 3.0
> jsonlite::toJSON(test, dataframe="columns", always_decimal=TRUE)
{"V1":[1.0,2.0,4.5,3.0]}
It doesn't seem to be strictly "always":
> test$V2 = 1:4
> jsonlite::toJSON(test, dataframe="columns", always_decimal=TRUE)
{"V1":[1.0,2.0,4.5,3.0],"V2":[1,2,3,4]}
but it looks like it does do some inspection of the column types:
> test$V3 = c(1,2,3,4.0)
> jsonlite::toJSON(test, dataframe="columns", always_decimal=TRUE)
{"V1":[1.0,2.0,4.5,3.0],"V2":[1,2,3,4],"V3":[1.0,2.0,3.0,4.0]}
and does decimals for "numeric" and not "integer" columns.
The following code does exactly what you want using format and round to specify the number of decimal places:
library(RJSONIO)
library(data.table)
test <- data.table(V1 = c(1.0, 2.0, 4.5, 3.0))
test$V1 <- format(round(test$V1, 2))
cat(toJSON(test))
results :
{
"V1": [ "1.0", "2.0", "4.5", "3.0" ]
}

Find Duplicate JSON Keys in Sublime Text 3

I have a JSON file that, for now, is validated by hand prior to being placed into production. Ideally, this is an automated process, but for now this is the constraint.
One thing I found helpful in Eclipse were the JSON tools that would highlight duplicate keys in JSON files. Is there similar functionality in Sublime Text or through a plugin?
The following JSON, for example, could produce a warning about duplicate keys.
{
"a": 1,
"b": 2,
"c": 3,
"a": 4,
"d": 5
}
Thanks!
There are plenty of JSON validators available online. I just tried this one and it picked out the duplicate key right away. The problem with using Sublime-based JSON linters like JSONLint is that they use Python's json module, which does not error on extra keys:
import json
json_str = """
{
"a": 1,
"b": 2,
"c": 3,
"a": 4,
"d": 5
}"""
py_data = json.loads(json_str) # changes JSON into a Python dict
# which is unordered
print(py_data)
yields
{'c': 3, 'b': 2, 'a': 4, 'd': 5}
showing that the first a key is overwritten by the second. So, you'll need another, non-Python-based, tool.
Even Python documentation says that:
The RFC specifies that the names within a JSON object should be
unique, but does not mandate how repeated names in JSON objects should
be handled. By default, this module does not raise an exception;
instead, it ignores all but the last name-value pair for a given name:
weird_json = '{"x": 1, "x": 2, "x": 3}'
json.loads(weird_json) {'x': 3}
The object_pairs_hook parameter can be used to alter this behavior.
So as pointed from docs:
class JsonUniqueKeysChecker:
def __init__(self):
self.keys = []
def check(self, pairs):
for key, _value in pairs:
if key in self.keys:
raise ValueError("Non unique Json key: '%s'" % key)
else:
self.keys.append(key)
return pairs
And then:
c = JsonUniqueKeysChecker()
print(json.loads(json_str, object_pairs_hook=c.check)) # raises
JSON is very easy format, not very detailed so things like that can be painful. Detection of doubled keys is easy but I bet it's quite a lot of work to forge plugin from that.

Pretty Printing Arbitrarily Nested Dictionaries & Lists in Vim

I've run into several scenarios where I lists & dictionaries of data in vim, with arbitrarily nested data structures, i.e.:
a = [ 'somedata', d : { 'one': 'x', 'two': 'y', 'three': 'z' }, 'moredata' ]
b = { 'one': '1', 'two': '2', 'three': [ 'x', 'y', 'z' ] }
I'd really like to have a way to 'pretty print' them in a tabular format. It would be especially helpful to simply treat them as JSON directly in vim. Any suggestions?
You may want to look at Tim Pope's Scriptease.vim which provides many niceties for vim scripting and plugin development.
Although I am not sure how pretty :PP is I have found it pretty enough for my uses.
It should also be noted that vim script dictionaries and arrays are very similar to JSON, so you could in theory use any JSON tools after some clean up.
If your text is valid json, you can turn to the external python -m json.tool
so, you just execute in vim: %!python -m json.tool.
Unfortunately your example won't work, if you take a valid json example with nested dict/lists:
Note
that in the screencast I have ft=json, so some quotes cannot be seen in normal mode, the text I used:
[{"test1": 1, "test2": "win", "t3":{"nest1":"foo","nest2":"bar"}}, {"test1": 1, "test2": "win", "t3":{"nest1":"foo","nest2":"bar"}}, {"test1": 1, "test2": "win", "t3":{"nest1":"foo","nest2":"bar"}}, {"test1": 1, "test2": "win", "t3":{"nest1":"foo","nest2":"bar"}}]