Related
I have two jsons/lazy maps in the format as shown below. I now need to compare them to find if there is any difference between them. The reason I combine each set of values in a string so that the comparison becomes faster as my actual inputs (i.e. json messages) are going to be really large.
reqJson:
[["B1": 100, "B2": 200, "B3": 300, "B4": 400],["B1": 500, "B2": 600, "B3": 700, "B4": 800], ["B1": 900, "B2": 1000, "B3": 2000, "B4": 3000], ["B1": 4000, "B2": 5000, "B3": 6000, "B4": 7000]]
respJson:
[["B1": 100, "B2": 200, "B3": 300, "B4": 400],["B1": 500, "B2": 600, "B3": 700, "B4": 800], ["B1": 900, "B2": 1000, "B3": 2000, "B4": 3000], ["B1": 4000, "B2": 5000, "B3": 6000, "B4": 7000], ["B1": 8000, "B2": 9000, "B3": 10000, "B4": 11000]]
My code looks something like as shown below but somehow I am unable to get the desired result. I am unable to figure out what is going wrong. I am taking each value from response Json and compare it with any value in request-Json to find if there is a difference or not.
def diffCounter = 0
Set diffSet = []
respJson.each { respJ ->
reqJson.any {
reqJ ->
if (respJ.B1+respJ.B2+respJ.B3+respJ.B4 != reqJ.B1+reqJ.B2+reqJ.B3+reqJ.B4) {
diffCounter += 1
diffSet << [
"B1" : respJ.B1,
"B2" : respJ.B2,
"B3" : respJ.B3,
"B4" : respJ.B4
]
}
}
}
println ("Difference Count: "+ diffCounter)
println ("Difference Set: "+ diffSet)
Actual Output:
Difference Count: 5
Difference Set: [[B1:100, B2:200, B3:300, B4:400], [B1:500, B2:600, B3:700, B4:800], [B1:900, B2:1000, B3:2000, B4:3000], [B1:4000, B2:5000, B3:6000, B4:7000], [B1:8000, B2:9000, B3:10000, B4:11000]]
Expected Output:
Difference Count: 1
Difference Set: [["B1": 8000, "B2": 9000, "B3": 10000, "B4": 11000]]
NOTE: It can also happen that the request-json is bigger than the response-json so in that case I need to store the difference obtained from request-json into the diffSet.
Any inputs/suggestions in this regard will be helpful.
As #daggett mentioned, if your JSONs become more nested/complicated, you will want to use a library to do this job for you.
In your use case of pure Lists of elements (with values that can be concatenated/added to form a unique key for that element) there is no problem with doing it 'manually'.
The problem with your code is that you check if any reqJson entry has a different count, which for 2+ different reqJson entries is always true.
What you really want to check is if there is any matching reqJson entry that has the same count. And if you can't find any matching entry, then you know that entry only exists in respJson.
def diffCounter = 0
Set diffSet = []
respJson.each { respJ ->
def foundMatching = reqJson.any { reqJ ->
respJ.B1 + respJ.B2 + respJ.B3 + respJ.B4 == reqJ.B1 + reqJ.B2 + reqJ.B3 + reqJ.B4
}
if (!foundMatching) {
diffCounter += 1
diffSet << [
"B1" : respJ.B1,
"B2" : respJ.B2,
"B3" : respJ.B3,
"B4" : respJ.B4
]
}
}
println ("Difference Count: "+ diffCounter)
println ("Difference Set: "+ diffSet)
You mention that reqJson can become bigger than respJson and that in that case you want to switch the roles of the two arrays in the comparison, so that you always get the unmatched elements from the larger array. A trick to do this is to start by swapping the two variables around.
if (reqJson.size() > respJson.size()) {
(reqJson, respJson) = [respJson, reqJson]
}
Note that the time complexity of this algorithm is O(m * n * 2i), meaning it grows linearly with the multiplication of the sizes of the two arrays (m and n, here 5 and 4), times the count of property accesses we do every loop on both elements (i for both elements, here 4 because there are 4 Bs), because we potentially check each element of the smaller array one time for each element of the bigger array.
So if the arrays are tens of thousands of elements long, this will become very slow. A simple way to speed it up to O(m * i + n * i) would be to:
make a Set smallArrayKeys out of the concatenates messages/added values of the smaller array
iterate through the bigger array, check if it's concatenated message is contained in the smallArrayKeys Set, and if not then it only exists in the bigger array.
I'm a new starter to node and sql. I have a question:
I have 3 tables in MySQL
1- Day
2- Exercises
3- Sets
I used Join SQL statement to retrieve data from all 3 of the tables. The problem is, I have 1 day, 1 exercise but 3 sets of reps. so with my statement, I got 3 day object with the same dayID and exerciceID but with a single set of reps.
Any ideea how to combine everithing in a single object when i have a single dayID ?
This is a little app which let me store daily exercises..
This is the postman response
[
{
"dayID": 11,
"dayName": "monday",
"exerciceID": 5,
"exName": "Biceps Curl braces",
"exComments": "close to body elbows",
"setID": 3,
"repNumbers": 12,
"timeBetween": "3",
"weights": 12,
"comments": "Add another 2 reps"
},
{
"dayID": 11,
"dayName": "monday",
"exerciceID": 5,
"exName": "Biceps Curl braces",
"exComments": "close to body elbows",
"setID": 4,
"repNumbers": 12,
"timeBetween": "3",
"weights": 12,
"comments": "Add another 2 reps"
},
{
"dayID": 11,
"dayName": "monday",
"exerciceID": 5,
"exName": "Biceps Curl braces",
"exComments": "close to body elbows",
"setID": 5,
"repNumbers": 12,
"timeBetween": "3",
"weights": 12,
"comments": "Add another 2 reps"
}
]
this is the statement i used =
const sql = SELECT DISTINCT * FROM workoutday w JOIN exercice e ON w.dayID = e.dayID JOIN sets s ON e.exerciceID = s.exerciceID;
Thanks
you have to use INNER JOIN not only join.
if this dosen't work your query code in node is probably executed 3 times (this at last, 90% is the first option)
Using the JSONiq to JavaScript implementation of JSONiq, say I have an array
let $a := [1,2,3]
I'd like to get the elements as a sequence, but all of these return the array itself -
return $a()
return $a[]
return members($a)
What is the correct way to extract the members of the array?
My ultimate goal is to convert objects in an array to strings, like so -
let $updates := [
{"address": "%Q0.1", "keys": ["OUT2", "output.2"], "value": 0},
{"address": "%Q0.7", "keys": ["OUT8", "output.8"], "value": 1}
]
for $update in $updates()
return "<timestamp>|address|" || $update.address
in order to convert an array of JSON objects to a set of strings like <timestamp>|address|%Q0.7, etc
Edit: Using Zorba the $a() syntax seems to work okay - is it an issue with the node jsoniq parser?
e.g.
jsoniq version "1.0";
let $updates := [
{"address": "%Q0.1", "keys": ["OUT2", "output.2"], "value": 0},
{"address": "%Q0.7", "keys": ["OUT8", "output.8"], "value": 1}
]
for $update in $updates()
return current-dateTime() || "|address|" || $update.address
returns
2021-02-19T23:10:13.434273Z|address|%Q0.1 2021-02-19T23:10:13.434273Z|address|%Q0.7
In the core JSONiq syntax, an array is turned into a sequence (i.e., its members are extracted) with an empty pair or square brackets, like so:
$array[]
Example:
[1, 2, 3, 4][]
returns the sequence:
(1, 2, 3, 4)
This means that the query would be:
let $updates := [
{"address": "%Q0.1", "keys": ["OUT2", "output.2"], "value": 0},
{"address": "%Q0.7", "keys": ["OUT8", "output.8"], "value": 1}
]
for $update in $updates[]
return "<timestamp>|address|" || $update.address
The function-call-like notation with an empty pair of parenthesis dates back to JSONiq's early days, as it was primarily designed as an extension to XQuery and maps and arrays were navigated with function calls ($object("foo"), $array(), $array(2)). As JSONiq started having its own life, though, more user-friendly and intuitive syntax for JSON navigation was introduced:
$array[[1]]
for array member lookup given a position
$object.foo
for object lookup given a key and
$array[]
for array unboxing.
While the JSONiq extension to XQuery still exists for scenarios in which users need both JSON and XML support (and is supported by Zorba 3.0, IBM Websphere, etc), the core JSONiq syntax is the main one for all engines that specifically support JSON, like Rumble.
Some engines (including Zorba 3.0) support both the core JSONiq syntax and the JSONiq extension to XQuery, and you can pick the one you want with a language version declaration:
jsoniq version "1.0";
[1, 2, 3, 4][]
vs.
xquery version "3.0";
[1, 2, 3, 4]()
Zorba is relatively lenient and will probably even accept both () and [] in its core JSONiq implementation.
(Warning: Zorba 2.9 doesn't support the latest core JSONiq syntax, in particular the try.zorba.io page still runs on Zorba 2.9. You need to download Zorba 3.0 and run it locally if you want to use it).
A final note: JSON navigation works in parallel, on sequences of arrays and objects, too:
(
{"foo":1},
{"foo":2},
{"foo":3},
{"foo":4}
).foo
returns
(1, 2, 3, 4)
while
(
[1, 2],
[3, 4, 5],
[6, 7, 8]
)[]
returns
(1, 2, 3, 4, 5, 6, 7, 8)
This makes it very easy and compact to navigate large sequences:
$collection.foo[].bar[[1]].foobar[].foo
Where I'm at
For this example, consider Friends.repo
Table Person has fields :id, :name, :age
Example Ecto query:
iex> from(x in Friends.Person, where: {x.id, x.age} in [{1,10}, {2, 20}, {1, 30}], select: [:name])
When I run this, I get relevant results. Something like:
[
%{name: "abc"},
%{name: "xyz"}
]
But when I try to interpolate the query it throws the error
iex> list = [{1,10}, {2, 20}, {1, 30}]
iex> from(x in Friends.Person, where: {x.id, x.age} in ^list, select: [:name])
** (Ecto.Query.CompileError) Tuples can only be used in comparisons with literal tuples of the same size
I'm assuming I need to do some sort of type casting on the list variable. It is mentioned in the docs here : "When interpolating values, you may want to explicitly tell Ecto what is the expected type of the value being interpolated"
What I need
How do I achieve this for a complex type like this? How do I type cast for a "list of tuples, each of size 2"? Something like [{:integer, :integer}] doesn't seem to work.
If not the above, any alternatives for running a WHERE (col1, col2) in ((val1, val2), (val3, val4), ...) type of query using Ecto Query?
Unfortunately, the error should be treated as it is stated in the error message: only literal tuples are supported.
I was unable to come up with some more elegant and less fragile solution, but we always have a sledgehammer as the last resort. The idea would be to generate and execute the raw query.
list = [{1,10}, {2, 20}, {1, 30}]
#⇒ [{1, 10}, {2, 20}, {1, 30}]
values =
Enum.join(for({id, age} <- list, do: "(#{id}, #{age})"), ", ")
#⇒ "(1, 10), (2, 20), (1, 30)"
Repo.query(~s"""
SELECT name FROM persons
JOIN (VALUES #{values}) AS j(v_id, v_age)
ON id = v_id AND age = v_age
""")
The above should return the {:ok, %Postgrex.Result{}} tuple on success.
You can do it with a separate array for each field and unnest, which zips the arrays into rows with a column for each array:
ids =[ 1, 2, 1]
ages=[10, 20, 30]
from x in Friends.Person,
inner_join: j in fragment("SELECT distinct * from unnest(?::int[],?::int[]) AS j(id,age)", ^ids, ^ages),
on: x.id==j.id and x.age==j.age,
select: [:name]
another way of doing it is using json:
list = [%{id: 1, age: 10},
%{id: 2, age: 20},
%{id: 1, age: 30}]
from x in Friends.Person,
inner_join: j in fragment("SELECT distinct * from jsonb_to_recordset(?) AS j(id int,age int)", ^list),
on: x.id==j.id and x.age==j.age,
select: [:name]
Update: I now saw the tag mysql, the above was written for postgres, but maybe it can be used as a base for a mySql version.
I'm trying to convert a data.table object to JSON. The columns, which have numeric values, should be converted to JSON values in an "uniform" way, i.e. all values should contain the decimal part, even if it's ".0". What I have is:
library(RJSONIO)
test <- data.table(V1 = c(1.0, 2.0, 4.5, 3.0))
cat(toJSON(test))
{
"V1": [ 1, 2, 4.5, 3 ]
}
However, what I'm trying to do is getting the following output:
{
"V1": [ 1.0, 2.0, 4.5, 3.0 ]
}
I've tried with other libraries such as rjson and I'm getting the same result. I can't seem to find any option that would control this. My last option would be to manually process the output the JSON string, adding the required ".0"s at the end, but I'm wondering if there is a better option. Any help would be greatly appreciated.
The jsonlite has an always_decimal option for this:
> test <- data.table(V1 = c(1.0, 2.0, 4.5, 3.0))
> test
V1
1: 1.0
2: 2.0
3: 4.5
4: 3.0
> jsonlite::toJSON(test, dataframe="columns", always_decimal=TRUE)
{"V1":[1.0,2.0,4.5,3.0]}
It doesn't seem to be strictly "always":
> test$V2 = 1:4
> jsonlite::toJSON(test, dataframe="columns", always_decimal=TRUE)
{"V1":[1.0,2.0,4.5,3.0],"V2":[1,2,3,4]}
but it looks like it does do some inspection of the column types:
> test$V3 = c(1,2,3,4.0)
> jsonlite::toJSON(test, dataframe="columns", always_decimal=TRUE)
{"V1":[1.0,2.0,4.5,3.0],"V2":[1,2,3,4],"V3":[1.0,2.0,3.0,4.0]}
and does decimals for "numeric" and not "integer" columns.
The following code does exactly what you want using format and round to specify the number of decimal places:
library(RJSONIO)
library(data.table)
test <- data.table(V1 = c(1.0, 2.0, 4.5, 3.0))
test$V1 <- format(round(test$V1, 2))
cat(toJSON(test))
results :
{
"V1": [ "1.0", "2.0", "4.5", "3.0" ]
}