I wish to match 2 json files based on common id
I've tried using awk, jq and the npm json package in quite a lot of different ways but nothing have gotten close to working.
The 2 json files are not sorted and do not contain all the same entries. they contain the common networkId, I only want the output to contain the entries from file2.
Hope somebody can help!
Here's an example.
file1.json:
[
{
"customerId": "id1",
"networkId": "L_653021545945744804"
},
{
"customerId": "id2",
"networkId": "L_653021545955724805"
},
{
"customerId": "id3",
"networkId": "L_655051945958724557"
},
{
"customerId": "id4",
"networkId": "L_655567989968735408"
}
]
file2.json:
[
{
"name": "a",
"networkId": "L_653021545945744804"
},
{
"name": "b",
"networkId": "L_655051945958724557"
}
]
Wanted output:
[
{
"customerId": "id1",
"name": "a",
"networkId": "L_653021545945744804"
},
{
"customerId": "id3",
"name": "b",
"networkId": "L_655051945958724557"
}
]
This is a task for INDEX, JOIN and add:
jq '[JOIN(INDEX(.networkId); input[]; .networkId; add)]' file1.json file2.json
[
{
"name": "a",
"networkId": "L_653021545945744804",
"customerId": "id1"
},
{
"name": "b",
"networkId": "L_655051945958724557",
"customerId": "id3"
}
]
Demo
Related
I'm needing to solve this with JQ. I have a large lists of arrays in my json file and am needing to do some sort | uniq -c types of stuff on them. Specifically I have a relatively nasty looking fruit array that needs to break down what is inside. I'm aware of unique and things like that, and imagine there is likely a simple way to do this, but I've been trying run down assigning things as variables and appending and whatnot, but I can't get the most basic part of counting the unique values per that fruit array, and especially not without breaking the rest of the content (hence the variable ideas). Please tell me I'm overthinking this.
I'd like to turn this;
[
{
"uid": "123abc",
"tID": [
"T19"
],
"fruit": [
"Kiwi",
"Apple",
"",
"",
"",
"Kiwi",
"",
"Kiwi",
"",
"",
"Mango",
"Kiwi"
]
},
{
"uid": "456xyz",
"tID": [
"T15"
],
"fruit": [
"",
"Orange"
]
}
]
Into this;
[
{
"uid": "123abc",
"tID": [
"T19"
],
"metadata": [
{
"name": "fruit",
"value": "Kiwi - 3"
},
{
"name": "fruit",
"value": "Mango - 1"
},
{
"name": "fruit",
"value": "Apple - 1"
}
]
},
{
"uid": "456xyz",
"tID": [
"T15"
],
"metadata": [
{
"name": "fruit",
"value": "Orange - 1"
}
]
}
]
Using group_by and length would be one way:
jq '
map(with_entries(select(.key == "fruit") |= (
.value |= (group_by(.) | map(
{name: "fruit", value: "\(.[0] | select(. != "")) - \(length)"}
))
| .key = "metadata"
)))
'
[
{
"uid": "123abc",
"tID": [
"T19"
],
"metadata": [
{
"name": "fruit",
"value": "Apple - 1"
},
{
"name": "fruit",
"value": "Kiwi - 4"
},
{
"name": "fruit",
"value": "Mango - 1"
}
]
},
{
"uid": "456xyz",
"tID": [
"T15"
],
"metadata": [
{
"name": "fruit",
"value": "Orange - 1"
}
]
}
]
Demo
I am trying to extract all name from computerScience and electronics using json path query
my input json is
{
"computerscience": [
{
"name": "john doe",
"grade": "B",
"year": "4"
},
{
"name": "Bjarne Stroustrup",
"grade": "A",
"year": "4"
},
{
"name": "Dennis Ritchie",
"grade": "A",
"year": "4"
}
],
"someProp1": "false",
"someProp2": "dirSync",
"electronics": [
{
"name": "thomas edison",
"grade": "B",
"year": "4"
},
{
"name": "nichola tesla",
"grade": "B",
"year": "4"
}
]
}
I want to figure out json path which give me below result
["john doe", "Bjarne Stroustrup", "Dennis Ritchie" , "thomas edison", "nichola tesla"]
I want to extract names from only computerScience and electronics
I am trying few jsonpaths here but it won't work
This pattern works:
$.[computerscience,electronics][*].name
This simpler pattern works as long as there aren't any other properties that you want to avoid selecting the names from:
$.*[*].name
Or if you really just want to collect all the names in the document, there's:
$..name
I have been playing around with jq to format a json file but I am having some issues trying to solve a particular transformation. Given a test.json file in this format:
[
{
"name": "A", // This would be the first key
"number": 1,
"type": "apple",
"city": "NYC" // This would be the second key
},
{
"name": "A",
"number": "5",
"type": "apple",
"city": "LA"
},
{
"name": "A",
"number": 2,
"type": "apple",
"city": "NYC"
},
{
"name": "B",
"number": 3,
"type": "apple",
"city": "NYC"
}
]
I was wondering, how can I format it this way using jq?
[
{
"key": "A",
"values": [
{
"key": "NYC",
"values": [
{
"number": 1,
"type": "a"
},
{
"number": 2,
"type": "b"
}
]
},
{
"key": "LA",
"values": [
{
"number": 5,
"type": "b"
}
]
}
]
},
{
"key": "B",
"values": [
{
"key": "NYC",
"values": [
{
"number": 3,
"type": "apple"
}
]
}
]
}
]
I have followed this thread Using jq, convert array of name/value pairs to object with named keys and tried to group the json using this expression
jq '. | group_by(.name) | group_by(.city) ' ./test.json
but I have not been able to add the keys in the output.
You'll want to group the items at the different levels and building out your result objects as you want.
group_by(.name) | map({
key: .[0].name,
values: (group_by(.city) | map({
key: .[0].city,
values: map({number,type})
}))
})
Just keep in mind that group_by/1 yields groups in a sorted order. You'll probably want an implementation that preserves that order.
def group_by_unsorted(key_selector):
reduce .[] as $i ({};
.["\($i|key_selector)"] += [$i]
)|[.[]];
I have the following db:
{
"a": [{
"name": "foo",
"thing": [{
"name": "bar",
"lyrics": ["1", "2", "3"]
}]
}, {
"name": "abc",
"thing": [{
"name": "123",
"list": ["one", "two"]
}]
}]
}
I can't seem to query it correctly. These two queries return the same thing, the entire db:
db.test.find({"a.name":"abc"})
db.test.find({"a.name":"foo"})
How do I find one collection instead of the whole db?
I would expect the first query to return:
{
"name": "abc",
"thing": [{
"name": "123",
"list": ["one", "two"]
}]
}
The two queries return the same document, because both queries match it.
This is one document
[{
"a": [{
"name": "foo",
"thing": [{
"name": "bar",
"lyrics": ["1", "2", "3"]
}]
}, {
"name": "abc",
"thing": [{
"name": "123",
"list": ["one", "two"]
}]
}]
}]
This is two documents
[{
"a": [{
"name": "foo",
"thing": [{
"name": "bar",
"lyrics": ["1", "2", "3"]
}]
}]
},
{
"a": [{
"name": "abc",
"thing": [{
"name": "123",
"list": ["one", "two"]
}]
}]
}]
You can get stats on of a collection like so
db.test.stats()
"count" will tell you how many documents are there.
Edit: To add to this, in your collection "test" a document has 1 field, which is "a" and is of type array that holds objects (documents). It has 2 array elements
First
{
"name": "foo",
"thing": [{
"name": "bar",
"lyrics": ["1", "2", "3"]
}]
}
Second
{
"name": "abc",
"thing": [{
"name": "123",
"list": ["one", "two"]
}]
}
Everything inside curly braces {..}, including the braces themselves, is a one single document, i.e. the whole your database contains only one document that you receive for any matching query. To receive the desired result, you have to re-write your JSON document as an array of documents inside square braces [..].
A friend wrote a program in VBA, which generates a json data. I am trying to visualize that data via the pack-layout. We extracted the rules by what the json data is being created from the json data here: http://bl.ocks.org/mbostock/7607535
I went through the data many times myself, I just can't seem to find the problem why it is not being visualized. The browser console claims a problem in line 33 with the token "]" but in my eyes the parenthesis are right and I can't seem to find another mistake.
The visualization works properly with the data from where we extracted the rules.
The question now is, which mistake in the json file prevents the code from being visualized?
Would be amazing if somebody can see this, since we cannot see it. Thanks in advance!
The generated json data looks like this:
{
"name": "While",
"children": [
{"name": "While", "size": 27},
{
"name": "If",
"children": [
{"name": "If", "size": 22},
{
"name": "If",
"children": [
{"name": "If", "size": 3}
]
},
{
"name": "If",
"children": [
{"name": "If", "size": 3}
]
},
{
"name": "If",
"children": [
{"name": "If", "size": 3}
]
},
{
"name": "If",
"children": [
{"name": "If", "size": 3}
]
},
]
},
]
}
You have two commas(,) at the end of some arrays within that JSON of yours - that makes it invalid and prone to errors.
Just edit it and it will work. Use https://jsonformatter.curiousconcept.com/ to check.
The error lies with the script that generates it :)
Here's the fixed version of your JSON:
{
"name": "While",
"children": [{
"name": "While",
"size": 27
}, {
"name": "If",
"children": [{
"name": "If",
"size": 22
}, {
"name": "If",
"children": [{
"name": "If",
"size": 3
}]
}, {
"name": "If",
"children": [{
"name": "If",
"size": 3
}]
}, {
"name": "If",
"children": [{
"name": "If",
"size": 3
}]
}, {
"name": "If",
"children": [{
"name": "If",
"size": 3
}]
}]
}]
}