I am working on a project where I want to query a JSon using Jquery but I am getting an error:
jq: error: syntax error, unexpected IDENT, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
..id
jq: 1 compile error
Error: Process completed with exit code 3.
I am using the command:
jq '..id' new_file.json
To query my JSON file which is named new_file.json
Here is my JSON file:
[
{
"type": "users_export",
"status": "pending",
"connection_id": "con_kmZIjREZWYzt39JI",
"format": "json",
"limit": 5,
"fields": [
{
"name": "user_id"
},
{
"name": "user_id",
"export_as": "provider"
},
{
"name": "username"
},
{
"name": "username",
"export_as": "provider"
},
{
"name": "nickname"
},
{
"name": "email"
},
{
"name": "email"
},
{
"name": "identities[0].connection",
"export_as": "provider"
},
{
"name": "email_verified",
"export_as": "provider"
}
],
"connection": "dev-default-evoyanbs",
"created_at": "2022-11-16T17:45:58.429Z",
"id": "job_aztDgKXWT8g8iZ5T"
}
]
I want the 'job_aztDgKXWT8g8iZ5T' as my output but I am getting the above mentioned error, can someone please help me out, thanks.
As you have an array, the command should be:
jq '.[].id' new_file.json
And the result will be: "job_aztDgKXWT8g8iZ5T".
If you want to get rid of double quotes in your output, you can add raw-output flag in your command, like this:
jq -r '.[].id' new_file.json
or
jq --raw-output '.[].id' new_file.json
Since yesterday, I am sitting fast to get it as .csv. but fails.
Tried with below commands, but failss with following error:
jq: error (at j.json:70): object ({"DCS_CCH_A...) is not valid in a csv row
jq -r '[.[]] | #csv' j.json
Tried also another way, still fails, with followingerror:
jq: error (at j.json:1): Cannot iterate over string ("{")
jq: error (at j.json:2): Cannot iterate over string (" \"vehi...)
jq: error (at j.json:3): Cannot iterate over string (" \"...).
jq -Rr '[.[]] | #csv' j.json
How I can convert it as csv?
OS is Linux Ubuntu
This is my JSON file:
{
"vehicleStatus": {
"DCS_CCH_Activation": "NA",
"DCS_CCH_Ongoing": false,
"cbsData": [
{
"cbsDescription": "Next service due when the stated distance has been covered or by the specified date.",
"cbsDueDate": "2021-02",
"cbsRemainingMileage": 18000,
"cbsState": "OK",
"cbsType": "OIL"
},
{
"cbsDescription": "Next change due at the latest by the stated date.",
"cbsDueDate": "2022-03",
"cbsState": "OK",
"cbsType": "BRAKE_FLUID"
},
{
"cbsDescription": "Next visual inspection due when the stated distance has been covered or by the stated date.",
"cbsDueDate": "2023-02",
"cbsRemainingMileage": 49000,
"cbsState": "OK",
"cbsType": "VEHICLE_CHECK"
}
],
"chargingLevelHv": 17,
"chargingStatus": "WAITING_FOR_CHARGING",
"connectionStatus": "CONNECTED",
"doorDriverFront": "CLOSED",
"doorDriverRear": "CLOSED",
"doorLockState": "SECURED",
"doorPassengerFront": "CLOSED",
"doorPassengerRear": "CLOSED",
"fuelPercent": 54.5,
"hood": "CLOSED",
"mileage": 13919,
"parkingLight": "OFF",
"position": {
"heading": 1,
"lat": 18.3,
"lon": 84.5,
"status": "OK"
},
"positionLight": "OFF",
"rearWindow": "INVALID",
"remainingFuel": 23,
"remainingRangeElectric": 4,
"remainingRangeElectricMls": 2,
"remainingRangeFuel": 260,
"remainingRangeFuelMls": 161,
"singleImmediateCharging": false,
"sunroof": "CLOSED",
"trunk": "CLOSED",
"updateReason": "DOOR_STATE_CHANGED",
"vin": "some_vin",
"windowDriverFront": "CLOSED",
"windowDriverRear": "CLOSED",
"windowPassengerFront": "CLOSED",
"windowPassengerRear": "CLOSED"
}
}
In this question, someone provided a generic JSON to CSV converter using jq. There are several other questions on here that explain more about the mechanics of the process, such as this one.
Hi I am new to JQ and Json. I am using
$ jq --version
jq-1.5
I am having a heck of a time trying to figure out how to select the values for id, attributes.name, attributes.albumName, and attributes.artistName
I am using the terminal app on a mac. I am running into some sort strange parsing problem
$ jq '.results.songs.data[0] | {id, attributes.name } ' t
jq: error: syntax error, unexpected FIELD, expecting '}' (Unix shell quoting issues?) at <top-level>, line 1:
.results.songs.data[0] | {id, attributes.name }
jq: 1 compile error
$
This example shows the structure of the data I am trying to filter looks like
$ jq '.results.songs.data[0] | {id, attributes } ' t
{
"id": "152471393",
"attributes": {
"previews": [
{
"url": "https://audio-ssl.itunes.apple.com/apple-assets-us-std-000001/AudioPreview71/v4/7d/c5/68/7dc56849-29b8-bd90-2bb1-51750e479569/mzaf_4742389090778091050.plus.aac.p.m4a"
}
],
"artwork": {
"width": 1449,
"height": 1449,
"url": "https://is5-ssl.mzstatic.com/image/thumb/Music/v4/7d/01/56/7d0156be-12cd-8724-a0ca-727b1013a81d/source/{w}x{h}bb.jpeg",
"bgColor": "ddcfc4",
"textColor1": "010100",
"textColor2": "422f10",
"textColor3": "2d2a27",
"textColor4": "614f34"
},
"artistName": "Gnarls Barkley",
"url": "https://itunes.apple.com/us/album/crazy/152471339?i=152471393",
"discNumber": 1,
"genreNames": [
"Alternative",
"Music",
"R&B/Soul",
"Rock",
"Soul",
"Hip-Hop/Rap",
"Rap",
"Hip-Hop",
"Adult Alternative",
"Neo-Soul",
"Alternative Rap",
"Underground Rap"
],
"durationInMillis": 178387,
"releaseDate": "2006-03-13",
"name": "Crazy",
"isrc": "USAT20611041",
"albumName": "St. Elsewhere",
"playParams": {
"id": "152471393",
"kind": "song"
},
"trackNumber": 2
}
}
Thanks
Andy
With your sample JSON as input, the following invocation:
jq '{id, name: .attributes.name }' input.json
produces:
{
"id": "152471393",
"name": "Crazy"
}
The filter above is short for:
{"id" : .id, "name": .attributes.name }
In any case, the keys must be appropriately specified.
For future reference, when asking questions on stackoverflow.com, please adhere to the http://stackoverflow.com/help/mcve guidelines as much as possible.
I using JQ 1.4 on Windows 64 bit machine.
Below are the contents input file (JSON.txt)
{
"name": "Google",
"location":
{
"street": "1600 Amphitheatre Parkway",
"city": "Mountain View",
"state": "California",
"country": "US"
},
"employees":
[
{
"name": "Michael",
"division": "Engineering"
},
{
"name": "Laura",
"division": "HR"
},
{
"name": "Elise",
"division": "Marketing"
}
]
}
At the output I exect to see two results "Google" and "Laura"
I am able to get them with idividual filters.
1) jq ."name" JSON.txt
"Google"
2) jq ."employees|map(select(.division==\"HR\"))"[0].name JSON.txt
"Laura"
When I combine these two filters using comma ,, I receive below error:
3) jq ."name",."employees|map(select(.division==\"HR\"))"[0].name JSON.txt
jq: error: Cannot iterate over string
null
"Laura"
Can someone please help to me to get below result:
"Google"
"Laura"
Use brackets around the second part of the expression, and shorthand key filter syntax as in:
jq ".name, (.employees|map(select(.division==\"HR\"))[0].name)" JSON.txt
Given a data.tsv file such :
id code name
1 AL Alabama
2 AK Alaska
4 AZ Arizona
5 AR Arkansas
6 CA California
... ... ...
Given a topojson.json file such : (the structure is correct, the numeral values are random)
{
"type":"Topology",
"transform":
{
"scale": [0.0015484881821515486,0.0010301030103010299],
"translate":[-5.491666666666662,41.008333333333354]
},
"objects":
{
"states":
{
"type":"GeometryCollection",
"geometries":
[
{"type":"Polygon","arcs":[[0]],"properties":{"code_2":"AL"}},
{"type":"Polygon","arcs":[[1]],"properties":{"code_2":"AK"}}
]
}
},
"arcs":
[
[[2466,9916],[-25,-5],[3,-13]],
[[2357,9852],[1,-2],[1,-2]]
]
}
How to use the common fields(1) to inject the values of an other field(2) into the json file ?
1]: data.txt#code and topojson.txt.objects.states.geometries.properties.code_2
2]: data.txt#name
The end result should contains :
{"type":"Polygon","arcs":[[0]],"properties":{"code_2":"AL", "name":"Alabama" }},
{"type":"Polygon","arcs":[[1]],"properties":{"code_2":"AK", "name":"Alaska" }},
EDIT: Accepted answer:
topojson -o final.json -e data.tsv --id-property=code_2,code -p code_2,state=name -- topojson.json
Try using this:
topojson -o final.json -e data.tsv \
--id-property=code_2,code -p code_2,state=name \
-- topojson.json
Which should output:
{
"type": "Topology",
"transform": {
"scale": [
0.000016880209206372492,
0.000007005401010148724
],
"translate": [ -1.8418800213354616, 51.15278777877789 ]
},
"objects": {
"states": {
"type": "GeometryCollection",
"geometries": [
{
"type": "Polygon",
"arcs": [
[ 0 ]
],
"id": "AK",
"properties": {
"code_2": "AK",
"state": "Alaska"
}
}
]
}
},
"arcs": [
[[2466,9916],[-25,-5],[3,-13]],
[[2357,9852],[1,-2],[1,-2]]
]
}
From the Command Line Reference wiki:
--id-property name of feature property to promote to geometry id
By using the code_2 property with this option, you promote it as the feature ID.
Prepend a + in front of the input property name to coerce its value to a number.
Plus:
If the properties referenced by --id-property are null or undefined,
they are omitted from the output geometry object. Thus, the generated
objects may not have a defined ID if the input features did not have a
property with the specified name.
So, when you are using +code and +code_2, they are probably undefined, as you can't convert the AK string value to a number.
Here, the input property "FIPS" is coerced to a number and used as the
feature identifier; likewise, the column named "FIPS" is used as the
identifier in the CSV file. (If your CSV file uses a different column
name for the feature identifier, you can specify multiple id
properties, such as --id-property=+FIPS,+id.)
That's why you have to add the code to the --id-property=code_2,code option. This is how the mapping is made (the code_2 from topojson.json and the code column from data.tsv).
Then, the output property "unemployment" is generated from the
external data file, unemployment.tsv, which defines the input property
"rate"
In our case, -p code_2,state=name specifies that we will preserve the code_2 property and we will rename the name property to state. The Properties and External Properties sections in the aforementioned documentation wiki are pretty informative on the matter.
The topojson package has been deprecated. The following steps are based on the command-line cartography workflow. These interfaces are more flexible, but a little bit more complicated to use.
Install dependencies:
npm install d3-dsv ndjson-cli
Add the node_modules/.bin directory to the path so that you can easily run the commands:
PATH=$(npm bin):$PATH
Convert the tsv file into a newline-delimited json file:
tsv2json data.tsv -n > data.ndjson
{"id":"1","code":"AL","name":"Alabama"}
{"id":"2","code":"AK","name":"Alaska"}
Parse the id column as a number:
ndjson-map '{id: +d.id, code: d.code, name: d.name}' < data.ndjson > data_parsed.ndjson
{"id":1,"code":"AL","name":"Alabama"}
{"id":2,"code":"AK","name":"Alaska"}
Extract the geometries of the topojson file:
ndjson-cat topojson.json | ndjson-split 'd.objects.states.geometries' > topojson_geometries.ndjson
{"type":"Polygon","arcs":[[0]],"properties":{"code_2":"AK"}}
{"type":"Polygon","arcs":[[1]],"properties":{"code_2":"AL"}}
Join both newline-delimited json files:
ndjson-join 'd.properties.code_2' 'd.code' topojson_geometries.ndjson data_parsed.ndjson > geometries_data_join.ndjson
[{"type":"Polygon","arcs":[[0]],"properties":{"code_2":"AK"}},{"id":2,"code":"AK","name":"Alaska"}]
[{"type":"Polygon","arcs":[[1]],"properties":{"code_2":"AL"}},{"id":1,"code":"AL","name":"Alabama"}]
Add the name column to the topojson properties and only keep the topojson geometries:
ndjson-map 'd[0].properties.name = d[1].name, d[0]' < geometries_data_join.ndjson > geometries_data_merge.ndjson
{"type":"Polygon","arcs":[[0]],"properties":{"code_2":"AK","name":"Alaska"}}
{"type":"Polygon","arcs":[[1]],"properties":{"code_2":"AL","name":"Alabama"}}
Convert the previous result into an array and concat it with the original topojson file:
ndjson-join <(ndjson-cat topojson.json) <(ndjson-reduce < geometries_data_merge.ndjson) > topojson_concat.ndjson
[{
"type": "Topology",
"transform": {
"scale": [0.0015484881821515486, 0.0010301030103010299],
"translate": [-5.491666666666662, 41.008333333333354]
},
"objects": {
"states": {
"type": "GeometryCollection",
"geometries": [{
"type": "Polygon",
"arcs": [[0]],
"properties": {
"code_2": "AK"
}
}, {
"type": "Polygon",
"arcs": [[1]],
"properties": {
"code_2": "AL"
}
}
]
}
},
"arcs": [[[2466, 9916], [-25, -5], [3, -13]], [[2357, 9852], [1, -2], [1, -2]]]
}, [{
"type": "Polygon",
"arcs": [[0]],
"properties": {
"code_2": "AK",
"name": "Alaska"
}
}, {
"type": "Polygon",
"arcs": [[1]],
"properties": {
"code_2": "AL",
"name": "Alabama"
}
}
]
]
Overwrite the geometries of original topojson file and save it as a normal json file:
ndjson-map 'd[0].objects.states.geometries = d[1], d[0]' < topojson_concat.ndjson > topojson_data.json
{
"type": "Topology",
"transform": {
"scale": [0.0015484881821515486, 0.0010301030103010299],
"translate": [-5.491666666666662, 41.008333333333354]
},
"objects": {
"states": {
"type": "GeometryCollection",
"geometries": [{
"type": "Polygon",
"arcs": [[0]],
"properties": {
"code_2": "AK",
"name": "Alaska"
}
}, {
"type": "Polygon",
"arcs": [[1]],
"properties": {
"code_2": "AL",
"name": "Alabama"
}
}
]
}
},
"arcs": [[[2466, 9916], [-25, -5], [3, -13]], [[2357, 9852], [1, -2], [1, -2]]]
}
All commands in one line:
ndjson-join <(ndjson-cat topojson.json) <(ndjson-join 'd.properties.code_2' 'd.code' <(ndjson-cat topojson.json | ndjson-split 'd.objects.states.geometries') <(tsv2json data.tsv -n | ndjson-map '{id: +d.id, code: d.code, name: d.name}') | ndjson-map 'd[0].properties.name = d[1].name, d[0]' | ndjson-reduce) | ndjson-map 'd[0].objects.states.geometries = d[1], d[0]' > topojson_data.json
Notes:
I swapped "AK" and "AL" in the topojson file to check if the join really works.
The last command (before the one-liner) only works on the original output and not on the given pretty-printed version, which has newlines in it.
I tested the workflow on the subsystem for Linux since ndjson-map does not seem to work properly on Windows currently.