Since yesterday, I am sitting fast to get it as .csv. but fails.
Tried with below commands, but failss with following error:
jq: error (at j.json:70): object ({"DCS_CCH_A...) is not valid in a csv row
jq -r '[.[]] | #csv' j.json
Tried also another way, still fails, with followingerror:
jq: error (at j.json:1): Cannot iterate over string ("{")
jq: error (at j.json:2): Cannot iterate over string (" \"vehi...)
jq: error (at j.json:3): Cannot iterate over string (" \"...).
jq -Rr '[.[]] | #csv' j.json
How I can convert it as csv?
OS is Linux Ubuntu
This is my JSON file:
{
"vehicleStatus": {
"DCS_CCH_Activation": "NA",
"DCS_CCH_Ongoing": false,
"cbsData": [
{
"cbsDescription": "Next service due when the stated distance has been covered or by the specified date.",
"cbsDueDate": "2021-02",
"cbsRemainingMileage": 18000,
"cbsState": "OK",
"cbsType": "OIL"
},
{
"cbsDescription": "Next change due at the latest by the stated date.",
"cbsDueDate": "2022-03",
"cbsState": "OK",
"cbsType": "BRAKE_FLUID"
},
{
"cbsDescription": "Next visual inspection due when the stated distance has been covered or by the stated date.",
"cbsDueDate": "2023-02",
"cbsRemainingMileage": 49000,
"cbsState": "OK",
"cbsType": "VEHICLE_CHECK"
}
],
"chargingLevelHv": 17,
"chargingStatus": "WAITING_FOR_CHARGING",
"connectionStatus": "CONNECTED",
"doorDriverFront": "CLOSED",
"doorDriverRear": "CLOSED",
"doorLockState": "SECURED",
"doorPassengerFront": "CLOSED",
"doorPassengerRear": "CLOSED",
"fuelPercent": 54.5,
"hood": "CLOSED",
"mileage": 13919,
"parkingLight": "OFF",
"position": {
"heading": 1,
"lat": 18.3,
"lon": 84.5,
"status": "OK"
},
"positionLight": "OFF",
"rearWindow": "INVALID",
"remainingFuel": 23,
"remainingRangeElectric": 4,
"remainingRangeElectricMls": 2,
"remainingRangeFuel": 260,
"remainingRangeFuelMls": 161,
"singleImmediateCharging": false,
"sunroof": "CLOSED",
"trunk": "CLOSED",
"updateReason": "DOOR_STATE_CHANGED",
"vin": "some_vin",
"windowDriverFront": "CLOSED",
"windowDriverRear": "CLOSED",
"windowPassengerFront": "CLOSED",
"windowPassengerRear": "CLOSED"
}
}
In this question, someone provided a generic JSON to CSV converter using jq. There are several other questions on here that explain more about the mechanics of the process, such as this one.
Related
I am working on a project where I want to query a JSon using Jquery but I am getting an error:
jq: error: syntax error, unexpected IDENT, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
..id
jq: 1 compile error
Error: Process completed with exit code 3.
I am using the command:
jq '..id' new_file.json
To query my JSON file which is named new_file.json
Here is my JSON file:
[
{
"type": "users_export",
"status": "pending",
"connection_id": "con_kmZIjREZWYzt39JI",
"format": "json",
"limit": 5,
"fields": [
{
"name": "user_id"
},
{
"name": "user_id",
"export_as": "provider"
},
{
"name": "username"
},
{
"name": "username",
"export_as": "provider"
},
{
"name": "nickname"
},
{
"name": "email"
},
{
"name": "email"
},
{
"name": "identities[0].connection",
"export_as": "provider"
},
{
"name": "email_verified",
"export_as": "provider"
}
],
"connection": "dev-default-evoyanbs",
"created_at": "2022-11-16T17:45:58.429Z",
"id": "job_aztDgKXWT8g8iZ5T"
}
]
I want the 'job_aztDgKXWT8g8iZ5T' as my output but I am getting the above mentioned error, can someone please help me out, thanks.
As you have an array, the command should be:
jq '.[].id' new_file.json
And the result will be: "job_aztDgKXWT8g8iZ5T".
If you want to get rid of double quotes in your output, you can add raw-output flag in your command, like this:
jq -r '.[].id' new_file.json
or
jq --raw-output '.[].id' new_file.json
The JSON output I am trying to parse:
{
"success": true,
"data": {
"aa": [
{
"timestamp": 123456,
"price": 1
},
{
"timestamp": 123457,
"price": 2
],
"bb": [
{
"timestamp": 123456,
"price": 3
},
{
"timestamp": 123457,
"price": 4
}
]
}
}
So after banging my head against the wall a million times, I just removed the "success": true", line from the output and I could easily do jq stuff with it. Otherwise if I ran for example:
cat jsonfile.json | jq -c .[].aa
I would get:
Cannot index boolean with string "aa"
Which makes sense, since the first key is boolean. But I have no clue how to skip it while processing with jq.
Goal is to filter only timestamp and price of "aa", without giving any care about the "success": true key/value pair.
You need to select the data field first: jq .data.aa[]
I know how to retrieve an entire JSON attribute with jq, but I want to extract only a specific substring. Consider the following sample input:
[
{
"name": "test",
"output": "",
"error": "",
"state": "unknown",
"startTime": 1571292623936,
"endTime": 0,
"extra": {},
"warning": "************************* test Warnings *************************\n*\n* \n*****************************************************************",
"hasWarning": false
},
{
"name": "npm run test",
"output": "\n> DISPLAY was set to: \":99\"\n\nCypress will attempt to fix the problem and rerun.\n\n\n Running: consumer/oct.js... (1 of 1) \nPROCESSING JS RESOURCE FILE FROM:/PMT1469/workspace/E2EI/cypress/e2e/consumer/kindle.js\n{\"dataFile\":\"scripts/regression/transfers/card/kindle.csv\"}\nSENDING JS RESOURCE FILE FROM: /PMT-1469/workspace/E2E-UI { startedTestsAt: '2019-10-17T06:10:59.339Z',\n endedTestsAt: '2019-10-17T06:11:53.542Z',\n totalDuration: 54203,\n totalSuites: 4,\n totalTests: 2,\n totalFailed: 2,\n totalPassed: 0,\n totalPending: 0,\n totalSkipped: 0,\n\n browserPath: '',\n browserName: 'electron',\n reporter: 'mochawesome',\n taskTimeout: 60000,\n video: true,\n known: true }\n",
"error": null,
"state": "success",
"startTime": 1571292631223,
"endTime": 1571292718780,
"extra": {},
"warning": "************************* npm run test Warnings *************************\n*\n* \n*************************************************************************",
"hasWarning": false
}
]
I just want to pick the following values in the above JSON payload which is in "output" attribute.
Expected output:
totalDuration: 54203
totalSuites: 4
totalFailed: 2
totalPassed: 0
totalSkipped: 0
We can easily fetch the attribute values using jq -r '.[].output', but I'm trying to only capture substrings of the form total<something>: <number>.
The inefficient-but-easy answer is to do the bulk of the work in a separate pipeline stage. Assuming GNU tools:
jq -r '.[].output' <in.json \
| grep -Eo '^[[:space:]]+(total[[:alpha:]]+: [[:digit:]]+)' \
| sed -re 's/^[[:space:]]+//'
However, with modern jq, one can do much better:
jq -r '.[].output | scan("total[[:alpha:]]+: [[:digit:]]+")' <in.json
Hi I am new to JQ and Json. I am using
$ jq --version
jq-1.5
I am having a heck of a time trying to figure out how to select the values for id, attributes.name, attributes.albumName, and attributes.artistName
I am using the terminal app on a mac. I am running into some sort strange parsing problem
$ jq '.results.songs.data[0] | {id, attributes.name } ' t
jq: error: syntax error, unexpected FIELD, expecting '}' (Unix shell quoting issues?) at <top-level>, line 1:
.results.songs.data[0] | {id, attributes.name }
jq: 1 compile error
$
This example shows the structure of the data I am trying to filter looks like
$ jq '.results.songs.data[0] | {id, attributes } ' t
{
"id": "152471393",
"attributes": {
"previews": [
{
"url": "https://audio-ssl.itunes.apple.com/apple-assets-us-std-000001/AudioPreview71/v4/7d/c5/68/7dc56849-29b8-bd90-2bb1-51750e479569/mzaf_4742389090778091050.plus.aac.p.m4a"
}
],
"artwork": {
"width": 1449,
"height": 1449,
"url": "https://is5-ssl.mzstatic.com/image/thumb/Music/v4/7d/01/56/7d0156be-12cd-8724-a0ca-727b1013a81d/source/{w}x{h}bb.jpeg",
"bgColor": "ddcfc4",
"textColor1": "010100",
"textColor2": "422f10",
"textColor3": "2d2a27",
"textColor4": "614f34"
},
"artistName": "Gnarls Barkley",
"url": "https://itunes.apple.com/us/album/crazy/152471339?i=152471393",
"discNumber": 1,
"genreNames": [
"Alternative",
"Music",
"R&B/Soul",
"Rock",
"Soul",
"Hip-Hop/Rap",
"Rap",
"Hip-Hop",
"Adult Alternative",
"Neo-Soul",
"Alternative Rap",
"Underground Rap"
],
"durationInMillis": 178387,
"releaseDate": "2006-03-13",
"name": "Crazy",
"isrc": "USAT20611041",
"albumName": "St. Elsewhere",
"playParams": {
"id": "152471393",
"kind": "song"
},
"trackNumber": 2
}
}
Thanks
Andy
With your sample JSON as input, the following invocation:
jq '{id, name: .attributes.name }' input.json
produces:
{
"id": "152471393",
"name": "Crazy"
}
The filter above is short for:
{"id" : .id, "name": .attributes.name }
In any case, the keys must be appropriately specified.
For future reference, when asking questions on stackoverflow.com, please adhere to the http://stackoverflow.com/help/mcve guidelines as much as possible.
I'm using Talend ETL Tool and extracting data from json files and storing them in Mysql database.
But I get the error while reading in very first json. For reading json I'm using tExtractJSONFileds component.
I'm sure about the configuation set up in talend etl tool its right. I believe there is some problem in json file.
While extracting the component shows error like this
Exception in component tExtractJSONFields_1
javax.xml.stream.XMLStreamException: java.io.IOException: Unexpected symbol: COMMA
at de.odysseus.staxon.base.AbstractXMLStreamReader.initialize(AbstractXMLStreamReader.java:218)
at de.odysseus.staxon.json.JsonXMLStreamReader.<init>(JsonXMLStreamReader.java:65)
at de.odysseus.staxon.json.JsonXMLInputFactory.createXMLStreamReader(JsonXMLInputFactory.java:148)
at de.odysseus.staxon.json.JsonXMLInputFactory.createXMLStreamReader(JsonXMLInputFactory.java:44)
at de.odysseus.staxon.base.AbstractXMLInputFactory.createXMLEventReader(AbstractXMLInputFactory.java:118)
I dont know how to deal with JSONs, So Acc to this error can anyone help me where could be the error in JSON file ?
Is there any value passed as NULL or something else ?
Sample JSON
[
[, {
"tstamp": "123456",
"event": "tgegfght",
"is_duplicate": false,
"farm": "dyhetygdht",
"uid": "tutyvbrtyvtrvy",
"clientip": "52351365136",
"device_os_label": "MICROSOFT_WINDOWS_7",
"device_browser_label": "MOZILLA_FIREFOX",
"geo_country_code": "MA",
"geo_region_code": "55",
"geo_city_name_normalized": "agadir",
"referer": "www.abc.com",
"txn": "etvevv5r",
"txn_isnew": true,
"publisher_id": 126,
"adspot_id": 11179502,
"ad_spot": 5188,
"format_id": 1611,
"misc": {
"PUBLISHER_FOLDER": "retvrect",
"NO_PROMO": "rctrctrc",
"SECTION": "evtrevr",
"U_COMMON_ALLOW": "0",
"U_Auth": "0"
},
"handler": "uint"
}, , ]
Thanks in advance !!
You have extra empty commas in your sample json.
Your Sample Json should look like
[{
"tstamp": "123456",
"event": "tgegfght",
"is_duplicate": false,
"farm": "dyhetygdht",
"uid": "tutyvbrtyvtrvy",
"clientip": "52351365136",
"device_os_label": "MICROSOFT_WINDOWS_7",
"device_browser_label": "MOZILLA_FIREFOX",
"geo_country_code": "MA",
"geo_region_code": "55",
"geo_city_name_normalized": "agadir",
"referer": "www.abc.com",
"txn": "etvevv5r",
"txn_isnew": true,
"publisher_id": 126,
"adspot_id": 11179502,
"ad_spot": 5188,
"format_id": 1611,
"misc": {
"PUBLISHER_FOLDER": "retvrect",
"NO_PROMO": "rctrctrc",
"SECTION": "evtrevr",
"U_COMMON_ALLOW": "0",
"U_Auth": "0"
},
"handler": "uint"
}]
OR
[
{
"somethinghere": "its value"
},
"somethingelse": "its value"
]
Your sample json is not valid json, due to the spurious extra commas on the second and last lines. Json only allows commas BETWEEN elements of a vector or object, and empty elements are not allowed.