Is there a way to convert JSONL to JSON in Linux with the full JSONL file depth? I found some methods based on jq but they don't work with full depth of JSONL file
Would something like this work?
#!/bin/sh
echo "[" >$1.json
perl -pe 's/$/,/' <$1 >>$1.json
echo "]" >>$1.json
I am quite confused as to what you want to do. But when it comes to jq, normally I process things line by line, with each line being an atomic JSON object. Something like
cat file | jq some-options 'some commands' > output.txt
Sometimes I get the output in tsv format and pipe it into awk. jq is very friendly with line-by-line objects.
To convert a large JSON list into line by line format, just parse the large object in any programming language, and serialize the inner objects back to json line by line.
But if you have already parsed the large object, I suggest you do the required processing you want to do in jq directly, without serializing the inner objects back...
Related
I have a json file that is getting created at runtime using the sh script within groovy code. The json file has below contents.
cat.json
{
"user1":"pass1",
"user2":"pass2",
"user3":"pass3"
}
Now I want to create a file at runtime which stores key value pairs in below format
test
user1:pass1
user2:pass2
user3:pass3
can some one help me out shell codes for writing this.
You have literally dozen ways to convert that JSON document to a tabular data file (pretty much like CSV/colon-SV) since you mentioned Java, Groovy, including Java-driven scripting engines (BeanShell, JavaScript, Groovy itself), but if you can use jq then you can extract k/v pairs at least for simple values that do not require any escaping:
#!/bin/sh
jq -r 'to_entries[] | "\(.key):\(.value)"' \
< cat.json
This answer is inspired by searching for extracting entries using jq (or converting a JSON file to a CSV file) and especially by the answer https://stackoverflow.com/a/50496145/12232870 by #peak.
I need to create a JSON file with the following contents:
{
"destinationVersion":"4",
"sourceVersion":"0",
"props":{
"METADATA_SIZE":"91669",
"METADATA_HASH":"O7CLdR2j7qoD0RI2k1AGc8b+xoWYn20Ic24eZ1ZWUWE=",
"FILE_SIZE":"980374602",
"FILE_HASH":"+XW4QKN5Y4ynTx43m4NYbMuk1x3P91f1biAVZBpj4fI="
}
}
The main snag with the props block.
These values for props must be read from the text file.
In it they are in the following format:
FILE_HASH=+XW4QKN5Y4ynTx43m4NYbMuk1x3P91f1biAVZBpj4fI=
FILE_SIZE=980374602
METADATA_HASH=O7CLdR2j7qoD0RI2k1AGc8b+xoWYn20Ic24eZ1ZWUWE=
METADATA_SIZE=91669
My task is to read the file and somehow create the final JSON (append formatted text to props block).
I try to do this through jq, but I don’t know how to convert the file to be added to the final JSON. Preferred way - bash + jq
Since the Q only indicates the source for the key=value strings, the following focuses on the conversion of these strings to a JSON object.
Invocation: jq -n -R -f program.jq props.txt
program.jq:
[inputs | capture("^(?<key>[^=]*)=(?<value>.*)")]
| from_entries
Using inputs here has several small advantages but is inessential.
I want to get value from JSON file:
Example:
{"name":"ghprbActualCommitAuthorEmail","value":"test#gmail.com"},{"name":"ghprbPullId","value":"226"},{"name":"ghprbTargetBranch","value":"master"},
My expect is :
I want to get test#gmail.com, 226 and master.
sed is the wrong tool for processing JSON.
Assuming you have a file tmp.json with valid JSON like
[{"name":"ghprbActualCommitAuthorEmail","value":"test#gmail.com"},
{"name":"ghprbPullId","value":"226"},
{"name":"ghprbTargetBranch","value":"master"}]
you can use jq '.[].value' tmp.son.
If the file instead contains
{"name":"ghprbActualCommitAuthorEmail","value":"test#gmail.com"}
{"name":"ghprbPullId","value":"226"}
{"name":"ghprbTargetBranch","value":"master"}
(i.e., just a stream of 3 separate JSON objects, you could use jq '.value' tmp.json, as jq will apply the filter to each object in succession. You can also use jq -s '.[].value' tmp.son, where the -s flag tells jq to read the entire input into an array first. This lets you use the same filter in both cases.
Is it possible to efficiently get the first record of a JSONL file without consuming the entire stream / file? One way I have been able to inefficiently do so is the following:
curl -s http://example.org/file.jsonl | jq -s '.[0]'
I realize that head could be used here to extract the first line, but assume that the file may not use a newline as the record separator and may simply be concatenated objects or arrays.
If I'm understanding correctly, the JSONL format just returns a stream of JSON objects which jq handles quite nicely. Best case scenario that you wanted the first item, you could just utilize the input filter to grab the first item.
I think you could just do this:
$ curl -s http://example.org/file.jsonl | jq -n 'input'
You need the null input -n to not process the input immediately then input just gets one input from the stream. No need to go through the rest of the input stream.
I still cannot parse JSON data in linux.
I need a linux command to parse Json data to readable string.
someone told me to use underscore-cli.(https://npmjs.org/package/underscore-cli)
I install and use it, still the result is unreadable.
my data:
"2005\u5e7405\u670812\u65e5(\u6728) 02\u664216\u5206"
according to this link
http://json.parser.online.fr/
the result is
"2005年05月12日(木) 02時16分"
Is there any other way to parse this Json data?
Please help.
Try jq: http://stedolan.github.com/jq/
echo '"2005\u5e7405\u670812\u65e5(\u6728) 02\u664216\u5206"' | ./jq .
"2005年05月12日(木) 02時16分"
jq takes escaped unicode and outputs it in utf-8.