I was trying to format my json in jsonformatter.org and I'm not sure why I'm getting the below error
Parse error on line 1:
...","topBrand":false}{"_id":{"$oid":"601c
----------------------^
Expecting 'EOF', '}', ',', ']', got '{'
[error][1]
Sample JSON:
{"_id":{"$oid":"601ac115be37ce2ead437551"},"barcode":"511111019862","category":"Baking","categoryCode":"BAKING","cpg":{"$id":{"$oid":"601ac114be37ce2ead437550"},"$ref":"Cogs"},"name":"test brand #1612366101024","topBrand":false}
{"_id":{"$oid":"601c5460be37ce2ead43755f"},"barcode":"511111519928","brandCode":"BUCKS","category":"Beverages","categoryCode":"BEVERAGES","cpg":{"$id":{"$oid":"5332f5fbe4b03c9a25efd0ba"},"$ref":"Cogs"},"name":"Starbucks","topBrand":false}
The JSON you posted contains two separate JSON documents, and is thus invalid. That's what the error is complaining about, it expected EOF (i.e. End of File), but it instead got the opening-brace from the second JSON document.
Either format them one at a time, or merge them into one document somehow.
For example, you could turn it into a list:
{
"data": [
{ ... },
{ ... }
]
}
Or a larger compound object:
{
"data1": { ... },
"data2": { ... }
}
Got the "syntaxError: JSON parse; unexpected non-whitespace character after JSON data at line 1 column 611 of the JSON data" error. While there are a lot of questions and answers about this and I see what is happening in my code, my question is:
Does this error originate with the construction of the array code here:
echo json_encode(array('html' => $html_string, 'cid' => $cid));
The input is:
$html_string = ['<div class="myclass">'.$value-> Customer_FName.'</div><div class="myclass">'.$value-> Customer_LName.'</div>'];
$cid[] = $value->id;
The output of the echo dumps as :
{"html":"<div class=\\\"myclass\\\">Bob<\/div><div class=\\\"myclass\\\">Smith<\/div>","cid":["44","45","46"]}0
Try escaping your special characters. Example:
{
"html": "<div class=\"flx2 waitlist-image waitlist-listing-img-pointer\">{$value-> Customer_FName}</div><div class=\"flx2 waitlist-image waitlist-listing-img-pointer\">{$value-> Customer_LName}</div>",
"cid": [
"44",
"45",
"46"
]
}
When trying to send my json formatted tcpdump to elasticsearch, I get the following error:
curl -X PUT --data-binary #myjson 'localhost:9200/_bulk?pretty'
{
"error" : {
"root_cause" : [
{
"type" : "parsing_exception",
"reason" : "Unknown key for a START_OBJECT in [layers].",
"line" : 1,
"col" : 95
}
],
"type" : "parsing_exception",
"reason" : "Unknown key for a START_OBJECT in [layers].",
"line" : 1,
"col" : 95
},
"status" : 400
}
The json file was obtained using tshark with the "-T json" option.
The json file was modified using jq with the filter "{index: .[]}" and the option -c since elasticsearch requires an entry to fit in a single line.
I am using elasticsearch 5.5.1 with the standard configuration.
jsonformatter marks the json object as valid.
A json object that produces the error looks as follows:
{"index":{"_index":"packets-2017-08-04","_type":"pcap_file","_score":null,"_source":{"layers":{"frame":{"frame.encap_type":"25","frame.time":"Aug 5, 2001 13:10:06.559762000 CEST","frame.offset_shift":"0.000000000","frame.time_epoch":"1501773006.559765000","frame.time_delta":"0.000000000","frame.time_delta_displayed":"0.000000000","frame.time_relative":"0.000000000","frame.number":"1","frame.len":"200","frame.cap_len":"200","frame.marked":"0","frame.ignored":"0","frame.protocols":"sll:ethertype:ip:tcp:data"},"sll":{"sll.pkttype":"4","sll.hatype":"65135","sll.halen":"0","sll.etype":"0x00000800"},"ip":{"ip.version":"4","ip.hdr_len":"20","ip.dsfield":"0x00000010","ip.dsfield_tree":{"ip.dsfield.dscp":"4","ip.dsfield.ecn":"0"},"ip.len":"184","ip.id":"0x000093f2","ip.flags":"0x00000002","ip.flags_tree":{"ip.flags.rb":"0","ip.flags.df":"1","ip.flags.mf":"0"},"ip.frag_offset":"0","ip.ttl":"64","ip.proto":"6","ip.checksum":"0x0000ef4b","ip.checksum.status":"2","ip.src":"0.0.00","ip.addr":"0.0.0.0","ip.src_host":"0.0.0.0","ip.host":"0.0.0.0","ip.dst":"0.0.0.0","ip.dst_host":"0.0.0.0","Source GeoIP: Germany":{"ip.geoip.src_country":"Germany","ip.geoip.country":"Germany","ip.geoip.src_city":"Frankfurt, 1","ip.geoip.city":"Berlin, 1","ip.geoip.src_asnum":"123","ip.geoip.asnum":"123","ip.geoip.src_lat":"701","ip.geoip.lat":"523,01","ip.geoip.src_lon":"2313,4","ip.geoip.lon":"12,13"},"Destination GeoIP: Germany":{"ip.geoip.dst_country":"Germany","ip.geoip.country":"Germany","ip.geoip.dst_asnum":"123","ip.geoip.asnum":"123","ip.geoip.dst_lat":"3321","ip.geoip.lat":"41","ip.geoip.dst_lon":"1","ip.geoip.lon":"2"}},"tcp":{"tcp.srcport":"41","tcp.dstport":"124","tcp.port":"234","tcp.stream":"3","tcp.len":"134","tcp.seq":"1","tcp.nxtseq":"133","tcp.ack":"4","tcp.hdr_len":"32","tcp.flags":"0x00000018","tcp.flags_tree":{"tcp.flags.res":"0","tcp.flags.ns":"0","tcp.flags.cwr":"0","tcp.flags.ecn":"0","tcp.flags.urg":"0","tcp.flags.ack":"1","tcp.flags.push":"1","tcp.flags.reset":"0","tcp.flags.syn":"0","tcp.flags.fin":"0","tcp.flags.str":"·······AP···"},"tcp.window_size_value":"223","tcp.window_size":"31","tcp.window_size_scalefactor":"-1","tcp.checksum":"0x0000b79c","tcp.checksum.status":"1","tcp.urgent_pointer":"0","tcp.options":"123","tcp.options_tree":{"No-Operation (NOP)":{"tcp.options.type":"1","tcp.options.type_tree":{"tcp.options.type.copy":"0","tcp.options.type.class":"0","tcp.options.type.number":"1"}},"Timestamps: TSval 1875055084, TSecr 5726840":{"tcp.option_kind":"8","tcp.option_len":"10","tcp.options.timestamp.tsval":"185084","tcp.options.timestamp.tsecr":"1116840"}},"tcp.analysis":{"tcp.analysis.bytes_in_flight":"123","tcp.analysis.push_bytes_sent":"133"}},"data":{"data.data":"01:01:02","data.len":"265"}}}}}
My question is: What is wrong with this json, so that elasticsearch rejects it?
This is not really a jq problem - Unknown key for a START_OBJECT is an elasticsearch error. The [layers] is a hint that the problem is in the object there which unfortunately was elided in the problem description so there's really not much to go on here.
Since the jq filter you specified is just {index:.[]}, jq is doing nothing to the part of the json elasticsearch is complaining about. If your workflow is expecting jq to correct that portion somehow you'll need to investigate the data closer and use a more sophisticated filter.
For reference, the elasticsearch test suite contains an example of this particular error:
---
"junk in source fails":
- do:
catch: /Unknown key for a START_OBJECT in \[junk\]./
reindex:
body:
source:
junk: {}
Hope this helps.
I try to find out why my JSON is not legal.
I use this site: http://jsonlint.com/
The first example which is good is:
{
"data": 1290,
"value": "a"
}
The second which is not good is:
{
"data": 1290,
"value": "a"
}
I dont understand why the second one does not work. Its the same as the first one.
EDIT
I found this sings at the end of the string.
How to remove them using PHP? the string source is from php.
There might be some additional output after you output your json. Try trimming output and terminating further execution.
$json = array('data' => 1290, 'value' => 'a');
echo json_encode($json);
die();
I've searched the forum and seen many folks with a similar problem, but not this exact one.
I think my question is the simplest form, and there must be something I'm missing because no one is asking it.
I have a shell script that calls a MongoDB script and gets the results in a file. I then want to parse that file with jq.
jq is breaking because the output from the query is not valid JSON. The offender is the ObjectId. I'm at a total loss as to how something that's "ALL JSON ALL THE TIME" produces invalid JSON.
I'm pretty sure there's something fundamental that I'm missing.
I have a file called MyMongoScript.js. Its contents look like this:
db.WorkflowJobs.find().sort({"STATUS":1}).forEach(printjson)
I call MyMongScript.js with the following command:
mongo -u $MONGO_U -p $MONGO_P $MONGO_DB -quiet --eval "var DATE_TO_RUN=\"$DATE_TO_RUN\"" MyMongoScript.js
Here's the results to STDOUT:
{
"_id" : ObjectId("52816fd50bc9efc3e6d8e33f"),
"WORK_TYPE" : "HIVE",
"Script_Name" : "upload_metrics_LANDING_to_HIST.sql",
"Stop_On_Fail" : true,
"STATUS" : "READY",
"START_TS" : "NULL",
"END_TS" : "NULL",
"DURATION" : "NULL",
"INS_TS" : "Mon Nov 11 2013 16:01:25 GMT-0800 (PST)"
}
Here's what jsonlint.com says about it:
Parse error on line 2:
{ "_id": ObjectId("52816fd50b
------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
Any help much appreciated.
Try this for your MyMongoScript.js:
db.WorkflowJobs.find().sort({"STATUS":1}).forEach(function(myDoc){myDoc._id=myDoc._id.valueOf();print(tojson(myDoc))});
The key is valueOf() which will set your ObjectId to a String.
EDITED Left out a paren.