Parsing this JSON snippet with Gson JsonReader streaming parser.
{
"host_id": "5001092",
"finding_number": "plsql:DeleteOrUpdateWithoutWhereCheck-BUG",
"scan_type": "SONARQUBE",
"justification_assignee": null,
"justification_text": "[{\"comment\":\"deleting the table for reload\",\"date\":1608120428,\"user\":\"n3vt\"},{\"comment\":\"Deleting the table for reload\",\"date\":1608120467,\"user\":\"n3vt\"}]",
"justification_status_id": "6",
"justification_verification_date": null,
"has_file": "0",
"security_comment_text": "Not yet provided.",
"justification_datetime": "2022-01-24 19:01:44",
"justification_external_issues": null,
"due_date": null,
"justification_status_name": "Accepted Risk",
"justification_status_mitigating": "1"
}
JsonReader.nextString() returns this value for "justification_text":
"justification_text": "[{"comment":"deleting the table for reload","date":1608120428,"user":"n3vt"},{"comment":"Deleting the table for reload","date":1608120467,"user":"n3vt"}]"
It's stripped out all the backslashes, essentially un-escaping all the escaped double quotes, so it's no longer valid JSON. Is there a way to stop this from happening?
Related
I want to create this JSON text with Delphi IDE:
{
"uygulamaninAdi": "ZGVuZW1l",
"uygulamaninSurumu1": "1",
"uygulamaninSurumu2": "0",
"uygulamaninSurumu3": "0",
"uygulamaninSurumu4": "0",
"uygulamaninMimarisi": "1",
"calistirilmaDuzeyi": "1",
"mesajGonderebilmeDurumu": "1",
"konum": "RDpcUHJvZ3JhbWxhbWE="
}
Here are my codes:
procedure TfrmManifestDosyasiOlusturucu.btnOluşturClick(Sender: TObject);
begin
jsGönderilecekMesaj := TJSONObject.Create;
jsGönderilecekMesaj.AddPair('uygulamaninAdi', TNetEncoding.Base64.Encode(edUygulamanınAdı.Text));
jsGönderilecekMesaj.AddPair('uygulamaninSurumu1', edUygulamanınSürümü1.Text);
jsGönderilecekMesaj.AddPair('uygulamaninSurumu2', edUygulamanınSürümü2.Text);
jsGönderilecekMesaj.AddPair('uygulamaninSurumu3', edUygulamanınSürümü3.Text);
jsGönderilecekMesaj.AddPair('uygulamaninSurumu4', edUygulamanınSürümü4.Text);
jsGönderilecekMesaj.AddPair('uygulamaninMimarisi', IntToStr(cbUygulamanınMimarisi.ItemIndex));
jsGönderilecekMesaj.AddPair('calistirilmaDuzeyi', IntToStr(cbÇalıştırılmaDüzeyi.ItemIndex));
jsGönderilecekMesaj.AddPair('mesajGonderebilmeDurumu', IntToStr(cbMesajGönderebilmeDurumu.ItemIndex));
jsGönderilecekMesaj.AddPair('konum', TNetEncoding.Base64.Encode(edManifestDosyasınınOluşturulacağıKonum.Text));
strKomutSatırıParametreleri := '/olustur ' + jsGönderilecekMesaj.ToString;
ShellExecute(0, 'open', 'bin\Manifest Dosyası Oluşturucu (Yardımcı Uygulama).exe', PWideChar(strKomutSatırıParametreleri), nil, SW_HIDE);
end;
But the problem is the Delphi IDE creates this:
{uygulamaninAdi:ZGVuZW1l,uygulamaninSurumu1:1,uygulamaninSurumu2:0,uygulamaninSurumu3:0,uygulamaninSurumu4:0,uygulamaninMimarisi:1,calistirilmaDuzeyi:1,mesajGonderebilmeDurumu:1,konum:RDpcUHJvZ3JhbWxhbWE=}
As far as I know all of keys should be nested with quotes and all of string values should be nested be quotes.
How can I fix my problem?
Try using TJSONObject.ToJSON instead of TNSONObject.ToString.
Also, JSON really isn't well-suited for passing around on the command line. If the target program expects to receive the JSON as a single parameter, and since the JSON has quotes in it and potentially also spaces, you should use AnsiQuotedStr() to add quotes around, and to escape quotes inside, of the JSON string.
I have field of type json. Example value is:
[{"id": "960287", "src_ip": "X.X.X.X", "filename": "XXX-20200408092811-0", "order_id": "199926", "download_ip": "", "datetime_add": "2020-04-09 09:16:48", "order_desk_id": null, "datarejestracji": null, "download_browser": "", "datetime_download": "0000-00-00 00:00:00"}, {"id": "960288", "src_ip": "2.128.4.33", "filename": "XXX-20200408101526-4", "order_id": "199926", "download_ip": "", "datetime_add": "2020-04-09 09:16:48", "order_desk_id": null, "datarejestracji": null, "download_browser": "", "datetime_download": "0000-00-00 00:00:00"}, {"id": "960751", "src_ip": "2.128.4.33", "filename": "20200409164205-24", "order_id": "199926", "download_ip": "", "datetime_add": "2020-04-09 20:02:46", "order_desk_id": null, "datarejestracji": null, "download_browser": "", "datetime_download": "0000-00-00 00:00:00"}]
How to select specified property in SQL query?
If you are using MySQL 5.7 or later, you can use the JSON_EXTRACT() function:
SELECT JSON_EXTRACT(myjsoncolumn, '$[0].src_ip') AS src_ip
FROM mytable;
There's also a shorthand syntax:
SELECT myjsoncolumn->'$[0].src_ip' AS src_ip
FROM mytable;
Note that this function returns JSON, not the scalar string value, so it'll look like "X.X.X.X" with quotes around it. If you want the raw value, use:
SELECT JSON_UNQUOTE(JSON_EXTRACT(myjsoncolumn, '$[0].src_ip')) AS src_ip
FROM mytable;
Or the shorthand:
SELECT myjsoncolumn->>'$[0].src_ip' AS src_ip
FROM mytable;
Read the documentation on JSON functions for more information: https://dev.mysql.com/doc/refman/8.0/en/json-functions.html
If you are not using at least MySQL 5.7, you should upgrade. Older versions are now past their end-of-life, and if you're storing JSON data in the database, it will be very difficult to do anything but return the whole JSON document to clients.
Given the structure of your example JSON, I think you might need to use JSON_TABLE().
It looks like you have an array of JSON objects with identical fields, so I wonder why you are using JSON at all, instead of storing these objects as rows with normal columns.
I am trying to create a stream in Apache/KAFKA KSQL
The topic contains (somewhat complex JSON)
{
"agreement_id": "dd8afdbe-59cf-4272-b640-b14a24d8234c",
"created_at": "2018-02-17 16:00:00.000Z",
"id": "6db276a8-2efe-4495-9908-4d3fc4cc16fa",
"event_type": "data",
"total_charged_amount": {
"tax_free_amount": null,
"tax_amounts": [],
"tax_included_amount": {
"amount": 0.0241,
"currency": "EUR"
}
}
"used_service_units": [
{
"amount": 2412739,
"currency": null,
"unit_of_measure": "bytes"
}
]
}
Now creating a stream is easy for just simple stuff like event_type and created_at. That would be like this
CREATE STREAM tstream (event_type varchar, created_at varchar) WITH (kafka_topic='usage_events', value_format='json');
But now I need to access the used_service_units....
and I would like to extract the "amount" in the JSON above
How would I do this ?
CREATE STREAM usage (event_type varchar,create_at varchar, used_service_units[0].amount int) WITH (kafka_topic='usage_events', value_format='json');
Results in
line 1:78: mismatched input '[' expecting {'ADD', 'APPROXIMATE', ...
And if I instead create a stream like so
CREATE STREAM usage (event_type varchar,create_at varchar, used_service_units varchar) WITH (kafka_topic='usage_events', value_format='json');
And then does a SQL SELECT on the stream like this
SELECT EXTRACTJSONFIELD(used_service_units,'$.amount') FROM usage;
SELECT EXTRACTJSONFIELD(used_service_units[0],'$.amount') FROM usage;
SELECT EXTRACTJSONFIELD(used_service_units,'$[0].amount') FROM usage;
Neither of these alternatives work...
This one gave me
SELECT EXTRACTJSONFIELD(used_service_units[0],'$.amount') FROM usage;'
Code generation failed for SelectValueMapper
It seems that ONE solution to this problem is to make the column datatype an array
i.e.
CREATE STREAM usage (event_type varchar,created_at varchar, total_charged_amount varchar, used_service_units array<varchar> ) WITH (kafka_topic='usage_events', value_format='json');
Now I am able to do the following:
SELECT EXTRACTJSONFIELD(used_service_units[0],'$.amount') FROM usage
So when I parse a file with EvaluateJSON the JSON looks like this:
{ "TEST_DATE": "", "T_DATE": "" }
When I do EvaluateJSON ($.TEST_DATE) to "test-date" on attributes... I get:
test-date: ""
Then when I do AttributesToJSON (flowfile-content destination) I get:
{ "test-date": "", "t-date": "" }
HOWEVER... I want it to be:
{ "test-date": null, "t-date": null }
I have tried every possible option. There is no way aside from "ReplaceText"-style dangerous regex to put NULL into the JSON.
Any updateAttribute fails to put "null" into it. I tried "replaceEmpty(null)", replaceEmpty("null") (which puts a string "null" instead). I tried "replaceEmpty(literal("null")) doesn't work.
It's like Nifi doesn't recognize null.
DExter,
You can replace the double quotes("") by null value in ReplaceText processor.
Afterwards you get below value.
{ "test-date": "", "t-date": "" }
Use ReplaceText processor to be search for empty double quotes and replace it with null.
search value:""
Replacement value:null
For your reference check this https://regexr.com/3kctp.
It will end like your required result;
{ "test-date": null, "t-date": null }
Please let me know, if you face any issues.
This is the my document which i store in the bucket.Also the Id(key) attribute is screenName.
{
"created": null,
"createdBy": null,
"updated": null,
"updatedBy": null,
"screenName": "steelers",
"appId": "100",
"domain": "100$APPINFOTEAM",
"alias": "steelers",
"devision": "1"
}
I have multiple documents in Couchbase in this format. So i need to fetch these documents in descending order. So this is the implementation I used it for,
Query query = new Query();
// Filter on the domain key
query.setIncludeDocs(true);
query.setDescending(true);
query.setInclusiveEnd(true);
query.setKey(domain);
List<AppInfoTeam> appInfoTeam = appinfoTeamService.getAppInfoTeamForApp(query);
This will give me the exact documents without sorting. This is my view
function (doc, meta) {
if (doc._class == "com.link.twitter.pojo.AppInfoTeam") {
emit(doc.domain, null);
}
}
Also I tried to Filter Results using Couchbase server interface. I tick the descending and inclusive_end values. Also put the domain as a key. Then when I click the show results button it will give me this error.
url: ?stale=false&descending=true&inclusive_end=true&key=domain&connection_timeout=60000&limit=10&skip=0
Error:
{"error":"bad_request","reason":"invalid UTF-8 JSON: {{error,{1,\"lexical error: invalid char in json text.\\n\"}},\n \"domain\"}"}
How can I fix this issue?
You need to wrap the key with double quotes:
<url>?stale=false&descending=true&inclusive_end=true&key="domain"&connection_timeout=60000&limit=10&skip=0