I have Json data as follows:
{
"template" : [
"{
"Id": "abc"
}"
]
}
I am using JSONPath to extract data from the Json above. I would like to extract the "Id" data from the Json using JsonPath.
The problem I see is, the data is being treated as a string and not as a Json as shown below.
"{
"Id": "abc"
}"
If there were no double-quotes I could have used JsonPath as follows:
$.template[0].Id
But due to the double-quotes, I am unable to access the "Id" data. I suspect there is a way to access this data using JsonPath-Expression but I am pretty much a novice here and no amount of research helped me out with a resolution.
How do I treat it as a Json and not as a string using JsonPath? Kindly help me out here.
JSON Path isn't going to be able to parse JSON that's encoded within a string. You need to perform three operations:
Get the string (use JSON Path or something else)
Parse the string as JSON.
Get the data you're looking for on that (JSON Path or something else)
Related
I have a JSON like below:
{
"abc":" hello",
"def":" hi"
}
I want to compare thus with JDBC request output. I have saved the output in variables "abc" and "def" so as to match my JSON. I am using groovy to compare. I tried using JsonSlurper but could not retrieve the elements in JSON by index so that I can iterate through the JSON response. Please help me on this.
I have this kind of data in a Kafka Topic:
{..., fields: { "a": "aval", "b": "bval" } }
If I create a Kafka Engine table, I get an error when using a field definition like this:
fields String
because it (correctly) doesn't recognize it as a String:
2018.07.09 17:09:54.362061 [ 27 ] <Error> void DB::StorageKafka::streamThread(): Code: 26, e.displayText() = DB::Exception: Cannot parse JSON string: expected opening quote: (while read the value of key fields): (at row 1)
As ClickHouse does not currently have a Map or JSONObject type, what would be the best way to work over it, provided I don't know in advance the name of the inner fields ("a" or "b" in the example - so I cannot see Nested structures helping)?
Apparently, at the moment ClickHouse does not support complex JSON parsing.
From this answer in ClickHouse Github:
Clickhouse uses quick and dirty JSON parser, which does not how to read complex deep structures. So it can't skip that field as it does not know where that nested structure ends.
Sorry. :/
So you should preprocess your json with some external tools, of you can contribute to Clickhouse and improve JSON parser.
currently I am stuck on the following problem:
I am reading messages from a Kafka Topic using KafkaConsumer. The messages are strings and have the following format:
{ "a" : "b", "a1" : "b1", "c2" : "c3" }
They are saved within the payload of the FlowFile.
I want to convert that string into json or ideally to csv, but cant figure out how to do it.
I am new to NiFi and researched as much as possible, but the answers I found were regarding conversions from json to avro or similar, but never string to json or avro.
I also found out that the Kafka message is in the payload of the FlowFile, not in the attributes, so I have no clue how to get my hands on it, since the examples are always involving the attributes.
So in short: Can I convert the payload of a FlowFile, which is a string, to json/cvs with some built-in processor.
if your message is in FlowFile, the following sequence could help:
1) Use AttributesToJson to convert payload message to Json.
2) Use EvaluateJsonPath to extract the payload message. In your case the kafka message. Then you can pass the extracted messages for csv generation.
This post can help to convert Json TO CSV: Convert Json To CSV
I ended up doing this:
ConsumeKafka gives me the string:
{ "a" : "b", "a1" : "b1" }
EvaluateJsonPath creates attributes by adding properties
a -> $.a //results in attribute named a with value b
a1 -> $.a1 //results in attribute named a1 with value b1
ReplaceText gets the attributes from EvaluateJsonPath to form one single csv formated:
Replacement value -> ${'a'},${'a1'}
This results as a single line, BUT NO NEW LINE:
b,b1
To add the new line appending \n, '\n', "\n" did not work.
What worked was pressing Shift+Enter while typing into the Replacement value field, which resulted in creating an empty new line.
I want to convert JSON to Avro via NiFi. Unfortunately the JSON has complex types as values that I want to see as a simple string!
JSON:
"FLAGS" : {"FLAG" : ["STORED","ACTIVE"]}
How can I tell AVRO to simply store "{"FLAG" : ["STORED","ACTIVE"]}" or "[1,2,3,"X"]" as a string?
Thank you sincerely!
The JSON to Avro conversion performed in NiFi's ConvertJSONToAvro processor does not really do transformation in the same step. There is a very limited ability to transform based on the Avro schema, mostly omitting input data fields in the output. But it won't coerce a complex structure to a string.
Instead, you should do a JSON-to-JSON transformation first, then convert your summarized JSON to Avro. I think what you are looking for is a structure like this:
{
"FLAGS": "{\"FLAG\":[\"STORED\",\"ACTIVE\"]}"
}
NiFi's JoltTransformJSON, ExecuteScript processors are great for this. If your records are simple enough, maybe even a combination of EvaluateJsonPath $.FLAGS and ReplaceText { "FLAGS": "${flags:escapeJson()}" }.
I am using JSON.NET and I want to convert from XML to JSON.
I am using JsonConvert.SerializeXNode(node) and I noticed that my json object has properties with # in front of their names:
So for example:
If I have:
<channel id="999" name="XXX" sid="8294" type="Digital TV" />
the JSON object is:
{ "#id": "999", #name="XXX" etc
Why am I getting "#" inserted in the JSON and is there a way I can avoid the "#" character being inserted?
I think thats just the way json.net works regarding the # signs. You can always run a regex on the json string and replace them. Theres an example here