I am using the following code to read the CSV file and parse its data to JSON.
File inputFile = new File("in.csv");
File outputFile = new File("out.json");
CsvSchema csvSchema = CsvSchema.builder().setUseHeader(true).build();
CsvMapper csvMapper = new CsvMapper();
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationFeature.INDENT_OUTPUT, true);
mapper.writerWithDefaultPrettyPrinter().writeValue(outputFile,
csvMapper.readerFor(Map.class).with(csvSchema).readValues(inputFile).readAll());
This is working fine and giving me output as follow,
[
{
"Nutrient" : "Calories",
"Amount" : " 289.00",
"Unit" : " kcal"
}, {
"Nutrient" : "Fat",
"Amount" : " 17.35",
"Unit" : " g"
}
]
But the required output is
{
{
"Nutrient" : "Calories",
"Amount" : " 289.00",
"Unit" : " kcal"
}, {
"Nutrient" : "Fat",
"Amount" : " 17.35",
"Unit" : " g"
}
}
Actually I need to read the JSONs files that I have converted from CSVs. Using the following code
String content = Files.readString(filePath);
JSONObject jsonObject1 = new JSONObject(content);
HashMap yourHashMap1 = new Gson().fromJson(jsonObject1.toString(), HashMap.class);
But while I am trying to do it gives me this error.
org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.json.JSONException: A JSONObject text must begin with '{' at 1 [character 2 line 1]
Meanwhile json file should start with { instead of [ and similarly it should end on } instead ].
My goal is to remove this error.
I think the result you are seeing is correct and that is the way JSON is supposed to be. JSON is a key value based format.
The output which you have pasted just means it is an array of json objects
[
{
"Nutrient" : "Calories",
"Amount" : " 289.00",
"Unit" : " kcal"
}, {
"Nutrient" : "Fat",
"Amount" : " 17.35",
"Unit" : " g"
}
]
The below JSON actually makes no sense, as there are no keys to the object. Even if you try to parse this kind of JSON with Jackson, it will throw an error Exception in thread "main" com.fasterxml.jackson.core.JsonParseException: Unexpected character ('{' (code 123)): was expecting double-quote to start field name
at [Source: (String)"{ . You can try this out
{
{
"Nutrient" : "Calories",
"Amount" : " 289.00",
"Unit" : " kcal"
}, {
"Nutrient" : "Fat",
"Amount" : " 17.35",
"Unit" : " g"
}
}
The other option would be to consider each json object as a unique node with a distinct name like below, see the keyword set1 and set2
{
"set1": {
"Nutrient" : "Calories",
"Amount" : " 289.00",
"Unit" : " kcal"
},
"set2": {
"Nutrient" : "Fat",
"Amount" : " 17.35",
"Unit" : " g"
}
}
For some reason if you really want to have {} instead of [] then just make a string operation and replace 1st "[" with "{" and last "]" with "}
Edited the answer to match the edited question:
Now as we know your JSON is an array of JSON objects, you have to read it as JSONArray instead of JSONObject, also you can no more read it to a hashmap, it has to be a list, where each element in the list will be a JSONObject which has your data. Working code snippet is below
String content = Files.readString(filePath);
JSONArray jsonArray = new JSONArray(content);
List<LinkedTreeMap> yourList = new Gson().fromJson(jsonArray.toString(), ArrayList.class);
for(LinkedTreeMap l : yourList) {
System.out.println(l.get("Amount"));
System.out.println(l.get("Nutrient"));
System.out.println(l.get("Unit"));
}
Related
I am totally new to groovy script and would like some help to solve this out. I have a JSON response I want to manipulate and get desired parameters back by avoiding duplication. The Json response does not have indexes like 0,1,2.. that I can iterate through.
Here is the response that I want to work with:
{
"AuthenticateV2" : {
"displayName" : "Verification of authentication",
"description" : "notification ",
"smsTemplate" : "authentication.v2.0_sms",
"emailHeaderTemplate" : "v2.0_header",
"emailBodyTemplate" : "html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE"
},
"supportedPlans" : [
"connectGo"
]
},
"PasswordRecovery" : {
"displayName" : "Verification of password recovery",
"description" : "notification",
"smsTemplate" : "recovery.v1.0_sms",
"emailHeaderTemplate" : "recovery.v1.0_header",
"emailBodyTemplate" : "recovery.v1.0_body_html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE",
"adminInitiated" : false,
"authnId" : "AUTHENTICATION_IDENTIFIER",
"authnType" : "EMAIL",
"user" : {
"displayName" : "USER_DISPLAY_NAME"
}
},
"supportedPlans" : [
"connectGo"
]
},
"PasswordReset" : {
"displayName" : "password reset",
"description" : "notification",
"smsTemplate" : "recovery.v1.0_sms",
"emailHeaderTemplate" : "recovery.v1.0_header",
"emailBodyTemplate" : "html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"user" : {
"displayName" : "USER_DISPLAY_NAME"
}
}
The expected output that I want to have:
{
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE",
"customToken" : "VERIFICATION_CODE",
"adminInitiated" : false,
"authnId" : "AUTHENTICATION_IDENTIFIER",
"authnType" : "EMAIL"
}
I need to retrieve all fields under parameters tag and also want to avoid duplication
You should first get familiar with parsing and producing JSON in Groovy.
Then, assuming the provided response is a valid JSON (it's not - there are 2 closing curlies (}) missing at the end) to get all the parameters keys merged into one JSON we have to convert the JSON string into a Map object first using JsonSlurper:
def validJsonResponse = '<your valid JSON string>'
Map parsedResponse = new JsonSlurper().parseText(validJsonResponse) as Map
Now, when we have a parsedResponse map we can iterate over all the root items in the response and transform them into the desired form (which is all the unique parameters keys) using Map::collectEntries method:
Map uniqueParameters = parsedResponse.collectEntries { it.value['parameters'] }
Finally, we can convert the uniqueParameters result back into a pretty printed JSON string using JsonOuput:
println JsonOutput.prettyPrint(JsonOutput.toJson(uniqueParameters))
After applying all the above we'll get the output
{
"displayName": "USER_DISPLAY_NAME",
"actionTokenURL": "VERIFICATION_LINK",
"customToken": "VERIFICATION_CODE",
"adminInitiated": false,
"authnId": "AUTHENTICATION_IDENTIFIER",
"authnType": "EMAIL",
"user": {
"displayName": "USER_DISPLAY_NAME"
}
}
If you want to get rid of user entry from the final output just remove it from the resulting uniqueParameters map (uniqueParameters.remove('user')) before converting it back to JSON string.
I am trying to output certain key values to show(WHICH I AM GETTING ALREADY). But for some reason it is returning as a struct. Basically, I am converting from query statement into api call. But for some reason I cant get it return from cfoutput to show as a struct from the api call or a json file. Can anyone tell me what I have done wrong. thanks for the help. here is my code:
Output: I get all my json fields showing in a struct. (with cfscript uncommented out)
Update output with current code: (I get my output of certain key value, but can I do iterate through a loop, I know the way I have it is not the best way)
<cfset jsonDatas = fileRead("c:\Users\Desktop\myApi.json" )>
<cfset jsonData = deserializeJSON(jsonDatas) />
<cfdump var="#jsonData#" abort="true">
<cfloop array="#jsonData#" index="prop">
<cfoutput>
<br>Output:
#prop.employeeId#
.....
</cfoutput>
</cfloop>
My Json:
[
{
"employeeId" : "77777",
"lastName" : "DOE",
"firstName" : "JOHN",
"middleName" : null,
"sex" : "Male",
"jobStatus" : "Active",
"jobStatusDate" : "2020-01 03 00:00:00.0",
"departmentNbr" : "5555",
}
]
Let's see if the third try does the trick.
I added more data so that we can see that each array is processed separately.
<cfscript>
json =
'[
{
"employeeId" : "77777",
"lastName" : "DOE",
"firstName" : "JOHN",
"middleName" : null,
"sex" : "Male",
"jobStatus" : "Active",
"jobStatusDate" : "2020-01 03 00:00:00.0",
"departmentNbr" : "5555"
},
{
"employeeId" : "123",
"lastName" : "Doe",
"firstName" : "Jane",
"middleName" : "H",
"sex" : "Female",
"jobStatus" : "Active",
"jobStatusDate" : "2021-01 03 00:00:00.0",
"departmentNbr" : "14"
}
]';
jsonData = DeserializeJSON(json);
writedump(jsonData);
for (row in jsonData) {
writeoutput("<hr />");
for (item in row) {
writeoutput(item & ": " & row[item] & "<br />");
}
}
</cfscript>
See: https://cffiddle.org/app/file?filepath=ca5836f8-af85-4878-a2d7-9406bdd4a884/dc41f7b8-3792-4141-bd0b-7a9839900a43/97dfe9de-53f2-4e82-8e8c-53fd5b5bfd36.cfm
Or From a file
<cfscript>
jsonDatas = fileRead("c:\Users\Desktop\myApi.json" );
jsonData = deserializeJSON(jsonDatas);
writedump(jsonData);
for (row in jsonData) {
writeoutput("<hr />");
for (item in row) {
writeoutput(item & ": " & row[item] & "<br />");
}
}
</cfscript>
I have an index/type in ES which has the following type of records:
body "{\"Status\":\"0\",\"Time\":\"2017-10-3 16:39:58.591\"}"
type "xxxx"
source "11.2.21.0"
The body field is a JSON.So I want to search for example the records that have in their JSON body Status:0.
Query should look something like this(it doesn't work):
GET <host>:<port>/index/type/_search
{
"query": {
"match" : {
"body" : "Status:0"
}
}
}
Any ideas?
You have to change the analyser settings of your index.
For the JSON pattern you presented you will need to have a char_filter and a tokenizer which remove the JSON elements and then tokenize according to your needs.
Your analyser should contain a tokenizer and a char_filter like these ones here:
{
"tokenizer" : {
"type": "pattern",
"pattern": ","
},
"char_filter" : [ {
"type" : "mapping",
"mappings" : [ "{ => ", "} => ", "\" => " ]
} ],
"text" : [ "{\"Status\":\"0\",\"Time\":\"2017-10-3 16:39:58.591\"}" ]
}
Explanation: the char_filter will remove the characters: { } ". The tokenizer will tokenize by the comma.
These can be tested using the Analyze API. If you execute the above JSON against this API you will get these tokens:
{
"tokens" : [ {
"token" : "Status:0",
"start_offset" : 2,
"end_offset" : 13,
"type" : "word",
"position" : 0
}, {
"token" : "Time:2017-10-3 16:39:58.591",
"start_offset" : 15,
"end_offset" : 46,
"type" : "word",
"position" : 1
} ]
}
The first token ("Status:0") which is retrieved by the Analyze API is the one you were using in your search.
i have this task to parse json data to object, as in
{
"AAA" : {
"A1" : "a1 comment",
"A2" : "a2 comment"
},
"BBB" : {
"B1" : "b1 comment"
},
"CCC" : {
"C1" : "c1 comment",
"C2" : "c2 comment",
"C3" : "c3 comment"
"C4" : "c4 cooment"
},
"DDD" : {
"D1" : "d1 comment"
}
}
need to be converted to objects like ,
class Schema {
String tableName;
List<Column> colData;
And
class Column {
String columnName;
String columnComments;
Since i am a beginner to json, i took to the approach of first parsing things to xml and if it works proceed to parsing xml to object.
but i am having trouble parsing the json data.
however if i replace all the single quotes with double quotes and and '=>' with ':' and only consider AAA, i am able to parse with below code, but if i use the whole json data it throws error.
any help is deeply appreciated.
the code that i have so far is :
FileInputStream fis = new FileInputStream(new File("schema.sql"));
Transformer transformer = TransformerFactory.newInstance().newTransformer();
InputSource source = new InputSource(new StringReader(IOUtils.toString(fis, "UTF-8")));
DOMResult result = new DOMResult();
transformer.transform(new SAXSource(new JsonXmlReader("Table"), source), result);
Node n=result.getNode();
System.out.println(result.toString());
StringWriter writer = new StringWriter();
// Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.transform(new DOMSource(n), new StreamResult(writer));
String xml = writer.toString();
System.out.println(xml);
which works for below data:
{
"AAA" : {
"A1" : "a1 comment",
"A2" : "a2 comment"
}
}
ofcourse i had to replace all ' with " and => with :
for the json data
{
"AAA" : {
"A1" : "a1 comment",
"A2" : "a2 comment"
},
"BBB" : {
"B1" : "b1 comment"
}
}
the error that i see is :
ERROR: 'Parsing error: HIERARCHY_REQUEST_ERR: An attempt was made to insert a node where it is not permitted. '
javax.xml.transform.TransformerException: net.javacrumbs.json2xml.JsonSaxAdapter$ParserException: Parsing error: HIERARCHY_REQUEST_ERR: An attempt was made to insert a node where it is not permitted.
at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown Source)
net.javacrumbs.json2xml.JsonSaxAdapter$ParserException: Parsing error: HIERARCHY_REQUEST_ERR: An attempt was made to insert a node where it is not permitted.
at net.javacrumbs.json2xml.JsonSaxAdapter.parse(JsonSaxAdapter.java:177)
at net.javacrumbs.json2xml.JsonXmlReader.parse(JsonXmlReader.java:152)
I am trying to parse below json. This has to throw me the error saying incorrect json format. But the parsor parses the json only till "value:15" and is not throwing any exception. How can I achieve this?
String json = { and : [{key: domain, value: cricket}, {key : STAT_CDE,value : 15}]}, { and : [{key: domain, value: football}, {key : STAT_CDE,value : 10}]}
Sample Code I am using:
import org.codehaus.jackson.map.ObjectMapper;
ObjectMapper mapper = new ObjectMapper();
mapper.readTree(json); //this line is not throwing me any exception
Here is the code snippet:
import org.codehaus.jackson.map.ObjectMapper;
public class JsonTestParse {
public static void main(String[] args) {
String json = "{ \"and\" : [{\"key\": \"domain\", \"op\": \"=\", \"value\": \"cricket\"}, {\"key\" : \"STAT_CDE\",\"op\" : \"=\",\"value\" : \"13\"}]},"+
"{ \"and\" : [{\"key\": \"domain\", \"op\": \"=\", \"value\": \"Football\"}, {\"key\" : \"STAT_CDE\",\"op\" : \"=\",\"value\" : \"10\"}]}";
System.out.println("json: " + json);
ObjectMapper mapper = new ObjectMapper();
try {
mapper.readTree(json);
} catch (Exception e) {
System.out.println("object mapper exp");
}
System.out.println("mapper complete");
}
}
And the output:
line 1: json: { "and" : [{"key": "domain", "op": "=", "value": "cricket"}, {"key" : "STAT_CDE","op" : "=","value" : "13"}]},{ "and" : [{"key": "domain", "op": "=", "value": "Football"}, {"key" : "STAT_CDE","op" : "=","value" : "10"}]}
line 2: mapper complete
The issue is that there is no problem with that json format! Syntactically, you're assigning a comma-delimited list to json, which is perfectly valid code - it will set json to the first item in the list and not do anything with the rest of the values.
After your code executes, json looks like this:
String json = { and : [{key: domain, value: cricket}, {key : STAT_CDE,value : 15}]}
And this value has been ignored entirely:
{ and : [{key: domain, value: football}, {key : STAT_CDE,value : 10}]}
As you can see, json is structured perfectly well.
Also, it looks like you're expecting a String object, but you're supplying a map instead.
Try the following:
String json = "{ and : [{key: domain, value: cricket}, {key : STAT_CDE,value : 15}]}, { and : [{key: domain, value: football}, {key : STAT_CDE,value : 10}]}"
And then parse json. This will most certainly fail because the keys are not enclosed with double quotes (the " character), which is a requirement of json format.
EDIT
To keep up to date with the question:
Here is a formatted view of the json you say you're working with:
{
"and" : [
{"key": "domain", "op": "=", "value": "cricket"},
{"key" : "STAT_CDE","op" : "=","value" : "15"}
]
},
{
"and" : [
{"key": "domain", "op": "=", "value": "football"},
{"key" : "STAT_CDE","op" : "=","value" : "10"}
]
}
The issue is that this is not a json object - it is TWO SEPARATE json objects, each of which is well-formed. I am guessing that ObjectMapper parses a full json structure, and ignores any trailing data without throwing an error.
If you'd like to capture the entire structure in json you will need to enclose them together, likely using an array:
[
{
"and" : [
{"key": "domain", "op": "=", "value": "cricket"},
{"key" : "STAT_CDE","op" : "=","value" : "15"}
]
},
{
"and" : [
{"key": "domain", "op": "=", "value": "football"},
{"key" : "STAT_CDE","op" : "=","value" : "10"}
]
}
]