Say I have some moderately complex JSON like
{
"revenue": 100,
"products":[
{"name": "Apple", "price": 50},
{"name": "Banana", "price": 50}
]
}
Obviously this this a bit contrived, but what's the best way to map this to pig using JsonLoader.
I've tried
a = LOAD 'test.json' USING
JsonLoader('revenue:int,products:[(name:chararray,price:int)]');
or
a = LOAD 'test.json' USING
JsonLoader('revenue:int,products:[{(name:chararray,price:int)]}');
However, when I DUMP A, I get (100,) for both.
I've also tried
a = LOAD '/json/complex.json'
USING JsonLoader('revenue:int,products:[{name:chararray,price:int}]');
which errors out with ERROR 1200: <line 1, column 28> mismatched input 'chararray' expecting LEFT_PAREN.
What's the best way to parse this for future use?
Thanks
For posterity,
a = LOAD 'test.json' USING
JsonLoader('revenue:int,products:{(name:chararray,price:int)}');
Related
I'm trying to load a json file from a server response and parsing it in flutter, the model i create is working for all the other fields but i'm in trouble with this class
this is a part of the JSON response:
"episodes": {
"1": [
{
"id": "63",
"episode_num": 1,
"title": "Some Name",
"container_extension": "mp4",
"info": {
"director": "",
"plot": "",
"cast": "",
"rating": "",
"releasedate": "",
"movie_image": "",
"genre": "",
"duration_secs": 6495,
"duration": "01:48:15"
}
}
]
}
in this case the entry under episodes is just one but this will represents a season and all the episode inside it, so under episodes many of this entry (undefined number during coding) can be present
At this time, using online json to dart converter tools i can be able to retrive just this one entry but if a response have more than 1 season i can't see it.
There is any way to handle this?
EDIT:
Solved using a for cicle with max value = (json['episodes'].length + 1).
For the info stored inside each 'episodes' value i can use
json['episodes']['$i']
Valid JSON is always convertible to a Dart data structure. But what you may be asking is "can I get nested objects from this?", and that just depends on how hard you want to work. Some JSON-to-Dart tools are better than others and some JSON values are impossible for any automated tool to make sense of. Only real answer is: "it depends".
I've noticed that some APIs use a format of sending a stripped down version of their data via a JSON array like the following:
[
"joe",
[
5,
2,
"yellow"
]
]
And store a set of keys like the following:
[
"name",
["some_data", [
"favorite_number",
"least_favorite_number",
"car_color"
]]
]
To turn the data from a bunch of random values to a readable set of data, like the following:
{
"name": "joe",
"some_data": {
"favorite_number": 5,
"least_favorite_number": 2,
"car_color": "yellow"
}
}
I was wondering how this could be done? I'd prefer it'd be in python, but I'm fine with programming my own libraries.
After grasping at more straws than I could fit in my mouth, I've figured it out. JSON schema is what I'm supposed to be using!
currently i am doing is represent relation ships of data using d3.js. need to represent it in a tree. My data stored at Neo4j server. And application is design under Express frame work.
var cypher = [
"match (b:Binary)-[r*..1]->(a:Binary)",
"where a.Key = '" + data + "'",
"return collect( distinct b) as dep"].join("\n");
execute this query and put the result in to a queue and sequentially execute it. this is for getting all children of the node. But I need to make this not as flat json something like depth. like `
{
"name": "flare",
"children": [
{
"name": "analytics",
"children": [
{
"name": "cluster",
"children": [
{"name": "AgglomerativeCluster", "size": 3938},
{"name": "CommunityStructure", "size": 3812},
{"name": "HierarchicalCluster", "size": 6714},
{"name": "MergeEdge", "size": 743}
]
},
...... how can i do it?
You should use d3's nest function to do this. This is actually mostly a JSON question, it seems your problem is that you need to start with the JSON output that the RESTful services Neo4J provides, and then transform that into a JSON structure suitable for tree representation in D3. The nest function will really help with that.
A second option you have is to use a tool like json2json, which is a more generic tool intended to help transform from one json structure into another. Under that approach, you write a set of template rules and then translate a data structure.
don't make down votes. i think you can have this answer chiran.
Generate (multilevel) flare.json data format from flat json
I have a large JSON file that looks similar to the code below. Is there anyway I can iterate through each object, look for the field "element_type" (it is not present in all objects in the file if that matters) and extract or write each object with the same element type to a file? For example each user would end up in a file called user.json and each book in a file called book.json?
I thought about using javascript but to my knowledge js can't write to files, I also tried to do it using linux command line tools by removing all new lines, then inserting a new line after each "}," and then iterating through each line to find the element type and write it to a file. This worked for most of the data; however, where there were objects like the "problem_type" below, it inserted a new line in the middle of the data due to the nested json in the "times" element. I've run out of ideas at this point.
{
"data": [
{
"element_type": "user",
"first": "John",
"last": "Doe"
},
{
"element_type": "user",
"first": "Lucy",
"last": "Ball"
},
{
"element_type": "book",
"name": "someBook",
"barcode": "111111"
},
{
"element_type": "book",
"name": "bookTwo",
"barcode": "111111"
},
{
"element_type": "problem_type",
"name": "problem object",
"times": "[{\"start\": \"1230\", \"end\": \"1345\", \"day\": \"T\"}, {\"start\": \"1230\", \"end\": \"1345\", \"day\": \"R\"}]"
}
]
}
I would recommend Java for this purpose. It sounds like you're running on Linux so it should be a good fit.
You'll have no problems writing to files. And you can use a library like this - http://json-lib.sourceforge.net/ - to gain access to things like JSONArray and JSONObject. Which you can easily use to iterate through the data in your JSON request, and check what's in "element_type" and write to a file accordingly.
var data = xhr.responseText;
When I output this console.log(xhr.responseText). Below is my output
["{id:1,name\":\"JOHN\",\"city\":\"null\"}"
,"{\"id\":2,\"name\":\"MICHEAL\,\"city\":\"null\"}"]
How do I get id, name. I tried like this data.id but I get this error
jquery JSON.parse: unexpected end of data.
Update
I am using code igniter with data mapper so my data mapper is giving that json response. Do you know, how I can resolve it.
You've already been told what the problem is in the comments: the JSON generated by the server is invalid. You are probably not using a library to encode your JSON, don't ever encode it by hand.
Your JSON should probably look like the following (when pretty printed) http://jsfiddle.net/7FKWr/
[
{"id": 1, "name": "JOHN", "city": null},
{"id": 2, "name": "MICHEAL", "city": null}
]