I am working with a much larger JSON file than I am used too and I am getting confused on how to parse the file to extract an object.
resultJSON['BACSDocument']['Data'][0]['ARUDD'][0]['Advice'][0]['OriginatingAccountRecords'][0]['OriginatingAccountRecord'][0]['ReturnedDebitItem']
This is what I used to extract the "ReturnedDebitItem" array. I cannot use this as the json will come in different schemas but will always contain a "ReturnedDebitItem" so I need to parse the document.
I have tried using a couple of for loops but I am just receiving errors - any insight would be much appreciated. I am working with node but any pseudo code would also be helpful!
Related
I'm new to the World of triplets :-) I'm trying to use DotNetRDF to load the SOLR searchresult into a Graph using DotNetRDF.
The URL I'm getting data from is:
https://nvv.entryscape.net/store/search?type=solr&query=rdfType:https%5C%3A%2F%2Fnvv.entryscape.net%2Fns%2FDocument+AND+context:https%5C%3A%2F%2Fnvv.entryscape.net%2Fstore%2F1
The format is supposed to be "RDF/JSON". No matter what parser or what I try - I only get "invalid URI". Have tried to load from the URL and also tried downloadning the result to a file and load from file, same error.
I'm using VS2017 and have "nugetted" the latest version of DotNetRdf.
Please help me, what am I missing?
Regards,
Lars Siden
It looks like the JSON being returned by that endpoint is not valid RDF/JSON. It does appear to contain some RDF/JSON fragments but they are wrapped up inside another JSON structure. The RDFJSONParser in dotNetRDF requires that your entire JSON document be a single, valid chunk of RDF/JSON.
The value at resource.children[*].metadata is an RDF/JSON object. So is the value at resource.children[*].info. The rest is wrapper using property names that are not valid IRIs (hence the parser error message).
Unfortunately there is no easy way to skip over the rest of the JSON document and only parse the valid bits. To do that you will need to load the JSON document using Newtonsoft.JSON and then serialize each valid RDF/JSON object you are interested in as a string and load that using the RDFJSONParser's Load(IGraph, TextReader) or Parse(IRdfHandler, TextReader) method.
I have an HDFS directory full of the following JSON file format:
https://www.hl7.org/fhir/bundle-transaction.json.html
What I am hoping to do is find an approach to flatten each individual file to become one df record or rdd tuple. I have tried everything I could think of using read.json(), wholeTextFiles(), etc.
If anyone has any best practices advice or pointers, it would be sincerely appreciated.
Load via wholeTextFiles something like this:
sc.wholeTextFiles(...) //RDD[(FileName, JSON)
.map(...processJSON...) //RDD[JsonObject]
Then, you can simply call the .toDF method so that it will infer from your JsonObject.
As far as the processJSON method, you could just use something like the Play json parser
mapPartitions is used when having to deal with data that is structured in a way that different elements can be on different lines. I've worked with both JSON and XML using mapPartitions.
mapPartitions works on an entire block of data at a time, as opposed to a single element. While you should be able to use the DataFrameReader API with JSON, mapPartitions can definitely do as you'd like. I don't have the exact code to flatten a JSON file, but I'm sure you can figure it out. Just remember the output must be an iterable type.
I'm really at my wit's end here... I'm using VB-JSON Parser (http://www.ediy.co.nz/vbjson-json-parser-library-in-vb6-xidc55680.html) and I have the following array :
[{"timestamp":1410001952,"tid":2834225,"price":"483.77"}]
The documentation is really minimal and I have no clue whatsoever of how to access the array, been searching for several hours now on how to resolve this.
How can I get the "price" value? I know that i can use .item("price") when there is no array but I don't know what to do when there's an array and there is no name before it.
First have a look at Parsing JSON in Excel VBA
It explains the JScript way of parsing JSON string.
Browsing through the net, I found it really hard to get a complete VBA based JSON parser.
Some options are available in the VB version and then there are few online parsers who promise to parse JSON and convert them in Excel. These ones work fine with simple JSON data structure. But once you feed in a complex data set with nested arrays and structures, they simply fail.
Using JavaScript features of parsing JSON, on top of ScriptControl, we can create a parser in VBA which will list each and every data point inside the JSON. No matter how nested or complex the data structure is, as long as we provide a valid JSON, this parser will return a complete tree structure.
JavaScript’s Eval, getKeys and getProperty methods provide building blocks for validating and reading JSON.
Coupled with a recursive function in VBA we can iterate through all the keys (up to nth level) in a JSON string. Then using a Tree control (used in this article) or a dictionary or even on a simple worksheet, we can arrange the JSON data as required.
Here, you can find a complete VBA example.
There is a JSON serializer in .NET: http://msdn.microsoft.com/en-us/library/system.runtime.serialization.json
I am loading data from a mongodb collection to a mysql table through Kettle transformation.
First I extract them using MongodbInput and then I use json input step.
But since json input step has very low performance, I wanted to replace it with a
javacript script.
I am a beginner in Javascript and even though i tried somethings, the kettle javascript script is not recognizing any keywords.
can anyone give me sample code to convert Json data to different columns using javascript?
To solve your problem you need to see three aspects:
Reading from MongoDB
Reading from JSON
Reading from (probably) String
Reading from MongoDB Except if you changed the interface, MongoDB returns not JSON but BSON files (~binary JSON). You need to see the MongoDB documentation about reading and writing BSON: probably something like BSON.to() and BSON.from() but I don't know it by heart.
Reading from JSON Once you have your BSON in JSON format, you can read it using JSON.stringify() which returns a String.
Reading from (probably) String If you want to use the capabilities of JSON (why else would you use JSON?), you also want to use JSON.parse() which returns a JSON object.
My experience is that to send a JSON object from one step to the other, using a String is not a bad idea, i.e. at the end of a JavaScript step, you write your JSON object to a String and at the beginning of the next JavaScript step (can be further down the stream) you parse it back to JSON to work with it.
I hope this answers your question.
PS: writing JavaScript steps requires you to learn JavaScript. You don't have to be a master, but the basics are required. There is no way around it.
you could use the json input step to get the values of this json and put in common rows
It might be a frequent question but i cannot figure out how to prevent errors in my parsing when the script can't find a property...
in XML was easy because even the empty properties were like <location/>
but now if location is not available JSON paser cant find it and it results in errors...OR
it may happen the json has different property or a children lost its father..... so for instance if you need to extract the LocalityName is no more under SubAdministrativeArea but under AddressLine...
any of you have any experience about? what the best way to solve it and to parse it correctly?
While answering one of your other question I had written the following javascript code to obtain the lat and lng from the JSON returned by the maps api without any validations for zero results.
$.getJSON("getjson.php?address="+address,
function(data){
lat=data.results[0].geometry.location.lat;
lng=data.results[0].geometry.location.lng;
//.....map initialization code
}
);
Now if I were to validate for zero results, I'd modify the code in the following way
$.getJSON("getjson.php?address="+address,
function(data){
if (data.result.length>0) {
for (count=0;count<data.result.length;count++){
lat=data.results[count].geometry.location.lat;
lng=data.results[count].geometry.location.lng;
//.....map initialization code
}
}
}
);
As you can see parsing JSON comes naturally to javascript and to many other languages for it resolves down to arrays/lists and objects/dictionary/hash.
If I get this right and you are using a library to convert to json like gson try to construct some kind of object array like arrayList in java and then convert to json so every object you retrieve later in javascript is distinct and thus clear during debugging.Also if you don't use firebug give it try as it shows json data clearly.cheers