I am attempting to get some JSON parsed with erlang-rfc4627 and struggling with the returned results
This is the JSON:
{
"people": [
{"name": "Toby"}
]
}
Using the erlang-rfc4627 library:
{ok, Json, []} = rfc4627:decode("...")
I can decode fine into Erlang as:
{obj,[
{"people",[
{obj,[
{"name",<<"Toby">>}
]},
{obj,[
{"name",<<"Blah">>}
]}
]}
]}
But then what happens?
How do I get an array of people out of this structure in an easy way?
(This is a very simplified model of the overall JSON).
Is there a better library I should be using for this?
Updated
I noticed that when pulling out arrays, each element of the arrays has the awful obj structure wrapped into it, which makes the process of manipulating arrays very clumsy.
Why on earth is this so complex in Erlang?
Reference: http://www.lshift.net/blog/2007/02/17/json-and-json-rpc-for-erlang
How about:
lookup(K, {obj, PL}) -> proplists:get_value(K, PL).
And then
People = lookup("people", JSON),
Names = [lookup("name", Obj) || Obj <- People].
The better way generalizes this idea into a query-compiler which can compile any query to a function which can then be applied to a JSON document. It will be way easier should you want to to rumaging inside JSON documents all the time.
It is also important to note that you should probably not be operating directly on the JSON structure, but embed it in something else inside the Erlang world.
Is there a better library I should be using for this?
Please, try use library like jiffy, because this library is easy to use:
1> Doc = {[{foo, [<<"bing">>, 2.3, true]}]}.
{[{foo,[<<"bing">>,2.3,true]}]}
2> BinJSON = jiffy:encode(Doc).
<<"{\"foo\":[\"bing\",2.3,true]}">>
3> JsonMap = jiffy:decode(BinJSON, [return_maps]).
#{<<"foo">> => [<<"bing">>,2.3,true]}
4> maps:get(<<"foo">>, JsonMap, undefined).
[<<"bing">>,2.3,true]
Related
For a json array like this:
[
my-json-obj1,
my-json-obj2,
my-json-obj3,
....
my-json-objN
]
And MyJsonObj class that represents a mapping of single object in array I can say:
val myJson = '''[...]'''
ZStream
.fromIterable(myJson.toSeq)
.via(JsonDecoder[MyJsonObj].decodeJsonPipeline(JsonStreamDelimiter.Array))
to parse that array in a "streaming" way, i.e. emit mapped objects as they are parsed form the input as opposed to reading all the input first and then extracting the objects.
How can I do the same if the array is nested inside a json object say like this?:
{
"hugeArray":
[
my-json-obj1,
my-json-obj2,
my-json-obj3,
....
my-json-objN
]
}
I trawled through zio-json source code, but I can't find any foothole there for this use case. I guess I could carve out that array from the json document and feed that to decodeJsonPipeline. Is there any better, json-syntax aware way of doing this? If not directly in zio-json perhaps with help of some other open source json libraries?
I need to create some JSON files for exporting data from a Django system to Google Big Query.
The problem is that Google BQ imposes some characteristics in the JSON file, for example, that each object must be in a different line.
json.dumps writes a stringified version of the JSON, so it is not useful for me.
Django serializes writes better JSON, but it put all in one line. All the information I found about pretty-printing is about json.dumps, which I cannot use.
I will like to know if anyone knows a way to create a JSON file in the format required by Big Query.
Example:
JSONSerializer = serializers.get_serializer("json")
json_serializer = JSONSerializer()
data_objects = DataObject.objects.all()
with open("dataobjects.json", "w") as out:
json_serializer.serialize(data_objects, stream=out)
json.dumps is OK. You have to use indent like this.
import json
myjson = '{"latitude":48.858093,"longitude":2.294694}'
mydata = json.loads(myjson)
print(json.dumps(mydata, indent=4, sort_keys=True))
Output:
{
"latitude": 48.858093,
"longitude": 2.294694
}
I've been asked to parse a JSON file to get all the buses that are over a specified speed inputed by the user.
The JSON file can be downloaded here
It's like this:
{
"COLUMNS": [
"DATAHORA",
"ORDEM",
"LINHA",
"LATITUDE",
"LONGITUDE",
"VELOCIDADE"
],
"DATA": [
[
"04-16-2015 00:00:55",
"B63099",
"",
-22.7931,
-43.2943,
0
],
[
"04-16-2015 00:01:02",
"C44503",
781,
-22.853649,
-43.37616,
25
],
[
"04-16-2015 00:11:40",
"B63067",
"",
-22.7925,
-43.2945,
0
],
]
}
The thing is: I'm really new to scala and I have never worked with json before (shame on me). What I need is to get the "Ordem", "Linha" and "Velocidade" from DATA node.
I created a case class to enclousure all the data so as to later look for those who are over the specified speed.
case class Bus(ordem: String, linha: Int, velocidade: Int)
I did this reading the file as a textFile and spliting. Although this way, I need to foreknow the content of the file in order to go to the lines after DATA node.
I want to know how to do this using a JSON parser. I've tried many solutions, but I couldn't adapt to my problem, because I need to extract all the lines from DATA node instead of nodes inside one node.
Can anyone help me?
PS: Sorry for my english, not a native speaker.
First of all, you need to understand the different JSON data types. The basic types in JSON are numbers, strings, booleans, arrays, and objects. The data returned in your example is an object with two keys: COLUMNS and DATA. The COLUMNS key has a value that is an array of strings and numbers. The DATA key has a value which is an array of arrays of strings.
You can use a library like PlayJSON to work with this type of data:
val js = Json.parse(x).as[JsObject]
val keys = (js \ "COLUMNS").as[List[String]]
val values = (js \ "DATA").as[List[List[JsValue]]]
val busses = values.map(valueList => {
val keyValues = (keys zip valueList).toMap
for {
ordem <- keyValues("ORDEM").asOpt[String]
linha <- keyValues("LINHA").asOpt[Int]
velocidade <- keyValues("VELOCIDADE").asOpt[Int]
} yield Bus(ordem, linha, velocidade)
})
Note the use of asOpt when converting the properties to the expected types. This operator converts the key-values to the provided type if possible (wrapped in Some), and returns None otherwise. So, if you want to provide a default value instead of ignoring other results, you could use keyValues("LINHA").asOpt[Int].getOrElse(0), for example.
You can read more about the Play JSON methods used here, like \ and as, and asOpt in their docs.
You can use Spark SQL to achieve it. Refer section under JSON Datasets here
In essence, Use spark APIs to load a JSON and register it as temp table.
You can run your SQL queries on the table from there.
As seen on #Ben Reich answer, that code works great. Thank you very much.
Although, my Json had some type problems on "Linha". As it can be seen on the JSON example that I put on the Question, there are "" and also numbers, e.g., 781.
When trying to do keyValues("LINHA").asOpt[Int].getOrElse(0), it was producing an error saying that value flatMap is not a member of Int.
So, I had to change some things:
case class BusContainer(ordem: String, linha: String, velocidade: Int)
val jsonString = fromFile("./project/rj_onibus_gps.json").getLines.mkString
val js = Json.parse(jsonString).as[JsObject]
val keys = (js \ "COLUMNS").as[List[String]]
val values = (js \ "DATA").as[List[List[JsValue]]]
val buses = values.map(valueList => {
val keyValues = (keys zip valueList).toMap
println(keyValues("ORDEM"),keyValues("LINHA"),keyValues("VELOCIDADE"))
for {
ordem <- keyValues("ORDEM").asOpt[String]
linha <- keyValues("LINHA").asOpt[Int].orElse(keyValues("LINHA").asOpt[String])
velocidade <- keyValues("VELOCIDADE").asOpt[Int]
} yield BusContainer(ordem, linha.toString, velocidade)
})
Thanks for the help!
I've been converting CF structs etc to JSON for a while now, and all good. Coldbox in particular makes this really easy.
However, I am currently working with a jQuery Datatable and need to pass it jSON in the format below.
I am starting with an array of objects.
I only want certain properties in each object to go into the final JSON String.
I'm running around in circles and possibly totally overcomplicating converting my data into this format JSON. Can anyone help, or suggest an easy way I might be able to do this..
Also worth mentioning I am building this in coldbox. Coldfusion 9.
{ "aaData": [ [ "Test1", "test#test1", "444444444", "<i class=''icon-pencil icon-large'' data-id=''s1''></i>" ],[ "Test2", "test#test2", "555555555", "<i class=''icon-pencil icon-large'' data-id=''s2''></i>" ],[ "Test3", "test#test3", "666666666", "<i class=''icon-pencil icon-large'' data-id=''s3''></i>" ] ]}
Many Thanks!
======================================================
Here is the code that game we what I needed:
var dataStruct = structNew();
var dataArray = arrayNew(1);
var subsArray = arrayNew(1);
var subs = prc.org.getSubscribers();
for (i=1; i<=arrayLen(subs); i++){
arrayAppend(subsArray,"#subs[i].getName()#");
arrayAppend(subsArray,"#subs[i].getEmail()#");
arrayAppend(subsArray,"#subs[i].getMobile()#");
arrayAppend(subsArray,"<i class='icon-pencil icon-large' data-id='s#subs[i].getID()#'></i>");
arrayAppend(dataArray,subsArray);
arrayClear(subsArray);
};
structInsert(dataStruct,'aaData',dataArray);
event.renderData('json',dataStruct);
OK, so you've got an array which has objects, and the objects contain all the properties which you need to end up in this JSONed array, yeah?
So do this:
create a new array
loop over the array of objects
create a struct
put all the values from each object you need to go into the JSON; be mindful to use associative array notation when setting the keys, to perserve the case of the keys
append the struct to the new array
/loop
serializeJson the new array
I don't think there's any simpler way of doing it.
I am getting json as response from server like below;
{"data":"<div align=\"left\"><select id =\"test\"><option id=\"1\" value=\"one\"><option id=\"2\" value=\"two\" selected></select></div>"};
I want to manipulate above json file using javascript to change option one to be selected instead of option two.
Any hints please.
Regards,
Raj
Your life would be easier if your JSON was actually data represented as JSON, instead of serialized DOM fragments embedded in a JSON value, e.g.,
[
{"value": "one"}
, {"value": "two", "selected": true}
]
Then when you turn that into an object, you could just do something like this (assume for the sake of the example that you named the result myArray):
myArray[0].selected = true; // Select the first element
myArray[1].selected = false; // Deselect the other element; in many cases, you'd probably need some sort of loop.