my json structure is something like this:
{
"posts": [],
"persons": [],
"organizations": [],
"meta": {
"sources": [
"http://loksabha.nic.in/",
"http://wikidata.org/",
"http://gender-balance.org/"
]
},
"memberships": [],
"events": [],
"areas": []
}
i want to read posts into a dataframe, wehre posts is an array of json objects. similarly other json arrays, except "meta". "sources" array inside "meta" json object should be read into another dataframe.
Is there anyway to achieve this with spark scala.
Any help is greatly appreciated.
Thanks in advance
Shakti
You could use the expand function.
I guess you have something like
val jsonDf = spark.read.json("your_json.json")
val postsDf = jsonDF.withColumn("post", explode(col("posts")).select("post")
Related
I want to get posts from Reddit API. Posts are in "children" node but each object has another object inside.
Can somebody help me write a function that convert this JSON to a list of dart objects?
Here is JSON string.
{
"kind": "Listing",
"data": {
"after": "t3_zzzhq4",
"dist": 2,
"children": [
{
"kind": "t3",
"data": {
"selftext": "blablabla",
"author_fullname": "3xblabla",
"title": "moreblabla",
"created": 1672515982,
"id": "10020p0"
}
},
{
"kind": "t3",
"data": {
"selftext": "blablabla",
"author_fullname": "3xblabla",
"title": "moreblabla",
"created": 1672515982,
"id": "10020p0"
}
}
],
"before": null
}
}
I tried all the tutorials on the topic of complex json parsing, but none of them met my needs. I would know how to parse simple json, but here it is deeply nested JSON, which bothers me a lot, and i cant quite grasp it. Appreciete any help.
Solution:
First go to json to dart and paste JSON string, this generator will make you all classes needed for your JSON.
Then you will need to decode string:
final jsonResponse = json.decode(jsonString);
And then deserialize your JSON like this:
List postslist = list.map((i) => Post.fromJson(i['data'])).toList();
For me, it was crucial i['data']. After adding that, i could deserialize all objects that were living inside that node. Thanks everyone! Hope that someone else this will be helpful! Cheers.
So how to read the local JSON data if the structure is like this-
{
"employees":
[ "id":1,
"name":Mike,
"location":[
"first-address":India,
"Sec-address":Bangalore
]
]
}
how to access location using the angular Framework?
You have not provided much info in the question but Im guessing you've got a string and you want to convert it to JSON which you do with JSON.parse
const obj = JSON.parse('{ "employees": [ "id":1, "name":Mike, "location":[ "first-address":India, "Sec-address":Bangalore ] ] }')
Now you can access as follows:
console.log(obj.employees[0].id)
Or if not in string format:
const obj = { "employees": [ "id":1, "name":Mike, "location":[ "first-address":India, "Sec-address":Bangalore ] ] }
console.log(obj.employees[0].id)
The question needs more clarification, according to my understanding
if you want to convert string format into JSON with JSON.parse(object) and store it in variable named obj
and you can access location attribute by
console.log(obj.employees[0].location)
import json
person = '{"name": "Bob", "languages": ["Italian", "English", "Fench"], "location": "Naples"}'
person_dict = json.loads(person)
print(list(person_dict.values())[1:1])
Hi guys i've a little issue handling json value in py3.
The question is simple.. considering the code above, how i can extract only 'Italian' value from 'languages' key?
The code is surely wrong because it give nothing:
[]
Any help will be appreciated.
Edit:
The imported pkgs:
from urllib3 import PoolManager, request
import certifi
import json2
the right api output in json format:
{
"error": [],
"result": {
"AUCTION1": {
"asks": [
[
"281.00000",
"0.163",
1609860353
]
],
"bids": [
[
"277.60000",
"0.100",
1609860353
]
]
}
}
}
And this is the function where i've the issue:
p_urll = PoolManager(ca_certs=certifi.where())
p_req = "https://api.auctionsite.com/0/public/Depth?pair=AUCTION1&count=1"
p_api_req = p_urll.request('GET', p_req)
p_api_res = json2.loads(p_api_req.data.decode('utf-8'))
print(p_api_res['result']['AUCTION1']['asks'][0])
It's not simple like the 1st example.. my fault..
Using [0] the code will give this result:
['26098.60000', '0.781', 1609861809]
i need only the auction price, so the 1st "asks" value (this "281.00000" without quotes)
I have this JSON that is returned from a REST-service I'm using.
{
"id": "6804",
"signatories": [
{
"id": "12125",
"fields": [
{
"type": "standard",
"name": "fstname",
"value": "John"
},
{
"type": "standard",
"name": "sndname",
"value": "Doe"
},
{
"type": "standard",
"name": "email",
"value": "john.doe#somwhere.com"
},
{
"type": "standard",
"name": "sigco",
"value": "Company"
}
]
}
]
}
Currently I'm looking into a way to parse this with json4s, iterating over the "fields" array, to be able to change the property "value" of the different objects in there. So far I've tried a few json libs and ended up with json4s.
Json4s allows me to parse the json into a JObject, which I can try extract the "fields" array
from.
import org.json4s._
import org.json4s.native.JsonMethods._
// parse to JObject
val data = parse(json)
// extract the fields into a map
val fields = data \ "signatories" \ "fields"
// parse back to JSON
println(compact(render(fields)))
I've managed to extract a Map like this, and rendered it back to JSON again. What I can't figure out though is, how to loop through these fields and change the property "value" in them?
I've read the json4s documentation but I'm very new to both Scala and it's syntax so I'm having a difficult time.
The question becomes, how do I iterate over a parsed JSON result, to change the property "value"?
Here's the flow I want to achieve.
Parse JSON into iterable object
Loop through and look for certain "names" and change their value, for example fstname, from John to some other name.
Parse it back to JSON, so I can send the new JSON with the updated values back.
I don't know if this is the best way to do this at all, I'd really appreciate input, maybe there's an easier way to do this.
Thanks in advance,
Best regards,
Stefan Konno
You can convert the json into an array of case class which is the easiest thing to do. For example: you can have case class for Fields like
case class Field(`type`: String, name: String, value: String)
and you can convert your json into array of fields like read[Array[Field]](json) where json is
[
{
"type": "standard",
"name": "fstname",
"value": "John"
},
...
]
which will give you an array of fields. Similarly, you can model for your entire Json.
As now you have an array of case classes, its pretty simple to iterate the objects and change the value using case classes copy method.
After that, to convert the array of objects into Json, you can simply use write(objects) (read and write functions of Json4s are available in org.json4s.native.Serialization package.
Update
To do it without converting it into case class, you can use transformField function
parse(json).transformField{case JField(x, v) if x == "value" && v == JString("Company")=> JField("value1",JString("Company1"))}
How to read the following json file and display in extjs grid columns?
{
"users": [
{
"id": 123,
"name": "Ed",
"orders": [
{
"id": 50,
"total": 100,
}
]
}
]
}
Could someone explain how to read the nested data in the json? I have tried a lot of options like renderer functions, using the '.' property, etc.
Edit: I would like to read the data within "orders", which are "id" and "total".
You are trying to read in an array nested in an object nested in an array.
So your root would have to be users[0].orders
Is that really what you are trying to do?
Working code:
http://jsfiddle.net/el_chief/s6Ynp/1/
Why is your code nested like this?