Now I have a string in format dict but as i can guess its a json format its look like:
{
"gid":"1201400250397201",
"memberships":[
"can be nested objects",
...
],
"name":"Name of task",
"parent":{
"gid":"1201400250397199",
"name":"name of parent task"
},
"permalink_url":"https://url...."
}
So first question: am i right? I used dumps() from json library but got unicode escape sequences, loads() didnt work for me, i got error "the JSON object must be str, bytes or bytearray, not dict".
Second question: if its not json format, how can i get comfortable view? I did it:
first of all i get dict-line, then I print a dictionary's key:
for key in task:
task
print(task[key])
output:
1201400250397201
[]
Name of task
{'gid': '1201400250397199', 'name': ''name of parent task'}
https://url....
At actually it would be great if I get something like that:
gid: 1201400250397201
name: Name of task
parent_name: 'Name of task' etc
But I dont know how to get it :(
Next question: as you can see for part "parent" (penultimate line) I also get dictionary, how can I extract it and get convenient format?
Or maybe you have your comfortable methods?
Like stated in your error, the object you are working with is already a dictionary. You can print it directly as json with json.dumps:
task = {'gid': '1201400250397201', 'memberships': [{}], 'name': 'Name of task', 'parent': {'gid': '1201400250397199', 'name': 'name of parent task'},'permalink_url': 'https://url....'}
print(json.dumps(task, indent=4))
Setting indent=4 makes it readable and you'll get:
{
"gid": "1201400250397201",
"memberships": [
{}
],
"name": "Name of task",
"parent": {
"gid": "1201400250397199",
"name": "name of parent task"
},
"permalink_url": "https://url...."
}
If you don't want unicode characters to be escaped, add the argument ensure_ascii=False:
print(json.dumps(task, indent=4, ensure_ascii=False))
Related
I am trying to parse the JSON file below. The problem is I cannot return "Mountpoint" as a key. It only gets parsed as a value. This is the command I am using to parse it json_data = JSON.parse(readjson). The reason I guess that it's a key is because if I run json_data.keys only EncryptionStatus and SwitchName are returned. Any help would be greatly appreciated.
{
"EncryptionStatus": [
{
"MountPoint": "C:",
"VolumeStatus": "FullyEncrypted"
},
{
"MountPoint": "F:",
"VolumeStatus": "FullyEncrypted"
},
{
"MountPoint": "G:",
"VolumeStatus": "FullyEncrypted"
},
{
"MountPoint": "H:",
"VolumeStatus": "FullyEncrypted"
}
],
"SwitchName": [
"LAN",
"WAN"
]
}
I tried using dig as a part of my JSON.parse but that didn't seem to help me.
JSON data can have multiple levels.
Your JSON document is a
Hash (Dictionary/Map/Object in other languages) that has two keys ("EncryptionStatus", "SwitchName"),
The value for the "EncryptionStatsu" key is an Array of Hashes (with keys "MountPoint" and "VolumeStatus").
# assuming your JSON is in a file called "input.json"
data = File.read("input.json")
json = JSON.parse(data)
json["EncryptionStatus"].each do |encryption_status|
puts "#{encryption_status["MountPoint"]} is #{encryption_status["VolumeStatus"]}"
end
This will print out
C: is FullyEncrypted
F: is FullyEncrypted
G: is FullyEncrypted
H: is FullyEncrypted
If you want to access a specific item you can look at the dig method. E.g.
json.dig("EncryptionStatus", 3)
Would return the information for mountpoint "H"
I have a JSON file arranged in this pattern:
[
{
"Title ID": "4224031",
"Overtime Status": "Non-Exempt",
"Shift rates": "No Shift rates",
"On call rates": "No On call rates"
},
[
{
"Step: 1.0": [
"$38.87",
"(38.870000)"
]
}
]
][
{
"Title ID": "4225031",
"Overtime Status": "Non-Exempt",
"Shift rates": "No Shift rates",
"On call rates": "No On call rates"
},
[
{
"Step: 1.0": [
"$38.87",
"(38.870000)"
]
}
]
]
I am trying to get it into a Pandas DataFrame. I have tried opening a connection to the JSON file and running JSON.load(s). Unfortunately, I get JSON decode errors like: "JSONDecodeError: Extra data: line 16 column 2 (char 182)". When running the JSON through a linter, I see that there might be an issue with the way the JSON is presented in the file. The parts between the brackets are valid but when wrapped in brackets, become invalid. I have then tried to get at the dictionaries with the wrapping brackets but have not been able to make much progress. Does anyone have tips on how I can successfully access this JSON data and get it into a pandas DataFrame?
The json is invalid beacuase it has more than one root in this representation.
This has to be like this
jsonObject = [{"1":"3"}], [{"4":"5"}]
Hacks that I am able to think of are replace these brackets ][ to this ],[ by find and replace in editor. You'll be able to then create a dataframe as its a list now.
Second, if its not a one time job, then you need to write a regex that can do this for you in text cleaning pipeline(or code). I'm not good at writing of working regex(sorry mate).
I found a solution.
First, after examining the JSON data in a linter, I found that I had some extra brackets and braces at different points. So, I am running the data through a regex that cleans out the unnecessary brackets and braces.
Next, I run each line, which now looks like a string dictionary through json.loads
Finally, I call pd.DataFrame(pd.json_normalize(data)) to get my desired pandas dataframe.
Thanks for the help from commenters.
I have a model like this:
class MyClass(models.Model):
typea = models.CharField(max_length=128)
typeb = models.CharField(max_length=128)
If for example, the resulting json from an API is like this:
{
"count": 75,
"results": [
{
"typea": "This tipe",
"typeb": "A B type",
"jetsons": [],
"data": [
"https://myurl.com/api/data/2/",
],
"created": "2014-12-15T12:31:42.547000Z",
"edited": "2017-04-19T10:56:06.685592Z",
},
I need to parse this result and save typea and typeb into the database, I'm kind of confused on how to do this.
I mean, there is the JSONField on Django, but I don't think that will work for me, since I need to save some specific nested string of the json dict.
Any example or idea on how to achieve this?
I mean, my confusion is on how to parse this and "extract" the data I need for my specific fields.
Thank You
You can always do an import json and use json.load(json_file_handle) to create a dictionary and extract the values you need. All you need is to open the .json file (you can use with open("file.json", "r") as json_file_handle) and load the data.
I am trying to dynamically build some json based on data I retrieve from a database. Up until the opening '[' is the "root" I guess you could say. The next parts with name and value are dynamic and will be based on the number of results I get from the db. I query the db and then the idea was to iterate through the result adding to the json. Can I use jsonBuilder for the root section and then loop with jsonSlurper to add each additional section? Most of the examples I have seen deal with a root and then a one time "slurp" and then joining the two so wasn't sure if I should try a different method for looping and appending multiple sections.
Any tips would be greatly appreciated. Thanks.
{
"hostname": "$hostname",
"path": "$path",
"extPath": "$extPath",
"appName": "$appName",
"update": {"parameter": [
{
"name": "$name",
"value": "$value"
},
{
"name": "$name",
"value": "$value"
}
]}
}
EDIT: So what I ended up doing was just using StringBuilder to create the initial block and then append the subsequent sections. Maybe not the most graceful way to do it, but it works!
//Create the json string
StringBuilder json = new StringBuilder("""{
"hostname": "$hostname",
"path": "$path",
"extPath": "$extPath",
"appName": "$appName",
"update": {"parameter": ["""
)
//Append
sql.eachRow("""<query>""",
{ params ->
json.append("""{ "name": "params.name", "value": "params.value" },""");
}
)
//Add closing json tags
json.append("""]}}""")
If I got your explanation correctly and if the data is not very big (it can live in memory), I'd build a Map object (which is very easy to work with in groovy) and convert it to JSON afterwards. Something like this:
def data = [
hostname: hostname,
path: path,
extPath: extPath,
appName: appName,
update: [parameter: []]
]
sql.eachRow(sqlStr) { row ->
data.update.parameter << [name: row.name, value: row.value]
}
println JsonOutput.toJson(data)
If you're using Grails and Groovy you can utilize grails.converters.JSON.
First, define a JSON named config:
JSON.createNamedConfig('person') {
it.registerObjectMarshaller(Person) {
Person person ->
def output = [:]
output['name'] = person.name
output['address'] = person.address
output['age'] = person.age
output
}
}
This will result in a statically defined named configuration for the Object type of person. Now, you can simply call:
JSON.use('person') {
Person.findAll() as JSON
}
This will return every person in the database with their name, address and age all in one JSON request. I don't know if you're using grails as well in this situation though, for pure Groovy go with another answer here.
tl;dr
I need to send the data from a LinkedHashMap in a GSP template to a Controller and preserve the order of the elements.
I'm assuming a structured data format like JSON is the ideal way to do this, but Grails' JSON converter doesn't create an ordered JSON object from a LinkedHashMap.
What is the best way to send a LinkedHashMap data structure from a GSP to a Controller so that I can preserve order, but do minimal work in parsing the data?
Long version
I'm developing a taglib to render search results in a table.
In the taglib, I construct a LinkedHashMap that specifies the data columns and the labels that the user wants to show for the column names. For example:
def tableFields = [firstName: "First Name", lastName: "Surname", unique_id: "Your Whizbang ID"]
That map gets sent to a view, which will then send it back to a controller to retrieve the search results from the database. I need to preserve the order of the elements (hence the use of a LinkedHashMap).
My first thought was to turn the LinkedHashMap into a JSON string, and then send it to the controller via a hidden form element. So,
import grails.converters.JSON
//taglib class and other code
def tableFields = [firstName: "First Name", lastName: "Surname", unique_id: "Your Whizbang ID"] as JSON
However, that creates a JSON Object like this in the HTML. I'm putting this in a hidden field's value attribute.
<input type="hidden" name="columns" value="{"firstName": "First Name", "lastName": "Surname", "unique_id": "Your Whizbang ID"}" id="columns">
Here's the JSON object by itself.
{"firstName": "First Name", "lastName": "Surname", "unique_id": "Your Whizbang ID"}
You can see that the JSON string's properties are in the same order as the LinkedHashMap in the JSON string. However, JSON Objects aren't really supposed to the preserve order of their properties. Thus, when my controller receives the columns parameter, and I use the JSON.parse() method on it, it creates a plain ol' unordered HashMap instead of a LinkedHashMap. As a result, the columns in my search results display in the wrong order when I render them into an HTML table.
At least one fellow has had a similar problem. Adding as LinkedHashMap after running JSON.parse() doesn't cut it, since the .parse() method screws up the order from the get go.
Daniel Woods, in his response to the above post, noted:
If it's a matter of the grails data binder not working for you, you should be able to override the implicit property setter to cast the object to your favorite Map implementation.
I assume that he's saying I could write my own parser, which would honor the order of the JSON elements (even though it technically shouldn't). I imagine I could also write my own converter so that the resulting JSON element would be something like:
{[{firstName: "First Name"}, {lastName: "Surname"}, {unique_id "Your Whizbang ID"}]}
I'm just about terrified of how the JSON parser would handle that, though. Would I get back a list of HashMaps?
Again, my real question is What is the best way to send a LinkedHashMap data structure from a GSP to a Controller so that I can preserve order, but do minimal work in parsing the data? I'm assuming that's JSON, but I'm more than happy to be told, "Why not just..."
I think the issue is a mismatch between the nature of Java/Groovy collections and the simple "it's a list or it's a map" nature of JSON. Without getting into custom parsing, I'd suggest shifting what you're sending a bit. Instead of trying to force Groovy notions of a LinkedHashMap into Javascriptland, maybe stick to an idiom Javascript understands, such as a list of maps.
In code, instead of:
def tableFields = [firstName: "First Name", lastName: "Surname", unique_id: "Your Whizbang ID"]
how about:
List tableFields = [
[ name: 'firstName', label: 'First Name' ],
[ name: 'lastName', label: 'Surname' ],
[ name: 'unique_id', label: 'Your Whizbang ID' ],
]
This shifts you to JSON that'd maintain the data (I think) you need while giving JSON something it understands is ordered (a list):
<input type="hidden" name="columns" id="columns" value="[
{ "name": "firstName", "label": "First Name" },
{ "name": "lastName", "label": "Surname" },
{ "name": "unique_id", "label": "Your Whizbang ID" }
]" />
Whatever handles this will be a slightly deeper iterator, but that's the price of going from a land of good collections to simpler types...
What I'm doing for now is passing both the current JSON object and a list that I can iterate through. In the GSP template, this looks like:
<g:hiddenField name="columns" value="${colJson}"/>
<g:hiddenField name="columnOrder" value="${columns.collect{it.key}}"/>
where columns is the LinkedHashMap.
Then, in the controller that gets those params, I do this:
def columnTitles = params.columnOrder.tokenize(",[] ")
def unorderedColumns = JSON.parse(params.columns)
def columns = columnTitles.collectEntries{ [(it): unorderedColumns[it]] }
Not elegant, but it does work, and it requires a bit less refactoring than Joe Rinehart's suggestion.