I have a pandas DataFrame with one feature, df['Computed Data'].
Computed Data
'{"stats":{"TypeCount":{"1
25":"31","8":"31"}},"plaintsCard":[{"root":"old","plaintsCount":1,"residencyCount":1}],"Count":62,"Status":{"activable":"10","activated":"18","inactivable":"3"},"Counta":0,"invoiCount":"31"}'
'{"Count":33,"invoiCount":"11","stats":{"TypeCount":{"1":"9","4":"22","11":"2"}},"plaintsCard":[],"Count":0,"Status":{"activated":"0","activable":"9","inactivable":"1"}}'
'{"Count":79,"invoiCount":"32","stats":{"TypeCount":{"1":"29","4":"32","18":"3","23":"15"}},"plaintsCard":[],"Count":0,"Status":{"activated":"0","activable":"28","inactivable":"2"}}'
'{"Count":80,"invoiCount":"32","stats":{"TypeCount":{"1":"31","4":"42","13":"1","23":"6"}},"plaintsCard":[],"Count":0,"Status":{"activated":"0","activable":"27","inactivable":"6"}}'
'{"stats": {"TypeCount": {"17": "27"}}, "plaintsCard": [], "parcelsCount": 27, "Status": {"activable": "9", "activated": "2", "inactivable": "16"}, "Count": 0, "invoiCount": "0"}'
I want to extract the "membersStatus", "activable" part from every string and to put it in a new column.
I have tried to use ast.literal_eval() and it is working but only when I apply it to one value
x = ast.literal_eval(df["Computed Data"][0])
x["membersStatus"]["activable"]
'10'
It gives me : '10'. Which is what I want but for every dict in "Computed Data" and to put it in a new column.
I tried to do it with a for loop :
for n, i in enumerate(df["Computed Data"]):
x = ast.literal_eval(df["Computed Data"][n])
ValueError: malformed node or string: <_ast.Name object at 0x13699c610>
I don't know how can I change what I did to make it work.
Can you Help please ?
you can use:
import json
df['Computed Data']=df['Computed Data'].apply(json.loads)
df['activable']=df['Computed Data'].apply(lambda x: x['membersStatus']['activable'])
if there are nan values:
df['Computed Data']=df['Computed Data'].apply(lambda x: json.loads(x) if pd.notna(x) else np.nan)
df['activable']=df['Computed Data'].apply(lambda x: x['membersStatus']['activable'] if pd.notna(x) else np.nan)
you can use apply methods to extract them:
df['member Status']=df['Computed Data'].apply(lambda x:eval(x)['membersStatus']['activable'])
Related
I want find min element of object (by days attribute) in a list of objects in Python, I have this code so far:
from operator import attrgetter
lists = json.loads("[" + data + "]")
print(lists)
maintenance_cycle = min(lists,key=lambda r: r.days)
Console output:
[{'type': 'runtime', 'days': 1}]
Error:
'dict' object has no attribute 'days'
Accessing dictionary keys is not like accessing class methods
Try this code
maintenance_cycle = min(lists, key=lambda r: r['days'])
Or
maintenance_cycle = min(lists, key=lambda r: r.get('days'))
Instead of
maintenance_cycle = min(lists,key=lambda r: r.days)
And tell me if its not working...
slurperresponse = new JsonSlurper().parseText(responseContent)
log.info (slurperresponse.WorkItems[0].WorkItemExternalId)
The above code helps me get the node value "WorkItems[0].WorkItemExternalId" using Groovy. Below is the response.
{
"TotalRecordCount": 1,
"TotalPageCount": 1,
"CurrentPage": 1,
"BatchSize": 10,
"WorkItems": [ {
"WorkItemUId": "4336c111-7cd6-4938-835c-3ddc89961232",
"WorkItemId": "20740900",
"StackRank": "0",
"WorkItemTypeUId": "00020040-0200-0010-0040-000000000000",
"WorkItemExternalId": "79853"
}
I need to append the string "WorkItems[0].WorkItemExternalId" (being read from a excel file) and multiple other such nodes dynamically to "slurperresponse" to get the value of nodes rather than directly hard coding as slurperresponse.WorkItems[0].WorkItemExternalId..
Tried append and "+" operator but i get a compilation error. What other way can I do this?
slurperrsesponse is an object its not a string that's why the concatenation does not work
Json Slurper creates an object out of the input string. This object is dynamic by nature, you can access it, you can add fields to it or alter the existing fields. Contatenation won't work here.
Here is an example:
import groovy.json.*
def text = '{"total" : 2, "students" : [{"name": "John", "age" : 20}, {"name": "Alice", "age" : 21}] }'
def json = new JsonSlurper().parseText(text)
json.total = 3 // alter the value of the existing field
json.city = 'LA' // add a totally new field
json.students[0].age++ // change the field in a list
println json
This yields the output:
[total:3, students:[[name:John, age:21], [name:Alice, age:21]], city:LA]
Now if I've got you right you want to add a new student dynamically and the input is a text that you've read from Excel. So here is the example:
json.students << new JsonSlurper().parseText('{"name" : "Tom", "age" : 25}')
// now there are 3 students in the list
Update
Its also possible to get the values without 'hardcoding' the property name:
// option 1
println json.city // prints 'LA'
// option 2
println json.get('city') // prints 'LA' but here 'city' can be a variable
// option 3
println json['city'] // the same as option 2
I have an issue with using jsonpickle. Rather, I believe it to be working correctly but it's not producing the output I want.
I have a class called 'Node'. In 'Node' are four ints (x, y, width, height) and a StringVar called 'NodeText'.
The problem with serialising a StringVar is that there's lots of information in there and for me it's just not necessary. I use it when the program's running, but for saving and loading it's not needed.
So I used a method to change out what jsonpickle saves, using the __getstate__ method for my Node. This way I can do this:
def __getstate__(self):
state = self.__dict__.copy()
del state['NodeText']
return state
This works well so far and NodeText isn't saved. The problem comes on a load. I load the file as normal into an object (in this case a list of nodes).
The problem loaded is this: the items loaded from json are not Nodes as defined in my class. They are almost the same (they have x, y, width and height) but because NodeText wasn't saved in the json file, these Node-like objects do not have that property. This then causes an error when I create a visual instance on screen of these Nodes because the StringVar is used for the tkinter Entry textvariable.
I would like to know if there is a way to load this 'almost node' into my actual Nodes. I could just copy every property one at a time into a new instance but this just seems like a bad way to do it.
I could also null the NodeText StringVar before saving (thus saving the space in the file) and then reinitialise it on loading. This would mean I'd have my full object, but somehow it seems like an awkward workaround.
If you're wondering just how much more information there is with the StringVar, my test json file has just two Nodes. Just saving the basic properties (x,y,width,height), the file is 1k. With each having a StringVar, that becomes 8k. I wouldn't care so much in the case of a small increase, but this is pretty huge.
Can I force the load to be to this Node type rather than just some new type that Python has created?
Edit: if you're wondering what the json looks like, take a look here:
{
"1": {
"py/object": "Node.Node",
"py/state": {
"ImageLocation": "",
"TextBackup": "",
"height": 200,
"uID": 1,
"width": 200,
"xPos": 150,
"yPos": 150
}
},
"2": {
"py/object": "Node.Node",
"py/state": {
"ImageLocation": "",
"TextBackup": "",
"height": 200,
"uID": 2,
"width": 100,
"xPos": 50,
"yPos": 450
}
}
}
Since the class name is there I assumed it would be an instantiation of the class. But when you load the file using jsonpickle, you get the dictionary and can inspect the loaded data and inspect each node. Neither node contains the property 'NodeText'. That is to say, it's not something with 'None' as the value - the attribute simple isn't there.
That's because jsonpickle doesn't know which fields are in your object normally, it restores only the fields passed from the state but the state doesn't field NodeText property. So it just misses it :)
You can add a __setstate__ magic method to achieve that property in your restored objects. This way you will be able to handle dumps with or without the property.
def __setstate__(self, state):
state.setdefault('NodeText', None)
for k, v in state.items():
setattr(self, k, v)
A small example
from pprint import pprint, pformat
import jsonpickle
class Node:
def __init__(self) -> None:
super().__init__()
self.NodeText = Node
self.ImageLocation = None
self.TextBackup = None
self.height = None
self.uID = None
self.width = None
self.xPos = None
self.yPos = None
def __setstate__(self, state):
state.setdefault('NodeText', None)
for k, v in state.items():
setattr(self, k, v)
def __getstate__(self):
state = self.__dict__.copy()
del state['NodeText']
return state
def __repr__(self) -> str:
return str(self.__dict__)
obj1 = Node()
obj1.NodeText = 'Some heavy description text'
obj1.ImageLocation = 'test ImageLocation'
obj1.TextBackup = 'test TextBackup'
obj1.height = 200
obj1.uID = 1
obj1.width = 200
obj1.xPos = 150
obj1.yPos = 150
print('Dumping ...')
dumped = jsonpickle.encode({1: obj1})
print(dumped)
print('Restoring object ...')
print(jsonpickle.decode(dumped))
outputs
# > python test.py
Dumping ...
{"1": {"py/object": "__main__.Node", "py/state": {"ImageLocation": "test ImageLocation", "TextBackup": "test TextBackup", "height": 200, "uID": 1, "width": 200, "xPos": 150, "yPos": 150}}}
Restoring object ...
{'1': {'ImageLocation': 'test ImageLocation', 'TextBackup': 'test TextBackup', 'height': 200, 'uID': 1, 'width': 200, 'xPos': 150, 'yPos': 150, 'NodeText': None}}
I get the following JSON response for Alpha vantage Stock API:
"Time Series (Daily)": {
"2018-07-09": {
"1. open": "142.6000",
...
},
I get different dates depending on the location of the stock exchange. On some, it might say "2018-07-09", and on others "2018-07-10".
Is there a way to reach the attributes of an object without typing the object, something along the lines of:
["Time Series (Daily)"]["first_child"]["1. open"]
Using:
["Time Series (Daily)"][0]["1. open"]
doesn't work.
ruby_hash["Time Series (Daily)"].values.first["1. open"]
Adapted from How can I get the key Values in a JSON array in ruby?
require 'json'
myData = JSON.parse('{"A": {"B": {"C": {"D": "open sesame"}}}}')
def getProperty(index, parsedData)
return parsedData[parsedData.keys[index]]
end
myDataFirstProperty = getProperty(0, myData) # => {"B"=>{"C"=>{"D"=>"open sesame"}}}
evenDeeper = getProperty(0, myDataFirstProperty) # => {"C"=>{"D"=>"open sesame"}}
This answer can also get the Nth property of the JSON, unlike the accepted answer which can only get the first.
I have a JSON dict in Python which I would like to parse into a CSV, my data and code looks like this:
import csv
import json
x = {
"success": 1,
"return": {
"variable_id": {
"var1": "val1",
"var2": "val2"
}...
f = csv.writer(open("foo.csv", "w", newline=''))
for x in x:
f.writerow([x["success"],
'--variable value--',
x["return"]["variable_id"]["var1"],
x["return"]["variable_id"]["var2"])
However, since variable_id's value is going to change I don't know how to refer to in the code. Apologies if this is trivial but I guess I lack the terminology to find the solution.
You can use the * (unpack) operator to do this, assuming only the values in your variable_id matter :
f.writerow([x["success"],
'--variable value--',
*[val for variable_id in x['return'].values() for val in variable_id.values()])
The unpack operator essentially takes everything in x["return"]["variable_id"].values() and appends it in the list you're creating as input for writerow.
EDIT this should now work if you don't know how to referencevariable_id. This will work best if you have several variable_ids in x['return'].
If you only have one variable_id, then you can also try this :
f.writerow([x["success"],
'--variable value--',
*list(x['return'].values())[0].values()])
Or
f.writerow([x["success"],
'--variable value--',
*next(iter(x['return'].values())).values()])
You can get variable_id's value using x['success']['return'].keys[0].