I hate json files. They are unwieldy and hard to handle :( Please tell me why the following doesn't work:
with open('data.json', 'r+') as file_object:
data = json.load(file_object)[user_1]['balance']
am_nt = 5
data += int(am_nt['amount'])
print(data)
file_object[user_1]['balance'] = data
Through trial and error (and many print statements), I have discovered that it opens the file, goes to the correct place, and then actually adds the am_nt, but I can't make the original json file update. Please help me :( :( . I get:
2000
TypeError: '_io.TextIOWrapper' object is not subscriptable
json is fun to work with as it is similar to python data structures.
The error is: object is not subscriptable
This error is for this line:
file_object[user_1]['balance'] = data
file_object is not json/dictionary data that can be updated like above. Hence the error.
Try to read the json data:
data=json.load(file_object)
Then manipulate the data as python dictionary. And save the file.
Related
I have a json file with about 1,200,000 records.
I want to read this file with pyspark as :
spark.read.option("multiline","true").json('file.json')
But it causes this error:
AnalysisException: Unable to infer schema for JSON. It must be specified manually.
When I create a json file with a smaller record count in the main file, this code can read the file.
I can read this json file with pandas, when I set the encoding to utf-8-sig:
pd.read_json("file.json", encoding = 'utf-8-sig')
How can I solve this problem?
Try this out:
spark.read.option("multiline","true").option("inferSchema", "true").json('file.json')
Since adding the encoding helps, maybe the following is what you need:
spark.read.json("file.json", multiLine=True, encoding="utf8")
Having a puzzling problem posting JSON to firebase programatically:
Original JSON retreived from firebase:
{'recipe1': {'abbie':2,'ben':0,'chris':1},'recipe2': {'abbie':1,'ben': 5,'chris':5}}
I then convert it into a dataframe using pandas to manipulate the data, before turning it back into JSON. Here is where I'm getting stuck.
Convert dataframe to JSON:
out = df.to_json()
Result printed in terminal:
{"recipe1":{"abbie":2,"ben":0,"chris":1},"recipe2":{"abbie":1,"ben":5,"chris":5}}
firebase.post("/testupdate", out)
Yet if I manually assign out to the same JSON structure:
out = {"recipe1":{"abbie":2,"ben":0,"chris":1},"recipe2":{"abbie":1,"ben":5,"chrisy":5}}
and post that,it works perfectly.
If anyone can help me out here it would be greatly appreciated!
Actually I've just figured it out myself, assumed it would be a pretty simple fix.
Anyone else having this difficulty simply use:
out = df.to_dict()
Instead of:
out = df.to_json()
When converting the dataframe.
I'm using the jsonlite library to try to get some JSON data and turn it into a data frame that I could use, but so far it hasn't worked. I've tried three methods so far:
testData = fromJSON("<json file location>")
Which outputs:
Error in feed_push_parser(readBin(con, raw(), n), reset = TRUE) :
parse error: trailing garbage
n_reply_to_status_id":null} {"contributors":null,"text":"Te
(right here) ------^
So I figured that if the quotes caused error, I'd just have to remove them:
singleString <- paste(readLines("<json file location>"), collapse=" ")
singleString = gsub('"', "",singleString)
singleString = fromJSON(singleString)
Which outputs:
Error: lexical error: invalid char in json text.
{contributors:null,text:A colour
(right here) ------^
It seems to be pointing to an 'o' in the text. If I delete that 'o' it just points to the 'n' that comes after it.
My last attempt I read on a forum post of someone having a similar problem that it should solve it:
stream = stream_in(file("<json file location>"))
I was pretty happy to see that this did not return an error and that a data frame named "stream" had been added to memory, but when I tried to view that data frame...
> View(stream)
Error in View : 'names' attribute [1] must be the same length as the vector [0]
I'm not exactly sure what this means or what I could do about it in this situation.
If you have an idea of I could get this data loaded correctly, I would deeply appreciate it. Thanks!
EDIT: Here is the JSON data in question: https://gist.github.com/geocachecs/f68d769aeed8e019a26cc230559bbf7f
I am using the smappR packageto retrieve Twitter user posts, specifically the getTimeline() function.
However, the problem is that the retrieved data, which has been stored in JSON format is not subsequently being read in as a JSON file by R.
The image below denotes the command and the corresponding error -
Was wondering if there is any other way we can read the files back into R for further processing?
Any help will be appreciated.
Edit 1 : Funnily enough, the file does not appear to be read in, even when I attempted the same in Python (2.7)
The Python Code is as follows -
with open('C:/Users/ABC/Downloads/twitter/profile/bethguido3.JSON') as data_file:
data = json.load(data_file)
The error that appeared is -
Basically, I'm trying to convert the query response from Solr server into a json object that can be passed to a third party api. However, as per my code below, I'm not able to do it:
import solr
import json
if __name__=='__main__':
s = solr.SolrConnection('http://localhost:8983/solr')
op = open('output.json','w')
for term in ['searchstring1','searchstring2','searcstring']:
t = s.query('title:%s'%term,rows=100, wt='json')
for news in t.results:
op.write(news)
Output: Traceback (most recent call last):
File "querying.py", line 11, in
op.write(news)
TypeError: expected a character buffer object
I have very briefly read about Solr and just found this solrpy library to store the query result in a json format. Any help in this regard will be much appreciated.
You're trying to write the actual python dictionary to the file, and not a json representation of it. The error message is trying to tell you that the object you're passing in can not be used as a character buffer directly. Try to use json.dumps to create a JSON representation as a string before writing it to a file.