I'm using the jsonlite library to try to get some JSON data and turn it into a data frame that I could use, but so far it hasn't worked. I've tried three methods so far:
testData = fromJSON("<json file location>")
Which outputs:
Error in feed_push_parser(readBin(con, raw(), n), reset = TRUE) :
parse error: trailing garbage
n_reply_to_status_id":null} {"contributors":null,"text":"Te
(right here) ------^
So I figured that if the quotes caused error, I'd just have to remove them:
singleString <- paste(readLines("<json file location>"), collapse=" ")
singleString = gsub('"', "",singleString)
singleString = fromJSON(singleString)
Which outputs:
Error: lexical error: invalid char in json text.
{contributors:null,text:A colour
(right here) ------^
It seems to be pointing to an 'o' in the text. If I delete that 'o' it just points to the 'n' that comes after it.
My last attempt I read on a forum post of someone having a similar problem that it should solve it:
stream = stream_in(file("<json file location>"))
I was pretty happy to see that this did not return an error and that a data frame named "stream" had been added to memory, but when I tried to view that data frame...
> View(stream)
Error in View : 'names' attribute [1] must be the same length as the vector [0]
I'm not exactly sure what this means or what I could do about it in this situation.
If you have an idea of I could get this data loaded correctly, I would deeply appreciate it. Thanks!
EDIT: Here is the JSON data in question: https://gist.github.com/geocachecs/f68d769aeed8e019a26cc230559bbf7f
Related
I'm trying to convert the page at https://lpl.qq.com/web201612/data/LOL_MATCH_DETAIL_7325.js to a json file so that I can parse it into a pandas table.
I'm using the python requests library and my function looks like this:
[1]def lpl_get_match_data(url, test):
[2] good_url = "https://lpl.qq.com/web201612/data/LOL_MATCH_DETAIL_7325.js"
[3] json_file = requests.get(good_url)
[4] json_content = json_file.json()
[5] if test == True:
[6] pprint.pprint(json_content)
[7] return(json_content)
When I run this function I get an error, "JSONDecodeError: Expecting value: line 1 column 1 (char 0)".
I did the obvious "google the error" and checked a bunch of the responses and it looked like in a lot of cases people had empty files or weren't getting responses. I tried changing line 3 to requests.get(good_url).content to confirm that I was getting the proper response and it looked like I was. I'm wondering if anyone could point me to why this page isn't readable as a json object.
The problem is that the page you are trying to read constains a Javascript object, instead of a JSON file. You can convert the object you are getting with demjson library, using: demjson.decode(json_file.content[12:-1]). Note that content is a string, and therefore you can slice the parts "var dataObj = {" and "}", with the given indexes. You can use this instead of your third line, and this should fix you problem.
I hate json files. They are unwieldy and hard to handle :( Please tell me why the following doesn't work:
with open('data.json', 'r+') as file_object:
data = json.load(file_object)[user_1]['balance']
am_nt = 5
data += int(am_nt['amount'])
print(data)
file_object[user_1]['balance'] = data
Through trial and error (and many print statements), I have discovered that it opens the file, goes to the correct place, and then actually adds the am_nt, but I can't make the original json file update. Please help me :( :( . I get:
2000
TypeError: '_io.TextIOWrapper' object is not subscriptable
json is fun to work with as it is similar to python data structures.
The error is: object is not subscriptable
This error is for this line:
file_object[user_1]['balance'] = data
file_object is not json/dictionary data that can be updated like above. Hence the error.
Try to read the json data:
data=json.load(file_object)
Then manipulate the data as python dictionary. And save the file.
i got some problems with my code. After i read a JSON received via socket by a Qt client in a python server, i want to get all the fields of that JSON so i can use it, but i got an error like this: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0). This is the code that cause the exception to raise.
data = connection.recv(1024)
temp = data.decode("utf-8")
jdata = json.loads(temp)
The exception is raised by json.loads(temp). I tried to be sure to have the right type for the loads function, i tried to copy the same string that i get from the socket into another str type and the function works well. Does anyone knows if there is something i overlooked?
update: I just found out that the JSON i get from the socket have a size that differs from a string with the same characters
the exception "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)" usually indicates that the data you're trying to load is not valid JSON.
See Python JSON Docs
"If the data being deserialized is not a valid JSON document, a JSONDecodeError will be raised." (Scroll down to json.loads function)
Can you print out your temp variable? You'll be able to see the value you're trying to JSONDecode.
Has anyone ever received this error when trying to web scrape a site:
Error in feed_push_parser(readBin(con, raw(), n), reset = TRUE) :
lexical error: invalid char in json text.
<!doctype html><html xmlns="htt
(right here) ------^
I do not understand why I am receiving this error when I scraped the first page of the site no problem with the same declaration on that first page. But the second page I get this error. Is there a way around this?
This works fine:
jsonlite::fromJSON("https://www.reddit.com/r/BestOfStreamingVideo/.json", flatten = TRUE)
Get the error here:
jsonlite::fromJSON("https://www.reddit.com/r/BestOfStreamingVideo/?count=25&after=t3_5fvgls/.json", flatten = TRUE)
The latter one does not return a JSON. It returns HTML. Enter both URL's in the browser and you'll see the difference.
I guess the URL you are looking for is:
https://www.reddit.com/r/BestOfStreamingVideo/.json?count=25&after=t3_5fvgls/
You need to put the ./json first and append the URL parameter after.
I am using fromJSON from the jsonlite package in [R] to call GetPlayerSummaries from the Steam API (https://developer.valvesoftware.com/wiki/Steam_Web_API) to get access to a user's data. For most calls it's working fine, but at some point I get an error:
Error in feed_push_parser(readBin(con, raw(), n), reset = TRUE) :
lexical error: invalid bytes in UTF8 string.
publicâ„¢ II: The Sith Lordsâ", "gameid": "208580" },
(right here) ------^
When I access the call in my browser I find a � on the spot where it is probably giving the error. I could Try-Catch but I'd really like to get this data. How to get around this?
For my purpose, reading with readLines and then parsing it seemed to work
readlines <- readLines(link, warn = FALSE)
parse <- fromJSON(readlines)
I have no idea why and how this works, and may hence be not the most clean solution, but it seems to be robust for my purposes.
You have to use use jsonlite's streaming function
json_file <- stream_in(file("abc.json"))
It has been answered in Stack Overflow here:
Error parsing JSON file with the jsonlite package
and here:
Export JSON from Spark and input into R