Related
I'm trying to convert a very large .json file to a .csv file. Here is a sample of the json file I have been using.
The file I'll be getting directly from a journal publisher in the same format.
The main purpose of this is to extract all the component from the .json file and put the information to our database.
Below is the code I have tried.
import csv, json, sys
if sys.argv[1] is not None and sys.argv[2] is not None:
fileInput = sys.argv[1]
fileOutput = sys.argv[2]
inputFile = open(fileInput, encoding="utf8") #open json file
outputFile = open(fileOutput, 'w') #load csv file
data = json.load(inputFile) #load json content
inputFile.close() #close the input file
output = csv.writer(outputFile) #create a csv.write
output.writerow(data[0].keys()) # header row
for row in data:
output.writerow(row.values()) #values row
I'm getting this error:
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 542)
that is not valid json. the opening bracket on byte offset 0 is closed with a closing bracket on byte offset 383, then another bracket is created on byte offset 386, the new backet outside of the closing bracket on offset 383 which is created on byte offset 386 is illegal in json, the only thing that would be legal after the closing bracket is whitespace (spaces, tabs, newlines)
it looks a lot like 100 separate json's that are all line-separated, though, but there is no easy way of parsing that, as valid jsons may also contain newlines. if the data provider can guarantee that their individual jsons NEVER contains newlines, or that all their newlines are encoded in some other way than using hex 0A bytes, for example encoded with hex 5C6E instead of hex 0A, then you could ofc split up the jsons by newlines.. but that approach is unreliable if the data provider's jsons may contain newlines. (and the json specificaion allows newlines, 0x0A bytes, in jsons, so that would require your data provider to only use a newline-lacking subset of json.. if your provider is looking for a quick-fix to this issue: use NULL-bytes, hex 00, as the separator instead of hex 0x0A, because json never contains null bytes, those always has to be encoded in json, to "\u0000", then you could reliably split up the jsons by null-bytes)
here is what happens when i try to parse all 100 lines as individual jsons, splitting them by the 0x0A byte, using the code:
<?php
$jsons=file_get_contents("https://pastebin.com/raw/p9NbH2tG");
json_decode($jsons);
echo json_last_error_msg(),PHP_EOL;
$jsons=explode("\n",$jsons);
foreach($jsons as $json){
json_decode($json);
echo json_last_error_msg(),PHP_EOL;
}
output:
$ php foo.php
Syntax error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
No error
as you can see, each individual line in your file contains valid json, but as a whole, it's not valid json. but splitting them by newlines is NOT a reliable way, it just happens to work here because there are no newlines in any of the 100 jsons in your test file.
This looks a lot like the question asked here Django convert JSON to CSV
Can you share a sample of the json response you are getting? Perhaps there is an issue with attempting to decode multiple dictionaries etc.
As per the EOSIO Docs, I ran the cleos wallet create command to create my first default wallet. However it is throwing following error.
3401237ms thread-0 main.cpp:2756 main ] Failed with error: Assert Exception (10)
response_content_length >= 0: Invalid content-length response
After going through few github issues, I figured out that following change to config.ini will fix this error.
Config File Location: /tmp/eosio/config, and File: config.ini, and the property http-validate-host should be changed from 1 (true) to 0 (false), as shown below:
http-validate-host=0
When I load Octave each time I get the below errors after the welcome/info message.
error: `prefix' undefined near line 12 column 35
error: evaluating argument list element number 2
error: called from:
error: C:\Octave\Octave3.4.3_gcc4.5.2\share\octave\site\m\startup/octaverc at line 12, column 1
octave:1>
Does anybody know why this is happening?
For that version of octave, the default settings of that file around that line are:
prefix=octave_config_info('prefix');
setenv('path',[getenv('path'),';',prefix,'\mingw32\bin;',prefix,'\msys\bin']);
setenv('path',[getenv('path'),';',prefix,'\gs\gs9.02\bin'])
So you should check if it has changed
I am trying to import json file into R. For the purpose I am using packages such as rjson, jsonlite, RJSONIO, etc. I have tried different things as shown below but I got errors with all of them.
mydata <- fromJSON(paste(readLines("result_prod_14-15_08_17.json"), collapse=""))
The error I got is:
Error: lexical error: invalid char in json text.
{ "_id" : ObjectId("59920f495401ac79452a0
(right here) ------^
Using 'RJSONIO' package:
md <- stream_in(file("result_prod_14-15_08_17.json"))
produces the following error:
opening fileconnectionoldClass input connection.
Error: parse error: premature EOF
{
(right here) ------^
closing fileconnectionoldClass input connection.
It must be something straightforward but since I am new to R I struggle to figure it out.
I am getting the following error message on my site.
"Notice: Undefined variable: successMessage in /home/sites/synapsetechnologies.com/public_html/Login.php on line 107 Notice: Undefined variable: error in /home/sites/synapsetechnologies.com/public_html/Login.php on line 107"
Code: https://jsbin.com/biwutacufe/edit?html,output
I took a look at my codes but cannot figure out what is wrong. I have tried the console in Chrome but it doesn't give me a lot of info.
Thanks