converting CSV/XLS to JSON? [closed] - json

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
Does anyone know if there is application that will let me convert preferably XLS to JSON?
I'll also settle for a converter from CSV since that's what I'll probably end up having to write myself if there is nothing around.

You can try this tool I made:
Mr. Data Converter
It converts to JSON, XML and others.
It's all client side, too, so your data never leaves your computer.

This worked perfectly for me and does NOT require a file upload:
https://github.com/cparker15/csv-to-json?files=1

Since Powershell 3.0 (shipped with Windows 8, available for Windows 7 and windows Server 2008 but not Windows Vista ) you can use the built-in convertto-json commandlet:
PS E:> $topicsjson = import-csv .\itinerary-all.csv | ConvertTo-Json
PS E:\> $topicsjson.Length
11909
PS E:\> $topicsjson.getType()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
Online Help Page on Technet

If you can't find an existing solution it's pretty easy to build a basic one in Java. I just wrote one for a client and it took only a couple hours including researching tools.
Apache POI will read the Excel binary.
http://poi.apache.org/
JSONObject will build the JSON
After that it's just a matter of iterating through the rows in the Excel data and building a JSON structure. Here's some pseudo code for the basic usage.
FileInputStream inp = new FileInputStream( file );
Workbook workbook = WorkbookFactory.create( inp );
// Get the first Sheet.
Sheet sheet = workbook.getSheetAt( 0 );
// Start constructing JSON.
JSONObject json = new JSONObject();
// Iterate through the rows.
JSONArray rows = new JSONArray();
for ( Iterator<Row> rowsIT = sheet.rowIterator(); rowsIT.hasNext(); )
{
Row row = rowsIT.next();
JSONObject jRow = new JSONObject();
// Iterate through the cells.
JSONArray cells = new JSONArray();
for ( Iterator<Cell> cellsIT = row.cellIterator(); cellsIT.hasNext(); )
{
Cell cell = cellsIT.next();
cells.put( cell.getStringCellValue() );
}
jRow.put( "cell", cells );
rows.put( jRow );
}
// Create the JSON.
json.put( "rows", rows );
// Get the JSON text.
return json.toString();

This works for me and runs client-side:
http://www.convertcsv.com/csv-to-json.htm

I just found this:
http://tamlyn.org/tools/csv2json/
( Note: you have to have your csv file available via a web address )

Take a try on the tiny free tool:
http://keyangxiang.com/csvtojson/
It utilises node.js csvtojson module

None of the existing solutions worked, so I quickly hacked together a script that would do the job. Also converts empty strings into nulls and and separates the header row for JSON. May need to be tuned depending on the CSV dialect and charset you have.
#!/usr/bin/python
import csv, json
csvreader = csv.reader(open('data.csv', 'rb'), delimiter='\t', quotechar='"')
data = []
for row in csvreader:
r = []
for field in row:
if field == '': field = None
else: field = unicode(field, 'ISO-8859-1')
r.append(field)
data.append(r)
jsonStruct = {
'header': data[0],
'data': data[1:]
}
open('data.json', 'wb').write(json.dumps(jsonStruct))

Instead of hard-coded converters, how about CSV support for Jackson (JSON processor): https://github.com/FasterXML/jackson-dataformat-csv. So core Jackson can read JSON in as POJOs, Maps, JsonNode, almost anything. And CSV support can do the same with CSV. Combine the two and it's very powerful but simple converter between multiple formats (there are backends for XML, YAML already, and more being added).
An article that shows how to do this can be found here.

See if this helps: Back to CSV - Convert CSV text to Objects; via JSON
This is a blog post published in November 2008 that includes C# code to provide a solution.
From the intro on the blog post:
As Json is easier to read and write then Xml. It follows that CSV (comma seperated values) is easier to read and write then Json. CSV also has tools such as Excel and others that make it easy to work with and create. So if you ever want to create a config or data file for your next app, here is some code to convert CSV to JSON to POCO objects

Related

Writing data from MATLAB to firebase database

I am using MATLAB to write data from MATLAB to firebase. I am using following lines of code to do so:
thingSpeakURL = 'https://hybrid-cabinet-265907.firebaseio.com/Ship A/Time Stamp.json';
lat = num2str(42);
lon = num2str(42);
data = struct('lat',lat,'lon',lon);
webwrite(thingSpeakURL,data)
Data is successfully written to Firebase. It is making my original JSON data as a child to a random string been generated on run-time.
For example, my JSON string is {lat: '40',lon:'40'} but instead it is creating a random string, let say, "Mxkkllslsll-1112", making that random string as parent and writing something like {"Mxkkllslsll-1112": lat:'40', lon:'40'} to the firebase database.
Please have a look at following image. It shows that for ship A, I have written data from MATLAB and it is not writing properly(I am facing the problem which I discussed above). I want to make it something like data written for Ship B.
I want to write the data without making any random string as a parent. Kindly assist me in that.
This is because webwrite uses the HTTP POST method by default.
As shown in the Firebase Realtime Database REST API documentation, if you do a POST you will push the data and therefore automatically generate a unique key every time a new child is added to the specified Firebase reference (the -MDJVMk..... value we can see in your question).
You need to use the PUT method.
I don't know matlab but a rapid look at the documentation shows that you need to use the RequestMethod option with a put value, in the weboptions object.
The above pushed me in the right direction (thanks!), and I had success with the following.
CAUTION: The following will overwrite everything in your database!
url = 'https://***.firebaseio.com/.json';
data.users(1) = struct('first','John','last','Locke');
data.users(2) = struct('first','Thomas','last','Hobbes');
data.users(3) = struct('first','Rene','last','Descartes');
headers = {'Content-Type' 'application/json'; 'Accept' 'application/json'};
options = weboptions('RequestMethod', 'put', 'HeaderFields', headers, 'ArrayFormat', 'json');
response = webwrite(url, data, options);
If your data is stored in a .json file (i.e., you don't want to create structures manually in Matlab), you can read it using "fileread" and pass in data as a string (instead of a structure).

convert excel to json file

I'm new in creating jsons and having bais knowledge of java.
I'm trying to convert database table data to json.
Having option to store table data in any format of file than convert that into json.
Here is my table data.
Table: PKGS
Price, pd, Id, Level
1 , 266 , 59098 , 5
2 , 247 , 59098 , 5
I want my table data in this json format. Its just an example...to show level in JSON
"Id":59098
"pd":266
"Level":5
"price":1
"Id":59098
"pd":247
"Level":5
"price":2
In this json there is two loops are going If am not wrong. I was able to do it for one loop in ETL..but couldnt do it for two loops.
Not getting values for reimbursementId and packageId
Have goggled alot but couldn't find any code to understand properly and approach for the same.
Code tried little bit
FileInputStream inp = new FileInputStream("D:/json.xlsx" );
Workbook workbook = WorkbookFactory.create( inp );
Sheet sheet = workbook.getSheetAt( 0 );
JSONObject json = new JSONObject();
JSONArray rows = new JSONArray();
but dont know what to next !!
can anyone tell me how to do this ?
I advice you to use a specific tool. This is not a whole new case.
Try Talend Open Studio. It is not so complicate to use if you want to convert a file (CSV, JSON, Database directly, etc) to another. Please see TalendForge for basics.
In your case, you can connect to your database, and send all data in JSON.
Edit:
Your representation is not following the same logic than JSON. Here how I see it (and this is probably wrong because I can't understand)
If you just want Excel to JSON without any changes:
{
"rows":[
{
"Price":"1",
"pd":"266",
"Id":"59098",
"Level":"5"
},
{
"Price":"1",
"pd":"266",
"Id":"59098",
"Level":"5"
},
//and again and again
{
"Price":"2",
"pd":"247",
"Id":"59098",
"Level":"5"
}
]
}
If you want to reorganize, then define what you want. Try to imagine a sample of your data in a Java Object using ArrayList, int, String and subclass or even better in a JavaScript Object.
For the last example it will give you:
public class myJson{
ArrayList<myObject> rows;
with
public class myObject{
String Price;
String pd;
String Id;
String Level; //Or int, or date, or whatever
If you want to reorganize your data model, please give us this model.
Converting Excel file data to Json format is a a bit complex process, it depends on the structure of data, we do not have exact online tool as such so far...
custom code is required, there are various technologies available to be used, but best suited should be VBA, because VBA fits within Excel and can generate Json file quickly and flexible to edit code compared to any other technology that requires to automate to excel and import data and then process.
We can find there are various websites provide code to generate json from excel data, here is one such site looks expertise in this area. http://www.xlvba.net/tools/excel-automation-to-convert-excel-data-to-json-format.html
Ragavendra

JSON reader was expecting a name but found ':'. in Mongodb Java

I am storing my data from external file to mongodb in localhost. it's quite huge dataset of volume 1.70GB with ~10 million tweets. While importing from file to mongodb it shows me the error "JSON reader was expecting a name but found ':'"
I dint have any error on previous files. But this I cant figure it out. The data is just a real time collection of tweets from streaming API in a json format.
BufferedReader br = new BufferedReader(new FileReader(file));
int counter = 0;
String line;
while ((line = br.readLine()) != null) {
System.out.println(line);
Document old_status = Document.parse(line);
// If it is a deleted tweet, then continue
if (old_status.containsKey("delete")) {
continue;
}
//populate original tweets
Document original_status = new Document();
if(line.contains("retweeted_status")){
Document retweets = (Document)old_status.get("retweeted_status");
original_status.append("status",retweets.get("text"));
original_status.append("Likes",retweets.getInteger("favorite_count"));}}
its a sample code for importing data from file to mongo collection.
Help me to solve this. I really stuck in this place and it takes my time.
Thanks in advance.
Late to the party, and also a little unrelated to this specific question, but google brought me here so going to leave an answer.
I got this error when I was trying to parse a mongodb document that was converted to JSON in Scala. Make sure that the variable's you're trying to parse are actually in the JSON you're parsing if it is a strict parsing.

Methods to convert CSV to unique JSON

I need to convert a .CSV to a specific .JSON format, and I decided to use the csvtojson package from NPM since it seemed to be designed for this sort of thing.
First, a little background on my problem. I have .CSV data that looks similar to this:
scan_type, date/time, source address, source-lat, source-lng, dest address, dest-lat, dest-lng
nexpose,2016-07-18 18:21:44,1008,40.585260,-10.124120,10.111.131.4,10.844880,-10.933360
I have a plain csv-to-json converter here , but there is a problem. It outputs a rather plain file, and I need to split the source/destination, and add some special formatting that I will show later on.
[
{
"scan_type": "nexpose",
"date/time": "2026-07-28 28:22:44",
"source_address": 2008,
"source-lat": 10.58526,
"source-lng": -105.08442,
"dest_address": "11.266.282.0",
"dest-lat": 11.83388,
"dest-lng": -111.82236
}
]
The first thing I need to do is be able to separate the "source" values, from the "destination" values. Here is an example of what I want the "source" values to look like:
(var destination will be exactly the same format)
var source={
id: "99.58926-295.09492",
"source-lat": 49.59926,
"source-lng": -209.98942,
"source_address": 2009,
x: {
valueOf: function() {
var latlng=[
49.58596,
-209.08442
];
var xy = map.FUNCTION_FOR_CONVERTING_LAT_LNG_TO_X_Y(latlng);
return xy[0]; //xy.x
},
y: {
valueOf: function(){
varlatlng=[
49.58596,
-209.08442
];
var xy = map.FUNCTION_FOR_CONVERTING_LAT_LNG_TO_X_Y(latlng);
return xy[1]; //xy.y
}
}
So, my question is, how should I approach converting my data? Should I convert everything with csvtojson? Or should I convert it from the plain .JSON file I generated?
Does anyone have any advice, or similar examples, they could share on how to approach this problem?
I do a lot of work with parsing CSV data and as I am sure you have seen, CSV is very hard to parse and work with correctly as there are a huge number of edge cases that can break even the most rugged of parsers (although it looks like your dataset is fairly plain so that isn't a huge concern). Not to mention you could potentially run into corruption by performing operations while reading from disk, so it is a much better idea to get the data from CSV into a JSON file and then make any manipulations to a JSON object loaded from that "plain" JSON file.
tl;dr: convert your data from the plain .JSON file

Json-Opening Yelp Data Challenge's data set

I am interested in data mining and I am writing my thesis about it. For my thesis I want to use yelp's data challenge's data set, however i can not open it since it is in json format and almost 2 gb. In its website its been said that the dataset can be opened in phyton using mrjob, but I am also not very good with programming. I searched online and looked some of the codes yelp provided in github however I couldn't seem to find an article or something which explains how to open the dataset, clearly.
Can you please tell me step by step how to open this file and maybe how to convert it to csv?
https://www.yelp.com.tr/dataset_challenge
https://github.com/Yelp/dataset-examples
data is in .tar format when u extract it again it has another file,rename it to .tar and then extract it.you will get all the json files
yes you can use pandas. Take a look:
import pandas as pd
# read the entire file into a python array
with open('yelp_academic_dataset_review.json', 'rb') as f:
data = f.readlines()
# remove the trailing "\n" from each line
data = map(lambda x: x.rstrip(), data)
data_json_str = "[" + ','.join(data) + "]"
# now, load it into pandas
data_df = pd.read_json(data_json_str)
Now 'data_df' contains the yelp data ;)
Case, you want convert it directly to csv, you can use this script
https://github.com/Yelp/dataset-examples/blob/master/json_to_csv_converter.py
I hope it can help you
To process huge json files, use a streaming parser.
Many of these files aren't a single json, but a stream of jsons (known as "jsons format"). Then a regular json parser will consider everything but the first entry to be junk.
With a streaming parser, you can start reading the file, process parts, and wrote them to the desired output; then continue writing.
There is no single json-to-csv conversion.
Thus, you will not find a general conversion utility, you have to customize the conversion for your needs.
The reason is that a JSON is a tree but a CSV is not. There exists no ultimative and efficient conversion from trees to table rows. I'd stick with JSON unless you are always extracting only the same x attributes from the tree.
Start coding, to become a better programmer. To succeed with such amounts of data, you need to become a better programmer.