I have this JSON which I want to convert to CSV, I have to use online tools for converting it but its not converting in proper CSV format according to the JSON format , but my JSON format is not in one layer, here is what I'm saying
because after converting into proper CSV i have to add some data and want to convert back to JSON like the original but with additional data so I can upload it on firebase
I suppose the first layer in the JSON is the first header name. The second layer has the value from 1st header as key and the other headers/values as value. This format is probably called "Hashed CSV". The format is something like below in Typescript:
type HashedCsv = {
[header1: string]: {
[value1: string]: {
[otherHeader: string]: string
}
}
}
So, to convert to CSV and from CSV you just need to have this type above in mind.
Below is a code snippet in JavaScript to convert the JSON to CSV. You can test it here on StackOverflow by clicking Run code snippet or with even with NodeJS.
function jsonToCsv (/** #type {{[header1: string]: {[value1: string]: {[otherHeader: string]: string}}}} */ json) {
// The first layer is the first header name
const header1 = Object.keys(json)[0]
const values = json[header1]
// In the 2nd layer are the other header names
const otherHeaders = Object.keys(Object.values(values)[0])
const headers = [header1, ...otherHeaders]
/** #type {string[]} */
const csvValues = []
for (const [value1, otherValues] of Object.entries(values)) {
csvValues.push([
value1,
// Escape commas and quotes
...otherHeaders.map(header => /[,"]/g.test(otherValues[header])
? `"${otherValues[header].replaceAll('"', '""')}"`
: otherValues[header])
].join(','))
}
return [
headers.join(','),
...csvValues
].join('\n')
}
const json = loadJson()
const csv = jsonToCsv(json)
console.log(csv)
// Loads the JSON. It could be a file or from memory like below
function loadJson () {
return {
"Quotes": {
"q1": {
"title": "The unexamined life is not worth living – Socrates"
},
"q2": {
"title": "Whereof one cannot speak, thereof one must be silent – Ludwig Wittgenstein"
},
"q3": {
"title": "Entities should not be multiplied unnecessarily – William of Ockham"
},
"q4": {
"title": "The mind is furnished with ideas by experience alone – John Locke"
},
"q5": {
"title": "We are too weak to discover the truth by reason alone – St. Augustine"
},
"q6": {
"title": "Man is the measure of all things – Protagoras"
},
"q7": {
"title": "Everything that exists is born for no reason, carries on living through weakness, and dies by accident – Jean-Paul Sartre"
},
"q8": {
"title": "To do as one would be done by, and to love one's neighbor as oneself, constitute the ideal perfection of utilitarian morality – John Stuart Mill"
},
"q9": {
"title": "Good and evil, reward and punishment, are the only motives to a rational creature – John Locke"
}
}
}
}
PS: Notice that I've stripped the JSON a bit.
try this
var jObject = (JObject)JObject.Parse(json)["Quotes"];
var arr = jObject.Properties().Select(a => new { title = a.Name, text = a.Value["title"] }).Dump();
var sb = new StringBuilder();
foreach (var item in arr)
sb.Append("\"" + item.title + "\"" + "," + "\"" + item.text + "\"" + Environment.NewLine);
File.WriteAllText("quotes.csv", sb.ToString());
Related
The Problem
Within an AJAX call, I am trying to retrieve a JSON object with an appended auto-generated identifier. Is it possible to use regex to select a JSON object that starts with a specific string?
E.g. below targeting announcements_414988813
jQuery
const parent = $('.c-banner');
let data;
$.getJSON('path_to_data.json', function (result) {
const data = result["jcr:content"]["parsys"];
const announcement = data["/^/announcements"];
let date = announcement.eventDate;
let _html = "";
_html += '<p>' + date + '</p>';
parent.append(_html);
});
JSON
{
"jcr:content": {
"parsys": {
"announcements_414988813": {
"eventDate": "Fri Jan 18 2019 00:00:00 GMT-1000",
"title": "Pizza Day!",
}
}
}
}
After you've extracted the data object, you should be able to loop through its keys and find the one that matches.
var announcement = {}
for (key in data) {
if (key.match(/^announcements/)) {
announcement = data[key];
}
}
Note that if your parsys object has multiple announcements, this would give you the last one. announcement will just remain an empty object if it doesn't find any. (Break out of the loop if you want the first one, or save them all in an array if you need them all.)
I am trying to get a JSON array from a webservice URL and parse it in JSON. The thing is the tutorial I was following shows receving one JSOn obj and parsing it but I need to know how to receive a JSON array and parse it. Below is the code I am working on, I am stuck.
Model
class Fact {
int id;
int fact_id;
String fact;
String image;
String reference;
Fact(this.id, this.fact_id, this.fact, this.image, this.reference);
Fact.fromJson(Map<String, dynamic> json)
: id = json['id'],
fact_id = json['fact_id'],
fact = json['fact'],
image = json['image'],
reference = json['reference'];
Map<String, dynamic> toJson() =>
{
'id' : id,
'fact_id': fact_id,
'fact': fact,
'image': image,
'reference': reference,
};
}
I don't get how to write this for the array of facts which I am getting from the webservice.
Fact Download Manager
class FactsManager {
var constants = Constants();
fetchFacts() {
final lastFactId = 0;
var fetchRequestUrl = constants.fetch_facts_url;
if (lastFactId == 0) {
fetchRequestUrl = fetchRequestUrl + "?count=" + constants.firstTimePostCount.toString();
} else {
fetchRequestUrl = fetchRequestUrl + "?count=" + constants.firstTimePostCount.toString() + "&last_id=" + lastFactId.toString();
}
Future<List<Fact>> fetchPost() async {
final response = await http.get(fetchRequestUrl);
if (response.statusCode == 200) {
return List<Fact>
}
}
}
}
Example Data which I am trying to parse.
[
{
"id": "407",
"fact": "Monsanto once tried to genetically engineer blue cotton, to produce denim without the use of dyes, reducing the pollution involved in the dyeing process. ",
"reference": null,
"image": "http:\/\/quickfacts.me\/wp-content\/uploads\/2015\/06\/fact492.png",
"fact_id": "1"
},
{
"id": "560",
"fact": "You can count from zero to nine hundred ninety-nine without ever having to use the letter \"a\" ",
"reference": null,
"image": "http:\/\/quickfacts.me\/wp-content\/uploads\/2015\/06\/fact04.png",
"fact_id": "2"
},
{
"id": "564",
"fact": "In order to keep the project a secret, the British army used the innocuous name \"mobile water carriers\" for a motorized weapons project - which is the reason we call them \"tanks\". ",
"reference": null,
"image": "http:\/\/quickfacts.me\/wp-content\/uploads\/2015\/06\/fact116.png",
"fact_id": "3"
},
{
"id": "562",
"fact": "In 2010 the mummified corpse of Sogen Kato, thought to be Tokyo's oldest man, was found in his bedroom by government officials. He had actually died in 1978. ",
"reference": null,
"image": "http:\/\/quickfacts.me\/wp-content\/uploads\/2015\/06\/fact216.png",
"fact_id": "4"
},
{
"id": "566",
"fact": "In 1927 the US Supreme Court ruled it constitutional for the government to forcefully sterilize mentally handicapped people ",
"reference": null,
"image": "http:\/\/quickfacts.me\/wp-content\/uploads\/2015\/06\/fact316.png",
"fact_id": "5"
}
]
You can do the following:
String receivedJson = "... Your JSON string ....";
List<dynamic> list = json.decode(receivedJson);
Fact fact = Fact.fromJson(list[0]);
In any case, you must consider the following in your json string and the Fact class that you have crafted:
In the json string the id and fact_id are Strings and you treat them as int. Either you change the json or the Fact class
Some strings inside the json string produce errors as the have additional quotation marks and this confuses the decoder.
A json string the works is the following:
String receivedJson = """
[
{
"id": 407,
"fact": "Monsanto once tried to genetically engineer blue cotton, to produce denim without the use of dyes, reducing the pollution involved in the dyeing process. ",
"reference": null,
"image": "http:\/\/quickfacts.me\/wp-content\/uploads\/2015\/06\/fact492.png",
"fact_id": 1
}
]
""";
Easily
String arrayText = '[{"name": "dart1","quantity": 12 },{"name": "flutter2","quantity": 25 }]';
var tagsJson = jsonDecode(arrayText);
List<dynamic> tags = tagsJson != null ? List.from(tagsJson) : null;
print(">> " + tags[0]["name"]);
print(">> " + tags[1]["name"]);
print(">> " + tags[0]["quantity"].toString());
print(">> " + tags[1]["quantity"].toString());
output
2021-04-28 18:55:28.921 22085-22225/com.example.flutter_applicationdemo08 I/flutter: >> dart1
2021-04-28 18:55:28.921 22085-22225/com.example.flutter_applicationdemo08 I/flutter: >> flutter2
2021-04-28 18:55:28.921 22085-22225/com.example.flutter_applicationdemo08 I/flutter: >> 12
2021-04-28 18:55:28.921 22085-22225/com.example.flutter_applicationdemo08 I/flutter: >> 25
I have a JSON file with a large amount of the following values:
"values": [
"Foo": 1,
"Bar": 2,
"Baz": 3,
...
],
How do I efficiently convert this into:
"values": [
{
"name": "Foo",
"value": 1
},
{
"name": "Bar",
"value": 2
},
{
"name": "Baz",
"value": 3
},
...
],
Any help would be appreciated!
Okay, so there are two problems with your input. The first is the fact that the given JSON is invalid, so can't directly be parsed. The square brackets after "values" should be curly brackets, to allow for a hash instead of an array:
let raw_old_data =
// Read the old file
fs.readFileSync('./input_data.json').toString()
// Remove all newlines which could interfere with the regex
.replace(/[\r\n]/g, '')
// Replace the square brackets after `"values"` with curly braces
.replace(/"values": \[(.+?)\]/g, '"values": { $1 }');
To convert this (now valid) string to a JSON object, you use JSON.parse:
let old_data = JSON.parse(raw_old_data);
The second problem is that the format in which the values are stored doesn't match your needs. You want to convert from { key: "value" } to [ name: "key", value: "value" ]. The following function can do that, assuming your version of Node supports ES6 (If not, look at Murillo's answer):
function fix_format(obj) {
// This is where we keep the new items in the correct format
let res = [];
// Loop over all values
Object.keys(obj.values).forEach(name => {
let value = obj.values[name];
// Change the format and add to resulting array
res.push({
// If the variable is the same as the key of the hash, it doesn't have to be specified
name,
value,
});
});
return res;
}
All that's then left to do is loop all data from the old object through that function with the Array.map function:
let new_data = old_data.map(fix_format);
And optionally write it back to a file to use with a different program:
fs.writeFileSync('./formatted_data.json', JSON.stringify(data, null, 2));
Note: The 2 in the JSON.stringify function indicates that the resulting JSON should be padded with 2 spaces, to keep it readable.
With ES6:
Object.keys(values).map(name => ({
name,
value: values[name]
}))
Without ES6:
var keys = Object.keys(values);
var newValues = [];
for(var i = 0; i < keys.length; i++){
newValues.push({
name: keys[i],
value: values[keys[i]]
})
}
If your intention is to use the received data i.e obtain data from DB (e.g MSSql, MySql...) using the connection.query(your_custom_sql_query, (err, rows, fields)
for more info:Node.js MySQL Select From Table
I'll recommend you to use:
const myJson = JSON.stringify(rows[0]);
I have a long list of data, in the following format:
[
{
"ID": "1234",
"date/time": "2016-07-18 18:21:44",
"source_address": "8011",
"lat": "40.585260",
"lng": "-105.084420",
}
]
And I am creating a script to extract the values out of each line. For example, if a line contains "ID": I want to be able to store the value "1234" into a variable, so I can store it in a different format.
Here is my code to detect "ID":
'use strict';
let lineReader = require('line-reader');
//.JSON variables from input file
let id;
//begin creating new object
console.log('var source = {');
//output the file
lineReader.eachLine('dataOut.json', function (line, last) {
//detect ID, and print it out in the new format
if (id =~ /^id:$/) {
console.log('id: "') + console.log('",');
}
//done
if (last) {
console.log('}');
return false; // stop reading
}
});
Once I detect the ID, I'm not sure how I can obtain the value that follows the "ID" on that line.
How can I store the values on a line, after I detect which line they are on?
Unless your json file is stupidly big, you can just require it and then it's an in memory JS object.
var obj = require('./dataOut.json');
// first element
console.log(obj[0]);
I know this should be easy, but I just can't work out how to do it despite having spent several hours looking at it today. There doesn't appear to be a straightforward example or tutorial online as far as I can tell.
I've got several "tables" of documents in a CouchDB database, with each "table" having a different value in a "schema" field in the document. All documents with the same schema contain an identical set of fields. All I want to do is be able to view the different "tables" in CSV format, and I don't want to have to specify the list of fieldnames in each schema.
The CSV output is going to be consumed by an R script, so I don't want any additional headers in the output if I can avoid them; just the list of fieldnames, comma separated, with the values in CSV format.
For example, two records in the "table1" format might look like:
{
"schema": "table1",
"field1": 17,
"field2": "abc",
...
"fieldN": "abc",
"timestamp": "2012-03-30T18:00:00Z"
}
and
{
"schema": "table1",
"field1": 193,
"field2": "xyz",
...
"fieldN": "ijk",
"timestamp": "2012-03-30T19:01:00Z"
}
My view is pretty simple:
"all": "function(doc) {
if (doc.schema == "table1") {
emit(doc.timestamp, doc)
}
}"
as I want to sort my records in timestamp order.
Presumably the list function will be something like:
"csv": "function(head, req) {
var row;
...
// Something here to iterate through the list of fieldnames and print them
// comma separated
for (row in getRow) {
// Something here to iterate through each row and print the field values
// comma separated
}
}"
but I just can't get my head around the rest of it.
If I want to get CSV output looking like
"timestamp", "field1", "field2", ..., "fieldN"
"2012-03-30T18:00:00Z", 17, "abc", ..., "abc"
"2012-03-30T19:01:00Z", 193, "xyz", ..., "ijk"
what should my CouchDB list function look like?
Thanks in advance
The list function that works with your given map should look something like this:
function(head,req) {
var headers;
start({'headers':{'Content-Type' : 'text/csv; charset=utf-8; header=present'}});
while(r = getRow()) {
if(!headers) {
headers = Object.keys(r.value);
send('"' + headers.join('","') + '"\n');
}
headers.forEach(function(v,i) {
send(String(r.value[v]).replace(/\"/g,'""').replace(/^|$/g,'"'));
(i + 1 < headers.length) ? send(',') : send('\n');
});
}
}
Unlike Ryan's suggestion, the fields to include in the list are not configurable in this function, and any changes in order or included fields would have to be written in. You would also have to rewrite any quoting logic needed.
Here some generic code that Max Ogden has written. While it is in node-couchapp form, you probably can get the idea:
var couchapp = require('couchapp')
, path = require('path')
;
ddoc = { _id:'_design/csvexport' };
ddoc.views = {
headers: {
map: function(doc) {
var keys = [];
for (var key in doc) {
emit(key, 1);
}
},
reduce: "_sum"
}
};
ddoc.lists = {
/**
* Generates a CSV from all the rows in the view.
*
* Takes in a url encoded array of headers as an argument. You can
* generate this by querying /_list/urlencode/headers. Pass it in
* as the headers get parameter, e.g.: ?headers=%5B%22_id%22%2C%22_rev%5D
*
* #author Max Ogden
*/
csv: function(head, req) {
if ('headers' in req.query) {
var headers = JSON.parse(unescape(req.query.headers));
var row, sep = '\n', headerSent = false, startedOutput = false;
start({"headers":{"Content-Type" : "text/csv; charset=utf-8"}});
send('"' + headers.join('","') + '"\n');
while (row = getRow()) {
for (var header in headers) {
if (row.value[headers[header]]) {
if (startedOutput) send(",");
var value = row.value[headers[header]];
if (typeof(value) == "object") value = JSON.stringify(value);
if (typeof(value) == "string") value = value.replace(/\"/g, '""');
send("\"" + value + "\"");
} else {
if (startedOutput) send(",");
}
startedOutput = true;
}
startedOutput = false;
send('\n');
}
} else {
send("You must pass in the urlencoded headers you wish to build the CSV from. Query /_list/urlencode/headers?group=true");
}
}
}
module.exports = ddoc;
Source:
https://github.com/kanso/kanso/issues/336