I am completely noobie in node.js and I am a university researcher.
I have a lot of files in XML and JSON format, I have a lot of folders and files, I need read it and create one standard (in JSON/CSV) and finally loading it in MySQL database, any tips? do you know npm packages for it or completely solutions?
There are many packages for converting xml to json like xmljs, xml2json, etc..
But looks like you need transforming to standard format for inserting into database as well.
I have this problem myself and i wrote camaro for this purpose: convert and transform xml to json
All i have to do is to write an output template that i would like my xml to be converted too using xpath syntax like below
Of course you can just flatten all the attribute you need ; the below is just example of what camaro can do.
const transform = require('camaro')
const fs = require('fs')
const xml = fs.readFileSync('examples/ean.xml', 'utf-8')
const template = {
cache_key: '/HotelListResponse/cacheKey',
hotels: ['//HotelSummary', {
hotel_id: 'hotelId',
name: 'name',
rooms: ['RoomRateDetailsList/RoomRateDetails', {
rates: ['RateInfos/RateInfo', {
currency: 'ChargeableRateInfo/#currencyCode',
non_refundable: 'boolean(nonRefundable = "true")',
price: 'number(ChargeableRateInfo/#total)'
}],
room_name: 'roomDescription',
room_type_id: 'roomTypeCode'
}]
}],
session_id: '/HotelListResponse/customerSessionId'
}
const result = transform(xml, template)
Thanks #tuananh
Is it possible to use attr of a tag as key or value? Some of my data value i have in attr like <name type="sub-name">Hotel name</name> or <hotelName id='12345'>Hotel name </name> it is possible to do in camaro?
And I have doesn't pure data like <name type="sub-name">Hotel <nameHotel>Hilton</nameHotel></name> It is possible to remove tag nameHotel and save it as {name: 'Hotel Hilton'}
Thank you very much
Related
I need to create some JSON files for exporting data from a Django system to Google Big Query.
The problem is that Google BQ imposes some characteristics in the JSON file, for example, that each object must be in a different line.
json.dumps writes a stringified version of the JSON, so it is not useful for me.
Django serializes writes better JSON, but it put all in one line. All the information I found about pretty-printing is about json.dumps, which I cannot use.
I will like to know if anyone knows a way to create a JSON file in the format required by Big Query.
Example:
JSONSerializer = serializers.get_serializer("json")
json_serializer = JSONSerializer()
data_objects = DataObject.objects.all()
with open("dataobjects.json", "w") as out:
json_serializer.serialize(data_objects, stream=out)
json.dumps is OK. You have to use indent like this.
import json
myjson = '{"latitude":48.858093,"longitude":2.294694}'
mydata = json.loads(myjson)
print(json.dumps(mydata, indent=4, sort_keys=True))
Output:
{
"latitude": 48.858093,
"longitude": 2.294694
}
I want to convert the following line of a file into JSON, I want to save that into an mongoose schema.
>HWI-ST700660_96:2:1101:1455:2154#5#0/1
GAA…..GAATG
Should be:
{“>HWI-ST700660_96:2:1101:1455:2154#5#0/1”: “GAA…..GAATG”}
I have tried several options, one sample below, but no success, any suggestion?
const parser = require("csv-parse/lib/sync");//import parser
const fs = require("fs");//import file reader
const path = require("path");//for join paths
const sourceData = fs.readFileSync(path.join(__dirname, "Reads.txt"), "utf8");//read the file, locally stored
console.log(sourceData);//print out for checking
const documents = parser(sourceData);//parsing, it works for other situations I have tested, in a column like data
console.log(documents);//printing out
This code give me an output as following:
[ [ '>HWI-ST700660_96:2:1101:1455:2154#5#0/1' ],
[ 'GAATGGAATGAAATGGATAGGAATGGAATGGAATGGAATGGATTGGAATGGATTAGAATGGATTGGAATGGAATGAAATTAATTTGATTGGAATGGAATG' ],...
Similar question: fasta file reading python
Because you are using the default config of the parser, it does simply output arrays of arrays in that configuration.
If you want to receive objects you will need to give the parser some options (columns) first. Take a look at the doc.
When using the sync parsing mode (like you are using) you can provide options like this:
const documents = parse(sourceData, {columns: true})
columns:true will infer the column names from the first line of the input csv.
So i tried anything to get a specific key:value. But it seems that i cant find a solution for this. Any help?
{
"contracts":[
{
"contractid":"BemDRHtv17",
"cid":"",
"category":"WORK",
"mainCategory":"Grundeinkommen",
"configured":false,
"customMainCategory":null,
"customSubCategory":null,
"customContractPartner":null,
"amount":209200,
"interval":"MONTHLY",
"runTime":null,
"periodOfNotice":null,
"cancelationAlert":null,
"extensionPeriod":null,
"contractPartner":{
"creditorId":null,
"name":null,
"__typename":"ContractPartner"
},
"__typename":"Contract"
},
P.s i'm trying to access the key and value of mainCategory / search for its specific values.
You can access certain key using the following approach
using libraries like rjson , jsonlite
calling the json data into R using fromJSON() function. Lets say you saved the data like this
library(jsonlite)
jsonData = fromJSON("PATH") #Link to the file or html link
Now you can parse what you what using this approach and save the result in a matrix (or whatever data class you want to use)
variable <- as.matrix(jsonData$contracts$mainCategory)
Rather than converting your whole JSON to an R object, you can use library(jqr) to access specific elements of raw JSON
library(jqr)
jq(js, ".contracts[].mainCategory")
# "Grundeinkommen"
Data
js <- '{
"contracts":[
{
"contractid":"BemDRHtv17",
"cid":"",
"category":"WORK",
"mainCategory":"Grundeinkommen",
"configured":false,
"customMainCategory":null,
"customSubCategory":null,
"customContractPartner":null,
"amount":209200,
"interval":"MONTHLY",
"runTime":null,
"periodOfNotice":null,
"cancelationAlert":null,
"extensionPeriod":null,
"contractPartner":{
"creditorId":null,
"name":null,
"__typename":"ContractPartner"
},
"__typename":"Contract"
}
]}
'
Hi I'm new to Backbone but have to solve a problem.
There is a working webapp - online shop. On the CartView browsers gets a JSON file from /lampy/api/filter.
I need to save it somehow into model because I need some value from it?
How should I do this?
Thanks in advance.
Let's say your JSON object looks like:
{
item1: 'some string',
item2: 'another string',
item3: 'and another'
}
When you say:
to save it somehow into model
I'm assuming you mean instantiate a model with the values of your JSON object. To do this you can simply do:
var myModel = new Backbone.Model(yourJsonObject)
This will just be a plain Backbone Model without any Url so you won't be able to interact with the server with it, but as you said you need some value from it so you can use:
myModel.get('item1') // returns 'some string'
I am referring to this example: http://bl.ocks.org/mbostock/1153292 ,
specifically to these beautiful lines of code:
var links = [
{source: "Microsoft", target: "Amazon", type: "licensing"},
...
{source: "Nokia", target: "Qualcomm", type: "suit"}
];
var nodes = {};
// Compute the distinct nodes from the links.
links.forEach(function(link) {
link.source = nodes[link.source] || (nodes[link.source] = {name: link.source});
link.target = nodes[link.target] || (nodes[link.target] = {name: link.target});
});
I'd like to know what data format 'var links' here has, and how I would parse the following java string (with essentially the same content)
String someLinks = "[{source: \"Microsoft\", target: ... }, ...]"
into something equal to the javascript 'var links' above, from which I can then create the graph.
I've tried
var links = JSON.parse( string );
But that doesn't do the job... 'var links' doesn't seem to be JSON?
EDIT: Or should I use a different java format, e.g. some kind of Array? (Doesn't have to be a String)
Your value in String someLinks is no valid JSON. It lacks the quotes on the attribute names. JSON is more pedantic than javascript itself.
JSON.parse('[{"source":"A"},{"source":"B"}]')
will work (note the " around source.
Also you should avoid writing "protocols" by hand. If you need to build a JSON string in javascript, you can use
JSON.stringify([{source:'A'},{source:'B'}])
Note that JSON e.g. also escapes / for security reasons. An oversight like this can quickly end in errors or worse.
I believe this is JSON in a string. Since JSON is simply JavaScript, you might be able to simply "run" the code to create the corresponding data structure. So try
var links = eval(someLinks)
Even though I have the suspicion if we take a step back, things might be handled differently and you wouldn't need eval in the first place (since it's generally considered bad style)