I am trying to get MySQL database data into json format. My file needs to be in the format of nested parents and children. (see snippet) This is because I am trying to use a d3 data visualization (collapsible tree layout) that requires that the json data be formatted in a certain way. Below I have attached a snippet of what I the json file to look like. I am having some issues trying to write nested sql statements to do this. I have also attached a visual of my database to see what I am dealing with. Any advice will help greatly. Thanks!
The json format I am trying to get is something like this:
{
"name": "Projects",
"children": [
{
"name":"category_name#1", "description":"category",
"children": [
{
"name":"sub_category_name1",
"description":"category description text here",
"children":[
{"name": "project1",
"description":"project 1 text goes here",
"children":[
{"name":"mike", "email":"xxx#mail.com"},
{"name":"dan", "email":"xxx#gmail.com"}
]
},
{
"name": "project2",
"description":"project 2 text goes here",
"children":[
{"name":"steve", "email":"xxx#mail.com"},
{"name":"chris", "email":"xxx#gmail.com"}
]
}
]
},
{
"name": "sub_category_name2",
"description":"sub category description text goes here..",
"children": [
{"name": "project3",
"description":"project3 text goes here ",
"children":[
{"name":"Alex", "email":"xxx#gmail.com"}
]
}
]
}
]
},
.
. //more categories with children of subcategories and sub_cat children of projects
.
}
I've had success with this tutorial in the past:
http://www.d3noob.org/2013/02/using-mysql-database-as-source-of-data.html
In short what you'll do is create a php file that connects to MySql, performs a query and translates the results into json. You'll then use this file in place of the data source in your d3 code.
Related
I have a nested JSON in a field that contains multiple important keys that I would like to retrieve as an array:
{
"tasks": [
{
"id": "task_1",
"name": "task_1_name",
"assignees": [
{
"id": "assignee_1",
"name": "assignee_1_name"
}
]
},
{
"id": "task_2",
"name": "task_2_name",
"assignees": [
{
"id": "assignee_2",
"name": "assignee_2_name"
},
{
"id": "assignee_3",
"name": "assignee_3_name"
}
]}]}
All the queries that I've tried so far fx ( $.tasks.*.assignees..id) and many others have returned
[
"assignee_1",
"assignee_2",
"assignee_3"
]
But what I need is:
[
["assignee_1"],
["assignee_2", "assignee_3"]
]
Is it possible to do with JSONPath or any script inside of it, without involving 3rd party tools?
The problem you're facing is that tasks and assignees are arrays. You need to use [*] instead of .* to get the items in the array. So your path should look like
$.tasks[*].assignees[*].id
You can try it at https://json-everything.net/json-path.
NOTE The output from my site will give you both the value and its location within the original document.
Edit
(I didn't read the whole thing :) )
You're not going to be able to get
[
["assignee_1"],
["assignee_2", "assignee_3"]
]
because, as #Tomalak mentioned, JSON Path is a query language. It's going to remove all structure and return only values.
I have the following JSON:
[
{
"date": "29/11/2021",
"Name": "jack",
},
{
"date": "30/11/2021",
"Name": "Adam",
},
"date": "27/11/2021",
"Name": "james",
}
]
Using Talend, I wanna add 2 lines to have something like:
[
{
"company": "AMA",
"service": "BI",
"date": "29/11/2021",
"Name": "jack",
},
{
"company": "AMA",
"service": "BI",
"date": "30/11/2021",
"Name": "Adam",
},
"company": "AMA",
"service": "BI",
"date": "27/11/2021",
"Name": "james",
}
]
Currently, I use 3 components (tJSONDocOpen, tFixedFlowInput, tJSONDocOutput) but I can't have the right configuration of components in order to get the job done !
If you are not comfortable with json .
Just do these steps :
In the metaData just create a FileJson like this then paste it in your job as a tFileInputJson
Your job design and mapping would be
In your tFileOutputJson don't forget to change in the name of the data block "Data" with ""
What you need to do there according to the Talend practices is read your JSON. Then extract each object of it, add your properties and finally rebuild your JSON in a file.
An efficient way to do this is using tMap componenent like this.
The first tFileInputJSON will have to specify what properties it has to read from the JSON by setting your 2 objects in the mapping field.
Then the tMap will simply add 2 columns to your main stream, here is an example with hard coded string values. Depending on you needs, this component will also offer you the possibility to assign dynamic data to your 2 new columns, it's a powerful tool for manipulating the structure of a data stream.
You will find more infos about this component in the official documentation : https://help.talend.com/r/en-US/7.3/tmap/tmap; especially the "tMap scenarios" part.
Note
Instead of using the tMap, if you are comfortable with Java, you can use a tjavaRow instead. Using this, you can setup your 2 new columns with whatever java code you want to put as long as you have defined the output schema of the component.
output_row.Name = input_row.Name;
output_row.date = input_row.date;
output_row.company = "AMA";
output_row.service = "BI";
I have a large JSON file that looks similar to the code below. Is there anyway I can iterate through each object, look for the field "element_type" (it is not present in all objects in the file if that matters) and extract or write each object with the same element type to a file? For example each user would end up in a file called user.json and each book in a file called book.json?
I thought about using javascript but to my knowledge js can't write to files, I also tried to do it using linux command line tools by removing all new lines, then inserting a new line after each "}," and then iterating through each line to find the element type and write it to a file. This worked for most of the data; however, where there were objects like the "problem_type" below, it inserted a new line in the middle of the data due to the nested json in the "times" element. I've run out of ideas at this point.
{
"data": [
{
"element_type": "user",
"first": "John",
"last": "Doe"
},
{
"element_type": "user",
"first": "Lucy",
"last": "Ball"
},
{
"element_type": "book",
"name": "someBook",
"barcode": "111111"
},
{
"element_type": "book",
"name": "bookTwo",
"barcode": "111111"
},
{
"element_type": "problem_type",
"name": "problem object",
"times": "[{\"start\": \"1230\", \"end\": \"1345\", \"day\": \"T\"}, {\"start\": \"1230\", \"end\": \"1345\", \"day\": \"R\"}]"
}
]
}
I would recommend Java for this purpose. It sounds like you're running on Linux so it should be a good fit.
You'll have no problems writing to files. And you can use a library like this - http://json-lib.sourceforge.net/ - to gain access to things like JSONArray and JSONObject. Which you can easily use to iterate through the data in your JSON request, and check what's in "element_type" and write to a file accordingly.
I am a real dumb with HTML and JavaScript, so please excuse any dumbness.
I am using D3 Tree Diagram, but I need to load a JSON file instead of writing it inside the JS script, which the name of the file to be loaded will be chose by the user in a select tag. Here's the D3 code
First, how can I load/read a JSON file, lets say exampleNodes.json,
And then, how can I pass the name of the selected select tag so that it reads the appropriate JSON?
Thanks for your patience, and help. Thank you.
in code
var treeData = [
{
"name": "Top Level",
"parent": "null",
"children": [
{
"name": "Level 2: A",
"parent": "Top Level",
"children": [
{
"name": "Son of A",
"parent": "Level 2: A"
},
{
"name": "Daughter of A",
"parent": "Level 2: A"
}
]
},
{
"name": "Level 2: B",
"parent": "Top Level"
}
]
}
];
you have to save it on data.json file like
{
"treeData" : [ ... your data array ...]
}
after that in d3.json() function you will receive this object
d3.json("data.json",function(json){
// do your coding
// or all code put inside one function and call it after data loaded
});
if you are using google chrome than it will gave you error on data reading from json because security Google Chrome not allow read files from file system you can get data in Firefox. to make it run upload your code on some local server. i.e in WampServer or Apache tomcat etc.
I'm Working on AngularJS.
In this part of the project my goal is to obtain a JSON structure after filling a form with some particulars values.
Here's the fiddle of my simple form: Fiddle
With the form I will do a query to KairosDB, that is my NoSql Database, I will query data from it by a JSON object. The form is structured in this way:
a Name
a certain Number of Tags, with Tag Id ("ch" for example) and tag value ("932" for example)
a certain Number of Aggregators to manipulate data coming from DB
Start Timestamp and End Timestamp (now they are static and only included in the final JSON Object)
After filling this form, with my code I'll obtain for example this JSON object:
{
"metrics": [
{
"tags": [
{
"id": "ch",
"value": "932"
},
{
"id": "ch",
"value": "931"
}
],
"aggregators": {
"name": "sum",
"sampling": [
{
"value": "1",
"unit": "milliseconds",
"type": "SUM"
}
]
}
}
],
"cache_time": 0,
"start_absolute": 123,
"end_absolute": 1234
}
Unfortunately, KairosDB accepts a different structure, and as you could see, Tag id "ch" doesn't hase an "id" string before, or for example, Tag values coming from the same tag id are grouped together
{
"metrics": [
{
"tags": {
"ch": [
"932",
"931"
]
},
"name": "AIENR",
"aggregators": [
{
"name": "sum",
"sampling": {
"value": "1",
"unit": "milliseconds"
}
}
]
}
],
"cache_time": 0,
"start_absolute": 1367359200000,
"end_absolute": 1386025200000
}
My question is: Is there a way to obtain the JSON structure like the one accepted by Kairos DB with an Angular JS form?. Thanks to everyone.
I've seen this topic as the one more similar to mine but it isn't in AngularJS.
Personally, I'd do the refactoring work in the backend - Have what ever server interfaces sends and receives data do the manipulation - Otherwise you'll end up needing to refactor your data inside Angular anywhere you want to use that dataset.
Where as doing it in the backend would put it in a single access point.
Of course, you could do it in Angular, just replace userString in the submitData method with a copy of the array and replace the tags section with data in the new format, and likewise refactor the returned result to the correct format when you get a reply.