Create a dynamically populated table from JSON API data - json

I'm trying to populate a table with the output of several API sources having extracted several identical variables from each source. So far I have most of the pieces I need worked out, such as how to retrieve the data, create the table and import the data in order to create the table using an existing framework. What I don't have is a means of formatting the API data to meet the framework's requirement such as..
Required format:
[
{id:1, name:"Billy Bob", age:"12", gender:"male", height:1, col:"red", dob:"", cheese:1},
{id:2, name:"Mary May", age:"1", gender:"female", height:2, col:"blue", dob:"14/05/1982", cheese:true},
{id:3, name:"Christine Lobowski", age:"42", height:0, col:"green", dob:"22/05/1982", cheese:"true"},
]
Create a table from the formatted data...
<div id="example-table"></div>
$("#example-table").tablear({
columns:[
{title:"Name", field:"name"},
{title:"Age", field:"age"},
{title:"Gender", field:"gender"},
{title:"Height", field:"height"},
{title:"Favourite Color", field:"col"},
{title:"Date Of Birth", field:"dob"},
{title:"Cheese Preference", field:"cheese"},
],
});

Whenever you get json as response you need to parse the json and then play with that.
Example:
if(response)
{
var myObj = parse.JSON(response) // this will give you the javascript
//object
}
There is also another option of using jquery datatable using server side processing. This automatically creates a table for you.
Jquery Datatable

Related

Creating a Quasar Data Table with JSON Data Array

I am trying to use the JSON array to create a quasar table but I am not able to link the array data with the table data.
I have stored the data from my database into an array called "scanResults" and the result is as shown:
console.log(scanResults);
Results of scanResults
Among the three arrays (Items, Count, and ScannedCount), I only want the data from Items. Is there a way I could put the data from scanResults into tableData under data() in the section such that I will be able to display the data into the quasar table?
tableData: [
{
name: 'xxx',
frequency: 'xxx',
prescription: 'xxx',
time: 'xxx'
}
]
<q-table
title="Reminders"
no-data-label="You do not have any reminders yet!"
align="center"
:columns="columns"
:data="tableData"
row-key="name"
selection="single"
:selected.sync="selected"
/>
If I am reading your scanResults correctly, you can set
tableData = scanResults.0[1]
to put the contents of the second place (the data) of the first array ("0") into tableData.
Did you define the columns array? The bindings must include both arrays so that QTable knows which data to put in which column. See the script tab of the "basic" section at https://quasar.dev/vue-components/table .

Insert JSON to Hadoop using Spark (Java)

I'm very new in Hadoop,
I'm using Spark with Java.
I have dynamic JSON, exmaple:
{
"sourceCode":"1234",
"uuid":"df123-....",
"title":"my title"
}{
"myMetaDataEvent": {
"date":"10/10/2010",
},
"myDataEvent": {
"field1": {
"field1Format":"fieldFormat",
"type":"Text",
"value":"field text"
}
}
}
Sometimes I can see only field1 and sometimes I can see field1...field50
And maybe the user can add fields/remove fields from this JSON.
I want to insert this dynamic JSON to hadoop (to hive table) from Spark Java code,
How can I do it?
I want that the user can after make HIVE query, i.e: select * from MyTable where type="Text
I have around 100B JSON records per day that I need to insert to Hadoop,
So what is the recommanded way to do that?
*I'm looked on the following: SO Question but this is known JSON scheme where it isnt my case.
Thanks
I had encountered kind of similar problem, I was able to resolve my problem using this. ( So this might help if you create the schema before you parse the json ).
For a field having a string data type you could create the schema :-
StructField field = DataTypes.createStructField(<name of the field>, DataTypes.StringType, true);
For a field having a int data type you could create the schema :-
StructField field = DataTypes.createStructField(<name of the field>, DataTypes.IntegerType, true);
After you have added all the fields in a List<StructField>,
Eg:-
List<StructField> innerField = new ArrayList<StructField>();
.... Field adding logic ....
Eg:-
innerField.add(field1);
innerField.add(field2);
// One instance can come, or multiple instance of value comes in an array, then it needs to be put in Array Type.
ArrayType getArrayInnerType = DataTypes.createArrayType(DataTypes.createStructType(innerField));
StructField getArrayField = DataTypes.createStructField(<name of field>, getArrayInnerType,true);
You can then create the schema :-
StructType structuredSchema = DataTypes.createStructType(getArrayField);
Then I read the json using the schema generated using the Dataset API.
Dataset<Row> dataRead = sqlContext.read().schema(structuredSchema).json(fileName);

Best way to handle data list of REST web service with foreign key(one to many)

I am going to implement the REST base CRUD modal in my my app.I wan to display the list of product data with edit and delete link
Product
id, title, unit_id, product_type_id, currency_id,price
Q1: what should be json response look like?
There are two formats comes in my mind to place the data in Json as a response of REST Get call
[
{
id:1,
title:"T-Shirt",
unit_id:20,
unit_title: "abc"
product_type_id:30,
product_type_title:"xyz"
currency_id: 10,
currency_name: "USD"
min_price:20
},
{...}
]
and the another one is
[
{
id:1,
title:"T-Shirt",
unit: {
id: 20,
title: "abc"
},
product_type: {
id: 30,
title: "xyz"
},
currency_id: {
id:10,
name: "USD"
},
min_price:20
},
{...}
]
what is the better and standard way to handle the above scenario?
Furthermore, let suppose I have 10 more properties in product table which will never display on list page. but i needed it when user going to edit the specific item.
Q2: Should I the load all data once at the time of displaying product list and pass the data to edit component.
or
Load only the needed propeties of product table and pass the id to produt edit component and a new REST GET call with id to get the properties of product.
I am using React + Redux for my front end
Typically, you would create additional methods for API consumers to retrieve the values that populate the lists of currency, product_type and unit when editing in a UI.
I wouldn't return more data than necessary for an individual Product object.

Edit Web2py json output for Google Chart

I have managed to get the google charts plugin (http://www.web2pyslices.com/slice/show/1721/google-charts-plugin) to work with my web2py application. Using the JSON example data (data hard coded into default.py).
I am struggling with using my own data. The chart does not work with my JSON data which returns the information as:
{"data": [["2014-03-28", 1000], ["2014-03-25", 1100]]}
When I hardcode the data with the titles, the chart works:
data = [['Date','Sales'],["2014-03-28",1000],["2014-03-25",1100]]
This returns JSON as:
{"data": [["Date", "Sales"], ["2014-03-28", 1000], ["2014-03-25", 1100]]}
The code for this is:
def return_data():
data = [['Date','Sales'],["2014-03-28",1000],["2014-03-25",1100]]
return dict(data=data)
Below is the code I am using in default.py to return the information from the database, the query works, it's the chart that doesn't!:
def return_data():
sales = db().select(db.sales.quantity, db.sales.date)
data = [[row.date,row.quantity] for row in sales]
return dict(data=data)
Somehow, I think I need to add the 'date' and 'sales' labels to the start of the json data but I have not managed to do this - I think I need to do some sort of encode? - do I need to use the simplejson or can this be done without??
Many thanks

Iterating through couchbase keys without a view

In couchbase, I was wondering if there was a way - WITHOUT using a view - to iterate through database keys. The admin interface appears to do this, but maybe its doing something special. What I'd like to is make a call like this to retrieve an array of keys:
$result = $cb->get("KEY_ALBERT", "KEY_FRED");
having the result be an array [KEY_ALEX, KEY_BOB, KEY_DOGBERT]
Again, I don't want to use a view unless there's no alternative. Doesn't look like its possible, but since the "view documents" in the admin appears to do this, I thought i'd double-check. I'm using the php interface if that matters.
Based on your comments, the only way is to create a simple view that emit only the id as par of the key:
function(doc, meta) {
emit( meta.id );
}
With this view you will be able to create query with the various options you need :
- pagination, range, ...
Note: you talk about the Administration Console, the console use an "internal view" that is similar to what I have written above (but not optimized)
I don't know about how couchbase admin works, but there are two options. First option is to store your docs as linked list, one doc have property (key) that points to another doc.
docs = [
{
id: "doc_C",
data: "somedata",
prev: "doc_B",
next: "doc_D"
},
{
id: "doc_D",
data: "somedata",
prev: "doc_C",
next: "doc_E"
}
]
The second approach is to use sequential id. You should have one doc that contain sequence and increment it on each add. It would be something like this:
docs = [
{
id: "doc_1",
data: "somedata"
},
{
id: "doc_2",
data: "somedata"
}
...
]
In this way you can do "range requests". To do this you form array of keys on server side:
[doc_1, doc_2 .... doc_N]and execute multiget query. Here is also a link to another example
The couchbase PHP sdk does support multiget requests. For a list of keys it will return an array of documents.
getMulti(array $ids, array $cas, int $flags) : array
http://www.couchbase.com/autodocs/couchbase-php-client-1.1.5/classes/Couchbase.html#method_getMulti