I have read plenty of resources and questions here regarding nested JSON but none are asking the exact same.
I am trying to use IBM Instana to retrieve the page load metrics what I plan to load into a JQUERY DataTables table.
The nested JSON I get from Instana:
{
"items" : [ {
"name" : "/Content/Search.htm",
"earliestTimestamp" : 1674432701496,
"cursor" : {
"#class" : ".IngestionOffsetCursor",
"ingestionTime" : 1674509519588,
"offset" : 1
},
"metrics" : {
"pageLoads.sum" : [ [ 1674511200000, 79.0 ] ]
}
}, {
"name" : "/Content/Home.htm",
"earliestTimestamp" : 1674435256403,
"cursor" : {
"#class" : ".IngestionOffsetCursor",
"ingestionTime" : 1674509519588,
"offset" : 2
},
"metrics" : {
"pageLoads.sum" : [ [ 1674511200000, 45.0 ] ]
}
} ],
"canLoadMore" : false,
"totalHits" : 2,
"totalRepresentedItemCount" : 2,
"totalRetainedItemCount" : 2,
"adjustedTimeframe" : {
"windowSize" : 169200000,
"to" : 1674511200000
}
}
and the code to load it and add it to a table in DataTables:
$(document).ready(function () {
$('#example').DataTable({
ajax: {
url: 'data/daily.txt',
dataSrc: 'items',
},
columns: [
{ data: 'cursor.offset' },
{ data: 'name' },
{ data: 'earliestTimestamp' },
{ data: 'metrics[0]' },
],
deferRender: true
});
});
While the whole thing is straightforward, even the nested cursor.offset with the dot, I fail to assign the two value in metrics\pageLoads.sum, especially that the nested pageLoads.sum also contains a dot and the values are in double square brackets without quotation marks. The two values (date and number of page loads) I would like to have in two separate columns, but as fail to load them at all, I went on to load them somehow into one cell, but even that does not work.
I have tried to add pageLoads.sum (then as 'metrics."pageLoads.sum"')in quotes, but didn't help, DataTables threw the error message "Requested unknown parameter 'metrics.....<and the variation I have tried>.
I have also tried:
{ data: 'metrics' }, this returns [object Object] in the html cells
{ data: 'metrics[0]' }, no error from DT, but cell remains empty
{ data: 'metrics[, ]' },no error from DT, but cell remains empty
Is there any way to access the two values within metrics\pageLoads.sum or do I have to change JSON structure before this can be done?
To split the contents of your nested arrays...
"pageLoads.sum" : [ [ 1674511200000, 45.0 ] ]
...into two columns, you can use the following:
columns: [
{ data: 'cursor.offset' },
{ data: 'name' },
{ data: 'earliestTimestamp' },
{
data: 'metrics.pageLoads\\.sum',
render: function (data, type, row) {
return data[0][0];
}
},
{
data: 'metrics.pageLoads\\.sum',
render: function (data, type, row) {
return data[0][1];
}
}
]
This uses the double-backslash I mentioned in a comment (ref.) and also two column renderers which handle the splitting of the nested arrays into 2 separate fields.
The end result looks like this:
Related
I am using datatables and here is how my data looks like:
{
"data": [{
"request": {
"responsible": "Pete Jackson",
"valuta": " EUR",
"customer": "Jim Manner",
"office": "123 Houston",
"UNID": "9D574D34B9140D3CC1257B8E002A487E"
}
}, {
"request": {
"responsible": "Jane Awesome",
"valuta": " EUR",
"customer": "Christian Slater",
"office": "503 New York",
"UNID": "2444DAA352E89A44C1257B8E002A487F"
}
}]
}
The datatables columns I have defined as followed:
'columns': [{
data: 'request.office',
'render': function(data) {
return data;
}
}, {
data: 'request.responsible',
'render': function(data) {
return data;
}
}, {
data: 'request.customer',
'render': function(data) {
return data;
}
}
]
Now I want to apply rowGrouping according the following example that I have found: http://live.datatables.net/migixiqi/1/edit
However it uses the rows for grouping and its seems the columns defined as dataSrc are considered as objects cause I get 'No group' returned as row group label.
How can I send in a real value as source in the rowgroup definition instead of the (expected) column value?
Perhaps I dont understand what you are hoping to do, but you can just pass the JSON path to the rowGroup.dataSrc exactly as you do with columns.data :
rowGroup: {
dataSrc: 'request.customer' //just a guess you want to group by custumer
},
http://jsfiddle.net/tgsz78jk/
PS: render() callbacks are unnecessary unless you actually need to do something special with a columns content, sort, filter or search behaviour.
I am trying to fill the table (Bootstrap datatable) with the data from remote JSON file.
JSON file is located at https://ba.ekapija.com/company/tender-winner-json/103510/pobede-na-tenderima?hash=28cd4a0e334aec8f84a94f30bb340e7f
And this is the function I use:
$(document).ready( function() {
$('#twodotsmediatable').dataTable( {
"data": "https://ba.ekapija.com/company/tender-winner-json/103510/pobede-na-tenderima?hash=28cd4a0e334aec8f84a94f30bb340e7f",
"columns": [
{ "data": "tender" },
{ "data": "url" },
{ "data": "date" },
{ "data": "amount" },
{ "data": "company" },
{ "data": "address" }
]
} );
$('.dataTables_length').addClass('bs-select');
});
I have also tried with:
"ajax": "https://ba.ekapija.com/company/tender-winner-json/103510/pobede-na-tenderima?hash=28cd4a0e334aec8f84a94f30bb340e7f"
But with no luck in both cases. Please help me to find where am I making mistake.
You should always use ajax.url :
$('#twodotsmediatable').dataTable( {
ajax: {
url: 'https://ba.ekapija....'
},
columns: [ .. ]
})
You cannot overcome request blocking in the browser, but you can get the desired JSON through a serverside proxy. If your server support PHP a proxy.php could look like this :
<?
echo file_get_contents($_GET['url']);
?>
Get data via proxy :
$('#twodotsmediatable').dataTable( {
ajax: {
url: 'proxy.php?url=https://ba.ekapija....',
dataSrc: function(d) {
return d[0];
}
},
columns: [ .. ]
})
NB: Use of dataSrc is needed since the JSON seem to be on the form [[{ item, item, .. } ]]
Using MongoDb how do you get back the date, and 3rd "obs" back from below?
{ "data" : [
{ "val" : [
{ "obs" : "2/3/2016"
},
{ "obs" : 41.8599992990494
},
{ "obs" : 41.3111999630928
},
{ "obs" : 5.048
}
]
},
{ "val" : [
{ "obs" : "2/4/2016"
},
{ "obs" : 39.394998550415
},
{ "obs" : 41.8486998975277
},
{ "obs" : NumberInt(0)
}
]
},
{ "val" : [
{ "obs" : "2/5/2016"
},
{ "obs" : NumberInt(0)
},
{ "obs" : 40.2090013027191
},
{ "obs" : 24.2410004138947
},
{ "obs" : 3.629
}
]
}
]
}
Started with this:
db.myColl.find({},{"_id":0, "data.val.obs": 1, })
would like:
["2/3/2016", 41.3111], ["2/4/2016", 41.8486]
Here is how you could do this in MongoDB starting from v 3.4
db.getCollection('test').aggregate([
{
$addFields: {
data: {
$map: {
input: "$data",
as: "item",
in: {$concatArrays: [{$slice: ['$$item.val', 1]}, {$slice: ['$$item.val', 2, 1]}]}
}
}
}
}
]);
So basically I'm using $addFields not to lose other properties of a root document (as you might need them). If you don't need them you can switch to $project.
Example: collection records look like this: {_id: ..., data: [...], data_2: [...]}.
If you run the query as is you'll have 'data' array filtered. But you'll still have data_2 unchanged. If you replace $addFields with $project you'll lose data_2. (or you need to explicitly tell mongo to keep it by passing data_2: true)
Then I'm mapping each element of 'data' array and assign the result back to 'data' array so in fact data property is overridden by filtered array.
To get 1st and 3rd elements I use $slice (each $slice returns an array of one document). And then I join them into a single array by $concatArrays.
I have been using the following MySQL command to construct a heatmap from log data. However, I have a new data set that is stored in a Mongo database and I need to run the same command.
select concat(a.packages '&' b.packages) "Concurrent Packages",
count(*) "Count"
from data a
cross join data b
where a.packages<b.packages and a.jobID=b.jobID
group by a.packages, b.packages
order by a.packages, b.packages;
Keep in mind that the tables a and b do not exist prior to the query. However, they are created from the packages column of the data table, which has jobID as the field which I want to check for matches. In other words if two packages are within the same job I want to add an entry to the concurrent usage count. How can I generate a similar query in Mongo?
This is not a "join" of different documents; it is an operation within one document, and can be done in MongoDB.
You have a SQL TABLE "data" like this:
JobID TEXT,
package TEXT
The best way to store this in MongoDB will be a collection called "data", containing one document per JobID that contains an array of packages:
{
_id: <JobID>,
packages: [
"packageA",
"packageB",
....
]
}
[ Note: you could also implement your data table as only one document in MongoDB, containing an array of jobs which contain each an array of packages. This is not recommended, because you might hit the 16MB document size limit and nested arrays are not (yet) well supported by different queries - if you want to use the data for other purposes as well ]
Now, how to get a result like this ?
{ pair: [ "packageA", "packageB" ], count: 20 },
{ pair: [ "packageA", "packageC" ], count: 11 },
...
As there is no built-in "cross join" of two arrays in MongoDB, you'll have to program it out in the map function of a mapReduce(), emitting each pair of packages as a key:
mapf = function () {
that = this;
this.packages.forEach( function( p1 ) {
that.packages.forEach( function( p2 ) {
if ( p1 < p2 ) {
key = { "pair": [ p1, p2 ] };
emit( key, 1 );
};
});
});
};
[ Note: this could be optimized, if the packages arrays were sorted ]
The reduce function is nothing more than summing up the counters for each key:
reducef = function( key, values ) {
count = 0;
values.forEach( function( value ) { count += value } );
return count;
};
So, for this example collection:
> db.data.find()
{ "_id" : "Job01", "packages" : [ "pA", "pB", "pC" ] }
{ "_id" : "Job02", "packages" : [ "pA", "pC" ] }
{ "_id" : "Job03", "packages" : [ "pA", "pB", "pD", "pE" ] }
we get the following result:
> db.data.mapReduce(
... mapf,
... reducef,
... { out: 'pairs' }
... );
{
"result" : "pairs",
"timeMillis" : 443,
"counts" : {
"input" : 3,
"emit" : 10,
"reduce" : 2,
"output" : 8
},
"ok" : 1,
}
> db.pairs.find()
{ "_id" : { "pair" : [ "pA", "pB" ] }, "value" : 2 }
{ "_id" : { "pair" : [ "pA", "pC" ] }, "value" : 2 }
{ "_id" : { "pair" : [ "pA", "pD" ] }, "value" : 1 }
{ "_id" : { "pair" : [ "pA", "pE" ] }, "value" : 1 }
{ "_id" : { "pair" : [ "pB", "pC" ] }, "value" : 1 }
{ "_id" : { "pair" : [ "pB", "pD" ] }, "value" : 1 }
{ "_id" : { "pair" : [ "pB", "pE" ] }, "value" : 1 }
{ "_id" : { "pair" : [ "pD", "pE" ] }, "value" : 1 }
For more information on mapReduce consult: http://docs.mongodb.org/manual/reference/method/db.collection.mapReduce/ and http://docs.mongodb.org/manual/applications/map-reduce/
You can't. Mongo doesn't do joins. Switching from SQL to Mongo is a lot more involved than migrating your queries.
Typically, you would include all the pertinent information in the same record (rather than normalize the information and select it with a join). Denormalize!
I want to load nested JSON in a Store, but the data is not correctly mapped. No problem with a single regModel but I canĀ“t get the associations to work.
// JSON in test.json
{"message" : {
"returnCodes": [
{
"value": "0",
"code": "200",
"description": "OK"
},
{
"value": "0",
"code": "200",
"description": "OK"
}
]
}}
// Model with associations
Ext.regModel("ReturnCode", {
fields : [{
name : "value",
type : "string"
}, {
name : "code",
type : "string"
}, {
name : "description",
type : "string"
}],
belongsTo: "Message"
});
Ext.regModel("Message", {
hasMany: {
model : "ReturnCode",
name : "returnCodes"
}
});
// Store
var jobStore = new Ext.data.Store({
model : 'Message',
autoLoad: true,
proxy : {
type : 'ajax',
url: 'test.json',
reader : {
type : 'json',
root : 'message.returnCodes'
}
}});
// List
var list = Ext.extend( Ext.List, {
fullscreen : true,
store : jobStore,
grouped : false,
itemTpl : '<div>{code}</div>' // no output
});
When I look into the store every data is stored in the raw section of the store object but nothing in the data section. In the list for both returnCode Objects a listitem is created but they are not filled with data, because the mapping didn't succeed -> itemTpl gets no data.
Try declaring Message Model first, then add associationKey:'returnCodes' in hasMany{} within the Message Model. Also change the root of the reader to message.
This reference could also be of use to you.