How to output response of type atom/xml feed (from arxiv call) into Jquery DataTable?
I have the datatable working for a simple json from Ajax call to flask server example.
When i try to do it with the xml from an arxiv api response, i cant seem to get it to display in the datatable (though i can just print the raw xml using <pre lang="xml" > or json).
I also tried to convert to json first via python dictionary, but still couldnt get it formatted into datatable as im unsure how to access the properties properly in the Ajax call when theyre deeper than the first level as in the basic example linked.
The HTML in template:
<table id="arxivtable" class="display" style="width:100%">
<thead>
<tr>
<th>title</th>
<th>id</th>
<th>link</th>
<th>author</th>
<th>published</th>
</tr>
</thead>
</table>
I tried via xml :
$('#arxivtable').DataTable({
"ajax": {
// "url": "static/objects2.txt", // This works for the static file
"url": "/get_arxivpapers", // This now works too thanks to #kthorngren
"dataType": "xml",
"type":"GET",
"dataSrc": "{{name}}",
"contentType":"application/atom+xml"
},
"columns": [
{"data": "title"},
{
"data": "link",
"render": function(data, type, row, meta){
if(type === 'display'){
data = '' + data + '';
}
return data;
}
},
{ "data": "id" },
{ "data": "link" },
{ "data": "author" },
{ "data": "journal" },
{ "data": "published" },
{ "data": "summary" }
]
});
JSON from AJAX call:
{
"feed": {
"#xmlns": "http://www.w3.org/2005/Atom",
"link": {
"#href": "http://arxiv.org/api/query?search_query%3Dall%3Aeinstein%26id_list%3D%26start%3D0%26max_results%3D2",
"#rel": "self",
"#type": "application/atom+xml"
},
"title": {
"#type": "html",
"#text": "ArXiv Query: search_query=all:einstein&id_list=&start=0&max_results=2"
},
"id": "http://arxiv.org/api/vehKAQR+bheXtHwJw3qx/OG/XXw",
"updated": "2022-06-14T00:00:00-04:00",
"opensearch:totalResults": {
"#xmlns:opensearch": "http://a9.com/-/spec/opensearch/1.1/",
"#text": "36970"
},
"opensearch:startIndex": {
"#xmlns:opensearch": "http://a9.com/-/spec/opensearch/1.1/",
"#text": "0"
},
"opensearch:itemsPerPage": {
"#xmlns:opensearch": "http://a9.com/-/spec/opensearch/1.1/",
"#text": "2"
},
"entry": [
{
"id": "http://arxiv.org/abs/1801.05533v2",
"updated": "2018-11-22T14:04:43Z",
"published": "2018-01-17T03:05:51Z",
"title": "Einstein-Weyl structures on almost cosymplectic manifolds",
"summary": "",
"author": {
"name": "Xiaomin Chen"
},
"arxiv:comment": {
"#xmlns:arxiv": "http://arxiv.org/schemas/atom",
"#text": "accepted by Periodica Mathematica Hungarica, 14 pages, no figures"
},
"link": [
{
"#href": "http://arxiv.org/abs/1801.05533v2",
"#rel": "alternate",
"#type": "text/html"
},
{
"#title": "pdf",
"#href": "http://arxiv.org/pdf/1801.05533v2",
"#rel": "related",
"#type": "application/pdf"
}
],
"arxiv:primary_category": {
"#xmlns:arxiv": "http://arxiv.org/schemas/atom",
"#term": "math.DG",
"#scheme": "http://arxiv.org/schemas/atom"
},
"category": [
{
"#term": "math.DG",
"#scheme": "http://arxiv.org/schemas/atom"
},
{
"#term": "53D10, 53D15",
"#scheme": "http://arxiv.org/schemas/atom"
}
]
},
{
"id": "http://arxiv.org/abs/0802.2137v3",
"updated": "2008-04-01T04:36:21Z",
"published": "2008-02-15T04:40:56Z",
"title": "",
"summary": ".",
"author": {
"name": ""
},
"arxiv:comment": {
"#xmlns:arxiv": "http://arxiv.org/schemas/atom",
"#text": "18 pages, added Theorem 5"
},
"link": [
{
"#href": "http://arxiv.org/abs/0802.2137v3",
"#rel": "alternate",
"#type": "text/html"
},
{
"#title": "pdf",
"#href": "http://arxiv.org/pdf/0802.2137v3",
"#rel": "related",
"#type": "application/pdf"
}
],
"arxiv:primary_category": {
"#xmlns:arxiv": "http://arxiv.org/schemas/atom",
"#term": "math.DG",
"#scheme": "http://arxiv.org/schemas/atom"
},
"category": [
{
"#term": "math.DG",
"#scheme": "http://arxiv.org/schemas/atom"
},
{
"#term": "53C30; 53C25",
"#scheme": "http://arxiv.org/schemas/atom"
}
]
}
]
}
}
Or the original atom/xml:
<feed xmlns="http://www.w3.org/2005/Atom">
<link href="http://arxiv.org/api/query?search_query%3Dall%3Aeinstein%26id_list%3D%26start%3D0%26max_results%3D2" rel="self" type="application/atom+xml">
<title type="html">ArXiv Query: search_query=all:einstein&id_list=&start=0&max_results=2</title>
<id>http://arxiv.org/api/vehKAQR+bheXtHwJw3qx/OG/XXw</id>
<updated>2022-06-14T00:00:00-04:00</updated>
<opensearch:totalresults xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">36970</opensearch:totalresults>
<opensearch:startindex xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">0</opensearch:startindex>
<opensearch:itemsperpage xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/">2</opensearch:itemsperpage>
<entry>
<id>http://arxiv.org/abs/1801.05533v2</id>
<updated>2018-11-22T14:04:43Z</updated>
<published>2018-01-17T03:05:51Z</published>
<title></title>
<summary>
</summary>
<author>
<name></name>
</author>
<arxiv:comment xmlns:arxiv="http://arxiv.org/schemas/atom">accepted by Periodica Mathematica Hungarica, 14 pages, no figures</arxiv:comment>
<link href="http://arxiv.org/abs/1801.05533v2" rel="alternate" type="text/html">
<link title="pdf" href="http://arxiv.org/pdf/1801.05533v2" rel="related" type="application/pdf">
<arxiv:primary_category xmlns:arxiv="http://arxiv.org/schemas/atom" term="math.DG" scheme="http://arxiv.org/schemas/atom">
<category term="math.DG" scheme="http://arxiv.org/schemas/atom">
<category term="53D10, 53D15" scheme="http://arxiv.org/schemas/atom">
</category></category></arxiv:primary_category></entry>
<entry>
<id>http://arxiv.org/abs/0802.2137v3</id>
<updated>2008-04-01T04:36:21Z</updated>
<published>2008-02-15T04:40:56Z</published>
<title></title>
<summary>
</summary>
<author>
<name></name>
</author>
<arxiv:comment xmlns:arxiv="http://arxiv.org/schemas/atom"></arxiv:comment>
<link href="http://arxiv.org/abs/0802.2137v3" rel="alternate" type="text/html">
<link title="pdf" href="http://arxiv.org/pdf/0802.2137v3" rel="related" type="application/pdf">
<arxiv:primary_category xmlns:arxiv="http://arxiv.org/schemas/atom" term="math.DG" scheme="http://arxiv.org/schemas/atom">
<category term="math.DG" scheme="http://arxiv.org/schemas/atom">
<category term="53C30; 53C25" scheme="http://arxiv.org/schemas/atom">
</category></category></arxiv:primary_category></entry>
</feed>
The End Point:
#app.route('/get_arxivpapers')
def getArxivPapers(name="einstein"):
max_results = 2
searchterm = name.replace("_", " ")
url = 'http://export.arxiv.org/api/query?search_query=all:' + searchterm + '&start=0&' + 'max_results='+ str(max_results)
data = urllib.request.urlopen(url)
# data_dict = xmltodict.parse(data)
# json_data = json.dumps(data_dict)
# print(json_data)
# return jsonify(json_data)
return data.read().decode('utf-8')
I will use your JSON source data instead of the XML, since that is easier to handle in DataTables.
Here is a basic demo, to start with, followed by some explanatory notes:
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Demo</title>
<script src="https://code.jquery.com/jquery-3.5.1.min.js"></script>
<script src="https://cdn.datatables.net/1.10.21/js/jquery.dataTables.min.js"></script>
<link rel="stylesheet" type="text/css" href="https://cdn.datatables.net/1.10.21/css/jquery.dataTables.min.css">
</head>
<body>
<div style="margin: 20px;">
<table id="arxivtable" class="display" style="width:100%">
<thead>
<tr>
<th>title</th>
<th>id</th>
<th>link</th>
<th>author</th>
<th>published</th>
<th>summary</th>
</tr>
</thead>
</table>
</div>
<script type="text/javascript">
$(document).ready(function(){
$('#arxivtable').DataTable({
"ajax": {
url: "YOUR_URL_GOES_HERE",
dataSrc: "feed.entry"
},
"columns": [
{"data": "title"},
{ "data": "id" },
{ "data": "link[].#href" },
{ "data": "author.name" },
{ "data": "published" },
{ "data": "summary" }
]
});
});
</script>
</body>
</html>
Notes
1 - Because you have provided hard-coded HTML column headers, you need to make sure the number of those headers matches the number of columns defined in the DataTable. Alternatively, you can remove the HTML <thead> section and use the DataTables columns.title option.
2 - Your Ajax JSON source data contains an array [ ... ]. DataTables needs to know where this array is located in your JSON response, as part of the Ajax handling option, so that it can iterate over that array. Each element in the array will be used to create a row of HTML table data. The ajax.dataSrc option therefore needs to be set accordingly:
dataSrc: "feed.entry"
Once you have set the above Ajax JSON starting point correctly, then you can use field names for each separate column data value - as shown below.
3 - The author JSON value is actually an object:
"author": {
"name": "Xiaomin Chen"
},
Therefore you need to drill down into that to get the field you want to show in the DataTable:
{ "data": "author.name" },
4 - I removed your column renderer function to keep my initial demo simple, but it can be used to access fields and sub-fields - and concatenate strings and other values as needed (as in your example in the question).
5 - The link JSON value is actually an array of objects. For my basic demo, I just accessed the final entry in that array, and then took the href field:
{ "data": "link[].#href" },
This may not be what you want. You may want to only choose links of a certain type, or choose all links, or something different.
This is where DataTables is limited in what it can handle. It cannot display arbitrary nested JSON values of this type (not surprisingly).
In such cases, you would need to re-structure the JSON, prior to sending it to DataTables - or restructure it in a dataSrc function inside DataTables itself:
"dataSrc": function ( json ) { ...transform and return your JSON here... }
6 - I was not sure what you wanted to display for { "data": "journal" }. I did not see anything called journal in the JSON.
7 - Note that all the source JSON data outside of the feed.entry array is also not available to DataTables. DataTables can only iterate over that outer array. Anything you may also need which is not in that outer array would need to be added to the array, to be accessible to DataTables.
See also Nested object data (arrays) and Nested object data (objects) for more related notes.
Related
I'm trying to display JSON data on my webpage by using the v-for function in Vue.js and Axios to get the data. Below is my code and an example of the JSON data i'm trying to use.
I have kept the JSON data URL out on purpose as it's private, which is why i've provided an example of the data structure.
I can print the entire data set to my page, as it appears below but if i use my code below to print specific parts of data, like the id or name, that's when i get nothing on the page.
<div id="root">
<p v-for="item in items">{{ item.name }}</p>
</div>
<script>
var app = new Vue({
el: '#root',
data: {
items: []
},
mounted() {
axios.get("...")
.then(response => {this.items = response.data.data})
}
});
</script>
JSON data example:
json
{
"current_page": 1,
"data": [
{
"id": "83",
"name": "Name1",
},
{
"id": "78",
"name": "Name2",
},
{
"id": "720",
"name": "Name3",
},
{
"id": "707",
"name": "Name4",
},
{
"id": "708",
"name": "Name5",
}
],
"from": 1,
"prev_page_url": null,
"to": 20,
"total": 42
}
looking at the code everything looks ok, but i have an example in the application i am working on and i've noticed that your items array should contain response.data.data.data wich is verbose but you can work on that later
I'm working on a chemicals database web application. Using DataTables 1.10.16 and CakePHP 3.5
Cake is producing a JSON feed. A sample is below:
[
{
"id": 1,
"app_id": "ID000001",
"name": "Chromium",
"ecs": [
{
"id": 1,
"value": "123-456-7"
},
{
"id": 32,
"value": "222-333-444"
},
],
"cas": [
{
"id": 1,
"value": "987-654-3"
}
]
},
]
For certain chemicals there are multiple EC (ecs in the JSON) and CAS Numbers (cas in the JSON).
I don't know if/how it's possible to get DataTables to loop through these objects, outputting them in the appropriate columns, with a separator such as a <br> tag.
My table markup looks like this:
<table id="substancesTable" class="table responsive display table-striped" cellspacing="0" width="100%">
<thead>
<tr>
<th>Application ID</th>
<th>EC Number</th>
<th>CAS Number</th>
<th>Chemical Name</th>
</tr>
</thead>
</table>
And in my Javascript I'm doing this:
$('#substancesTable').DataTable({
"processing": true,
"serverSide": true,
"searching": false,
"ajax": {
"url" : "/get-substances.json",
"method" : "POST",
"dataSrc":""
},
"columns": [
{ "data": "app_id", "name" : "app_id" },
{ "data": "ecs", "name" : "ec_number" },
{ "data": "cas", "name" : "cas_number" },
{ "data": "name", "name" : "name" },
]
});
This works in terms of populating the "Application ID" and "Chemical Name" columns - but that's because there is a simple 1:1 mapping in the JSON (no arrays/objects).
The output for the "EC Number" and "CAS Number" columns is just [object Object] and is repeated the number of objects there are. In the example above there are 2 EC Numbers for this chemical so the output under "EC Number" is [object Object],[object Object]
Can anyone assist with this? I'd like the output to be generated by looping through my JSON and introducing a break between each item, e.g.
123-456-7<br>222-333-444
You'll be needing a render function like this:
{
"data": "ecs",
"name": "ec_number",
"title":"EC Number",
"render": function(d,t,r){
return d.map(function(e) {
return e.value;
}).join("<br/>");
}
}
Working JSFiddle here.
I am using DataTables ajax to fill a table. I was following this example and it works if my json looks like their example, however my 'data' is a nested object in my json and then it is not able to fill the table because fails in getting the values from the json.
This json works:
{
"data": [
{"id": "myid1", "name": "name1"},
{"id": "myid2", "name": "name2"}
]
}
But my json looks like this, and it does not work
{
"result": {
"data": [
{"id": "myid1", "name": "name1"},
{"id": "myid2", "name": "name2"}
]
}
}
That's my html (which works with the first json):
<table id="myTable" class="tabletable-bordered dataTable">
<thead>
<tr>
<th>name</th>
<th>id</th>
</tr>
</thead>
<tbody>
<td></td>
<td></td>
</tbody>
</table>
And that's my js:
$('#myTable').DataTable( {
"ajax": "/names",
"processing": true,
"dataSrc" : "result.data",
"columns": [
{ "data": "name" },
{ "data": "id" }
]
} );
As you can see I try to access to my nested data by setting dataSrc to result.data, but nothing is shown in the table.
I cannot change the format of the json response, do you know if then is not possible to use DataTables ajax with this json? Thanks!
MY SOLUTION
Finally (before getting an answer to my post) I went with the following solution, I first make an ajax call and then on success I create my DataTable.
so I only changed my js to this one:
$.ajax({
type: "GET",
url: '/names',
dataType: "json",
contentType: "application/json; charset=utf-8",
success: function (data) {
var myData = data.result.data;
$('#myTable').DataTable({
"data": myData,
"columns": [
{ "data": "id" },
{ "data": "name"},
]
});
}
});
I had more complex processing requirements so I implemented my own ajax function but I think you can get away with this:
$('#myTable').DataTable( {
"processing": true,
"ajax": {
"url": "/names",
"dataSrc" : function(jsonObj) {
// Just being safe here
if(!jsonObj.results || !json.results.data) {
return [];
}
return jsonObj.results.data;
}
},
"columns": [
{ "data": "name" },
{ "data": "id" }
]
});
Here is the reference for this: https://datatables.net/reference/option/ajax
First time poster, thanks for the attention.
Using web api to generate object to be consumed by fusioncharts (Multiseries). Object is produced as array of 2 classes (ChartCategories and ChartSeries) output seems fine and is retrieved in angular controller as data. yet after building up $scope.categories and $scope.dataset using data, I am unable to generate the chart with error 'No data to display'.
Partial Html template for chart:
<div id = "Div1">
<fusioncharts
width="400"
height="200"
type="mscolumn2d"
chart="{{attrs}}"
categories="{{categories}}"
dataset="{{dataset}}"
></fusioncharts>
</div>
data retrieved from web api: (copied + pasted)
[
[
{
"category": [
{
"label": "7/18/2014 9:30:01 AM"
},
{
"label": "7/18/2014 9:40:00 AM"
},
{
"label": "7/18/2014 9:50:00 AM"
}
]
},
null,
null
],
[
null,
{
"seriesname": "Free_Memory",
"renderas": "Line",
"data": [
{
"value": "6632"
},
{
"value": "5136"
},
{
"value": "6376"
}
]
},
{
"seriesname": "Page_Life_Exp",
"renderas": "Line",
"data": [
{
"value": "48859"
},
{
"value": "49458"
},
{
"value": "50057"
}
]
}
]
]
and in angular, I set the $scope.categories and $scope.dataset like so: $scope.attr is hard coded for the time being.
$scope.categories = data[0][0];
$scope.dataset = data[1][1];
What is needed in order to generate the graph?
Refer to http://jsfiddle.net/ayanonly1/yh1cvjqw/
I think following changes will make the chart live.
$scope.categories = [data[0][0]];
$scope.dataset = data[1].slice(1);
got this json file:
[
{
"name": "paprika",
"imgSrc": "img/paprika.jpg"
},
{
"name": "kurkku",
"imgSrc": "img/kurkku.jpg"
},
{
"name": "porkkana",
"imgSrc": "img/porkkana.jpg"
},
{
"name": "lehtisalaatti",
"imgSrc": "img/lehtisalaatti.jpg"
},
{
"name": "parsakaali",
"imgSrc": "img/parsakaali.jpg"
},
{
"name": "sipula",
"imgSrc": "img/sipuli.jpg"
},
{
"name": "peruna",
"imgSrc": "img/peruna.jpg"
},
{
"name": "soijapapu",
"imgSrc": "img/soijapapu.jpg"
},
{
"name": "pinaatti",
"imgSrc": "img/pinaatti.jpg"
}
]
Which I successfully fetch in a factory:
factory('getJson', ['$resource', function($resource) {
return $resource('json/vegs.json', {}, {
query: {method:'GET', isArray:true}
});
}]);
in my Controller I can get the json's file content:
var vegs = getJson.query();
$scope.vegs = vegs;
console.log(vegs)
console.log(typeof vegs)
The weird part is the first console.log produces an array of objects, as expected.
The second console says it's an "object", and not an array.
I can get the .json content to my view using {{vegs}}, and I can use ng-repeat as well, tho in the controller I can't do vegs[0] or vegs.length. It comes out empty.
I'm breaking my head on this for over 3 hours now :)
This isn't an 'answer'. Just an observation on one part of your issue. (Sorry, can't comment yet...new to stackoverflow).
Just a note on your comment that "The second console says it's an "object", and not an array." Using typeof on an array will always return "object".
There are various (and debated, it seems) ways to test if it's an array--Array.isArray(obj) for example.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/isArray