Representing matrix in JSON - json
I am trying to have matrix style data in JSON, but it doesn't seem to work. Can anyone help me understand what am I doing wrong?
{
"took": 12,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 44,
"max_score": 1,
"hits": [
{
"_index": "transactions",
"_type": "transaction",
"_id": "trans0007",
"_score": 1,
"_source": {
"fundRelation": "[1,0,0,1,0,0,1,0,1,1,0,1,0,1,1,1,0,1,0,0],
[0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,1,1,0],
[0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
[0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0],
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
[1,0,0,1,0,0,1,0,0,1,0,1,0,1,1,1,0,1,0,0],
[0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0],
[0,0,1,1,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0],
[0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0],
[0,0,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,1,1,0],
[1,0,0,1,0,0,1,0,1,1,0,1,0,0,1,1,0,1,1,0],
[1,0,0,1,0,0,1,0,1,1,0,1,0,1,0,1,0,1,1,0],
[1,0,0,1,0,0,1,0,1,1,0,1,0,1,1,0,0,1,0,0],
[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0],
[0,0,1,1,0,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0],
[0,0,1,0,0,0,0,0,0,0,1,1,0,0,0,0,0,1,0,0],
[0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0]
",
"fundName": ["Fund A","Fund B","Fund C","Fund D","Fund E","Fund F","Fund G","Fund H","Fund I","Fund J","Fund K","Fund L","Fund M","Fund N","Fund O","Fund P","Fund Q","Fund R","Fund S","Fund T"],
"fundColor": ["#9ACD32","#377DB8","#F5DEB3","#EE82EE","#40E0D0","#FF6347","#D8BFD8","#D2B48C","#4682B4","#00FF7F","#FFFAFA","#708090","#708090","#6A5ACD","#87CEEB","#A0522D","#FFF5EE","#2E8B57","#F4A460","#FA8072"]
}
} ]
}
}
Not sure what i am doing wrong.
I am getting the following error message:
> Parse error on line 19: ... "fundRelation": "[1,0,0,1,0,0,1,0,1,
> -----------------------^ Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['
at http://jsonlint.com/
Here problem is, you are trying to assign multi line string value to fundRelation which is not valid JSON.
....
"fundRelation": "[1,0,0,1,0,0,1,0,1,1,0,1,0,1,1,1,0,1,0,0],[0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]",
...
Or you can do something like this :
{
"took": 12,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 44,
"max_score": 1,
"hits": [
{
"_index": "transactions",
"_type": "transaction",
"_id": "trans0007",
"_score": 1,
"_source": {
"fundRelation": [
[1,0,0,1,0,0,1,0,1,1,0,1,0,1,1,1,0,1,0,0],
[1,0,0,1,0,0,1,0,1,1,0,1,0,1,1,1,0,1,0,0]
],
"fundName": ["Fund A","Fund B","Fund C","Fund D","Fund E","Fund F","Fund G","Fund H","Fund I","Fund J","Fund K","Fund L","Fund M","Fund N","Fund O","Fund P","Fund Q","Fund R","Fund S","Fund T"],
"fundColor":["#9ACD32","#377DB8","#F5DEB3","#EE82EE","#40E0D0","#FF6347","#D8BFD8","#D2B48C","#4682B4","#00FF7F","#FFFAFA","#708090","#708090","#6A5ACD","#87CEEB","#A0522D","#FFF5EE","#2E8B57","#F4A460","#FA8072"]
}
}
]
}
}
You can't have multi line strings in javascript (except in the newish engines using the ` operator). You need to escape the end of each line of fundRelation by adding a \ to end end of each line.
Alternatively, don't store the matrix data as a string. remove the quote at the beginning and end of the array and store it as a standard array
Looks like it's not ok with the " splitting in multiple lines.
In any case, shouldn't that be like this:
...
"fundRelation": [
[1,0,0,1,0,0,1,0,1,1,0,1,0,1,1,1,0,1,0,0],
[0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],
...
[0,0,1,1,0,0,1,0,0,0,0,0,0,0,0,0,0,1,0,0]
]
...
Note the extra enclosing brackets [...].
Related
format the results in kusto
example
in the following example we need to summerize the user logs.
datatable(user:string, dt:datetime,page: string, value:int)
[
'chmod', datetime(2019-07-15), "page1", 1,
'ls', datetime(2019-07-02), "page2", 2,
'dir', datetime(2019-07-22), "page3", 3,
'mkdir', datetime(2019-07-14), "page4", 4,
'rm', datetime(2019-07-27), "page5", 5,
'pwd', datetime(2019-07-25), "page6", 6,
'rm', datetime(2019-07-23), "page7", 7,
'pwd', datetime(2019-07-25), "page8", 8,
]
| summarize commands_details = make_list(pack('dt', dt, 'page', page, "value", value)) by user
results
the results in the last example query will be like
"user": pwd,
"commands_details": [
{
"dt": "2019-07-25T00:00:00.0000000Z",
"page": "page6",
"value": 6
},
{
"dt": "2019-07-25T00:00:00.0000000Z",
"page": "page8",
"value": 8
}
],
expected results
but i need the results to be like the following
"user": pwd,
"commands_details": [
{
"dt": "2019-07-25T00:00:00.0000000Z",
"data":[
{"page": "page6", "value": 6},
{"page": "page8", "value": 8}
]
}
],
question
is there any way in Kusto to formate the results like the expected section?
You can use next query to achieve this:
datatable(user:string, dt:datetime,page: string, value:int)
[
'chmod', datetime(2019-07-15), "page1", 1,
'ls', datetime(2019-07-02), "page2", 2,
'dir', datetime(2019-07-22), "page3", 3,
'mkdir', datetime(2019-07-14), "page4", 4,
'rm', datetime(2019-07-27), "page5", 5,
'pwd', datetime(2019-07-25), "page6", 6,
'rm', datetime(2019-07-23), "page7", 7,
'pwd', datetime(2019-07-25), "page8", 8,
]
| summarize commands_details = make_list(pack('page', page, "value", value)) by user, dt
| project result = pack('user', user, 'data', pack('dt', dt, 'data', commands_details))
A result (for 'pwd'):
{
"user": "pwd",
"data": {
"dt": "2019-07-25T00:00:00Z",
"data": [
{
"page": "page6",
"value": 6
},
{
"page": "page8",
"value": 8
}
]
}
}
What is the best way to get access to nested JSON in D?
Now I am using vibed JSON module, but I do not know how to get access to nested elements without iteration it with foreach.
Here is my JSON:
{
"hasMore": false,
"result": [{
"ip": "127.0.0.1",
"passedtests": "[firsttest8,firsttest8,firsttest8,firsttest8]",
"guid": ""
}],
"code": 201,
"extra": {
"stats": {
"writesIgnored": 0,
"scannedIndex": 0,
"scannedFull": 1,
"executionTime": 0,
"filtered": 0,
"writesExecuted": 0
},
"warnings": []
},
"error": false,
"cached": false
}
I would like to do something like: result.passedtests. But here result is an array.
I found perfect solution: Json resultPassedtestsJson = visitorsInfo["result"][0]["passedtests"]; // "[firsttest8,firsttest8,firsttest8,firsttest8]" [0] is because result is array, and we access to it's first element.
How to insert json to the elasticsearch field?
I want to index document that include json in one field. I used folloeing code.
JSONOBject myjson=new JSONObject(myJSONstring);
IndexResponse response = client.prepareIndex("twitter", "tweet", "1")
.setSource(jsonBuilder()
.startObject()
.field("jsondata",myjson )
.field("postDate", "date")
.field("message", "trying out Elasticsearch")
.endObject()
)
.get();
this document successfully added. when I try to query data ,myjson get as string.But I want to query inside the myjson also. can anyone help me. Thank you.
query using kibana sense,
POST /twitter/tweet/_search
{
"query": {
"match": {
"jsondata.amount": 0
}
}
}
this gives null,
{
"took": 14,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
Nested JSON not working in d3.js
I am using nested JSON file in my data visualization using d3.js. I was referring to a previous Smilar Queastion. I created a jsfiddle of the answer of the question to see whether things are working. But i am confused why the code is not working. I have similar type of problem in my project. How can i solve that. Here is sample code I am using for printing the data in the form of a list
d3.select("body").append("ul").selectAll("li")
.data(data).enter().append("li").each(function() {
var li = d3.select(this);
li.append("p")
.text(function(d) { return d.date; });
li.append("ul").selectAll("li")
.data(function(d) { return d.hours; }) // second level data-join
.enter().append("li")
.text(function(d) { return d.hour + ": " + d.hits; });
});
It looks like the JSON wasn't formed correctly. I fixed the JSON like this -
var data=[{ "date": "20120927",
"hours": [
{ "hour": 0, "hits": 823896 },
{ "hour": 1, "hits": 654335 },
{ "hour": 2, "hits": 548812 },
{ "hour": 3, "hits": 512863 },
{ "hour": 4, "hits": 500639 }
],
"totalHits": "32,870,234",
"maxHits": "2,119,767",
"maxHour": 12,
"minHits": "553,821",
"minHour": 3 },
{ "date": "20120928",
"hours": [
{ "hour": 0, "hits": 1235923 },
{ "hour": 1, "hits": 654335 },
{ "hour": 2, "hits": 1103849 },
{ "hour": 3, "hits": 512863 },
{ "hour": 4, "hits": 488506 }
],
"totalHits": "32,870,234",
"maxHits": "2,119,767",
"maxHour": 12,
"minHits": "553,821",
"minHour": 3 }];
Also, I think there was a problem with the HTTP d3.js library not being loaded. I added an external HTTPS reference and it is working.
Here is a working version of your fiddle - https://jsfiddle.net/ferlin_husky/wzuvob6m/
Delete element of Solr
I deleted an item you do not need to solr, but I solr response still appears.
The json:
{
"responseHeader": {
"status": 0,
"QTime": 1,
"params": {
"facet": "true",
"q": "*:*",
"facet.limit": "-1",
"facet.field": "manufacturer",
"wt": "json",
"rows": "0"
}
},
"response": {
"numFound": 84,
"start": 0,
"docs": []
},
"facet_counts": {
"facet_queries": {},
"facet_fields": {
"manufacturer": [
"Chevrolet",
0,
"abarth",
1,
"audi",
7,
"austin",
1,
"bmw",
2,
"daewoo",
2,
"ford",
1,
"fso",
1,
"honda",
1,
"hyundai",
1,
"jaguar",
3,
"lexus",
1,
"mazda",
1,
"mitsubishi",
1,
"nissan",
1,
"pontiac",
1,
"seat",
1
]
},
"facet_dates": {},
"facet_ranges": {}
}
}
the deleted item is "chevrolet", now this to '0 'but it still appears.
"manufacturer":["Chevrolet",0,
I wish I could delete the item completely, is that possible.. Thanks.
Here is a two step approach I would follow: Make sure changes(deletion) is committed. You may issue a commit If it still shows facets with zero count, you may append &facet.mincount=1 to your query &facet.mincount=1 will make sure facets with zero count do not show up. For more details, please refer to: http://wiki.apache.org/solr/SimpleFacetParameters#facet.mincount
In your case probably it is because of uninverted index created by solr. Pass facet.mincount=1 in your query to get rid of this problem.