This is the my document which i store in the bucket.Also the Id(key) attribute is screenName.
{
"created": null,
"createdBy": null,
"updated": null,
"updatedBy": null,
"screenName": "steelers",
"appId": "100",
"domain": "100$APPINFOTEAM",
"alias": "steelers",
"devision": "1"
}
I have multiple documents in Couchbase in this format. So i need to fetch these documents in descending order. So this is the implementation I used it for,
Query query = new Query();
// Filter on the domain key
query.setIncludeDocs(true);
query.setDescending(true);
query.setInclusiveEnd(true);
query.setKey(domain);
List<AppInfoTeam> appInfoTeam = appinfoTeamService.getAppInfoTeamForApp(query);
This will give me the exact documents without sorting. This is my view
function (doc, meta) {
if (doc._class == "com.link.twitter.pojo.AppInfoTeam") {
emit(doc.domain, null);
}
}
Also I tried to Filter Results using Couchbase server interface. I tick the descending and inclusive_end values. Also put the domain as a key. Then when I click the show results button it will give me this error.
url: ?stale=false&descending=true&inclusive_end=true&key=domain&connection_timeout=60000&limit=10&skip=0
Error:
{"error":"bad_request","reason":"invalid UTF-8 JSON: {{error,{1,\"lexical error: invalid char in json text.\\n\"}},\n \"domain\"}"}
How can I fix this issue?
You need to wrap the key with double quotes:
<url>?stale=false&descending=true&inclusive_end=true&key="domain"&connection_timeout=60000&limit=10&skip=0
Related
I have field of type json. Example value is:
[{"id": "960287", "src_ip": "X.X.X.X", "filename": "XXX-20200408092811-0", "order_id": "199926", "download_ip": "", "datetime_add": "2020-04-09 09:16:48", "order_desk_id": null, "datarejestracji": null, "download_browser": "", "datetime_download": "0000-00-00 00:00:00"}, {"id": "960288", "src_ip": "2.128.4.33", "filename": "XXX-20200408101526-4", "order_id": "199926", "download_ip": "", "datetime_add": "2020-04-09 09:16:48", "order_desk_id": null, "datarejestracji": null, "download_browser": "", "datetime_download": "0000-00-00 00:00:00"}, {"id": "960751", "src_ip": "2.128.4.33", "filename": "20200409164205-24", "order_id": "199926", "download_ip": "", "datetime_add": "2020-04-09 20:02:46", "order_desk_id": null, "datarejestracji": null, "download_browser": "", "datetime_download": "0000-00-00 00:00:00"}]
How to select specified property in SQL query?
If you are using MySQL 5.7 or later, you can use the JSON_EXTRACT() function:
SELECT JSON_EXTRACT(myjsoncolumn, '$[0].src_ip') AS src_ip
FROM mytable;
There's also a shorthand syntax:
SELECT myjsoncolumn->'$[0].src_ip' AS src_ip
FROM mytable;
Note that this function returns JSON, not the scalar string value, so it'll look like "X.X.X.X" with quotes around it. If you want the raw value, use:
SELECT JSON_UNQUOTE(JSON_EXTRACT(myjsoncolumn, '$[0].src_ip')) AS src_ip
FROM mytable;
Or the shorthand:
SELECT myjsoncolumn->>'$[0].src_ip' AS src_ip
FROM mytable;
Read the documentation on JSON functions for more information: https://dev.mysql.com/doc/refman/8.0/en/json-functions.html
If you are not using at least MySQL 5.7, you should upgrade. Older versions are now past their end-of-life, and if you're storing JSON data in the database, it will be very difficult to do anything but return the whole JSON document to clients.
Given the structure of your example JSON, I think you might need to use JSON_TABLE().
It looks like you have an array of JSON objects with identical fields, so I wonder why you are using JSON at all, instead of storing these objects as rows with normal columns.
I have a sqlite database and in one of the fields I have stored complete json object . I have to make some json select requests . If you see my json
the ALL key has value which is an array . We need to extract some data like all comments where "pod" field is fb . How to extract properly when sqlite json has value as an array ?
select json_extract(data,'$."json"') from datatable ; gives me entire thing . Then I do
select json_extract(data,'$."json"[0]') but i dont want to do it manually . i want to iterate .
kindly suggest some source where i can study and work on it .
MY JSON
{
"ALL": [{
"comments": "your site is awesome",
"pod": "passcode",
"originalDirectory": "case1"
},
{
"comments": "your channel is good",
"data": ["youTube"],
"pod": "library"
},
{
"comments": "you like everything",
"data": ["facebook"],
"pod": "fb"
},
{
"data": ["twitter"],
"pod": "tw",
"ALL": [{
"data": [{
"codeLevel": "3"
}],
"pod": "mo",
"pod2": "p"
}]
}
]
}
create table datatable ( path string , data json1 );
insert into datatable values("1" , json('<abovejson in a single line>'));
Simple List
Where your JSON represents a "simple" list of comments, you want something like:
select key, value
from datatable, json_each( datatable.data, '$.ALL' )
where json_extract( value, '$.pod' ) = 'fb' ;
which, using your sample data, returns:
2|{"comments":"you like everything","data":["facebook"],"pod":"fb"}
The use of json_each() returns a row for every element of the input JSON (datatable.data), starting at the path $.ALL (where $ is the top-level, and ALL is the name of your array: the path can be omitted if the top-level of the JSON object is required). In your case, this returns one row for each comment entry.
The fields of this row are documented at 4.13. The json_each() and json_tree() table-valued functions in the SQLite documentation: the two we're interested in are key (very roughly, the "row number") and value (the JSON for the current element). The latter will contain elements called comment and pod, etc..
Because we are only interested in elements where pod is equal to fb, we add a where clause, using json_extract() to get at pod (where $.pod is relative to value returned by the json_each function).
Nested List
If your JSON contains nested elements (something I didn't notice at first), then you need to use the json_tree() function instead of json_each(). Whereas the latter will only iterate over the immediate children of the node specified, json_tree() will descend recursively through all children from the node specified.
To give us some data to work with, I have augmented your test data with an extra element:
create table datatable ( path string , data json1 );
insert into datatable values("1" , json('
{
"ALL": [{
"comments": "your site is awesome",
"pod": "passcode",
"originalDirectory": "case1"
},
{
"comments": "your channel is good",
"data": ["youTube"],
"pod": "library"
},
{
"comments": "you like everything",
"data": ["facebook"],
"pod": "fb"
},
{
"data": ["twitter"],
"pod": "tw",
"ALL": [{
"data": [{
"codeLevel": "3"
}],
"pod": "mo",
"pod2": "p"
},
{
"comments": "inserted by TripeHound",
"data": ["facebook"],
"pod": "fb"
}]
}
]
}
'));
If we were to simply switch to using json_each(), then we see that a simple query (with no where clause) will return all elements of the source JSON:
select key, value
from datatable, json_tree( datatable.data, '$.ALL' ) limit 10 ;
ALL|[{"comments":"your site is awesome","pod":"passcode","originalDirectory":"case1"},{"comments":"your channel is good","data":["youTube"],"pod":"library"},{"comments":"you like everything","data":["facebook"],"pod":"fb"},{"data":["twitter"],"pod":"tw","ALL":[{"data":[{"codeLevel":"3"}],"pod":"mo","pod2":"p"},{"comments":"inserted by TripeHound","data":["facebook"],"pod":"fb"}]}]
0|{"comments":"your site is awesome","pod":"passcode","originalDirectory":"case1"}
comments|your site is awesome
pod|passcode
originalDirectory|case1
1|{"comments":"your channel is good","data":["youTube"],"pod":"library"}
comments|your channel is good
data|["youTube"]
0|youTube
pod|library
Because JSON objects are mixed in with simple values, we can no longer simply add where json_extract( value, '$.pod' ) = 'fb' because this produces errors when value does not represent an object. The simplest way around this is to look at the type values returned by json_each()/json_tree(): these will be the string object if the row represents a JSON object (see above documentation for other values).
Adding this to the where clause (and relying on "short-circuit evaluation" to prevent json_extract() being called on non-object rows), we get:
select key, value
from datatable, json_tree( datatable.data, '$.ALL' )
where type = 'object'
and json_extract( value, '$.pod' ) = 'fb' ;
which returns:
2|{"comments":"you like everything","data":["facebook"],"pod":"fb"}
1|{"comments":"inserted by TripeHound","data":["facebook"],"pod":"fb"}
If desired, we could use json_extract() to break apart the returned objects:
.mode column
.headers on
.width 30 15 5
select json_extract( value, '$.comments' ) as Comments,
json_extract( value, '$.data' ) as Data,
json_extract( value, '$.pod' ) as POD
from datatable, json_tree( datatable.data, '$.ALL' )
where type = 'object'
and json_extract( value, '$.pod' ) = 'fb' ;
Comments Data POD
------------------------------ --------------- -----
you like everything ["facebook"] fb
inserted by TripeHound ["facebook"] fb
Note: If your structure contained other objects, of different formats, it may not be sufficient to simply select for type = 'object': you may have to devise a more subtle filtering process.
python for loop:
actions = json_data['actions']
for a in actions:
if 'causes' in a:
for cause in a['causes']:
if 'userId' in cause:
self.user = cause['userId']
How do I do this in groovy?
def jenkins_data = new JsonSlurper().parseText(obj)
Using this json, I'm not sure how to drill down to grab userId. I imagine I need to use a for loop to check for each element in the list for the 'cause' key and then repeat for the 'userId' key.
Here is an example payload I'm dealing with.
self.payload_a = {"number": 3585, "url": "https://test.m.test.com/job/gfdsgdsf/3585/",
"displayName": "master_3585", "timestamp": 1516992464686,
"actions": [{"something": "nothing"}, {"causes": [{"userId": "build"}]}]}
Using an example payload I am able to to echo jenkins_data.actions.causes and see output
however echoing jenkins_data.actions.causes.userId is null (even though the userId is definitely in the payload)
When I run
echo jenkins_data.actions.causes
I get
[null, [[_TestIdCause, shortDescription:Started by user, B, userId:valueweneed, userName:test, B]], null, null, null, null, null, null, null, null, null, null, null, null, null, null]
The answer is yes . You need to put a loop to traverse all nodes in json.
Parsing json is quire complex.
According to Json : Any data type can be returned from a same node like ('userId') while traversing. like shown below
"userId" : "A"
...
"userId" : { ... }
...
"userId" : [ ... ]
You need to handle this return data as per you wish. As you ask in the question to get the userId node, just use the xpath like shown below
def json = new JsonSlurper().parseText(jsontxt)
println json.actions.causes.userId
In your case it returns list. so just flatten it.
println json.actions.causes.userId.flatten()
I have an index with a couple of fields of type Edm.String and Collection(Edm.String). I want to have another index with the same fields plus another field of type Edm.Double. When I create such an index and try to add the same values (plus the newly added Edm.Double value) as I did to the first index, I'm getting the following error:
{
"error": {
"code": "",
"message": "The request is invalid. Details: parameters : An unexpected 'StartArray' node was found when reading from the JSON reader. A 'PrimitiveValue' node was expected.\r\n"
}
}
Does anyone know what this error means? I tried looking for it on the Internet but I couldn't find anything related to my situation. A sample request I'm sending to the new index looks like this:
POST https://myservicename.search.windows.net/indexes/newindexname/docs/index?api-version=2016-09-01
{
"value": [{
"#search.action": "upload",
"keywords": ["red", "lovely", "glowing", "cute"],
"name": "sample document",
"weight": 0.5,
"id": "67"
}]
}
The old index is the same but it doesn't have the "weight" parameter.
Edit: I created the index using the portal, so I don't have the exact JSON to create the index, but the fields are roughly like this:
Field Type Attributes Analyzer
---------------------------------------------------------------------------------------
id Edm.String Key, Retrievable
name Edm.String Searchable, Filterable, Retrievable Eng-Microsoft
keywords Collection(Edm.String) Searchable, Filterable, Retrievable Eng-Microsoft
weight Edm.Double Filterable, Sortable
The reason I got the error was because I made a mistake and was trying to send a Collection(Edm.String) when the actual type on the index was Edm.String.
When I query Couchbase using N1QL I always get the results under an object named 'default'. I searched through the documentation but couldn't find anything mentioning this object. Also none of the examples in the documentation show this object in the query results. Where does it come from and how can I get rid of it? Is it something to do with the 'default' bucket?
Sample query result:
"default": {
"$type": "MetaPage, Core",
"Datasources": [
{
"Data": null,
"EntityTypeName": "book",
"Name": "book_data",
"Query": "SELECT * FROM `default` WHERE Id = 'lotr' AND Type = 'entity_type';"
}
]
}
Yes, it is the default bucket in your query.
A couple of solutions.
SELECT default.* ...
SELECT RAW default FROM default ...