I'm using mongolite package to connect and retrieve data from MongoDB.How to pass value in mongolite find query
##connecting mongodb
library(mongolite)
mongo<-mongolite::mongo(collection = "Sample", db = "Test", url =
"mongodb://User:123#Wyyuyu:13333/ty2_U",verbose = TRUE)
## getting all data from collection data from collection below query is working fine.
values <- mongo$find()
## But I want to filter specific value by passing value.
for(i in c("process","check","queue"))
{
values <- mongo$find('{"field" : i}',)
}
if I tried above code i'm getting getting below issues . please help me to resolve
Error: Invalid JSON object: {"field" : i}
Given your i is a variable, you need to create the string using something like paste0:
values <- mongo$find(paste0('{"field" : ', i, '}') )
but rather than a loop you could also use
values <- mongo$find('{"field" : { "$in" : [ "process", "check", "queue" ] } }')
Related
I have the following table:
local my_table = {data = {{value1 = "test1", value2 = "test2"}, {value3 = "test3", value4 = "test4"}}}
I want to convert this table to json format and save to a file. But, when I tried
json.encode(my_table)
I got an error: bad argument #1 to 'encode' (["data"] => string index expected, got number)
I expect the json:
{
"data":[
{
"value1":"test1",
"value2":"test2"
},
{
"value3":"test3",
"value4":"test4"
}
]
}
It works!
local json = require'json'
local my_table = {data = {{value1 = "test1", value2 = "test2"}, {value3 = "test3", value4 = "test4"}}}
print(json.encode(my_table)) -- {"data":[{"value1":"test1","value2":"test2"},{"value4":"test4","value3":"test3"}]}
I'm using this repo
Probably, the implementation you are using requires special syntax to treat Lua table as JSON array instead of JSON dictionary.
The implementation I'm using makes this decision (is it an array or a dictionary) automatically.
this is my test json file.
{
"item" : {
"fracData" : [ ],
"fractimeData" : [ {
"number" : "1232323232",
"timePeriods" : [ {
"validFrom" : "2021-08-03"
} ]
} ],
"Module" : [ ]
}
}
This is how I read the json file.
starhist_test_df = spark.read.json("/mapr/xxx/yyy/ttt/dev/rawdata/Test.json", multiLine=True)
starhist_test_df.createOrReplaceTempView("v_test_df")
This query works.
df_test_01 = spark.sql("""
select item.fractimeData.number from v_test_df""")
df_test_01.collect();
Result
[Row(number=['1232323232'])]
But this query doesn't work.
df_test_01 = spark.sql("""
select item.fractimeData.timePeriods.validFrom from v_test_df""")
df_test_01.collect();
Error
cannot resolve 'v_test_df.`item`.`fractimeData`.`timePeriods`['validFrom']' due to data type mismatch: argument 2 requires integral type, however, ''validFrom'' is of string type.; line 3 pos 0;
What do I have to change, to read the validFrom field?
dot notation to access values works with struct or array<struct> types.
The schema for field number in item.fractimeData is string and accessing it via dot notation returns an array<string> since fractimeData is an array.
Similarly, the schema for field timePeriods in item.fractimeData is <array<struct<validFrom>>, and accessing it via dot notation wraps it into another array, resulting in final schema of array<array<struct<validFrom>>>.
The error you get is because the dot notation can work on array<struct> but not on array<array>.
Hence, flatten the result from item.fractimeData.timePeriods to get back an array<struct<validFrom>> and then apply the dot notation.
df_test_01 = spark.sql("""
select flatten(item.fractimeData.timePeriods).validFrom as validFrom from v_test_df""")
df_test_01.collect()
"""
[Row(validFrom=['2021-08-03', '2021-08-03'])]
"""
slurperresponse = new JsonSlurper().parseText(responseContent)
log.info (slurperresponse.WorkItems[0].WorkItemExternalId)
The above code helps me get the node value "WorkItems[0].WorkItemExternalId" using Groovy. Below is the response.
{
"TotalRecordCount": 1,
"TotalPageCount": 1,
"CurrentPage": 1,
"BatchSize": 10,
"WorkItems": [ {
"WorkItemUId": "4336c111-7cd6-4938-835c-3ddc89961232",
"WorkItemId": "20740900",
"StackRank": "0",
"WorkItemTypeUId": "00020040-0200-0010-0040-000000000000",
"WorkItemExternalId": "79853"
}
I need to append the string "WorkItems[0].WorkItemExternalId" (being read from a excel file) and multiple other such nodes dynamically to "slurperresponse" to get the value of nodes rather than directly hard coding as slurperresponse.WorkItems[0].WorkItemExternalId..
Tried append and "+" operator but i get a compilation error. What other way can I do this?
slurperrsesponse is an object its not a string that's why the concatenation does not work
Json Slurper creates an object out of the input string. This object is dynamic by nature, you can access it, you can add fields to it or alter the existing fields. Contatenation won't work here.
Here is an example:
import groovy.json.*
def text = '{"total" : 2, "students" : [{"name": "John", "age" : 20}, {"name": "Alice", "age" : 21}] }'
def json = new JsonSlurper().parseText(text)
json.total = 3 // alter the value of the existing field
json.city = 'LA' // add a totally new field
json.students[0].age++ // change the field in a list
println json
This yields the output:
[total:3, students:[[name:John, age:21], [name:Alice, age:21]], city:LA]
Now if I've got you right you want to add a new student dynamically and the input is a text that you've read from Excel. So here is the example:
json.students << new JsonSlurper().parseText('{"name" : "Tom", "age" : 25}')
// now there are 3 students in the list
Update
Its also possible to get the values without 'hardcoding' the property name:
// option 1
println json.city // prints 'LA'
// option 2
println json.get('city') // prints 'LA' but here 'city' can be a variable
// option 3
println json['city'] // the same as option 2
I'm ingesting a large simple json dataset from Azure Blob and moving data into a "stage" called "cities_stage" with FILE_FORMAT = json like so.
(Here is the error steps are below "Error parsing JSON: unknown keyword "Hurzuf", pos 7.")
create or replace stage cities_stage
url='azure://XXXXXXX.blob.core.windows.net/xxxx/landing/cities'
credentials=(azure_sas_token='?st=XXXXX&se=XXX&sp=racwdl&sv=XX&sr=c&sig=XXX')
FILE_FORMAT = (type = json);
I then take this stage location and dump it into a table with a single variant column like so. The file I'm ingesting is larger than 16mb so I create individual rows for each object by using type = json strip_outer_array = true
create or replace table cities_raw_source (
src variant);
copy into cities_raw_source
from #cities_stage
file_format = (type = json strip_outer_array = true)
on_error = continue;
When I select * from cities_raw_source each row looks like the following.
{
"coord": {
"lat": 44.549999,
"lon": 34.283333
},
"country": "UA",
"id": 707860,
"name": "Hurzuf"
}
When I add a reference to "country" or "name" that's where the issues come in. Here is my query (I did not use country in this one but it produces the same result).
select parse_json(src:id),
parse_json(src:coord:lat),
parse_json(src:coord:lon),
parse_json(src:name)
from cities_raw_source;
ERROR:
Error parsing JSON: unknown keyword "Hurzuf", pos 7.
ID, Lat, and Lon all come back as expected if I remove "src:name"
Any help is appreciated!
It turns out I had everything correct except for the query itself.
When querying a VARIANT column you do not need to PARSE_JSON so the correct query would look like this.
select src:id,
src:coord:lat,
src:coord:lon,
src:name
from cities_raw_source;
I'm using the latest cassandra version and trying to save JSON like below and was successful,
INSERT INTO mytable JSON '{"username": "myname", "country": "mycountry", "userid": "1"}'
Above query saves the record like,
"rows": [
{
"[json]": "{\"userid\": \"1\", \"country\": \"india\", \"username\": \"sai\"}"
}
],
"rowLength": 1,
"columns": [
{
"name": "[json]",
"type": {
"code": 13,
"type": null
}
}
]
Now I would like to retrieve the record based on userid:
SELECT JSON * FROM mytable WHERE userid = fromJson("1") // but this query throws error
All this occurs in a node/express app and I'm using dse-driver as the client driver.
The CQL command worked like below,
SELECT JSON * FROM mytable WHERE userid="1";
However if it has to be executed via the dse-driver then the below snippet worked,
let query = 'SELECT JSON * FROM mytable WHERE userid = ?';
client.execute(query, ["1"], { prepare: true });
where client is,
const dse = require('dse-driver');
const client = new dse.Client({
contactPoints: ['h1', 'h2'],
authProvider: new dse.auth.DsePlainTextAuthProvider('username', 'pass')
});
If your Cassandra version is 2.1x and below, you can use the Python-based approach. Write a python script using Cassandra-Python API
Here you have to get your row first and then use python json's loads method, which will convert your json text column value into JSON object which will be dict in Python. Then you can play around with Python dictionaries and extract your required nested keys. See the below code snippet.
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
import json
if __name__ == '__main__':
auth_provider = PlainTextAuthProvider(username='xxxx', password='xxxx')
cluster = Cluster(['0.0.0.0'],
port=9042, auth_provider=auth_provider)
session = cluster.connect("keyspace_name")
print("session created successfully")
rows = session.execute('select * from user limit 10')
for user_row in rows:
#fetchign your json column
column_dict = json.loads(user_row.json_col)
print(column_dict().keys()
Assuming user-id is the partition key, and assuming you want to retrieve a JSON object corresponding to user of id 1, you should try:
SELECT JSON * FROM mytable WHERE userid=1;
If userid is of type text, you will need to add some quotes.