How to send multiple documents using RMongo - json

I am following the conventions from http://docs.mongodb.org/manual/reference/method/db.collection.insert/
to send a batch of multiple documents in one call of RMongo::dbInsertDocument.
data=data.frame(A=c(1,2),B=c(3,4))
L=lapply(split(data,rownames(data)),as.list)
names(L)=NULL
dataJSON = toJSON(L)
cat(dataJSON)
which gives the following result:
[
{
"A":1,
"B":3
},
{
"A":2,
"B":4
}
]
Then
dbInsertDocument(rmongo.object=myRmongo.object, collection=myCollection, doc=dataJSON)
returns the following error:
Error in ls(envir = envir, all.names = private) :
invalid 'envir' argument
Note that if I replace
L = L[[1]
Then
cat(dataJSON)
gives the following result:
{
"A":1,
"B":3
}
and the same call to dbInsertDocument works with no error (and the data is indeed sent to the database)

Has anyone figured this out? I would really like a better way to do this, but for now am just looping over the list (not ideal)
data=data.frame(A=c(1,2),B=c(3,4))
L=lapply(split(data,rownames(data)),as.list)
names(L)=NULL
for (i in 1:NROW(L)) {
dataJSON = toJSON(L[[i]])
output <- dbInsertDocument(mongo, "test_data7", dataJSON)
}

Related

Send JSON from MySQL query in NodeJS?

Currently my NodeJS code sends JSON back in the following format;
{"data":
[
{"audioname":"mytalk.m4a","name":"Josie ","email":"josie#gmail.com"},
{"audioname":"mytalk40.m4a","name":"Jessie James","email":"jesse#gmail.com"},
{"audioname":"mytalk.m4a","name":"Joan Bologney","email":"joan#gmail.com"}
]
}
But I'd like to get rid of the "data" and send back just;
[
{"audioname":"mytalk.m4a","name":"Josie ","email":"josie#gmail.com"},
{"audioname":"mytalk40.m4a","name":"Jessie James","email":"jesse#gmail.com"},
{"audioname":"mytalk.m4a","name":"Joan Bologney","email":"joan#gmail.com"}
]
Here's the query;
query = mysql.format(query);
connection.query(query,function(err,data){
if(err) {
res.json({"Error" : true, "Message" : "Error executing MySQL query"});
} else {
res.json({data});
}
});
If the object shown above is available though the data variable, we can pass only the data array by using:
res.json(data.data);
Where the second data references to the data array in the data variable.
Small example:
const data = {"data": [{"audioname":"mytalk.m4a","name":"Josie ","email":"josie#gmail.com"}, {"audioname":"mytalk40.m4a","name":"Jessie James","email":"jesse#gmail.com"}, {"audioname":"mytalk.m4a","name":"Joan Bologney","email":"joan#gmail.com"} ] };
console.log(data.data);
Removing the inner brackets seemed to work;
res.json(data);

Get array from json in Bigquery

I'm trying to get the data from a JSON in BigQuery. This JSON is Stored in a one-column table.
So far, I've been able to get only the "variables" array, with the following:
Select JSON_QUERY_ARRAY(Column1, '$.sessions[0].variables') FROM Table
How can I get the other values/arrays (sessionMessage and events)? I can't make it work..
I've tried with:
JSON_VALUE(Column1, '$.sessions[0].conversation')
JSON_QUERY_ARRAY(Column1, '$.sessions[0].sessionMessages')
But I get only empty values (The original json has values inside this arrays..)
{
"fromDate":"2020-04-10T23:47:17.161Z",
"pageRows":151,
"sessions":[
{
"variables":[],
"sessionDate":"2020-04-10T23:47:17.161Z",
"botMessages":2,
"userHasTalked":"true",
"topics":[
"TOPIC1"
],
"sessionId":"WXXXSXSXSXXXQ_2020-01-00T23:47:17.161Z",
"platformContactId":"XXXXXXX-XXXXXXX-XXXXXXXXXXXXXX",
"sessionMessages":[.....],
"queues":[
"QUEUE1",
"QUEUE2"
],
"customerId":"SSDSDS",
"userMessages":2,
"operatorMessages":1,
"sessionMessagesQty":2,
"sessionStartingCause":"Organic",
"channelId":"IDCHANEL",
"conversation":"https://url.com",
"events":[.....]
}
],
"toDate":"2020-04-10T23:47:17.161Z",
"hasMore":true,
"pageToken":"XXXXXXXXXXXXXX"
}
There is nothing wrong with the function and JSONPath that you used, but your sample JSON file has some unexpected thing, like [.....], removing/replacing those and query below works fine:
WITH a as (select
"""
{
"fromDate":"2020-04-10T23:47:17.161Z",
"pageRows":151,
"sessions":[
{
"variables":[],
"sessionDate":"2020-04-10T23:47:17.161Z",
"botMessages":2,
"userHasTalked":"true",
"topics":[
"TOPIC1"
],
"sessionId":"WXXXSXSXSXXXQ_2020-01-00T23:47:17.161Z",
"platformContactId":"XXXXXXX-XXXXXXX-XXXXXXXXXXXXXX",
"sessionMessages":[1,2,3],
"queues":[
"QUEUE1",
"QUEUE2"
],
"customerId":"SSDSDS",
"userMessages":2,
"operatorMessages":1,
"sessionMessagesQty":2,
"sessionStartingCause":"Organic",
"channelId":"IDCHANEL",
"conversation":"https://url.com",
"events":[],
}
],
"toDate":"2020-04-10T23:47:17.161Z",
"hasMore":true,
"pageToken":"XXXXXXXXXXXXXX"
}
""" data)
SELECT JSON_VALUE(data, '$.sessions[0].conversation'),
JSON_QUERY_ARRAY(data, '$.sessions[0].sessionMessages')
FROM a;

Python: create json query

I'm trying to get python to create a json formatted like :
[
{
"machine_working": true
},
{
"MachineName": "TBL165-169",
"MachineType": "Rig Test"
}
]
However, i can seam to do it, this is the code i have currently but its giving me error
this_is_a_dict_too=[]
this_is_a_dict_too = dict(State="on",dict(MachineType="machinetype1",MachineName="MachineType2"))
File "c:\printjson.py", line 40
this_is_a_dict_too = dict(Statedsf="test",dict(MachineType="Rig Test",MachineName="TBL165-169")) SyntaxError: non-keyword arg after
keyword arg
this_is_a_dict_too = [dict(machine_working=True),dict(MachineType="machinetype1",MachineName="MachineType2")]
print(this_is_a_dict_too)
You are trying to make dictionary in dictionary, the error message say that you try to add element without name (corresponding key)
dict(a='b', b=dict(state='on'))
will work, but
dict(a='b', dict(state='on'))
won't.
The thing that you presented is list, so you can use
list((dict(a='b'), dict(b='a')))
Note that example above use two dictionaries packed into tuple.
or
[ dict(a='b'), dict(b='a') ]

Parsing json : Test is json key existing

I'm request with API REST a JIRA filter since Excel and I return my result in a json object.
I'm parsing this object and I Try to show my result (in a msgbox for now) but I have a problem when the json Key doesn't exist !
A extract of my json :
{
"expand":"schema,names",
"startAt":0,
"maxResults":500,
"total":2,
"issues":[
{
"expand":"operations,versionedRepresentations,editmeta,changelog,renderedFields",
"id":"00001",
"fields":{
"components":[
{
"id":"01",
"name":"component_1"
},
{
"id":"02",
"name":"component_02"
}
]
}
},
{
"expand":"operations,versionedRepresentations,editmeta,changelog,renderedFields",
"id":"00002",
"fields":{
"components":[
]
}
},
]
}
As you cans see, in my first issue (id 00001) I have a 2 components key but in my second issus (id 0002) I don't have component key, because this fields is empty in JIRA for this issue.
So, a part of my code to show my result :
For Each Item In jsonObject("issues")
issueId = Item("id")
compoId1 = Item("fields")("components")(1)("id")
compoId2 = Item("fields")("components")(2)("id")
i = i + 1
'PRINT_OF_MY_RESULT
Next
My problem :
If my issue (00001) has a "component" value, it's OK and I can return my result but ... if my issus (00002) hasn't a result, my code failled to define compoId ... and my code crash.
Did you have a simple solution ? I try somethings with Exists, isEmpty, etc etc ... but nothing concluent for me :(
You can modify your solution some what like this,
For Each Item In jsonObject("issues")
issueId = Item("id")
For Each componentItem In jsonObject(Item("fields")("components"))
If componentItem("id")==1 then
compoId1 = componentItem("id")
EndIf
If componentItem("id")==1 then
compoId2 = componentItem("id")
EndIf
Next
i = i + 1
'PRINT_OF_MY_RESULT
Next

Azure tables unable to store flattened JSON

I am using the npm flat package, and arrays/objects are flattened, but object/array keys are surrounded by '' , like in 'task_status.0.data' using the object below.
These specific fields do not get stored into AzureTables - other fields go through, but these are silently ignored. How would I fix this?
var obj1 = {
"studentId": "abc",
"task_status": [
{
"status":"Current",
"date":516760078
},
{
"status":"Late",
"date":1516414446
}
],
"student_plan": "n"
}
Here is how I am using it - simplified code example: Again, it successfully gets written to the table, but does not write the properties that were flattened (see further below):
var flatten = require('flat')
newObj1 = flatten(obj1);
var entGen = azure.TableUtilities.entityGenerator;
newObj1.PartitionKey = entGen.String(uniqueIDFromMyDB);
newObj1.RowKey = entGen.String(uniqueStudentId);
tableService.insertEntity(myTableName, newObj1, myCallbackFunc);
In the above example, the flattened object would look like:
var obj1 = {
studentId: "abc",
'task_status.0.status': 'Current',
'task_status.0.date': 516760078,
'task_status.1.status': 'Late',
'task_status.1.date': 516760078,
student_plan: "n"
}
Then I would add PartitionKey and RowKey.
all the task_status fields would silently fail to be inserted.
EDIT: This does not have anything to do with the actual flattening process - I just checked a perfectly good JSON object, with keys that had 'x.y.z' in it, i.e. AzureTables doesn't seem to accept these column names....which almost completely destroys the value proposition of storing schema-less data, without significant rework.
. in column name is not supported. You can use a custom delimiter to flatten your objects instead.
For example:
newObj1 = flatten(obj1, {delimiter: '__'});