I have Json data like below
{
"!type": "alarm",
"$": {
"12279": {
"!type": "alarm",
"title": "Default",
"$": {
"5955": {
"!type": "alarm",
"name": "Wake",
"day": "SUN",
"startTime": "06:00"
},
"29323": {
"!type": "alarm",
"name": "Away",
"day": "SUN",
"startTime": "08:00"
},
"2238": {
"!type": "alarm",
"name": "Home",
"day": "SUN",
"startTime": "18:00"
}
}
}
}
}
My fbs looks like this
namespace space.alarm;
table Atom{
!type:string;
name:string;
startDay:string;
startTime:string; }
table AtomShell{
key:string (required, key);
value: Atom; }
table Alarm{
!type:string;
title:string;
$:[AtomShell]; }
table AlarmShell{
key:string (required, key);
value:Alarm; }
table Weeklyalarm{
!type:string;
$:[AlarmShell]; } root_type Weeklyalarm;
Im trying to implement google flat buffers but I'm getting errors like
alarm.fbs:4:0: error: illegal character: !
alarm.fbs:23:0: error: illegal character: $ (i have removed ! from
!type and changed $ to dollar to test the working of flat buffers
but i can't change the dynamic ids )
Sample.json:25:0: error: unknown field: 12279
Now my question,
Is it possible to use dynamic ids in flat buffers, if possible how
shall i proceed?
Can is use special characters in ids, if possible how to do it?
Thanks in advance.
You can't have characters like ! and $ in field names. Just use type instead of !type, etc.
Not sure what you mean by dynamic ids. All field names (keys) have to be declared in the schema, so they can't be dynamic. You can still achieve similar results though, if you make your JSON look something like this:
{
"type": "alarm",
"data": [
{
id: "12279",
"type": "alarm",
"title": "Default",
"data": [
{
"id": "5955",
"type": "alarm",
"name": "Wake",
"day": "SUN",
"startTime": "06:00"
},
{
"id": "29323",
"type": "alarm",
"name": "Away",
"day": "SUN",
"startTime": "08:00"
},
{
"id": "2238",
"type": "alarm",
"name": "Home",
"day": "SUN",
"startTime": "18:00"
}
]
}
]
}
And then make the corresponding schema.
Note that I made the "dynamic" list into a vector, and moved the id into the object itself.
Other tip: string values that are not dynamic (like "alarm") will take up way less space if you make them into an enum instead.
Related
I'm trying to limit the map in my view to a specific set of documents by either having the id "startsWith" a string or based on there being a specific node in the JSON> I can't seem to get a result set once I add an IF statement. The reduce is a simple _count:
function(doc, meta) {
if (doc.metricType == "Limit_Exceeded") {
emit([doc.ownedByCustomerNumber, doc.componentProduct.category], meta.id);
}
}
I've also tried if (doc.metricType) and also if(meta.id.startsWith("Turnaway:")
Example Doc:
{
"OvidUserId": 26105400,
"id": "Turnaway:00005792:10562440",
"ipAddress": "111187081038",
"journalTurnawayNumber": 10562440,
"metricType": "Limit_Exceeded",
"oaCode": "OA_Gold",
"orderNumber": 683980,
"ovidGroupID": 3113900,
"ovidGroupName": "tnu999",
"ovidUserName": "tnu999",
"ownedByCustomerNumber": 59310,
"platform": "Lippincott",
"samlString": "",
"serialName": "00005792",
"sessionID": "857616ee-dab7-43d0-a08b-abb2482297dd",
"soldProduct": {
"category": "Multidisciplinary Subjects",
"name": "Custom Collection For CALIS - LWW TA 2020",
"productCode": "CCFCCSI20",
"productNumber": 33410,
"subCategory": "",
"subject": "Multidisciplinary Subjects"
},
"soldToCustomer": {
"customerNumber": 59310,
"keyAccount": false,
"name": "Tongji University"
},
"turnawayDateTime": "2022-05-04T03:01:44.600",
"usedByCustomer": {
"customerNumber": 59310,
"keyAccount": false,
"name": "Tongji University"
},
"usedByCustomerNumber": 59310,
"yearMonth": "202205"
},
"id": "Turnaway:00005792:10562440"
}
Thanks,
Gerry
Found it (of course after posting the question) The second component of the Key in the emit has to exist. I entered doc.componentProduct.category instead of doc.soldProduct.Category.
I am trying to use Pentaho Kettle to read the Json file with the structure below and insert the data into the DW (Redshift).
{
"_id": {
"_data": "11111111111111"
},
"operationType": "insert",
"clusterTime": {
"$timestamp": {
"t": 1599495064,
"i": 1
}
},
"ns": {
"db": "abc",
"coll": "abc"
},
"documentKey": {
"_id": {
"$uuid": "abcabcabcabcabcabc"
}
},
"fullDocument": {
"_id": {
"$uuid": "abcabcabcabcabcabc"
},
"orderNumber": "1234567",
"externalOrderId": "12345678",
"orderDateTime": "2020-09-11T08:06:26Z[UTC]",
"attraction": "abc",
"entryDate": {
"$date": 1599523200000
},
"entryTime": {
"$date": 1599472800000
},
"requestId": "abc",
"ticketUrl": "abc",
"tickets": [
{
"passId": "1111111",
"externalTicketId": "1234567"
},
{
"passId": "222222222",
"externalTicketId": "122442492"
}
],
"_class": "abc"
}
}
As we see above both columns "entry_date" and "entry_time" are in Unix format. I need to somehow take the date component from "entry_date" and the time component from "entry_time" and transform both into a concatenated unique field that will give me the following output: "GMT: Monday, September 7, 2020 4:11:04 PM".
Also I would like to achieve the same for the field "orderDateTime" - is there any way I could use Pentaho to transform that into the same format as the above "GMT: Monday, September 7, 2020 4:11:04 PM"?
Below you can see how the 3 fields above ("entry_time", "entry_date" and "orderDatetime") are currently set up for the "Select Values" step. From there I would then take the data into the DW with the Table Output step as seen in the illustration below. Any help is appreciated.
I've got a Select-Operation on an an object that contains one key more than once.
It's practically two versions of one object in one JSON object.
I want to get the id of both of those Objects.
When i inspect the Object, i can clearly see the two different Id's, but the Select-Operation returns only one of them twice.
This is the original Object:
[
{
"Created": "2020-06-05T11:47:42",
"ID": 9,
},
{
"Created": "2020-06-05T11:06:04",
"ID": 10,
}
]
The Select-Operation looks like this:
{
"inputs": {
"from": "#body('Rest')?['value']",
"select": {
"ID": "#triggerBody()?['ID']",
"Created": "#triggerBody()?['Created']"
}
}
}
And it returns:
[
{
"Created": "2020-06-05T11:47:42",
"ID": 9,
},
{
"Created": "2020-06-05T11:47:42",
"ID": 9,
}
]
I don't really understand what's going on.
"select": {
"ID": "#triggerBody()?['ID']",
"Created": "#triggerBody()?['Created']"
}
is wrong, it should select item()?['ID']
I have a big query table that has a JSON object as one of the field in the table. How can I extract the data from the JSON object using app script. The object itself is nested. It looks like this
{
"uid": "124551",
"subjects": [
{
"tid": 37,
"title": "Algebra",
"html_id": "algebra",
"selected": true
},
{
"tid": 214853,
"title": "Trigonometry",
"html_id": "trigonometry",
"selected": true
},
{
"tid": 38,
"title": "Geometry",
"html_id": "geometry",
"selected": true
}
],
"cellphone": "09178854579",
"educations": [
{
"index": 0,
"schoolname": "University of the Philippines - Los BaƱos",
"degree": "BS Mathematics",
"major": "Mathematics",
"eduFrom": "2009-05-31T16:00:00.000Z",
"eduTo": "2013-04-26T16:00:00.000Z",
"eduFromTs": 1243785600,
"eduToTs": 1366992000
}
],
"info": {
"os": "Windows",
"internet": "ADSL",
"browser": "Chrome",
"network": "Wireless",
"speed": "",
"timezone": "Asia/Hong_Kong"
}
}
I want to extract all school names from education field. Any ideas?
Working with JSON objects is similar to working with XML, except that parsing or encoding a JSON object is much easier.
Once this string is retrieved, simply call JSON.parse() on the string to get a native object representation.
var data = JSON.parse(json);
Browser.msgBox(data.info.os);
Other sample code is at https://developers.google.com/apps-script/advanced/bigquery#run_query
I have the following document structure:
{
"some_field": "some_data",
"entries": {
{"id": "some_id", "type": "some_type", "value": "some_value"},
{"id": "another_id", "type": "another_type", "value": {"foo": 1, "bar": "two"}
}
}
So I would like to map entries based on the "type" field.
Which maping type or flag should I use?
Or maybe I need to change my doc structure?
Could you use this one
{
"some_field":"some_data",
"entries":[{
"id":"some_id",
"type":"some_type",
"value":"some_value"
},
{
"id":"another_id",
"type":"another_type",
"value":{
"foo":1,
"bar":"two"
}
}]
}