Method to assign object IDs to imported JSON in Firebase - json

Firebase is organizing an imported JSON file in the following way:
But the imported file (and exported file from Firebase) is organized this way:
{
"features" : [ {
"geometry" : {
"coordinates" : [ -77.347191, 36.269321 ],
"type" : "Point"
},
"properties" : {
"name" : "Branch Chapel",
"osm_id" : "262661",
"religion" : "christian"
},
"type" : "Feature"
},
...
It appears that Firebase assigns an internal number to each object in the array of "features". This is nice, but it makes it hard to reference each object without knowing how Firebase is naming it- and I have 400k+ objects.
Is there a way to assign an id to each object to prevent Firebase from generating its own? Or is there a way to programmatically rename/reorganize the data after it's been imported? The optimal outcome would have the object named by its osm_id, rather than some arbitrary number Firebase assigns.
Any help is appreciated.

get rid of the square brackets and replace with squiggley brackets
this
{
"flags": {
"1": {
"information": "blah",
},
"2": {
"information": "It is great!",
},
"3": {
"information": "Amazing!",
}
}
}
not this
[
{
"1": {
"information": "blah",
}
},
{
"2": {
"information": "It is great!",
}
},
{
"3": {
"information": "Amazing!",
}
}
]

Related

Azure CI/CD Json transformation on array

When I'm creating a Task for an Azure DevOps release, can I transform the value of an object in an array, based on the "key" of the object?
As an example, I'm copying the example Json file from this article, which is this.
{
"Data": {
"DefaultConnection": {
"ConnectionString": "Data Source=(LocalDb)\\MSDB;AttachDbFilename=aspcore-local.mdf;"
},
"DebugMode": "enabled",
"DBAccess": {
"Administrators": ["Admin-1", "Admin-2"],
"Users": ["Vendor-1", "vendor-3"]
},
"FeatureFlags": {
"Preview": [
{
"newUI": "AllAccounts"
},
{
"NewWelcomeMessage": "Newusers"
}
]
}
}
}
What I have is something more like this, look at the "FeatureFlags" array. My array contains a list of "key"/"value" objects similar to this. I need to be able to transform the object in the array that matches a key, and have the transform process replace the "value" property value.
{
"Data": {
"DefaultConnection": {
"ConnectionString": "Data Source=(LocalDb)\\MSDB;AttachDbFilename=aspcore-local.mdf;"
},
"DebugMode": "enabled",
"DBAccess": {
"Administrators": ["Admin-1", "Admin-2"],
"Users": ["Vendor-1", "vendor-3"]
},
"FeatureFlags": {
"Preview": [
{
"key": "newUI",
"value": "AllAccounts"
},
{
"key": "NewWelcomeMessage",
"value": "Newusers"
}
]
}
}
}
For example, I want to change the value of "Newusers" in the object with a key of NewWelcomeMessage, to "AllUsers".
In the Azure Release "Variables" tab, can I simply use this as the "Name", Data.FeatureFlags.Preview.NewWelcomeMessage, and AllUsers as the value for it to transform the value for the correct object, or will this fail?
This pattern seems to work with XML, see this.

Difference between Set and List for DynamoDB

I'm uploading data to my Dynamo Db table with the sensor's data. I created a List for sensors locations, however, I heard that it might be better to create a set and I could not find a difference between the way I upload data and the way it would be presented. Currently if I use List("L":) I have [ { "S" : "Culpeper VA" }, { "S" : "Colorado Springs Co" } ] in my table. Would it be different if I use Set instead and what attribute on the left I would use instead of "L" for list?
{
"Sensor" : {
"S": "Sensor1"
},
"SensorDescription": {
"S" : "Sensor to meassure water temperature"
},
"ImageFile" : {
"S" : "/Sensors/images/acoustic-elementarray.jpg"
},
"SampleRate":{
"N" : "2048"
},
"Locations" : {
"L": [
{
"S" : "Culpeper VA"
},
{
"S": "Colorado Springs Co"
}
]
}
}
That is my JSON that I use with put item API call
Now I figured out, the best way in my case would be to use String Set instead, update JSON is :
"Locations" : {
"SS": [ "Colorado Springs Co" , "Culpeper VA"
]
}

How to use JSON.parse with bucket_script?

I have a data field saved with JSON string and I need to count the average value of price in "{price: 10}", how do I use JSON.parse with bucket_script to compute this with elastic search?
There is no JSON parsing class in painless So you cannot perform this while querying. You should parse JSOn while indexing this will make your search queries faster.
Ingest
You can use JSON Precessor
{
"json" : {
"field" : "string_source",
"target_field" : "json_target"
}
}
Pipeline
PUT _ingest/pipeline/my-pipeline
{
"description": "describe pipeline",
"processors": [
{
"json": {
"field": "string_source",
"target_field": "json_target"
}
}
]
}
Index document using ingest pipeline
POST json_index/_doc?pipeline=my-pipeline
{
"string_source":"{\"price\":10}"
}
Document
"hits" : [
{
"_index" : "json_index",
"_type" : "_doc",
"_id" : "m7t3gXEB1B5aJp__0oos",
"_score" : 1.0,
"_source" : {
"json_target" : {
"price" : 10
},
"string_source" : """{"price":10}"""
}
}
]
If you don't want to keep original string in index you can use
PUT _ingest/pipeline/my-pipeline
{
"description": "describe pipeline",
"processors": [
{
"json": {
"field": "string_source",
"target_field": "json_target"
},
"remove": {
"field": "string_source"
}
}
]
}
2. Logstash
This is a JSON parsing filter. It takes an existing field which contains JSON and expands it into an actual data structure within the Logstash event.
filter {
json {
source => "message"
}
}

JSON-LD context with array of objects

I am trying to define a JSON-LD context that includes an array of objects.
Does anyone know, why the output is empty?
{
"#context": {
"testobjects": {
"#id" : "http://example.org/arrayOfObjects",
"type" : "array",
"items" : {
"type" : "object",
"properties" : {
"attr1": { "type" : "number", "default" : 1},
"attr2": { "type" : "string", "default" : "foo"}
}
}
}
},
"testobjects": [
{
"attr1": 216,
"attr2": "test"
},
{
"attr1": 329,
"attr2": "test2"
}
]
}
Output:
[
{
"http://example.org/arrayOfObjects": [
{},
{}
]
}
]
See JSON-LD Playground for trying yourself.
Note that data does not go in a context, so having items as a part of the "testobjects" term definition won't do. There is a proposal to allow multi-dimensional arrays to have individual types, that may be taken up in the future.
For the example you've provided to generate anything, both "attr1" and "attr2" would need to be defined as terms at the top-level of the context, not under an existing context definition.

How to fetch the data in the mongodb

How to fetch the data from the json file using mongoshell
I want to fecch the Data by policyID
Say in the json file I sent the PolicyID is 3148
I tried could of ways to write the command but say 0 rows fetched.
db.GeneralLiability.find({"properties.id":"21281"})
db.GeneralLiability.find({properties:{_id:"21281"}})
Do i need to set any thing else?index,cursors etc?
Sample json
{
"session": {
"data": {
"account": {
"properties": {
"userName": "abc.com",
"_dateModified": "2014-10-01",
"_manuscript": "Carrier_New_Rules_2_1_0",
"_engineVersion": "2.0.0",
"_cultureCode": "en-US",
"_cultureName": "United States [english]",
"_context": "Underwriter",
"_caption": "Carrier New Rules (2.1.0)",
"_id": "p1CEB08012E51477C9CD0E89FE77F5E51"
},
"properties": {
"_xmlns:xsd": "http://www.w3.org/2001/XMLSchema",
"_xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"_id": "3148",
"_HistoryID": "5922",
"_Type": "onset",
"_Datestamp": "2014-10-01T04:46:33",
"_TransactionType": "New",
"_EffectiveDate": "2014-01-01",
"_Charge": "1599",
"_TransactionGroup": "t4CE4FA751F9C400D9007E692A883DA66",
"_PolicyID": "3148",
"_Index": "1",
"_Count": "1",
"_Sequence": "1"
}
}
}
This will return the document with _PolicyID = "3148":
db.GeneralLiability.find({
"session._PolicyID": "3148"
}).pretty();
You have some issues in your document formatting. First off I am pretty sure that using underscores are reserved for mongo (I could be wrong). Either way it is bad form. I have restructured your data for you. I am not sure why you wanted to nest your data so much, but I am guessing you had a good reason for it.
You will notice that I am using the ObjectID from Mongo for my _id:
{
"_id" : ObjectId("56e1c1f53bac31a328e3682b"),
"session" : {
"data" : {
"account" : {
"properties" : {
"xmlns:xsd" : "http://www.w3.org/2001/XMLSchema",
"xmlns:xsi" : "http://www.w3.org/2001/XMLSchema-instance",
"HistoryID" : "5922",
"Type" : "onset",
"Datestamp" : "2014-10-01T04:46:33",
"TransactionType" : "New",
"EffectiveDate" : "2014-01-01",
"Charge" : "1599",
"TransactionGroup" : "t4CE4FA751F9C400D9007E692A883DA66",
"PolicyID" : "3148",
"Index" : "1",
"Count" : "1",
"Sequence" : "1"
}
}
}
}
}
Now if you run this command it will return your document:
{ "session.data.account.properties.PolicyID": "3148" }