Is there any way to get the length of an array found in a JSON object parsed by cloud watch log insights?
For example, when sending a JSON object of the following structure to log insights:
{
names: ['john', 'doe', 'joe', 'schmoe']
}
it gets parsed into the following fields:
names.0: john
names.1: doe
names.2: joe
names.3: schmoe
and can be accessed by
fields #timestamp, names.0, names.1, ...
In this example, is there a way to get a field called number_of_names?
e.g., | parse get_length(names) as number_of_names
Here is an ugly workaround for smaller arrays where the max length is known:
fields #timestamp, ispresent(names.0) + ispresent(names.1) + ispresent(names.2) + ... + ispresent(names.10) as names_length
Related
I am trying to build up a dictionary / json object of sensor values in esphome. I have a sensor that sends me key / value pairs (e.g. one sensor reading could be { “temperature“: 25.1 }, another one could be { “speed“: 50.1 } and so forth) in very high frequency (milliseconds). What I would like to do is to collect data for these key / value pairs for a certain time span, for simplicity say ten seconds, and only then take the dictionary and post it to a web service. It would also somehow combine the readings for the same key if sent multiple times within the ten seconds time span for example by averaging them out, using a filter or whatever. So the final dictionary to be posted to the web service would then look like
{
“temperature“: 26.3,
“speed“: 52.5,
…
}
How could I achieve this - any idea / proposal?
Thanks and best regards
Dear stackoverflow community,
I found a solution to this issue. I am now using a global variable in esphome. This can be defined as follows:
globals:
- id: "my_dict"
type: std::map<std::string, std::string>
With this, I have a global map which I can use to store the key / value pairs. Adding a new key / value pair via a lambda works as simple as shown in the following (where in this example, the key is stored in the variable key and the value is stored in the variable value:
lambda: |-
id(my_dict)[key] = value;
Every ten seconds, I post the dictionary content to the web service and then clear the dictionary again:
interval:
- interval: 10s
then:
- http_request.post:
url: "https://<URL>"
json: |-
for ( auto item : id(my_dict) ) {
root[item.first] = item.second;
}
- lambda: |-
id(my_dict).clear();
I'm new to Neo4j and Cypher and I'm trying to import some data from csv that includes an array of IDs. I have the query below working but as Cypher defaults to strings, I've been unable to find the best way to convert the array of placeIDs to integers.
LOAD CSV WITH HEADERS FROM 'http://localhost:11001/project-ca45d786-e360-4e3b-b4b4-eb8fe62a7b55/People-Gridv2.csv' AS row
CREATE (:People {peopleID: toInteger(row.peopleID), nickname: row.nickname, firstName: row.firstName, lastName: row.lastName, relationship: row.relationship, firstMemory: row.firstMemory, lastMemory: row.lastMemory, placeID: split(row.placeID,";")})
I hoped that I'd be able to do something like the following, but it doesn't work:
placeID: toInteger(split(row.placeID,";"))
Can anyone point me in the right direction?
that would probably something like
placeID : REDUCE(array=[] , s IN split(row.placeID,";") | array+[toInteger(s)] )
to get an array of integers
example
with '123;456' as placeID
return REDUCE(array=[] , s IN split(placeID,";") | array+[toInteger(s)] )
will return
[123,456]
and even shorter :)
with '123;456' as placeID
return [s IN split(placeID,";") | toInteger(s)]
I have a JSON response like this
Members":
[
{
"id":"ABC",
"name":"XXXX",
"XXX":"XXX",
"XXXX":"XXXX",
"Managers":
[
{
"id":XYZ,
"name":"XXX",
"XXXX":XXXX,
}
],
I need to get the value ABC and XYZ from the above response and I am using 2 JSON extractor to fetch the value and storing it in different variable.
JSON Extractor 1 expression:-
$..Members.[*].id
JSON Extractor 2 expression:-
$.Members..Managers.[*].id
But the above code picks the value from different arrays like sometime it picks the Members id as ABC but picks the Managers ID from different array. I want it to pick the value from same array value.
Any suggestions?
Assuming that you need to extract first member and his first manager:
Add JSON Extractor as a child of the request which returns above JSON and configure it as follows:
Variable names: memberId; ManagerId
JSON Path expressions: $.Members[0].id; $.Members[0].Managers[0].id
Match No: 1; 1
Default Values: NOT_FOUND; NOT_FOUND
Refer ABC as ${memberId} and XYZ as ${ManagerId} where required. You can see JMeter Variables using Debug Sampler and View Results Tree Listener combination
I have a JSON file contains fields such as machine_id, category, and ... Category contains states of machines such as "alarm", "failure". I simply like to see how many times each machine_id has been reported using rmr2.
For example, if I have the following:
machine_id, state
48, alarm
39, failure
48, utilization
I like to see this result:
48,2
39,1
What I did:
I wrote a simple mapreduce to read the value of JSON file and used it as an input in the second mapreduce. Code is:
mp = function(k,v){
machine_id=v$machine_id
keyval(machine_id,1) }
rd = function(k,v) keyval(k,length(v))
mapreduce(input = mapreduce(input='\user\cloudera\sample.json', input.format="json" , map=function(k,v) keyval(k,v)) , map=mp, reduce = rd)
Unfortunately, it returns only the last two values of JSON file. It seems that it doesn't read entire of the value of the JSON file. I would appreciate any help.
I am selecting nodes in a JSON input but can't find a way to include parent object detail for each array entry that I am querying. I am using pentaho data integration to query the data using JSON input form a mongodb input.
I have also tried to create a mongodb query to achieve the same but cannot seem to do this either.
Here are the two fields/paths that display the data:
$.size_break_costs[*].size
$.size_break_costs[*].quantity
Here is the json source format:
{
"_id" : ObjectId("4f1f74ecde074f383a00000f"),
"colour" : "RAVEN-SMOKE",
"name" : "Authority",
"size_break_costs" : [
{
"quantity" : NumberLong("80"),
"_id" : ObjectId("518ffc0697eee36ff3000002"),
"size" : "S"
},
{
"quantity" : NumberLong("14"),
"_id" : ObjectId("518ffc0697eee36ff3000003"),
"size" : "M"
},
{
"quantity" : NumberLong("55"),
"_id" : ObjectId("518ffc0697eee36ff3000004"),
"size" : "L"
}
],
"sku" : "SK3579"
}
I currently get the following results:
S,80
M,14
L,55
I would like to get the SKU and Name as well as my source will have multiple products (SKU/Description):
SK3579,Authority,S,80
SK3579,Authority,M,14
SK3579,Authority,L,55
When I try To include using $.sku, I the process errors.
The end result i'm after is a report of all products and the available quantities of their various sizes. Possibly there's an alternative mongodb query that provides this.
EDIT:
It seems the issue may be due to the fact that not all lines have the same structure. For example the above contains 3 sizes - S,M,L. Some products come in one size - PACK. Other come in multiple sizes - 28,30,32,33,34,36,38 etc.
The error produced is:
*The data structure is not the same inside the resource! We found 1 values for json path [$.sku], which is different that the number retourned for path [$.size_break_costs[].quantity] (7 values). We MUST have the same number of values for all paths.
I have tried the following mongodb query separately which gives the correct results, but the corresponding export of this doesn't work. No values are returned for the Size and Quantity.
Query:
db.product_details.find( {}, {sku: true, "size_break_costs.size": true, "size_break_costs.quantity": true}).pretty();
Export:
mongoexport --db brandscope_production --collection product_details --csv --out Test01.csv --fields sku,"size_break_costs.size","size_break_costs.quantity" --query '{}';
Shortly after I added my own bounty, I figured out the solution. My problem has the same basic structure, which is a parent identifier, and some number N child key/value pairs for ratings (quality, value, etc...).
First, you'll need a JSON Input step that gets the SKU, Name, and size_break_costs array, all as Strings. The important part is that size_break_costs is a String, and is basically just a stringified JSON array. Make sure that under the Content tab of the JSON Input, that "Ignore missing path" is checked, in case you get one with an empty array or the field is missing for some reason.
For your fields, use:
Name | Path | Type
ProductSKU | $.sku | String
ProductName | $.name | String
SizeBreakCosts | $.size_break_costs | String
I added a "Filter rows" block after this step, with the condition "SizeBreakCosts IS NOT NULL", which is then passed to a second JSON Input block. This second JSON block, you'll need to check "Source is defined in a field?", and set the value of "Get source from field" to "SizeBreakCosts", or whatever you named it in the first JSON Input block.
Again, make sure "Ignore missing path" is checked, as well as "Ignore empty file". From this block, we'll want to get two fields. We'll already have ProductSKU and ProductName with each row that's passed in, and this second JSON Input step will further split it into however many rows are in the SizeBreakCosts input JSON. For fields, use:
Name | Path | Type
Quantity | $.[*].quantity | Integer
Size | $.[*].size | String
As you can see, these paths use "$.[*].FieldName", because the JSON string we passed in has an array as the root item, so we're getting every item in that array, and parsing out its quantity and size.
Now every row should have the SKU and name from the parent object, and the quantity and size from each child object. Dumping this example to a text file, I got:
ProductSKU;ProductName;Size;Quantity
SK3579;Authority;S; 80
SK3579;Authority;M; 14
SK3579;Authority;L; 55