I need to retrieve only the values from an array of a payload, using dataweave in mule.
I have tried using the ++ technique, but it returns errors or when I put the values variable set an an array I get " in the results.
Input:
{
"Shops":
[{
"StoreName": "Store1",
"Sales":
[{"dayDate": "01/01/2019",
"product": "A",
"quantity": 2
},
{"dayDate": "02/01/2019",
"product": "B",
"quantity": 1
}
]
}]
}
I expect the Output:
[Store1, [01/01/2019, A, 2], [02/01/2019, B, 1]]
but actual is
["Store1, [01/01/2019, A, 2], [02/01/2019, B, 1]"]
How do I remove the " or if there is a better way of obtaining my expected output?
You need the pluck function, which transforms an object into an array (docs here).
%dw 2.0
output application/json
var shops = {
"Shops": [
{
"StoreName": "Store1",
"Sales": [
{
"dayDate": "01/01/2019",
"product": "A",
"quantity": 2
},
{
"dayDate": "02/01/2019",
"product": "B",
"quantity": 1 }]}]}
var res = shops.Shops map (shop) ->
using (sales = shop.Sales map (sale) -> sale pluck $)
[shop.StoreName] ++ sales
---
flatten(res)
This goes a bit beyond what you've described so that it can also handle multiple shops. If you need to be able to handle multiple shops, you probably won't want to use flatten, but it is there so the output matches what you asked for.
Output:
[
"Store1",
[
"01/01/2019",
"A",
2
],
[
"02/01/2019",
"B",
1
]
]
Related
Consider following array:
{
"A": 100,
"B": 200,
"C": "ccc",
"arr": [
{
"result": ".R1.R3",
"fmt": "%s::%s::baz",
"vals": [".A", ".B"]
},
{
"result": ".R2.R4",
"fmt": "%s/%s",
"vals": [".A", ".C"]
}
]
}
I need to replace keys according some format with values from other keys whose are in strings of some array.
Desired output:
{
"A": 100,
"B": 200,
"C": "ccc",
"R1": {"R3": "100::200::baz"},
"R2": {"R4": "100/ccc"}
}
You didn't specify what language you use in the .vals array items to reference into the document. If you were trying to execute (arbitrary) jq code from there, know that jq cannot do that with code provided as a string value. jq also doesn't provide printf-style substitutions using %s (and others). Therefore, you either need to re-implement a whole bunch of (third-party) functionality, or revert to a simpler scheme describing your references and substitutions.
For the sake of simplicity, this solution just removes the first character (the dot) from the .vals array items and treats the result as top-level field name, and then simply replaces each occurrence of a literal %s with the next value. This should give you an overview of the general technique.
. as $top | reduce .arr[] as $a (del(.arr); .[$a.result] = (
reduce $a.vals[][1:] as $val ($a.fmt; sub("%s"; $top[$val] | #text))
))
{
"A": 100,
"B": 200,
"C": "ccc",
".R1": "100::200::baz",
".R2": "100/ccc"
}
Demo
One quite simple way of improving the reference language is to instead use path expressions jq provides functions for. They are represented as arrays with field names as strings items and array indices as number items. .A would become ["A"], .A[3].B would become ["A",3,"B"], and so on. Thus, assume your input looked like this:
{
"A": 100,
"B": 200,
"C": "ccc",
"arr": [
{
"result": ".R1",
"fmt": "%s::%s::baz",
"vals": [["A"], ["B"]]
},
{
"result": ".R2",
"fmt": "%s/%s",
"vals": [["A"], ["C"]]
}
]
}
Then you could use getpath to evaluate the given path expressions as above:
. as $top | reduce .arr[] as $a (del(.arr); .[$a.result] = (
reduce $a.vals[] as $path ($a.fmt; sub("%s"; $top | getpath($path) | #text))
))
{
"A": 100,
"B": 200,
"C": "ccc",
".R1": "100::200::baz",
".R2": "100/ccc"
}
Demo
Edit: As the question has been modified with the .result value now also being subject to reference interpretation, measures taken for .vals therefore apply to it as well. This implies changing the suggested source document format to use path expressions as in "result": ["R1", "R3"], and changing the assignment in the suggested code from .[$a.result] = ... to setpath($a.result; ...):
{
"A": 100,
"B": 200,
"C": "ccc",
"arr": [
{
"result": ["R1", "R3"],
"fmt": "%s::%s::baz",
"vals": [["A"], ["B"]]
},
{
"result": ["R2", "R4"],
"fmt": "%s/%s",
"vals": [["A"], ["C"]]
}
]
}
. as $top | reduce .arr[] as $a (del(.arr); setpath($a.result;
reduce $a.vals[] as $path ($a.fmt; sub("%s"; $top | getpath($path) | #text))
))
{
"A": 100,
"B": 200,
"C": "ccc",
"R1": {
"R3": "100::200::baz"
},
"R2": {
"R4": "100/ccc"
}
}
Demo
Considering the dataframe below:
timestamp coordinates
0 [402, 404] [[2.5719,49.0044], [2.5669,49.0043]]
1 [345, 945] [[2.5719,49.0044], [2.5669,49.0043]]
I'd like to generate a json file like below:
[
{
"vendor": 1,
"path": [
[2.5719,49.0044],
[2.5669,49.0043]
],
"timestamps": [402, 404]
},
{
"vendor": 1,
"path": [
[2.5719,49.0044],
[2.5669,49.0043]
],
"timestamps": [345, 945]
}]
To do so, my idea is:
For each row of my df, generate a new column geometry
containing row json data
Then append all geometries in a json
However, my function below doesn't work.
df["geometry"] = df.apply(lambda row: {
"vendor": 1,
"path": row["coordinates"],
"timestamps": row["timestamp"]
},
axis = 1)
Indeed, the result is (for example):
Note the quote marks (') around arrays in path
{
'vendor': 1,
'path': ['[2.5719,49.0044]', '[2.5669,49.0043]'],
'timestamps': [402, 404]
}
Any idea?
Thanks
Presumably the values in coordinates column are of type string. You can use ast.literal_eval to convert it to list:
from ast import literal_eval
df["geometry"] = df.apply(
lambda row: {
"vendor": 1,
"path": literal_eval(row["coordinates"]),
"timestamps": row["timestamp"],
},
axis=1,
)
print(df)
Prints:
timestamp coordinates geometry
0 [402, 404] [[2.5719,49.0044], [2.5669,49.0043]] {'vendor': 1, 'path': [[2.5719, 49.0044], [2.5669, 49.0043]], 'timestamps': [402, 404]}
1 [345, 945] [[2.5719,49.0044], [2.5669,49.0043]] {'vendor': 1, 'path': [[2.5719, 49.0044], [2.5669, 49.0043]], 'timestamps': [345, 945]}
New to jq but I've managed to group a load of data which I would now like summarized in a different format. Original data after group by and mapping:
[
{
"Agents": "a",
"Count": 1
},
{
"Agents": "b",
"Count": 50
},
{
"Agents": "c",
"Count": 25
},
{
"Agents": "d",
"Count": 1
},
{
"Agents": "e",
"Count": 4
},
{
"Agents": "f",
"Count": 4
},
{
"Agents": "g",
"Count": 4
}
]
and I would like this output:
{
"Count": 7,
"Agents": {
"a": 1,
"b": 50,
"c": 25,
"d": 1,
"e": 4,
"f": 4,
"g": 4
}
}
How exactly might I do this in jq please because it requires mapping the values as field names?
Another variant using reduce would be to do. The reduce expression takes the array as input and puts the key as the Agents and the value as its corresponding Count value.
jq '{ Count: length, Agents: (reduce .[] as $d ({}; .[$d.Agents] = $d.Count)) }'
The Object Value Iterator .[] used to construct the JSON. For a given .["a"], it returns "a" which is how the keys are constructed in the final JSON.
Use map to create an input for from_entries:
map({key: .Agents, value: .Count}) | {Count: length, Agents: from_entries}
map produces a list of objects like [{"key": "a", "value": 1}, ...]. from_entries turns that into a single object {"a": 1, ...}. In the final object, both length and from_entries get the same array as input, and their outputs are used to create the final object with Count and Agents keys.
E.g. Given the following JSON:
{
"success": true,
"message": "''",
"result": [
{
"buy": [
{
"quantity": 12.37,
"rate": 32.55412402
}
],
"sell": [
{
"quantity": 12.37,
"rate": 32.55412402
}
]
}
]
}
How would I go about retrieving the 'Buy' and 'Sell' to store into variables?
I can get the result via:
d = json.loads(string)
print(d['result'])
However I fail to see how to retrieve the buy object now. E.g. I've tried:
#print(d['result']['buy'])
#print(d['result'].buy)
#print(d['result'].indexOf('buy'))
All to no avail.
Try:
print(d['result'][0]['buy'])
should give you the buy object:
[{u'rate': 32.55412402, u'quantity': 12.37}]
If you inspect the type of d['result']:
print(type(d['result'])) # <type 'list'>
It's a list of length 1, so the [0] at the end of d['result'][0] will return that first and only item in the list which is the dictionary you are expecting that you can get by key ['buy'] as you did in your first try:
#print(d['result']['buy'])
there just need to index the list by [0]
#print(d['result'][0]['buy'])
I am using MySQL 5.7+ with the native JSON data type. Sample data:
[
{
"code": 2,
"stores": [
{
"code": 100,
"quantity": 2
},
{
"code": 200,
"quantity": 3
}
]
},
{
"code": 4,
"stores": [
{
"code": 300,
"quantity": 4
},
{
"code": 400,
"quantity": 5
}
]
}
]
Question: how do I extract an array where code = 4?
The following (working) query has the position of the data I want to extract and the search criterion hardcoded:
SELECT JSON_EXTRACT(data_column, '$[0]')
FROM json_data_table
WHERE data_column->'$[1].code' = 4
I tried using a wildcard (data_column->'$[*].code' = 4) but I get no results in return.
SELECT row FROM
(
SELECT data_column->"[*]" as row
FROM json_data_table
WHERE 4 IN JSON_EXTRACT(data_column, '$[*].code')
)
WHERE row->".code" = 4
... though this would be much easier to work with if this wasn't an unindexed array of objects at the top level. You may want to consider some adjustments to the schema.
Note:
If you have multiple rows in your data, specifying "$[i]" will pick that row, not the aggregate of it. With your dataset, "$[1].code" will always evaluate to the value of code in that single row.
Essentially, you were saying:
$ json collection
[1] second object in the collection.
.code attribute labeled "code".
... since there will only ever be one match for that query, it will always eval to 4...
WHERE 4 = 4
Alternate data structure if possible
Since the entire purpose of "code" is as a key, make it the key.
[
"code2":{
"stores": [
{
"code": 100,
"quantity": 2
},
{
"code": 200,
"quantity": 3
}
]
},
"code4": {
"stores": [
{
"code": 300,
"quantity": 4
},
{
"code": 400,
"quantity": 5
}
]
}
]
Then, all it would require would be:
SELECT datacolumn->"[code4]" as code4
FROM json_data_table
This is what you are looking for.
SELECT data_column->'$[*]' FROM json_data_table where data_column->'$[*].code' like '%4%'.
The selected data will have [] around it when selecting from an array thus data_column->'$[*].code' = 4 is not possible.