%dw 2.0
output application/csv quoteValues=true, separator=";", header=true
---
[
{
Aaa: payload.problem.Aaa[0],
Bbb: payload.problem.Bbb[0]
}
]
And this write it to CSV like this:
Aaa;Bbb
1234; Test1
But this payload looks like this:
"problem": {
"Aaa": [
"1234",
"1567",
"105"
],
"Bbb": [
"Test1",
"Test2",
"Test3"
]
}
And I would like to write all 3 value or more depends from my request to my CSV. But now I can do it only if I specify which element from the array (example payload.issues.Aaa[0]). Because under my key Aaa I don't know exactly how many value I will get to response.
This solution assumes that the data for the first column has the same number of rows as all the others.
First I capture the names of columns from the key names in payload.problem, using the function namesOf(), and store the array of names in the variable column.
Then I map over the first 'column' to get the index of each row. Using that index I iterate over each column name obtaining the value for that column at that index, and transforming it into a key-value (column name, value of that column for the row). Finally I use reduce() to concatenate all the key-values from the same row into a single object, which represents the row as DataWeave expects to transform to CSV. If you change the output type to JSON you will see the structure more clearly.
%dw 2.0
output application/csv quoteValues=true, separator=";", header=true
var columns=namesOf(payload.problem)
---
payload.problem[columns[0]] map ((item, index) -> (
columns map ($): payload.problem[$][index])
reduce ($$++$)
)
Output:
Aaa;Bbb
"1234";"Test1"
"1567";"Test2"
"105";"Test3"
If the columns are not of the same size you can use this alternative that will find first a column with the maximum length and use it as the first iteration:
%dw 2.0
output application/csv quoteValues=true, separator=";", header=true
import firstWith from dw::core::Arrays
var columns=namesOf(payload.problem)
var maxColumnSize=max(columns map sizeOf(payload.problem[$]))
var maxColumnName=columns firstWith (sizeOf(payload.problem[$]) == maxColumnSize)
---
payload.problem[maxColumnName]
map ((item, index) -> (
columns map ($): payload.problem[$][index])
reduce ($$++$)
)
Related
I'm trying to process some JSON output and modify a value but struggling to get anywhere.
I have no control over the source data, which looks like this:
[
[
"dave",
"likes",
"rabbits"
],
[
"brian",
"likes",
"fish"
]
]
In pseudo code, I need to:
Select the subarray with value "brian" at index 0
Change the value at index [2] in the selected array to "cats"
Return the complete modified array
I've managed to use map and select to get the subarray I want (jq -r -c 'map(select(.[]=="brian"))), but not build that into anything more useful...
Help much appreciated!
Update the required value by specifying the array indices and using the |= update select construct
map(select(.[0] == "brian")[2] |= "cats" )
This also populates [2] with "cats" even if previously there was no value at the specific index.
Of course it goes without saying, the indices could be dynamically arrived at as well
map(select(any(.[]; . == "brian"))[2] |= "cats")
If you want to stringify column A,B,C for a few rows it makes sense that JSON.stringify returns something like [ ["1a","2a","3a"], ["1b","2b", "3b"] ].
However if you are using just one column i.e. a 1 dimensional array, then what JSON.stringify does is terrible: [ ["1a"], ["1b"] ]
What my API expects is ["1a","1b"]
What I am missing?: How can I tell it to properly format it?
From the question
However if you are using just one column i.e. a 1 dimensional array, then what JSON.stringify does is terrible: [ ["1a"], ["1b"] ]
It looks that you have a misconception, as getValues() returns a bi-dimensional no matter if the range refers to a single row or a single column. Anyway, one way to convert the bi-dimentional array into a one-dimension array is by using Array.prototype.flat().
let column = [[1],[2],[3]]
console.log(column.flat())
I am trying to extract data from some JSON with JQ - I have already got it down to the last level of data that I need to extract from, but I am completely stumped as to how to proceed with how this part of the data is formatted.
An example would be:
{
"values": [
[
1483633677,
42
]
],
"columns": [
"time",
"count_value"
],
"name": "response_time_error"
}
I would want to extract just the value for a certain column (e.g. count_value) and I can extract it by using [-1] in this specific case, but I want to select the column by its name in case they change in the future.
If you're only extracting a single value and the arrays will always correspond with eachother, you could find the index in the columns array then use that index into the values array.
It seems like values is an array of rows with those values. Assuming you want to output the values of all rows with the selected column:
$ jq --arg col 'count_value' '.values[][.columns | index($col)]' input.json
If the specified column name does not exist in .columns, then Jeff's filter will fail with a rather obscure error message. It might therefore be preferable to check whether the column name is found. Here is an illustration of how to do so:
jq --arg col count_value '
(.columns | index($col)) as $ix
| if $ix then .values[][$ix] else empty end' input.json
If you want an informative error message to be printed, then replace empty with something like:
error("specified column name, \($col), not found")
I have a column of text type be contain JSON value.
{
"customer": [
{
"details": {
"customer1": {
"name": "john",
"addresses": {
"address1": {
"line1": "xyz",
"line2": "pqr"
},
"address2": {
"line1": "abc",
"line2": "efg"
}
}
}
"customer2": {
"name": "robin",
"addresses": {
"address1": null
}
}
}
}
]
}
How can I extract 'address1' JSON field of column with query?
First I am trying to fetch JSON value then I will go with parsing.
SELECT JSON customer from text_column;
With my query, I get following error.
com.datastax.driver.core.exceptions.SyntaxError: line 1:12 no viable
alternative at input 'customer' (SELECT [JSON] customer...)
com.datastax.driver.core.exceptions.SyntaxError: line 1:12 no viable
alternative at input 'customer' (SELECT [JSON] customer...)
Cassandra version 2.1.13
You can't use SELECT JSON in Cassandra v2.1.x CQL v3.2.x
For Cassandra v2.1.x CQL v3.2.x :
The only supported operation after SELECT are :
DISTINCT
COUNT (*)
COUNT (1)
column_name AS new_name
WRITETIME (column_name)
TTL (column_name)
dateOf(), now(), minTimeuuid(), maxTimeuuid(), unixTimestampOf(), typeAsBlob() and blobAsType()
In Cassandra v2.2.x CQL v3.3.x Introduce : SELECT JSON
With SELECT statements, the new JSON keyword can be used to return each row as a single JSON encoded map. The remainder of the SELECT statment behavior is the same.
The result map keys are the same as the column names in a normal result set. For example, a statement like “SELECT JSON a, ttl(b) FROM ...” would result in a map with keys "a" and "ttl(b)". However, this is one notable exception: for symmetry with INSERT JSON behavior, case-sensitive column names with upper-case letters will be surrounded with double quotes. For example, “SELECT JSON myColumn FROM ...” would result in a map key "\"myColumn\"" (note the escaped quotes).
The map values will JSON-encoded representations (as described below) of the result set values.
If your Cassandra version is 2.1x and below, you can use the Python-based approach.
Write a python script using Cassandra-Python API
Here you have to get your row first and then use python json's loads method, which will convert your json text column value into JSON object which will be dict in Python. Then you can play around with Python dictionaries and extract your required nested keys. See the below code snippet.
from cassandra.cluster import Cluster
from cassandra.auth import PlainTextAuthProvider
import json
if __name__ == '__main__':
auth_provider = PlainTextAuthProvider(username='xxxx', password='xxxx')
cluster = Cluster(['0.0.0.0'],
port=9042, auth_provider=auth_provider)
session = cluster.connect("keyspace_name")
print("session created successfully")
rows = session.execute('select * from user limit 10')
for user_row in rows:
customer_dict = json.loads(user_row.customer)
print(customer_dict().keys()
I am trying to use JSX to convert a list of tuples to a JSON object.
The list items are based on a record definition:
-record(player, {index, name, description}).
and looks like this:
[
{player,1,"John Doe","Hey there"},
{player,2,"Max Payne","I am here"}
]
The query function looks like this:
select_all() ->
SelectAllFunction =
fun() ->
qlc:eval(qlc:q(
[Player ||
Player <- mnesia:table(player)
]
))
end,
mnesia:transaction(SelectAllFunction).
What's the proper way to make it convertable to a JSON knowing that I have a schema of the record used and knowing the structure of tuples?
You'll have to convert the record into a term that jsx can encode to JSON correctly. Assuming you want an array of objects in the JSON for the list of player records, you'll have to either convert each player to a map or list of tuples. You'll also have to convert the strings to binaries or else jsx will encode it to a list of integers. Here's some sample code:
-record(player, {index, name, description}).
player_to_json_encodable(#player{index = Index, name = Name, description = Description}) ->
[{index, Index}, {name, list_to_binary(Name)}, {description, list_to_binary(Description)}].
go() ->
Players = [
{player, 1, "John Doe", "Hey there"},
% the following is just some sugar for a tuple like above
#player{index = 2, name = "Max Payne", description = "I am here"}
],
JSON = jsx:encode(lists:map(fun player_to_json_encodable/1, Players)),
io:format("~s~n", [JSON]).
Test:
1> r:go().
[{"index":1,"name":"John Doe","description":"Hey there"},{"index":2,"name":"Max Payne","description":"I am here"}]