jq iterate over a array of values a subset at a time - json

Question

I have json (that actually starts as csv) of the form of an array of elements of the form:
{
"field1" : "value1"
"field2.1; Field2.2 Field2.3" : "Field2.1Value0; Field2.2Value0; Field2.3Value0; Field2.1Value1; Field2.2Value1; Field2.3Value1; ..."
}
...
I would like to iterate over the string of the field "field2.1; Field2.2 Field2.3", three ";" separated items at a time to produce an array of key value pairs
{
"field1" : "value1"
"newfield" : [
{ "Field2.1": "Field2.1Value0",
"Field2.2": "Field2.2Value0",
"Field2.3": "Field2.1Value0" },
{ "Field2.1": "Field2.1Value1",
"Field2.2": "Field2.2Value1",
"Field2.3": "Field2.3Value1"},
...
]
}
...
note that there are actually a couple of keys that need to be expanded like this. Each with a variable number of "sub-keys".
In other words, the original CSV file contains some columns that represent tuples of field values separated by semicolons.
I know how to get down to the "field2.1; Field2.2 Field2.3" and say split it on the ";" but then I'm stuck trying to iterate through that 3 (or however many) items at a time to produce the separate 3 tuples.
The real world example/context is the format of the CSV from catalog export from the Google Play Store.
For example Field2.1 is Locale, Field2.2 is Title and Field3.3 is Description:
jq '."Locale; Title; Description" |= split(";") '
If possible, then it would be nice if the iteration is based on the number of semicolon separated "subfields" in the key value. There is another column that has a similar format for the price in each country.

Answer 1

The following assumes the availability of splits/1 for splitting a string based on a regex. If your jq does not have it, and if you cannot or don't want to upgrade, you could devise a workaround using split/1, which only works on strings.
First, let's start with a simple variant of the problem that does not require recycling the headers. If the following jq program is in a file (say program.jq):
# Assuming header is an array of strings,
# create an object from an array of values:
def objectify(headers):
. as $in
| reduce range(0; headers|length) as $i ({}; .[headers[$i]] = ($in[$i]) );
# From an object of the form {key: _, value: _},
# construct an object by splitting each _
def devolve:
if .key|index(";")
then .key as $key
| ( [.value | splits("; *")] ) | objectify([$key | splits("; *")])
else { (.key): .value }
end;
to_entries | map( devolve )
and if the following JSON is in input.json:
{
"field1" : "value1",
"field2.1; Field2.2; Field2.3" : "Field2.1Value0; Field2.2Value0; Field2.3Value0"
}
then the invocation:
jq -f program.jq input.json
should yield:
[
{
"field1": "value1"
},
{
"field2.1": "Field2.1Value0",
"Field2.2": "Field2.2Value0",
"Field2.3": "Field2.3Value0"
}
]
It might make sense to add some error-checking or error-correcting code.
Recycling the headers
Now let's modify the above so that headers will be recycled in accordance with the problem statement.
def objectifyRows(headers):
(headers|length) as $m
| (length / $m) as $n
| . as $in
| reduce range(0; $n) as $i ( [];
.[$i] = (reduce range(0; $m) as $h ({};
.[headers[$h]] = $in[($i * $m) + $h] ) ) );
def devolveRows:
if .key|index(";")
then .key as $key
| ( [.value | splits("; *")] )
| objectifyRows([$key | splits("; *")])
else { (.key): .value }
end;
to_entries | map( devolveRows )
With input:
{
"field1" : "value1",
"field2.1; Field2.2; Field2.3" :
"Field2.1Value0; Field2.2Value0; Field2.3Value0; Field2.4Value0; Field2.5Value0; Field2.6Value0"
}
the output would be:
[
{
"field1": "value1"
},
[
{
"field2.1": "Field2.1Value0",
"Field2.2": "Field2.2Value0",
"Field2.3": "Field2.3Value0"
},
{
"field2.1": "Field2.4Value0",
"Field2.2": "Field2.5Value0",
"Field2.3": "Field2.6Value0"
}
]
]
This output can now easily be tweaked along the lines suggested by the OP, e.g. to introduce a new key, one could pipe the above into:
.[0] + { newfield: .[1] }
Functional definitions
Here are reduce-free but efficient (assuming jq >= 1.5) implementations of objectify and objectifyRows:
def objectify(headers):
[headers, .] | transpose | map( {(.[0]): .[1]} ) | add;
def objectifyRows(headers):
def gather(n):
def g: if length>0 then .[0:n], (.[n:] | g ) else empty end;
g;
[gather(headers|length) | objectify(headers)] ;

Answer 2

Here is my almost final solution that inserts the new key as well as uses the first element of the ";" list as the key for sorting the array.
def objectifyRows(headers):
(headers|length) as $m
| (headers[0]) as $firstkey
| (length / $m) as $n
| . as $in
| reduce range(0; $n) as $i ( [];
.[$i] = (reduce range(0; $m) as $h ({};
.[headers[$h]] = $in[($i * $m) + $h] ) ) )
;
def devolveRows:
if .key|index(";")
then .key as $multikey
| ( [.value | splits("; *")] )
# Create a new key with value being an array of the "splits"
| { ($multikey): objectifyRows([$multikey | splits("; *")])}
# here "arbitrarily" sort by the first split key
| .[$multikey] |= sort_by(.[[$multikey | splits("; *")][0]])
else { (.key): .value }
end;
to_entries | map( devolveRows )