Using jq to build new document with paths passed as args - json

Question

As a follow up to my question that was answered so well --
Given this sample input --
[
{
"x1": "abc123",
"x2": "Larry",
"attr": {
"f1": "one",
"f2": ["two", "2"],
"f3": "three",
"x1": "Never included in below examples as this is .attr.x1, not .x1"
}
},
{
"x1": "xyz789",
"x2": "Curly",
"attr": {
"f1": ["one", "111"],
"f2": "two"
}
},
{
"x1": "def456",
"x2": "moe",
"attr": {
"f4": {
"a": 1,
"b": "two"
}
}
}
]
I need a jq program that will allow me to pass the paths I want in the output document as --args on the command line. All non-target paths are removed. Examples --
jq -f program.jq test.json --args x1 attr.f1 attr.f3 would produce ---
[
{
"x1": "abc123",
"attr": {
"f1": "one",
"f3": "three"
}
},
{
"x1": "xyz789",
"attr": {
"f1": ["one", "111"],
"f3": null
}
},
{
"x1": "def456",
"attr": {
"f1": null,
"f3": null
}
}
]
whereas jq -f program.jq test.json --args x1 x2 attr.f2 attr.f4 would produce --
[
{
"x1": "abc123",
"x2": "Larry",
"attr": {
"f2": ["two", "2"],
"f4": null
}
},
{
"x1": "xyz789",
"x2": "Curly",
"attr": {
"f2": "two",
"f4": null
}
},
{
"x1": "def456",
"x2": "moe",
"attr": {
"f2": null,
"f4": {
"a": 1,
"b": "two"
}
}
}
]
Notes:
I'm not particularly concerned about the syntax. If --argsjson, a different way to specify the path, piping multiple commands together or really any syntax is more appropriate, that's fine.
Key names can be duplicated at different levels of the hierarchy, but only the path matching "exactly" should be included. .attr.x1 is always ignored in the examples provided. In those examples, I want .x1, not .attr.x1.
This example is fairly trivial, a real input document may have up to ~5 or so levels of hierarchy and I need to retrieve key/value's from any depth (ie. .attr.level2.level3.level4.mykey.
In practice, the input might be a small document with a dozen entries and < 20 keys to a reasonably large one with 100k entries and ~100 keys. Performance is important, but not critical.
EDIT - I also wouldn't be adverse to using some type of templating (Jinja2, etc) to generate the jq program at runtime. Not ideal, but perfectly fine if that's the answer.
Any ideas? I've hacked around with map, with_entries and many schemes and can't find the right syntax given the need to pass the "target" path as args.
Thanks!

Answer 1

Read or convert the path arguments as path arrays and Use getpath and setpath to filter and build your objects:
jq '
map(. as $item | reduce ($ARGS.positional[] / ".") as $p ({};
setpath($p; $item | getpath($p))
))
' file.json --args x1 attr.f1 attr.f3
[
{
"x1": "abc123",
"attr": {
"f1": "one",
"f3": "three"
}
},
{
"x1": "xyz789",
"attr": {
"f1": [
"one",
"111"
],
"f3": null
}
},
{
"x1": "def456",
"attr": {
"f1": null,
"f3": null
}
}
]
Demo
jq '
map(. as $item | reduce ($ARGS.positional[] / ".") as $p ({};
setpath($p; $item | getpath($p))
))
' file.json --args x1 x2 attr.f2 attr.f4
[
{
"x1": "abc123",
"x2": "Larry",
"attr": {
"f2": [
"two",
"2"
],
"f4": null
}
},
{
"x1": "xyz789",
"x2": "Curly",
"attr": {
"f2": "two",
"f4": null
}
},
{
"x1": "def456",
"x2": "moe",
"attr": {
"f2": null,
"f4": {
"a": 1,
"b": "two"
}
}
}
]
Demo