How to flip lines and change keys in JQ - json

I'm creating a interactive map of my campus, the ideia is to replicate what I did on uMaps, in this link. The geojson was downloaded from UMap and I'm using the coordinates that came with it.
My first issue is my coordinates in the json, originally were a GeoJson, are sorted wrongly, my long came first then lat, thus when parse Google Maps can't read properly.
Json:
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {
"name": "Almoxarifado / Patrimônio"
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-52.163317,
-32.075462
],
[
-52.163884,
-32.075467
],
[
-52.163883,
-32.075336
],
[
-52.163321,
-32.075332
],
[
-52.163317,
-32.075462
]
]
]
}
},
{
...
},
{
...
},
...
]
}
So, I have to flip the coordinates lines to proper put in my Google Maps Api.
And my second issue is chaging the "type" key to "layer", for a better sepation layers in my app.
I've tried:
.features[] | .["type"] |= .["new value"]
How ever that changes the value and only accepts float values
Any help, advice or guidance would be greatly appreciated.

Part 1
flip the coordinates lines
For clarity and ease of testing, let's define a helper function:
# input: a JSON object with a coordinates array of arrays of pairs
def flip: .coordinates |= map(map(reverse)) ;
or even better - before invoking reverse, check the array has the expected length, e.g.:
def flip:
.coordinates
|= map(map(if length == 2 then reverse
else error("incorrect length: \(length)")
end)) ;
To flip the coordinates, we can now simply write:
.features[].geometry |= flip
Part 2
change the "type" key to "layer"
{layer: .type} + .
| del(.type)
Putting it together
{layer:.type} + .
| del(.type)
| .features[].geometry |= flip

Related

Merging JSON with array to create new file

I am trying to make a map of the U.S. with Mapbox that shows median home price by county. I have a .json file that contains all the counties and is already accepted by Mapbox tileset -
{
"type": "Topology",
"transform": {
"scale": [
0.035896170617061705,
0.005347309530953095
],
"translate": [
-179.14734,
17.884813
]
},
"objects": {
"us_counties_20m": {
"type": "GeometryCollection",
"geometries": [
{
"type": "Polygon",
"arcs": [],
"id": "0500000US01001"
},
{
"type": "Polygon",
"arcs": [],
"id": "0500000US01009"
},
{
"type": "Polygon",
"arcs": [],
"id": "0500000US01017"
},
{
"type": "Polygon",
"arcs": [],
"id": "0500000US01021"
}
]
}
}
}
Basically, it's a json file with "type" (Polygon), "arcs" (to map the county), and "id", which is an ID for the county.
This is great and accepted by Mapbox Tilesets to give me a visualization by county, but I need to add in median home price by county (in order to get colors by county, based on price).
I have a second json file that is more like an array, which has
[
{
"0500000US01001": 51289.0,
"0500000US01009": 46793.0,
"0500000US01017": 39857.0,
"0500000US01021": 48859.0
}
]
and so on, but basically it has the ID -> median home price per county. The ID's are the same between these 2 files, and of the same quantity. So I need get a 3rd json file out of these, which has "type", "arcs", "id", and "PRICE" (the addition).
These files are huge - any suggestions? I tried using jq but received an error that
jq: error ... object ({"type":"To...) and array ([{"0500000U...) cannot be multiplied
Thanks in advance!
A straightforward approach would be saving the second file into a variable and using it as a reference while updating the first file. E.g:
jq 'add as $prices | input
| .objects.us_counties_20m.geometries[] |= . + {PRICE: $prices[.id]}' file2 file1
add can be substituted with .[0] if the array in file2 contains only one object.
Online demo

Create a new json string from jq output elements

My jq command returns objects in brackets but without comma separators. But I would like to create a new json string from it.
This call finds all elements of arr that have a FooItem in them and then returns texts from the nested array at index 3:
jq '.arr[] | select(index("FooItem")) | .[3].texts'
on this json (The original has more elements ):
{
"arr": [
[
"create",
"w199",
"FooItem",
{
"index": 0,
"texts": [
"aBarfoo",
"avalue"
]
}
],
[
"create",
"w200",
"NoItem",
{
"index": 1,
"val": 5,
"hearts": 5
}
],
[
"create",
"w200",
"FooItem",
{
"index": 1,
"texts": [
"mybarfoo",
"bValue"
]
}
]
]
}
returns this output:
[
"aBarfoo",
"avalue"
]
[
"mybarfoo",
"bValue"
]
But I'd like to create a new json from these objects that looks like this:
{
"arr": [
[
"aBarfoo",
"avalue"
],
[
"mybarfoo",
"bValue"
]
]
}
Can jq do this?
EDIT
One more addition: Considering that texts also has strings of zero length, how would you delete those/not have them in the result?
"texts": ["",
"mybarfoo",
"bValue",
""
]
You can always embed a stream of (zero or more) JSON entities within some other JSON structure by decorating the stream, that is, in the present case, by wrapping the STREAM as follows:
{ arr: [ STREAM ] }
In the present case, however, we can also take the view that we are simply editing the original document, and accordingly use a variation of the map(select(...)) idiom:
.arr |= map( select(index("FooItem")) | .[3].texts)
This latter approach ensures that the context of the "arr" key is preserved.
Addendum
To filter out the empty strings, simply add another map(select(...)):
.arr |= map( select(index("FooItem"))
| .[3].texts | map(select(length>0)))

How to use jq to reconstruct complete contents of json file, operating only on part of interest?

All the examples I've seen so far "reduce" the output (filter out) some part. I understand how to operate on the part of the input I want to, but I haven't figured out how to output the rest of the content "untouched".
The particular example would be an input file with several high level entries "array1", "field1", "array2", "array3" say. Each array contents is different. The specific processing I want to do is to sort "array1" entries by a "name" field which is doable by:
jq '.array1 | sort_by(.name)' test.json
but I also want this output as "array1" as well as all the other data to be preserved.
Example input:
{
"field1": "value1",
"array1":
[
{ "name": "B", "otherdata": "Bstuff" },
{ "name": "A", "otherdata": "Astuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
Expected output:
{
"field1": "value1",
"array1":
[
{ "name": "A", "otherdata": "Astuff" },
{ "name": "B", "otherdata": "Bstuff" }
],
"array2" :
[
array2 stuff
],
"array3" :
[
array3 stuff
]
}
I've tried using map but I can't seem to get the syntax correct to be able to handle any type of input other than the array I want to be sorted by name.
Whenever you use the assignment operators (=, |=, +=, etc.), the context of the expression is kept unchanged. So as long as your top-level filter(s) are assignments, in the end, you'll get the rest of the data (with your changes applied).
In this case, you're just sorting the array1 array so you could just update the array.
.array1 |= sort_by(.name)

Invalid GeoJSON for polygon

I am trying to plot a polygon on a google map using geojson. Here is the PHP code where I am trying to build the polygon using four bounds values return from the query result array:
$arr3_poly = Array(
"type" => "Polygon",
"coordinates" => Array()
);
foreach ($q->result_array() as $row) {
$arr3_poly["coordinates"][] = Array(
floatval($row['v3_bounds_sw_lat']),
floatval($row['v3_bounds_sw_lng']),
floatval($row['v3_bounds_ne_lat']),
floatval($row['v3_bounds_ne_lng']),
);
}
When I then do json_encode($arr3_poly, JSON_PRETTY_PRINT);, this is the resulting output:
{
"type": "Polygon",
"coordinates": [
[
43.8526377,
-79.0499898,
43.8526509,
-79.0499877
],
[
43.8526546,
-79.0501977,
43.8526678,
-79.0501957
],
[
43.8526716,
-79.0504057,
43.8526848,
-79.0504037
]
]
}
There must be something wrong with this geojson because when I try to validate it at geojsonlint.com it returns with this error saying Failed to validate field 'coordinates' list schema.:
Any ideas what I am doing wrong? Thanks.
This works for me in geojsonlint.com (changed your points slightly to make it not look like a straight line):
{
"type": "Polygon",
"coordinates": [
[
[43.8526377,-79.0499898],
[43.854,-79.051],
[43.8526716,-79.0504057],
[43.8526377,-79.0499898]
]
]
}
However, looking closer at that map, they are in Antarctica, you probably wanted this, which is in Canada, near Toronto:
{
"type": "Polygon",
"coordinates": [
[
[-79.0499898,43.8526377],
[-79.051,43.854],
[-79.0504057,43.8526716],
[-79.0499898,43.8526377]
]
]
}
GeoJSON coordinates:
The order of elements must follow x, y, z order (longitude, latitude, altitude for
coordinates in a geographic coordinate reference system).
Which is the opposite order from a google.maps.LatLng object (that is Latitude, Longitude).

Using jq to list keys in a JSON object

I have a hierarchically deep JSON object created by a scientific instrument, so the file is somewhat large (1.3MB) and not readily readable by people. I would like to get a list of keys, up to a certain depth, for the JSON object. For example, given an input object like this
{
"acquisition_parameters": {
"laser": {
"wavelength": {
"value": 632,
"units": "nm"
}
},
"date": "02/03/2525",
"camera": {}
},
"software": {
"repo": "github.com/username/repo",
"commit": "a7642f",
"branch": "develop"
},
"data": [{},{},{}]
}
I would like an output like such.
{
"acquisition_parameters": [
"laser",
"date",
"camera"
],
"software": [
"repo",
"commit",
"branch"
]
}
This is mainly for the purpose of being able to enumerate what is in a JSON object. After processing the JSON objects from the instrument begin to diverge: for example, some may have a field like .frame.cross_section.stats.fwhm, while others may have .sample.species, so it would be convenient to be able to interrogate the JSON object on the command line.
The following should do exactly what you want
jq '[(keys - ["data"])[] as $key | { ($key): .[$key] | keys }] | add'
This will give the following output, using the input you described above:
{
"acquisition_parameters": [
"camera",
"date",
"laser"
],
"software": [
"branch",
"commit",
"repo"
]
}
Given your purpose you might have an easier time using the paths builtin to list all the paths in the input and then truncate at the desired depth:
$ echo '{"a":{"b":{"c":{"d":true}}}}' | jq -c '[paths|.[0:2]]|unique'
[["a"],["a","b"]]
Here is another variation uing reduce and setpath which assumes you have a specific set of top-level keys you want to examine:
. as $v
| reduce ("acquisition_parameters", "software") as $k (
{}; setpath([$k]; $v[$k] | keys)
)