Get json object that has latest timestamp using jq - json

I have a below json file but I'm struggling to only display the description with the latest createdDate.
I tried with
>
<
todateiso8601?
now
and a few more but I can't get this to work.
Would anyone be able to help?
JSON:
{
"items": [
{
"createdDate": 1543585940,
"id": "awefef",
"description": "this is description 1"
},
{
"createdDate": 1555324487,
"id": "hjkvhuk",
"description": "this is description 2"
},
{
"createdDate": 1547034297,
"id": "xdfxdfv",
"description": "this is description 3"
}
]
}

Simply sort by .createdDate and (assuming you only want one value even if there is more than one with the greatest .createdDate value), select the last one:
.items
| sort_by(.createdDate)[-1].description
Ties
If you want all the descriptions in the case of ties:
.items
| sort_by(.createdDate)
| (.[-1].createdDate) as $max
| .[]
| select($max == .createdDate)
| .description

EDIT: use peaks answer it is superior
Here is a simple script that does this in 2 commands. Probably can be done in 1 but alas my nooblet skills were not enough
You can pipe to max with an array of numbers in JQ and it will return the largest value in the input array.
Then we use select to grab the object containing the max value and output the description.
We will also use arg which allows us to reference a local environment variable, and we need to cast it to a number or JQ thinks it's a string.
maxDate=$(cat tmp.json | jq '[.items[].createdDate] | max')
cat tmp.json | jq --arg maxDate "$maxDate" '.[][] | select(.createdDate == ($maxDate | tonumber)).description'
Output:
"this is description 2"
In the future, please post your desired output as well as your question so responders can be confident they are solving the problem to your liking

Related

How do I print a specific value of an array given a condition in jq if there is no key specified

I am trying to output the value for .metadata.name followed by the student's name in .spec.template.spec.containers[].students[] array using the regex test() function in jq.
I am having trouble to retrieve the individual array value since there is no key specified for the students[] array.
For example, if I check the students[] array if it contains the word "Jeff", I would like the output to display as below:
student-deployment: Jefferson
What i have tried:
I've tried the command below which somewhat works but I am not sure how to get only the "Jefferson" value. The command below would print out all of the students[] array values which is not what I want. I am using Powershell to run the command below.
kubectl get deployments -o json | jq -r '.items[] | select(.spec.template.spec.containers[].students[]?|test("\"^Jeff.\"")) | .metadata.name, "\":\t\"", .spec.template.spec.containers[].students'
Is there a way to print a specific value of an array given a condition in jq if there is no key specified? Also, would the solution work if there are multiple deployments?
The deployment template below is in json and I shortened it to only the relevant parts.
{
"apiVersion": "v1",
"items": [
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"name": "student-deployment",
"namespace": "default"
},
"spec": {
"template": {
"spec": {
"containers": [
{
"students": [
"Alice",
"Bob",
"Peter",
"Sally",
"Jefferson"
]
}
]
}
}
}
}
]
}
For this approch, we introduce a variable $pattern. You may set it with --arg pattern to your regex, e.g. "Jeff" or "^Al" or "e$" to have the student list filtered by test, or leave it empty to see all students.
Now, we iterate over all .item[] elements (i.e. over "all deployments"). For each found, we output the content of .metadata.name followed by a literal colon and a space. Then we iterate again over all .spec.template.spec.containers[].students[], perform the pattern test and concatenate the outcome.
To print out raw strings instead of JSON, we use the -r option when calling jq.
kubectl get deployments -o json \
| jq --arg pattern "Jeff" -r '
.items[]
| .metadata.name + ": " + (
.spec.template.spec.containers[].students[]
| select(test($pattern))
)
'
To retrieve the "students" array(s) in the input, you could use this filter:
.items[]
| paths(objects) as $p
| getpath($p)
| select( objects | has("students") )
| .students
You can then add additional filters to select the particular student(s) of interest, e.g.
| .[]
| select(test("Jeff"))
And then add any postprocessing filters, e.g.
| "student-deployment: \(.)"
Of course you can obtain the students array in numerous other ways.

jq: grab specific repeating key-value from first N elements

I am trying to parse https://api.weather.gov/gridpoints/PHI/47,91/forecast/hourly, with mild success.
{
"number": 1,
"name": "",
"startTime": "2020-12-16T13:00:00-05:00",
"endTime": "2020-12-16T14:00:00-05:00",
"isDaytime": true,
"temperature": 30,
"temperatureUnit": "F",
"temperatureTrend": null,
"windSpeed": "15 mph",
"windDirection": "NE",
"icon": "https://api.weather.gov/icons/land/day/snow,40?size=small",
"shortForecast": "Chance Light Snow",
"detailedForecast": ""
}
I started with jq '.properties.periods[0]' to grab the first element, worked with jq '.properties.periods[0].shortForecast' and I figured out that jq '.properties.periods[0,1,2,3]' gets me the first 4 elements in the array.
However, I run into a syntax error if I try jq '.properties.periods[:3]'
jq: error: syntax error, unexpected '['
which I thought would be a shorthand for 0-3.
Additionally, I only want (the same, repeating) specific K/V pairs from each element (eg: shortForecast, temperature, etc.), but I have not been able to figure out how to combine it all into one jq statement.
So how do I grab specific values from the first X elements of an array? (I dont really need the keys, just the values.)
Bonus: would be great to have all values from each element on a single line.
Sample:
"2020-12-16T14:00:00-05:00" 30 "Chance Light Snow"
"2020-12-16T15:00:00-05:00" 30 "Snow"
"2020-12-16T16:00:00-05:00" 29 "Heavy Snow"
.properties.periods[:3] evaluates to an array of the three items, whereas .properties.periods[0,1,2] produces an itemization. So the abbreviation of the latter would be:
.properties.periods[:3][]
Selection
There are numerous possibilities, e.g. to get a specific set of key-value pairs on a single line:
jq -c '.properties.periods[:3][]
| {shortForecast, temperature}' input.json
To select just the values as CSV:
.properties.periods[:3][]
| {shortForecast, temperature}
| [.[]]
| #csv
You might like to use #tsv instead, or join(" "), or ....
Bonus
To get all the values in the order in which they are given, you could simply omit the selection line: | {....}
However, that would not be so robust. The following would be safer:
.properties.periods[:3]
| (.[0] | keys_unsorted) as $keys
| .[]
| [.[$keys[]]]
| #tsv

Search and extract value using JQ command line processor

I have a JSON file very similar to the following:
[
{
"uuid": "832390ed-58ed-4338-bf97-eb42f123d9f3",
"name": "Nacho"
},
{
"uuid": "5b55ea5e-96f4-48d3-a258-75e152d8236a",
"name": "Taco"
},
{
"uuid": "a68f5249-828c-4265-9317-fc902b0d65b9",
"name": "Burrito"
}
]
I am trying to figure out how to use the JQ command line processor to first find the UUID that I input and based on that output the name of the associated item. So for example, if I input UUID a68f5249-828c-4265-9317-fc902b0d65b9 it should search the JSON file, find the matching UUID and then return the name Burrito. I am doing this in Bash. I realize it may require some outside logic in addition to JQ. I will keep thinking about it and put an update here in a bit. I know I could do it in an overly complicated way, but I know there is probably a really simple JQ method of doing this in one or two lines. Please help me.
https://shapeshed.com/jq-json/#how-to-find-a-key-and-value
You can use select:
jq -r --arg query Burrito '.[] | select( .name == $query ) | .uuid ' tst.json

Count records with missing keys using jq

Below is a sample output that is returned when calling an API:
curl "https://mywebsite.com/api/cars.json&page=1" | jq '.'
Using jq, how would one count the number or records where the charge key is missing? I understand that the first bit of code would include jq '. | length' but how would one filter out objects that contain or don't contain a certain key value ?
If applied to the sample below, the output would be 1
{
"current_page": 1,
"items": [
{
"id": 1,
"name": "vehicleA",
"state": "available",
"charge": 100
},
{
"id": 2,
"name": "vehicleB",
"state": "available",
},
{
"id": 3,
"name": "vehicleB",
"state": "available",
"charge": 50
}
]
}
Here is a solution using map and length:
.items | map(select(.charge == null)) | length
Try it online at jqplay.org
Here is a more efficient solution using reduce:
reduce (.items[] | select(.charge == null)) as $i (0;.+=1)
Try it online at jqplay.org
Sample Run (assuming corrected JSON data in data.json)
$ jq -M 'reduce (.items[] | select(.charge == null)) as $i (0;.+=1)' data.json
1
Note that each of the above takes a minor shortcut assuming that the items won't have a "charge":null member. If some items could have a null charge then the test for == null won't distinguish between those items and items without the charge key. If this is a concern the following forms of the above filters which use has are better:
.items | map(select(has("charge")|not)) | length
reduce (.items[] | select(has("charge")|not)) as $i (0;.+=1)
Here is a solution that uses a simple but powerful utility function worthy perhaps of your standard library:
def sigma(stream): reduce stream as $s (null; . + $s);
The filter you'd use with this would be:
sigma(.items[] | select(has("charge") == false) | 1)
This is very efficient as no intermediate array is required, and no useless additions of 0 are involved. Also, as mentioned elsewhere, using has is more robust than making assumptions about the value of .charge.
Startup file
If you have no plans to use jq's module system, you can simply add the above definition of sigma to the file ~/.jq and invoke jq like so:
jq 'sigma(.items[] | select(has("charge") == false) | 1)'
Better yet, if you also add def count(s): sigma(s|1); to the file, the invocation would simply be:
jq 'count(.items[] | select(has("charge") | not))'
Standard Library
If for example ~/.jq/jq/jq.jq is your standard library, then assuming count/1 is included in this file, you could invoke jq like so:
jq 'include "jq"; count(.items[] | select(has("charge") == false))'

Select entries based on multiple values in jq

I'm working with JQ and I absolutely love it so far. I'm running into an issue I've yet to find a solution to anywhere else, though, and wanted to see if the community had a way to do this.
Let's presume we have a JSON file that looks like so:
{"author": "Gary", "text": "Blah"}
{"author": "Larry", "text": "More Blah"}
{"author": "Jerry", "text": "Yet more Blah"}
{"author": "Barry", "text": "Even more Blah"}
{"author": "Teri", "text": "Text on text on text"}
{"author": "Bob", "text": "Another thing to say"}
Now, we want to select rows where the value of author is equal to either "Gary" OR "Larry", but no other case. In reality, I have several thousand names I'm checking against, so simply stating the direct or conditional (e.g. cat blah.json | jq -r 'select(.author == "Gary" or .author == "Larry")') isn't sufficient. I'm trying to do this via the inside function like so but get an error dialog:
cat blah.json | jq -r 'select(.author | inside(["Gary", "Larry"]))'
jq: error (at <stdin>:1): array (["Gary","La...) and string ("Gary") cannot have their containment checked
What would be the best method for doing something like this?
inside and contains are a bit weird. Here are some more straightforward solutions:
index/1
select( .author as $a | ["Gary", "Larry"] | index($a) )
any/2
["Gary", "Larry"] as $whitelist
| select( .author as $a | any( $whitelist[]; . == $a) )
Using a dictionary
If performance is an issue and if "author" is always a string, then a solution along the lines suggested by #JeffMercado should be considered. Here is a variant (to be used with the -n command-line option):
["Gary", "Larry"] as $whitelist
| ($whitelist | map( {(.): true} ) | add) as $dictionary
| inputs
| select($dictionary[.author])
IRC user gnomon answered this on the jq channel as follows:
jq 'select([.author] | inside(["Larry", "Garry", "Jerry"]))'
The intuition behind this approach, as stated by the user was: "Literally your idea, only wrapping .author as [.author] to coerce it into being a single-item array so inside() will work on it." This answer produces the desired result of filtering for a series of names provided in a list as the original question desired.
You can use objects as if they're sets to test for membership. Methods operating on arrays will be inefficient, especially if the array may be huge.
You can build up a set of values prior to reading your input, then use the set to filter your inputs.
$ jq -n --argjson names '["Larry","Garry","Jerry"]' '
(reduce $names[] as $name ({}; .[$name] = true)) as $set
| inputs | select($set[.author])
' blah.json