I need to download application JAR for execution, and I managed to figure it out up to this step part in my script:
jq '.files[] | select(.uri | contains("RELEASE") and contains(".jar"))'
This gives me a bunch of results that looks like multiple blocks of this:
{
other versions
}
{
"uri": "/2.0.6-RELEASE/app-name-2.0.6-RELEASE.jar",
"size": 32981192,
"lastModified": "2020-02-05T14:21:06.728-05:00",
"folder": false,
"sha1": "whatever",
"sha2": "whatever2",
"mdTimestamps": {
"properties": "2020-02-05T14:21:13.468-05:00"
}
}
Based on the documentation, I tried to extend this to:
jq '.files[] | select(.uri | contains("RELEASE") and contains(".jar")) | max_by(.lastModified)'
But it returns error
jq: error (at <stdin>:3284): Cannot index string with string "lastModified"
Documentation is vague on combining all the different search filters together.. I am not sure what is the correct syntax, please.
My goal: Retrieve only one element, ideally just the URI string so that I can use that to fetch the JAR directly on my next command without further processing.
I also tried the documented syntax for max_by with suffix: | {uri} - same error.
max_by requires an array as input, so in the spirit of your attempt, you might wish to consider:
.files
| map( select(.uri | contains("RELEASE")
and contains(".jar")))
| max_by(.lastModified)
p.s.
In future, please follow the guidelines at
http://stackoverflow.com/help/mcve
Related
The data in the column looks like the below:
{
"activity": {
"token": "e7b64be4-74d4-7a6d-a74b-xxxxxxx",
"route": "http://example.com/enroll/confirmation",
"url_parameters": {
"Success": "True",
"ContractNumber": "003992314W",
"Barcode": "1908Y10Z",
"price": "8.99"
},
"server_info": {
"cookie": [
"_ga=xxxx; _fbp=xxx; _hjid=xxx; XDEBUG_SESSION=XDEBUG_ECLIPSE;"
],
"upgrade-insecure-requests": [
"1"
],
},
"campaign": "Unknown/None",
"ip": "192.168.10.1",
"entity": "App\\Models\\User",
"entity_id": "1d9f3066-13ce-4659-b10d-xxxxx",
},
"time": "2021-05-21 20:15:02"
}
My code that I am using is below:
SELECT *
FROM websote.stored_events
WHERE JSON_EXTRACT(event_properties, '$.route') = 'http://example.com/enroll/confirmation'
ORDER BY created_at DESC LIMIT 500;
The code works on the other the json values just not the url ones. I've tried escaping the values in MySQL like the below:
SELECT *
FROM websote.stored_events
WHERE JSON_EXTRACT(event_properties, '$.route') = 'http:///example.com//enroll//confirmation'
ORDER BY created_at DESC LIMIT 500;
But still no luck. Any help on this would be appreciated!
Route is a nested property; I would have expected the path to be
JSON_EXTRACT(event_properties, '$.activity.route')
Your example data isn't valid JSON. You can't have a comma after the last element in an object or array:
"entity_id": "1d9f3066-13ce-4659-b10d-xxxxx",
},
^ here
If I remove that and other similar cases, I can test your data inserts into a JSON column and I can extract the object element you described:
mysql> select json_extract(event_properties, '$.activity.route') as route from stored_events;
+------------------------------------------+
| route |
+------------------------------------------+
| "http://example.com/enroll/confirmation" |
+------------------------------------------+
Note the value is returned with double-quotes. This is because it's returned as a JSON document, a scalar string. If you want the raw value, you have to unquote it:
mysql> select json_unquote(json_extract(event_properties, '$.activity.route')) as route from stored_events;
+----------------------------------------+
| route |
+----------------------------------------+
| http://example.com/enroll/confirmation |
+----------------------------------------+
If you want to search for that value, you would have to do a similar expression:
select * from stored_events
where json_unquote(json_extract(event_properties, '$.activity.route'))
= 'http://example.com/enroll/confirmation'
Searching based on object properties stored in JSON has disadvantages.
It requires complex expressions that force you (and anyone else you needs to maintain your code) to learn a lot of details about how JSON works.
It cannot be optimized with an index. This query will run a table-scan. You can add virtual columns with indexes, but that adds to complexity and if you need to ALTER TABLE to add virtual columns, it misses the point of JSON to store semi-structured data.
The bottom line is that if you find yourself using JSON functions in the WHERE clause of a query, it's a sign that you should be storing the column you want to search as a normal column, not as part of a JSON document.
Then you can write code that is easy to read, easy for your colleagues to maintain, and can be optimized easily with indexes:
SELECT * FROM stored_events
WHERE route = 'http://example.com/enroll/confirmation';
You can still store other properties in the JSON document, but the ones you want to be searchable should be stored in normal columns.
You might like to view my presentation How to Use JSON in MySQL Wrong.
Is it possible to use a variable to hold a dot notation path? (I'm probably not using the correct term.)
For example, given the following json:
{
"people": [
{
"names": {
"given": "Alice",
"family": "Smith"
},
"id": 47
},
{
"id": 42
}
]
}
Is it possible to construct something like:
.names.given as $ng | .people[] | select(.id==47) | ($ng)
and output "Alice"?
The idea is to allow easier modification of a complex expression. I've tried various parens and quotes with increasing literal results ('.names.given' and '$ng')
The answer is no and yes: as you've seen, once you write an expression such as .names.given as $ng, $ng holds the JSON values, not the path.
But jq does support path expressions in the form of arrays of strings and/or non-negative integers. These can be used to access values in conjunction with the built-in getpath/1.
So you could, for example, write something along the lines of:
["names", "given"] as $ng
| .people[]
| select(.id==47)
| getpath($ng)
Converting jq paths to JSON arrays
It's possible to convert a "dot notation" path into an "array path" using path/1; e.g. the assignment to $ng above could be written as:
(null | path(.names.given)) as $ng
Your question and the example you provided seems very confusing to me. The jist that I got is that you want to assign a name to a value obtained from dot notation and then use it at a later point in time.
See if this is of any help -
.people | map(select(.id = 47))[0].names.given as $ng | $ng
I am learning jq. I have content as below:
{
"amphorae_links":[
],
"amphorae":[
{
"status":"BOOTING",
"loadbalancer_id":null,
"created_at":"2020-06-23T08:56:56",
"vrrp_id":null,
"id":"6d66935e-6d39-40c9-bb0d-dd6a734dc77b"
},
{
"status":"ALLOCATED",
"loadbalancer_id":"79970c9a-b0ba-4cde-a7e6-16b61641a7b8",
"created_at":"2020-06-25T06:41:56",
"vrrp_id":1,
"id":"872c08ee-9b21-4b26-9550-c2ffb4a1ad59"
}
]
}
I want to have an output like
"ALLOCATED=872c08ee-9b21-4b26-9550-c2ffb4a1ad59,79970c9a-b0ba-4cde-a7e6-16b61641a7b8"
I tried to use below, but I don't know how to remove the line with status as "BOOTING".
.amphorae[] | "\(.status)=\(.id),\(.loadbalancer_id)"
Thanks anyone for the helping.
You have a couple of options:
You can specify the second index of the amphorae array, that is if you are sure the "ALLOCATED" part will always be the second index of the amphorae array
.amphorae[1] | "\(.status)=\(.id),\(.loadbalancer_id)"
This is a little bit more complex, but will always ensure you get only the element of the amphorae array where the status key is 'ALLOCATED'
.amphorae | map(select(.status == "ALLOCATED"))[] | "\(.status)=\(.id),\(.loadbalancer_id)"
On the jq manual page there are a few examples of output formatting, particularly some shortcuts for when you want to just echo exactly what was in the input JSON.
What if I want to echo exactly what was in the input, but only for keys that match a certain pattern?
For example, given input like so ...
[
{"Name":"Widgets","Size":10,"SymUS":"Widg","SymCN":"Zyin","SymJP":"Kono"},
{"Name":"Blodgets","Size":400,"SymUS":"Blodg","SymAU":"Blod","SymJP":"Kado"},
{"Name":"Fonzes","Size":11,"SymRU":"Fyet","SymBR":"Foao"}
]
Say I want to select all objects where the Name ends in "ets" and then display the Name and all attributes of the form Sym*. All I know about those attributes is that there will be one or more per JSON object, and the names have the format Sym followed by a two-letter ISO country code.
I would like to just do this:
jq '.[] | select(.Name | endswith("ets")) | {Name, Sym*}'
but that's not a thing.
Is this just not something jq is designed to handle in a single operation? Should I do a first pass through the file to collect all the possible keys and then list them all explicitly via a slurpfile?
The key to a simple solution to the problem is to_entries, as described in the online manual. With your example data, the following filter produces the output shown below, in accordance with what I understand to be the expectations:
.[]
| select(.Name | test("ets$"))
| {Name} + (to_entries | map(select(.key|test("^Sym"))) | from_entries)
You might want to refine the regex tests, and/or make other minor adjustments.
Output:
{
"Name": "Widgets",
"SymUS": "Widg",
"SymCN": "Zyin",
"SymJP": "Kono"
}
{
"Name": "Blodgets",
"SymUS": "Blodg",
"SymAU": "Blod",
"SymJP": "Kado"
}
EDIT:
My below question still stands but I appreciate that it's hard to answer without sifting through a pile of code. Therefore, to ask a somewhat similar question, does anyone have any examples of Menhir being used to implement an AST? Preferably not "toy" projects like a calculator but I would appreciate any help I could get.
Original Question:
I'm trying to implement an abstract syntax tree using Menhir and there's an issue I can't seem to solve. My set up is as follows:
The AST's specification is generated using atdgen. This is basically a file with all of my grammar rules translated to to the ATD format. This allows me to serialize some JSON which is what I use to print out the AST.
In my parser.mly file I have a long list of production. As I'm using Menhir I can link these productions up to AST node creation, i.e. each production from the parser corresponds with an instruction to record a value in the AST.
The second point is where I'm really struggling to make progress. I have a huge grammar (the ast.atd file is ~600 lines long and the parser.mly file is ~1000 files long) so it's hard to pin down where I'm going wrong. I suspect I have a type error somewhere along the way.
Snippets of Code
Here's what my ast.atd file looks like:
...
type star = [ Star ]
type equal = [ Equal ]
type augassign = [
| Plusequal
| Minequal
| Starequal
| Slashequal
| Percentequal
| Amperequal
| Vbarequal
| Circumflexequal
| Leftshiftequal
| Rightshiftequal
| Doublestarequal
| Doubleslashequal
]
...
Here's what my parser.mly file looks like:
...
and_expr // Used in: xor_expr, and_expr
: shift_expr
{ $1 }
| and_expr AMPERSAND shift_expr
{ `And_shift ($1, `Ampersand, $3) } ;
shift_expr // Used in: and_expr, shift_expr
: arith_expr
{ $1 }
| shift_expr pick_LEFTSHIFT_RIGHTSHIFT arith_expr
{ `Shift_pick_arith ($1, $2, $3) } ;
pick_LEFTSHIFT_RIGHTSHIFT // Used in: shift_expr
: LEFTSHIFT
{ `Leftshift }
| RIGHTSHIFT
{ `Rightshift } ;
...
The error I get when I try to compile the files with
ocamlbuild -use-menhir -tag thread -use-ocamlfind -quiet -pkgs
'core,yojson,atdgen' main.native
is a type error, i.e.
This expression has type [GIANT TYPE CONSTRUCTION] but an expression
was expected of type [DIFFERENT GIANT TYPE CONSTRUCTION]
I realise that this question is somewhat difficult to answer in the abstract like this, and I'm happy to provide a link to the dropbox of my code, but I'd really appreciate if anyone could point me in the right direction.
Possibly of interest: I have some productions in parser.mly that were initially "empty" which I dealt with by using the ocaml option type (Some and None). Perhaps I could be having issues here?
About examples of code using menhir, you can have a look at the list on the right on the OPAM menhir page - all those depend on menhir.