Trying to query az repos policy list with query of:
[?type.displayName=='Build' && contains(settings.displayName, 'Test')] returning message:
Invalid jmespath query supplied for `--query`: In function contains(), invalid type for value: None, expected one of: ['array', 'string'], received: "null"
Data is present while i query the fields, but when trying to query the value of them it is complaining.
tried with --query "[?contains(settings.displayName, 'Test')]" resulting in the same error.
I came across a link to bug query will traceback while attempting to filter a nullable field which contained instead a helpful explanation:
The reason for to_string() is that some service principals have their homepage set to null which is not an array or string. So the jmespath library fails.
This can be remedied by calling to_string on each object's homepage field so that homepage is always a string for the purposes of contains()
so i updated my query: az repos policy list --query "[?type.displayName=='Build' && contains(to_string(settings.displayName), 'Test') ]"
which returned my exspected output.
Thanks.
One of the workaround you can follow to resolve the above issue;
You can specify the space between )& ] in the last section as suggested in this SO THREAD by #rcarrington
For example:-
--query "[?contains(settings.displayName, 'Test') ]"
If the above does not work then please try to give a space between ? & contains .
For more information please refer this MICROSOFT DOCUMENTATION| How to query Azure CLI command output using a JMESPath query .
Related
I am building a stats table that tracks user data points. The JSON is dynamic and can grow for multiple levels. I'm basically getting an error about invalid JSON using json_merge_patch, which I have used often before. I can not figure out why this is giving me the following error:
ERROR: Invalid JSON text in argument 1 to function json_merge_patch: "Invalid value." at position 0.
insert into
stats.daily_user_stats
VALUES
(null,'2022-02-02',1,18,3,'{"pageviews":{"user":1}}')
on duplicate key update
jdata =
if(
json_contains_path(jdata, 'one', '$.pageviews.user'),
json_set(jdata, '$.pageviews.user', cast(json_extract(jdata, '$.pageviews.user')+1 as UNSIGNED)),
json_merge_patch('jdata','{"pageviews":{"user":1}}')
)
Any help on identifying why the JSON I'm passing to the json_merge_function is not correct?
Solved. The json_merge_patch should look like this:
json_merge_patch(jdata,'{"pageviews":{"user":1}}')
Good day,
Hope you are well.
I am trying to use the Vincere API, and trying to query the response to only return where private_job:0. I am using Postman to test the API.
When I use the below request, doing my best to follow the instructions on the Documentation:
https://domain.vincere.io/api/v2/job/search/fl=job_title,private_job;sort=published_date asc?q=private_job:0
I get the following response:
"Parse exception Unexpected end of input, expected term_char, ws0, term or term_end (line 1, pos 14):\nprivate_job:0\n ^\n"
If I remove ?q=private_job:0, I get a valid response.
I am clearly doing something wrong. Please assist.
in query parameter the key name is q ,
q=private_job:0
but in documentation it says instead of q it should be fq
https://domain.vincere.io/api/v2/job/search/fl=job_title,private_job;sort=published_date asc?fq=private_job:0
Also if you are using special character q=private_job:0 # , then give the value in the query parameter session of postman it will url encode it automatically for you
This stumped me as well, turns out my issue was twofold.
Firstly, this error refers to their URL parser expecting to see the end character %23, so your query string needs to end with that.
Secondly, I was attempting to query the job_type and using the actual string value ie. job_type:PERMANENT%23. This actually needs to be the enum value (1 in this case).
as the title suggests I am having a problem trying to use JPA Criteria with Spring Boot for a specific case.
Generally everything works but trying to search for stored data with a String property having newLine character embedded ( \n ) doesn't seem to work.
I am able to Get the data, edit them, save them through my front end, create new with multiple lines etc. but trying to search for them when for example a column is equals with 'hello\nworld' it wont work even though running this query in MySQL Workbench works returning the desired data :
select * from kerkinidb.ct_thhlastika_press_threats where description_en = 'hello\nworld';
To clarify, the way I do the search is by waiting in a Get request an argument called search which has all the properties that the user filtered. I am matching it with a Regex (which also has inside the Java 8.0 new Regex \\\\R for matching with multilines (and it works) ) then I am giving the the Service layer the Search Criteria that I matched which then passes to the Jpa Criteria Repository to parse them and generating the Predicates (matching again with Regex and \\\\R to create a final Predicate with OR and ANDs for the filtering) then triggering the query, then making another query called count to implement Pagination and finally mapping to a custom object and returning it.
I debugged every step and the final Predicate does generate the query I want, but the db doesn't return the expected data. So I am really confused since as I said the query does work in MySQL Workbench.
This is an example of the logging (I turned Spring Boot logging for MySQL logs on) generated when the Request is being triggered (in this case the stored data I have in my Table ct_thhlastika_press_threats in column description_en is a\ns\ndd so I am searching for this one as you can see instead of the example I said earlier hello\nworld :
2019-02-12 16:01:01.929 DEBUG 18368 --- [nio-8080-exec-2] org.hibernate.SQL : select ctthhlasti0_.id as col_0_0_, ctthhlasti0_.act_code as col_1_0_, ctthhlasti0_.description_en as col_2_0_, ctthhlasti0_.remarks as col_3_0_ from ct_thhlastika_press_threats ctthhlasti0_ where 1=1 and ctthhlasti0_.description_en=? order by ctthhlasti0_.id asc limit ?
2019-02-12 16:01:01.933 TRACE 18368 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [VARCHAR] - [a\ns\ndd]
2019-02-12 16:01:01.944 DEBUG 18368 --- [nio-8080-exec-2] org.hibernate.SQL : select count(ctthhlasti0_.id) as col_0_0_ from ct_thhlastika_press_threats ctthhlasti0_ where 1=1 and ctthhlasti0_.description_en=?
2019-02-12 16:01:01.944 TRACE 18368 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicBinder : binding parameter [1] as [VARCHAR] - [a\ns\ndd]
2019-02-12 16:01:01.946 TRACE 18368 --- [nio-8080-exec-2] o.h.type.descriptor.sql.BasicExtractor : extracted value ([col_0_0_] : [BIGINT]) - [0]
For anyone interested to look farther into the code you could find it Github repo. The project is for my University thesis. I made comments for adding to the Regexes (both of them) for ctThhlastikaPressThreats Controller and Repository. The db (the standard as well as the secondary for testing) need to be generated (can be generated by changing the auto-dll in application.properties).
Update
I tried using System.lineSeparator() (as suggested by #Hermann Steidel in the answer bellow) replacing the text newLine \n. But even though in the logs we can see now clearly that the value for the equals Predicate has line separations it still doesn't return the data from the DB.
Farther Explanation of my implementation
A correct Get request for the dynamic searching is like this :
http://localhost:8080/v1/ctThhlastikaPressThreats/search?search=descriptionEn~hello\nworld;#&size=10&page=0&sort=Asc
As you can see I am using a path variable called search (which has all the properties filtering requested from the user) along with 3 more for the size, page and sort.
For the search variable I am using 3 distinct characters to implement OR and AND Predicates for the final query. Those are the ~ which is to know that the property before it needs to use an equal predicate, the ; which is to be able to have multiple values for each property requested by the user and finally the # which triggers the end of this property's filtering.
Those 3 characters are being Regexed in two places. In the Controller and in the SearchRepository of (for example since we specifically started with this one -> the CtThhlastasikaPressThreats)
Finally in the SearchRepository you can see that I am triggering 2 queries, one is to get the filtered data from the db and another is to get the count of the data for Pagination purposes while also in the end mapping to a custom DTO.
Steps to reproduce :
After generating the db, change for the 2 Regexes of CtThhlastikaPressThreats. For the Controller to be (\w+?)(~|<|>)([(!-/.\\\\R 0-9\p{L});]+)?# and for the SearchRepository to be ([(!-/.\\\\R 0-9\p{L})]+).
Then you can use the example of the request I have above when having in the db saved for the specific table and for column descriptionEn with value of hello\nworld for example or whatever value you put (also change it into the request).
Final thing I tried but wasn't the solution :
Put in the CtThhlastikaPressThreatsSearchRepository in the method search (it is after line 61) above the :
predicate = builder.equal(root.get(param.getKey()), match.toString());
Make it as :
match = match.toString().replace("\\n", System.lineSeparator());
predicate = builder.equal(root.get(param.getKey()), match.toString());
This will basically change the value from being hellow\nworld to become hello\r\nworld so I guess it still isn't the desired solution. Thinking that in the db it is stored as \n for the lineSeparators.
Now in the logs you can see that when you trigger the Get Request again the VARCHAR value of descriptionEn is indeed now with line separations instead of the \n (which still should be recognized by MySQL) text.
Final thoughts
I believe since even this
select * from kerkinidb.ct_thhlastika_press_threats where description_en = 'hello\nworld';
works in MySQL Workbench, that something in between might be ruining the request when trying to also include newLine char or lineSeparators.
If there is any idea of why it doesn't work as intended please share to try it out.
Thank you for your time
I got it to work. Probably not the answer you wanted but, if you replace your incoming "\n"s with the system line separator, it will get you want you want.
search = search.replace("\\n", System.getProperty("line.separator"));
try {
return ctThhlastikaPressThreatsService.searchCtThhlastikaPressThreats(producedSearchCriterias(search), size, page, sort);
If you look at the params value logging, you'll see the param is now
extracted value ([col_2_0_] : [VARCHAR]) - [hello
world]
instead of
extracted value ([col_2_0_] : [VARCHAR]) - [hello\nworld]
It's like I suspected, somewhere in search pipeline, it's escaping the "\n". I'm not suggesting you do what I suggest right there on the controller, I just wanted to show you that this is what needs to happen (somewhere) for it to work.
I use Loopback (v3) to manage endpoints with a MySQL DB. From my app, I use Axios to get end point data.
In the DB, I have this kind of record:
id name value //other columns
1 Sally 0.00
2 Sally 135.00
3 Sally null
I can query each value in SQL:
select * from Tab where name = 'Sally' and value = 0;
select * from Tab where name = 'Sally' and value is null;
select * from Tab where name = 'Sally' and value > 0;
Problem: I can't replicate some of these queries with Axios from the app (nor with my curl tests). For example, I'd like to get only the null values and can't find the correct syntax for "is null" to put in a Loopback filter.
Here are the curls I tried:
curl -g http://my_ip/api/tabs <-- works as expected
curl -g http://my_ip/api/tabs\?filter\[where\]\[value\]\=null <-- returns only the 0 values and none of the null ones!
So, to spot only the null ones, I tried this solution:
curl -g http://my_ip/api/tabs\?filter\[where\]\[value\]\[eq\]\=null | jq .
It returns an error (and anyway, I can't find a "eq" operator in the list provided by the loopback docs.
How could I get the distinct 0 and null values with loopback, replicating what I got directly in MySQL? Is there a way to make a "is null" filter with Loopback?
Disclaimer: I am a co-author and maintainer of LoopBack.
Query string parameters have always a string value, it's not possible to tell whether the value null was meant as a string "null" or as JavaScript keyword null.
Fortunately LoopBack supports also JSON encoding for complex query string arguments. With JSON, it's easy to distinguish between numbers, strings and null values.
Here is the query to list all model instances with NULL value:
curl -g 'http://my_ip/api/tabs?filter={"where":{"value":null}}'
If you are building the request URL in JavaScript, just convert your filter argument from an object into a string via JSON.stringify and let your HTTP client library to URL-encode the value.
Here is a typical request you will get as a result:
http://my_ip/api/tabs?filter=%7B%22where%22%3A%7B%22value%22%3A%20null%7D%7D
I'm trying to make a N1QL based query on Spring Data Couchbase. The documentation says
#n1ql.fields will be replaced by the list of fields (eg. for a SELECT clause) necessary to reconstruct the entity.
My repository implementation is this one:
#Query("#{#n1ql.fields} WHERE #{#n1ql.filter}")
List<User> findAllByFields(String fields);
And I'm calling this query as follows:
this.userRepository.findAllByFields("SELECT firstName FROM default");
I'm getting this error:
Caused by: org.springframework.data.couchbase.core.CouchbaseQueryExecutionException: Unable to execute query due to the following n1ql errors:
{"msg":"syntax error - at AS","code":3000}
After a little bit of researching, I also tryed:
#Query("SELECT #{#n1ql.fields} FROM #{#n1ql.bucket} WHERE #{#n1ql.filter}")
With this query, I don't get an error, I get all the documents stored but only the ID the other fields are set to null, when my query tries to get the firstName field.
this.userRepository.findAllByFields("firstName");
Anyone knows how to do such a query?
Thank you in advance.
You're misunderstanding the concept, I encourage you to give the documentation more time and see more examples. I'm not sure what exactly you're trying to achieve but I'll throw some examples.
Find all users (with all of their stored data)
#Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter}")
List<User> findAllUsers();
This will basically generate SELECT meta().id,_cas,* FROM bucket WHERE type='com.example.User'
Notice findAllUsers() does not take any parameters because there are no param placeholders defined in the #Query above.
Find all users where firstName like
#Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter} AND firstName like $1")
List<User> findByFirstNameLike(String keyword);
This will generate something like the above query but with an extra where condition firstName like
Notice this method takes a keyword because there is a param placeholder defined $1.
Notice in the documentation it says
#{#n1ql.selectEntity} WHERE #{#n1ql.filter} AND test = $1
is equivalent to
SELECT #{#n1ql.fields} FROM #{#n1ql.bucket} WHERE
#{#n1ql.filter} AND test = $1
Now if you don't want to fetch all the data for user(s), you'll need to specify the fields being selected, read following links for more info
How to fetch a field from document using n1ql with spring-data-couchbase
https://docs.spring.io/spring-data/couchbase/docs/2.2.4.RELEASE/reference/html/#_dto_projections
I think you should try below query, that should resolve the issue to get fields based parameter you have sent as arguments.
Please refer blow query.
#Query("SELECT $1 FROM #{#n1q1.bucket} WHERE #{#n1ql.filter}")
List findByFirstName(String fieldName);
Here, bucket name resolve to the User entity and and n1ql.filter would be a default filter.