jq, search for a 'sections' basing on a value of an attribute - unique

I am new in this stuff,
I have input json:
{
"2017-09-15": {
"key": "stops",
"node": {
"lastUpdate": "2017-09-15 05:15:26",
"stops": [
{
"stopId": 11000,
"stopCode": null,
"stopName": null,
"stopShortName": "1000",
"stopDesc": "Tom Tail",
"subName": "1000",
"zoneId": null,
"zoneName": null,
"stopUrl": "",
"locationType": null,
"parentStation": null,
"stopTimezone": "",
"wheelchairBoarding": null,
"virtual": null,
"nonpassenger": null,
"depot": null,
"ticketZoneBorder": null,
"onDemand": null,
"activationDate": "2017-09-14"
},
{
"stopId": 1100,
"stopCode": "04",
"stopName": "One Three",
"stopShortName": "1100",
"stopDesc": "Big Wall",
"subName": "04",
"zoneId": 1,
"zoneName": "Demo2",
"stopUrl": "",
"locationType": null,
"parentStation": null,
"stopTimezone": "",
"wheelchairBoarding": null,
"virtual": 0,
"nonpassenger": 0,
"depot": 0,
"ticketZoneBorder": 0,
"onDemand": 0,
"activationDate": "2017-09-24"
}
]
}
}
}
I can get all stops by filter:
jq -r '.[].node.stops' json
but how to search for a specific stop, for example basing on stopDescr - exact name or containing a string?
thx.

You're on the right track ...
.[].node.stops[] | select(.stopDesc == "Big Wall")
unique
Regarding the supplementary question:
1) sort and unique both expect arrays as input
2) calling unique obviates the need to call sort
3) unique produces an array, so you might want to use unique[]
4) One way to convert a stream of (JSON) values to an array is to wrap the expression producing the stream in square brackets.
Here is an example:
[.[].node.stops[] | select(.stopDesc == "Big Wall") | .zoneName]
| unique

Related

How to count total number of a specific key from a JSON Object in Groovy?

I dont have much experience in using JSONSlurper in Groovy and I have below payload:
{
"header": {
"quoteType": "YQT",
"salesOrg": "2040",
"channel": "02",
"atpMethod": "01",
"distributionChannel": "00",
"division": "00",
"deliveryDateHeader": "2022-10-08T00:00Z",
"isCompleteDeliveryRequired": "",
"shippingCondition": "01",
"pricingDate": "2022-09-02T08:56Z",
"currencyCode": "EUR"
},
"partner": [
{
"partnerNumber": "131761",
"partnerFunction": "SP"
},
{
"partnerNumber": "131762",
"partnerFunction": "WE"
},
{
"partnerNumber": "131761",
"partnerFunction": "RE"
}
],
"materials": [
{
"materialNumber": "29221136",
"requestedDeliveryDate": "2022-10-08T00:00Z",
"deliveryGroup": 0,
"isPartialDeliveryAccepted": "",
"deliveryPlant": null,
"deliveryStorageLocation": null,
"quantity": 1,
"unitOfMeasure": null,
"deliveryPriority": null,
"uomType": "Sales Unit of Measurement"
},
{
"materialNumber": "29233777",
"requestedDeliveryDate": "2022-10-08T00:00Z",
"deliveryGroup": 0,
"isPartialDeliveryAccepted": "",
"deliveryPlant": null,
"deliveryStorageLocation": null,
"quantity": 1,
"unitOfMeasure": null,
"deliveryPriority": null,
"uomType": "Sales Unit of Measurement"
}
]
}
How can I count the total number of "materialNumber"? I have also a code (please check below) but my code does not seem to work as the count is always 0 whenever I run it and I am not also sure on what is wrong.
import com.sap.gateway.ip.core.customdev.util.Message
import groovy.json.JsonSlurper
def Message countMATNR(Message message) {
def reader = message.getBody(InputStream)
def jsonIN = new JsonSlurper().parse(reader)
def property_matnrCounter = 0
jsonIN.materialNumber.each { materials ->
if (materials.materialNumber?.trim() && materials.materialNumber != 'null') {
property_matnrCounter++;
}
}
message.setProperty("numberOfMaterial", property_matnrCounter)
return message;
}
You are addressing the wrong properties, the JSON has a list called materials, and each element in the list is an object which contains the key materialNumber!
If you change it to this, then the result is 2.
jsonIN.materials.each { material ->
if (material.materialNumber?.trim() && material.materialNumber != 'null') {
property_matnrCounter++;
}
}
You can use the out-of-box count() method in a one-liner.
I left irrelevant parts of your json out for brevity.
import groovy.json.JsonSlurper
def json = new JsonSlurper().parseText '''\
{
"materials": [
{
"materialNumber": "29221136",
"requestedDeliveryDate": "2022-10-08T00:00Z",
"deliveryGroup": 0,
"isPartialDeliveryAccepted": "",
"deliveryPlant": null,
"deliveryStorageLocation": null,
"quantity": 1,
"unitOfMeasure": null,
"deliveryPriority": null,
"uomType": "Sales Unit of Measurement"
},
{
"materialNumber": "29233777",
"requestedDeliveryDate": "2022-10-08T00:00Z",
"deliveryGroup": 0,
"isPartialDeliveryAccepted": "",
"deliveryPlant": null,
"deliveryStorageLocation": null,
"quantity": 1,
"unitOfMeasure": null,
"deliveryPriority": null,
"uomType": "Sales Unit of Measurement"
},
{},
{"materialNumber": " "},
{"materialNumber": "null"},
{"materialNumber": null}
]
}'''
int count = json.materials*.materialNumber.count{ it?.trim() && it != 'null' }
assert count == 2

Function from backend returns json object, but when received it is also an array

I'm trying to create a function that will retrieve some objects for my in a very specific format. Here's the code for that function:
pool.getConnection(function(err, connection) {
if (err) {
return res.status(500).json({
success: false,
data: err
});
}
connection.query("select cqq.* from coursequizquestions as cqq, coursequiz as cq where cqq.quiz = ? and cq.quizTitle = cqq.quiz and cq.course in (select courseUUID from courses where courseName = ?)", [quizName, courseName], function(err1, results) {
connection.release();
if (err1) {
return res.status(500).json({
success: false,
data: err1
});
}
let r = {};
for(const i in results) {
r[results[i].quizQuestion] = results[i];
}
return res.status(201).json({
success: true,
data: r
});
});
})
Where "pool" is an SQL pool object created using the mysql.createpool function. When I request this data, I'm expecting to get something that looks like this:
{
"Question 1": {
"id": 1,
"quiz": "Vue.js Introduction",
"quizQuestion": "Question 1",
"quizAnswer": null,
"dateCreated": "2021-06-17T04:00:00.000Z",
"deletedDate": null,
"videoUUID": null,
"questionScore": 50,
"questionOrder": 1,
"type": "Input"
},
"Question 2": {
"id": 2,
"quiz": "Vue.js Introduction",
"quizQuestion": "Question 2",
"quizAnswer": null,
"dateCreated": "2021-06-17T04:00:00.000Z",
"deletedDate": null,
"videoUUID": null,
"questionScore": 50,
"questionOrder": 2,
"type": "Input"
}
}
and it seems like it is exactly what I get when monitoring from the backend. When it arrives on the frontend, however, I get this:
{
"0": {
"questionOrder": 1,
"type": "input",
"quizQuestion": "0",
"questionScore": 50,
"dateCreated": "2021-06-17",
"deletedDate": null,
"videoUUID": null,
"choices": {}
},
"1": {
"questionOrder": 2,
"type": "input",
"quizQuestion": "1",
"questionScore": 50,
"dateCreated": "2021-06-17",
"deletedDate": null,
"videoUUID": null,
"choices": {}
},
"Question 1": {
"id": 1,
"quiz": "Vue.js Introduction",
"quizQuestion": "Question 1",
"quizAnswer": null,
"dateCreated": "2021-06-17T04:00:00.000Z",
"deletedDate": null,
"videoUUID": null,
"questionScore": 50,
"questionOrder": 1,
"type": "Input"
},
"Question 2": {
"id": 2,
"quiz": "Vue.js Introduction",
"quizQuestion": "Question 2",
"quizAnswer": null,
"dateCreated": "2021-06-17T04:00:00.000Z",
"deletedDate": null,
"videoUUID": null,
"questionScore": 50,
"questionOrder": 2,
"type": "Input"
}
}
There are added keys that are integers as if my object was an array. I don't see this type of output at any other endpoint. Oddly, when I log this object (in Google Chrome) the integer keys do not appear in the preview, but they do appear when I expand the entire object. Also, when I loop over this object using this code:
fetch("http://localhost:3000/api/courses/getQuestions?quizName=" + encodeURI(document.getElementById("quiz_title").value) + "&courseName=" + encodeURI(course)).then(j => j.json())
.then(j => {
console.log(j.data);
questionData = j.data;
let questionList = document.getElementById("question-list");
questionList.innerHTML = "";
for (const i in sortDict(questionData, "questionOrder")) {
makeQuestionElement(i, questionList);
}
}
then it only loops over the integer keys, essentially ignoring the portion of the output that was expected. Is there any insight as to what's happening here?
I've solved this problem - It turns out that my sortDict function was implemented incorrectly. Instead of returning an object with sorted keys, it returned a list of keys in the sorted order. The reason that the keys were populating like an array is that I was actually iterating through an array when running my loop. I've adjusted the code with this information in mind, and the problem was fixed.

want to get approval status for files

I'm trying to get the status, type, and set that a document is in. I'm making the call to the data/v1/projects/{projectId}/items/{itemId}/versions endpoint
While looking at the UI I noticed that one of the endpoints the UI uses
https://developer.api.autodesk.com/dm/v1/projects/801f898a-4cc9-42cd-a56b-4e4b9ca0a054/folders/urn%3Aadsk.wipemea%3Afs.folder%3Aco.mgQJS4w7SUOx5EbM4gVJBA/documents?entity_types=SEED_FILE&limit=200&offset=0
contain some of the info I need
{
"urn": "urn:adsk.wipemea:dm.lineage:sdpW01kQRVyGOhKFIvkELA",
"create_time": "2021-02-01T08:42:06+0000",
"create_user_id": "SE6ZLLDDEY9R",
"create_user_name": "MICHAEL LEIPPER",
"last_modified_time": "2021-02-01T08:42:07+0000",
"last_modified_user_id": "SE6ZLLDDEY9R",
"last_modified_user_name": "MICHAEL LEIPPER",
"delete_by": null,
"delete_at": null,
"title_block_id": null,
"title_block_image": null,
"file_name": "happy-familyNHS (1).jpg",
"id": "b1da56d3-5910-455c-863a-128522f9042c",
"name": "happy-familyNHS (1).jpg",
"description": "",
"hidden": false,
"move_state": null,
"cde_is_master": null,
"custom_attributes": [],
"current_version": {
"create_time": "2021-02-01T08:42:06+0000",
"update_time": "2021-02-09T10:42:15+0000",
"created_by": "MICHAEL LEIPPER",
"updated_by": "MICHAEL LEIPPER",
"create_user_name": "MICHAEL LEIPPER",
"create_user_id": "SE6ZLLDDEY9R",
"update_user_id": "SE6ZLLDDEY9R",
"storage_urn": "urn:adsk.objects:os.object:wip.dm.emea.2/4b48ee2d-78fd-4e17-876c-48400c9cf6b6.jpg",
"urn": "urn:adsk.wipemea:fs.file:vf.sdpW01kQRVyGOhKFIvkELA?version=1",
"states": [
"CONTENT_AVAILABLE"
],
"name": "happy-familyNHS (1).jpg",
"title": "happy-familyNHS (1).jpg",
"mime_type": null,
"revision_number": 1,
"title_block_id": null,
"original_name": null,
"file_name": "happy-familyNHS (1).jpg",
"custom_attributes": [],
"sets": [],
"approve_status": {
"id": "2715ab6c-46b8-4493-b48a-426056e93cae",
"value": "approved",
"label": "Approved As built",
"iconValue": "approved-check",
"buildIn": false,
"approverUserId": "SE6ZLLDDEY9R",
"approverUserName": "MICHAEL LEIPPER"
},
"translation_states": "none",
"translation_has_thumbnail": false,
"forge_type": null,
"forged_is_composite_design": null,
"entity_type": "SEED_FILE",
"bubble_viewable_guid": null,
"bubble_viewable_id": null,
"bubble_viewable_order": null,
"bubble_viewable_resource_mimetype": null,
"bubble_viewable_resource_urn": null,
"bubble_urn": "urn:adsk.wipemea:fs.file:vf.sdpW01kQRVyGOhKFIvkELA?version=1",
"process_state": "PROCESSING_COMPLETE",
"extraction_state": "SUCCESS",
"splitting_state": "NOT_SPLIT",
"review_id": null,
"review_state": "NOT_IN_REVIEW",
"review_new_document_count": null,
"review_matched_document_count": null,
"review_deleted": null,
"action": null,
"partial_extraction_failure": null,
"move_state": null,
"dm_command_id": "ad123f4d-04df-4620-b733-316d346856cf",
"id": "6986e92e-7a83-4769-9101-e8a5615568c8",
"file_size": 136688,
"process_result": "PROCESSING_SUCCESS"
},
"current_set_version": 0,
"reserved": false,
"reserved_user_name": null,
"reserved_user_id": null,
"reserved_at": null,
"latest_version": 1,
"latest_version_file_state": [
"CONTENT_AVAILABLE"
],
"latest_version_create_time": "2021-02-01T08:42:06+0000",
"current_version_urn": "urn:adsk.wipemea:fs.file:vf.sdpW01kQRVyGOhKFIvkELA?version=1",
"latest_storage_urn": "urn:adsk.objects:os.object:wip.dm.emea.2/4b48ee2d-78fd-4e17-876c-48400c9cf6b6.jpg",
"parent_folder_urn": "urn:adsk.wipemea:fs.folder:co.mgQJS4w7SUOx5EbM4gVJBA",
"origin_folder_urn": null,
"folder_set_urn": null,
"latest_versioned_file_urn": "urn:adsk.wipemea:fs.file:vf.sdpW01kQRVyGOhKFIvkELA?version=1"
},
As can be seen this has approve_status in the return object. I know that this endpoint is private and not available publicly, however, is there a way to get these approval statuses for each file.
Any help is greatly appreciated.
Sounds like you are looking for BIM 360 Docs Review and Set API. Unfortunately, there is no public API and they are still under wish:
ALEX-37150: “API wish: expose Docs Review API”
ALEX-24690: “Expose BIM 360 SET via DM API”
Sorry for the bad news.
I will add your name to the wish log above. In this context, you are interested in read access to those?

How to parse unnest json data using cypher query language to neo4j database?

In the below json file I want to access "934934507945312256", "934934503604174848",.... and then the keys inside them.
But after using UNWIND clause I am unable to access the data of these keys(quote_count,reply_count,etc.) as these keys("934934507945312256" ,"934934503604174848",...) are randomly generated.
{
"934934507945312256": {
"quote_count": 0,
"reply_count": 0,
"hashtags": null,
"datetime": "2017-11-26 23:58:51",
"date": "2017-11-26",
"like_count": 0,
"verified": "False",
"sentiment": 0,
"author": "Judy💯The Resistance",
"location": "Hollywood, California USA🇺🇸",
"tid": "934934507945312256",
"retweet_count": 0,
"type": "retweet",
"media_list": null,
"quoted_source_id": null,
"url_list": null,
"tweet_text": "RT #kylegriffin1: Reminder: The Senate Judiciary Committee gave Jared Kushner a November 27 deadline to turn over the missing records… ",
"author_profile_image": "https://pbs.twimg.com/profi...",
"author_screen_name": "jgirl66",
"author_id": "23737528",
"lang": "en",
"keywords_processed_list": [
"reminder",
"senate judiciary committee",
"kushner november",
"deadline"
],
"retweet_source_id": "934872065471115264",
"mentions": [
"kylegriffin1"
],
"replyto_source_id": null
},
"934934503604174848": {
"quote_count": 0,
"reply_count": 2,
"hashtags": [
"MissUniverse",
"Thailand"
],
"datetime": "2017-11-26 23:58:50",
"date": "2017-11-26",
"like_count": 38,
"verified": "False",
"sentiment": 0,
"author": "P'Hmee7.5",
"location": "Bangkok, Thailand",
"tid": "934934503604174848",
"retweet_count": 105,
"type": "Tweet",
"media_list": null,
"quoted_source_id": null,
"url_list": null,
"tweet_text": "รอโหวต มรญ #MissUniverse #Thailand",
"author_profile_image": "
Thumbnail
",
"author_screen_name": "Peehmee75",
"author_id": "700720806972624897",
"lang": "th",
"keywords_processed_list": null,
"retweet_source_id": null,
"mentions": null,
"replyto_source_id": null
},
"934934336381636608": {
"quote_count": 0,
"reply_count": 0,
"hashtags": null,
"datetime": "2017-11-26 23:58:10",
"date": "2017-11-26",
"like_count": 0,
"verified": "False",
"sentiment": 0,
"author": "selfresqingprncess",
"location": "Maine, USA",
"tid": "934934336381636608",
"retweet_count": 0,
"type": "retweet",
"media_list": null,
"quoted_source_id": null,
"url_list": null,
"tweet_text": "RT #kylegriffin1: Reminder: The Senate Judiciary Committee gave Jared Kushner a November 27 deadline to turn over the missing records… ",
"author_profile_image": "https://pbs.twimg.com/profi...",
"author_screen_name": "slfresqngprncss",
"author_id": "100536014",
"lang": "en",
"keywords_processed_list": [
"reminder",
"keywords_processed_list": [
"reminder",
"senate judiciary committee",
"kushner november",
"deadline"
],
"retweet_source_id": "934872065471115264",
"mentions": [
"kylegriffin1"
],
"replyto_source_id": null
}
}
There's I have tried :-
query = """
with {json} as data UNWIND data as doc FOREACH( l in doc| MERGE (label1:Label1 {author:l.author}))
"""
But I am getting error:- Cannot merge node using null property value for author.
Your JSON file is malformed.
A JSON string cannot contain control characters (like carriage returns or linefeeds). And your JSON contains an extraneous section (that is probably a copy/paste error):
"keywords_processed_list": [
"reminder",
Furthermore, UNWIND can only be used with lists. You cannot use it to get the properties from a map. Once you have fixed your JSON errors, try the following query (I assume that json is a parameter whose value is your JSON data):
UNWIND KEYS($json) AS k
MERGE(label1:Label1 {author: $json[k].author})

Stripe API parsing JSON objects

I'm getting a bit confused when retrueving card information. I get the card data like this:
card_data = customer.sources.list(limit=3, object='card')
card = card_data['data']
print(card)
And this is what is printed:
[<Card card id=card_1DPKYtAuBx2mXUsrmQG0gHMz at 0x54d8420> JSON: {
"address_city": null,
"address_country": null,
"address_line1": null,
"address_line1_check": null,
"address_line2": null,
"address_state": null,
"address_zip": "42424",
"address_zip_check": "pass",
"brand": "Visa",
"country": "US",
"customer": "cus_Dqyu8HKCIQnUIA",
"cvc_check": "pass",
"dynamic_last4": null,
"exp_month": 4,
"exp_year": 2024,
"fingerprint": "TuMlU4wS6zLVngGc",
"funding": "credit",
"id": "card_1DPKYtAuBx2mXUsrmQG0gHMz",
"last4": "4242",
"metadata": {},
"name": null,
"object": "card",
"tokenization_method": null
}]
I'm unable to parse this data as if it were normal json "card['id']". How do I go about getting data from the fields?
The card you got from card = card_data['data'] is an array. You could get the id by card[0]['id']
Give it a try
Thanks