Get Object from http get API result - json

I know question has already asked but I can't do it with my get result.
My service get data from API return this kind of result :
{
"nhits": 581,
"parameters": {
"dataset": "communes-belges-2019",
"rows": 1,
"start": 0,
"facet": [
"niscode",
"region",
"nis_code_region"
],
"format": "json",
"timezone": "UTC"
},
"records": [
{
"datasetid": "communes-belges-2019",
"recordid": "65d40b7bc42f766b4fdb04c4a985766dc8b51717",
"fields": {
"shape_area": 79397718.576,
"mun_name_upper_fr": "ÉTALLE",
"arr_name_fr": "Virton",
"region": "Région wallonne",
"niscode": "85009",
"mun_off_language": "FR",
"geo_shape": {
"coordinates": [
[
[
5.678490965,
49.687217222
],
[
5.678462422,
49.6873304
]
]
],
"type": "Polygon"
},
"prov_name_fr": "Luxembourg",
"namefre": "Étalle",
"nom_commune": "Étalle",
"nis_code_region": "03000",
"mun_name_fr": "Étalle",
"reg_name_fr": "Région wallonne",
"mun_area_code": "BEL",
"modifdate": "2007-01-05",
"mun_type": "Commune/Gemeente/Gemeinde",
"mun_name_lower_fr": "étalle",
"prov_code": "80000",
"year": "2021-01-01",
"geo_point_2d": [
49.6639160352,
5.60896600843
]
},
"geometry": {
"type": "Point",
"coordinates": [
5.60896600843,
49.6639160352
]
},
"record_timestamp": "2019-05-24T09:44:14.333000+00:00"
}
],
"facet_groups": [
{
"name": "niscode",
"facets": [
{
"name": "11001",
"count": 1,
"state": "displayed",
"path": "11001"
},
{
"name": "33021",
"count": 1,
"state": "displayed",
"path": "33021"
},
{
"name": "33029",
"count": 1,
"state": "displayed",
"path": "33029"
},
{
"name": "33037",
"count": 1,
"state": "displayed",
"path": "33037"
}
]
},
{
"name": "region",
"facets": [
{
"name": "Région flamande",
"count": 299,
"state": "displayed",
"path": "Région flamande"
},
{
"name": "Région wallonne",
"count": 262,
"state": "displayed",
"path": "Région wallonne"
},
{
"name": "Région de Bruxelles-Capitale",
"count": 19,
"state": "displayed",
"path": "Région de Bruxelles-Capitale"
}
]
},
{
"name": "nis_code_region",
"facets": [
{
"name": "02000",
"count": 299,
"state": "displayed",
"path": "02000"
},
{
"name": "03000",
"count": 262,
"state": "displayed",
"path": "03000"
},
{
"name": "01000",
"count": 19,
"state": "displayed",
"path": "01000"
}
]
}
]
}
For the moment my service get data like this :
getProvinces(): Observable<any[]>{
return this._http.get<any>(this.apiProvinces)
.pipe(
map((response: any) => response.records as any[]),
catchError(this.handleError)
)
}
It returns a Observable<any[]> but I would like to get an object.
Therefore I defined a class with the below properties.
export class Record{
region : string;
nom_commune : string;
prov_name_fr: string;
records?: [];
constructor(reg: string, commune: string, prov: string) {
this.region = reg;
this.nom_commune = commune;
this.prov_name_fr = prov;
}
}
To reach that I try by replacing any[] by Record[] like below but it doesn't work.
getProvinces(): Observable<Record[]>{
return this._http.get<Record[]>(this.apiProvinces)
.pipe(
map(response => response as Record[]),
catchError(this.handleError)
)
}
In my component I defined an Observable<Record[]> like this :
public results$!: Observable<Record[]>;
And call the service like this :
this.results$ = this.apiService.getProvinces();
And try to see content in the html part:
<div *ngFor="let p of (results$ | async)">
{{p | json}}
</div>
I have the following error message : "Cannot find a differ supporting object '[object Object]' of type 'object'. NgFor only supports binding to Iterables such as Arrays."
And can't access to my object.
Any suggestions is helpfull because. I'm absolutely new to ANgular.
Thanks

When you create an observable, you don't actually make a request to your api. To make the request and get the data from the observable, you need to subscribe.
this.apiService.getProvinces().subscribe((res) => (this.results = res));
This will execute the get request and return your data to the callback function. Get requests are finite observables, so you don't need to unsubscribe, but you will need to subscribe again if you want to make another request.

Related

Check if a key exists and return another key

I need help with jq syntax on how to return the Gitlab job ID if it contains an artifact. The JSON output looks like this (removed a lot of unrelated info from it and added [...]):
[{
"id": 3219589880,
"status": "success",
"stage": "test",
"name": "job_with_no_artifact",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.119Z",
"started_at": "2022-10-24T18:21:25.986Z",
"finished_at": "2022-10-24T18:21:38.464Z",
"duration": 12.478682,
"queued_duration": 0.499786,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
"pipeline": {
"id": 123456789,
[...]
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts": [],
"runner": {
"id": 12270859,
[...]
},
"artifacts_expire_at": null,
"tag_list": []
}, {
"id": 3219589878,
"status": "success",
"stage": "test",
"name": "create_artifact_job_2",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.111Z",
"started_at": "2022-10-24T18:21:25.922Z",
"finished_at": "2022-10-24T18:21:39.090Z",
"duration": 13.168405,
"queued_duration": 0.464364,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
},
"pipeline": {
"id": 675641982,
[...],
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts_file": {
"filename": "artifacts.zip",
"size": 223
},
"artifacts": [{
"file_type": "archive",
"size": 223,
"filename": "artifacts.zip",
"file_format": "zip"
}, {
"file_type": "metadata",
"size": 153,
"filename": "metadata.gz",
"file_format": "gzip"
}],
"runner": {
"id": 12270845,
[...]
},
"artifacts_expire_at": "2022-10-25T18:21:35.859Z",
"tag_list": []
}, {
"id": 3219589876,
"status": "success",
"stage": "test",
"name": "create_artifact_job_1",
"ref": "main",
"tag": false,
"coverage": null,
"allow_failure": false,
"created_at": "2022-10-24T18:21:25.103Z",
"started_at": "2022-10-24T18:21:25.503Z",
"finished_at": "2022-10-24T18:21:41.407Z",
"duration": 15.904028,
"queued_duration": 0.098837,
"user": {
"id": 123456789,
[...]
},
"commit": {
"id": "5e0e1f287d20daf2036a3ca71c656dce55999265",
[...]
},
"pipeline": {
"id": 123456789,
[...]
},
"web_url": "WEB_URL",
"project": {
"ci_job_token_scope_enabled": false
},
"artifacts_file": {
"filename": "artifacts.zip",
"size": 217
},
"artifacts": [{
"file_type": "archive",
"size": 217,
"filename": "artifacts.zip",
"file_format": "zip"
}, {
"file_type": "metadata",
"size": 152,
"filename": "metadata.gz",
"file_format": "gzip"
}],
"runner": {
"id": 12270857,
},
"artifacts_expire_at": "2022-10-25T18:21:37.808Z",
"tag_list": []
}]
I've been trying to do either of the following using jQ:
Either:
Check if artifacts_file key exists in each iteration and if it does return the (job) id (so .[].id)
Check if artifacts array is empty in each iteration and if it is empty return the (job) id.
In both cases I'm able to do the first part but I am not sure how to return the .id key.
Related stackoverflow questions that I've been trying to utilize and adapt to my case:
jq - return array value if its length is not null
How to check for presence of 'key' in jq before iterating over the values
What I have so far: jq '[.[].artifacts[]|select(length > 0)] | .[]' which returns all the artifacts found (but it doesn't contain the .id of the job).
Checking the existence of a field using has:
.[] | select(has("artifacts_file")).id
3219589878
3219589876
Demo
Checking if a field is an empty array by comparing it to []:
.[] | select(.artifacts == []).id
3219589880
Demo

Json Path Read from a Kafka Message

I have a kafka message like below, where im trying to read the data from the json path. However im having a challenge when reading some of the attributes from the json path. here is the sample message.
sample1:
{
"header": {
"bu": "google",
"id": "12345",
"bum": "google",
"originTimestamp": "2021-10-09T15:17:09.842+00:00",
"batchSize": "0",
"jobType": "Batch"
},
"payload": {
"derivationdetails": {
"Id": "6783jhvvh897u31y283y",
"itemid": "1234567",
"batchid": 107,
"attributes": {
"itemid": "1234567",
"lineNbr": "1498",
"cat": "5929",
"Id": "6783jhvvh897u31y283y",
"indicator": "false",
"subcat": "3514"
},
"Exception": {
"values": [
{
"type": "PICK",
"value": "blocked",
"Reason": [
"RULE"
],
"rules": [
"439"
]
}
],
"rulesBagInfo": [
{
"Idtype": "XXXX",
"uniqueid": "7889423rbhevfhjaufdyeuiryeukjbdafvjd",
"rulesMatch": [
"439"
]
}
]
}
}
}
}
sample 2: Same message but see the difference in "Payload"
{
"header": {
"bu": "google",
"id": "12345",
"bum": "google",
"originTimestamp": "2021-10-09T15:17:09.842+00:00",
"batchSize": "0",
"jobType": "Batch"
},
"payload": {
"Id": "6783jhvvh897u31y283y",
"itemid": "1234567",
"batchid": 107,
"attributes": {
"itemid": "1234567",
"lineNbr": "1498",
"cat": "5929",
"Id": "6783jhvvh897u31y283y",
"indicator": "false",
"subcat": "3514"
},
"Exception": {
"values": [
{
"type": "PICK",
"value": "blocked",
"Reason": [
"RULE"
],
"rules": [
"439"
]
}
],
"rulesBagInfo": [
{
"Idtype": "XXXX",
"uniqueid": "7889423rbhevfhjaufdyeuiryeukjbdafvjd",
"rulesMatch": [
"439"
]
}
]
}
}
}
If you observe, sometimes the message has "derivationdetails", and sometimes it doesn't. But irrespective of its existence, i need to read the values of id,itemid and batchid. I tried using
$.payload[*].id
$.payload[*].itemid
$.payload[*].batchid
But i see that for batchid is returning null even though it has a value in the message, and the attributes under "attributes" return null if im using the above. For fields under "attributes" im using this(example):
$.payload.attributes.itemId
And, completely blank on how to read the below part.
"Exception": {
"values": [
{
"type": "PICK",
"value": "blocked",
"Reason": [
"RULE"
],
"rules": [
"439"
]
}
],
"rulesBagInfo": [
{
"Idtype": "XXXX",
"uniqueid": "7889423rbhevfhjaufdyeuiryeukjbdafvjd",
"rulesMatch": [
"439"
]
Im new to this and need some suggestions on how to read the attributes properly. Any help would be much appreciated.Thanks
Use ..(recursive descent, Deep scan. JSONPath borrows this syntax from E4X.) to get the values. But It will return a list if there are multiple entries with same key nested in deep.
Below jsonpath expressions will return a list with one item each for both sample1 and sample2
$.payload..attributes.Id
$.payload..attributes.itemid
$.payload..batchid
$.payload..Exception

POWERSHELL - How to access multilevel child elements in JSON file with condtion

can someone please send me solution or link for PowerShell 5 and 7 how can I access child elements if specific condition is fulfilled for JSON file which I have as output.json. I haven't find it on the net.
I want to retrieve value of the "children" elements if type element has value FILE and to put that into some list. So final result should be [test1.txt,test2.txt]
Thank you!!!
{
"path": {
"components": [
"Packages"
],
"parent": "",
"name": "Packages",
},
"children": {
"values": [
{
"path": {
"components": [
"test1.txt"
],
"parent": "",
"name": "test1.txt",
},
"type": "FILE",
"size": 405
},
{
"path": {
"components": [
"test2.txt"
],
"parent": "",
"name": "test2.txt",
},
"type": "FILE",
"size": 409
},
{
"path": {
"components": [
"FOLDER"
],
"parent": "",
"name": "FOLDER",
},
"type": "DIRECTORY",
"size": 1625
}
]
"start": 0
}
}
1.) The json is incorrect, I assumt that this one is the correct one:
{
"path": {
"components": [
"Packages"
],
"parent": "",
"name": "Packages"
},
"children": {
"values": [
{
"path": {
"components": [
"test1.txt"
],
"parent": "",
"name": "test1.txt"
},
"type": "FILE",
"size": 405
},
{
"path": {
"components": [
"test2.txt"
],
"parent": "",
"name": "test2.txt"
},
"type": "FILE",
"size": 409
},
{
"path": {
"components": [
"FOLDER"
],
"parent": "",
"name": "FOLDER"
},
"type": "DIRECTORY",
"size": 1625
}
],
"start": 0
}
}
2.) The structure is not absolute clear, but for your example this seems to me to be the correct solution:
$element = $json | ConvertFrom-Json
$result = #()
$element.children.values | foreach {
if ($_.type -eq 'FILE') { $result += $_.path.name }
}
$result | ConvertTo-Json
Be aware, that the used construct $result += $_.path.name is fine if you have up to ~10k items, but for very large items its getting very slow and you need to use an arraylist. https://adamtheautomator.com/powershell-arraylist/

Parsing Git Json with Regular Express

I am taking a Github json file and parsing it with Java's regular expression library JsonPath. I am having a problem parsing arrays that do not have labels.
I need to send a email every time a particular file is changed in our repository.
Here is the Git Json:
{
"trigger": "push",
"payload": {
"type": "GitPush",
"before": "xxxxxxxx",
"after": "yyyyyyyy",
"branch": "branch-name",
"ref": "refs/heads/branch-name",
"repository": {
"id": 42,
"name": "repo",
"title": "repo",
"type": "GitRepository"
},
"beanstalk_user": {
"type": "Owner",
"id": 42,
"login": "username",
"email": "user#example.org",
"name": "Name Surname"
},
"commits": [
{
"type": "GitCommit",
"id": "ffffffff",
"message": "Important changes.",
"branch": "branch-name",
"author": {
"name": "Name Surname",
"email": "user#example.org"
},
"beanstalk_user": {
"type": "Owner",
"id": 42,
"login": "username",
"email": "user#example.org",
"name": "Name Surname"
},
"changed_files": {
"added": [
"NEWFILE",
],
"deleted": [
"Gemfile",
"NEWFILE"
],
"modified": [
"README.md",
"NEWFILE"
],
"copied": [
]
},
"changeset_url": "https://subdomain.github.com/repository-name/changesets/ffffffff",
"committed_at": "2014/08/18 13:30:29 +0000",
"parents": [
"afafafaf"
]
}
]
}
}
This is the expression I am using: to get the commits
$..changed_files
This return the whole changed files part but I can not explicitly choose the name "NEWFILE"
I tried
$..changed_files.*[?(#.added == "NEWFILE")]
$..changed_files.*[?(#.*== "NEWFILE")]
It just returns a empty array.
I just want it to return Newfile and what type of change. Any Ideas?
You can use the following JsonPath to retrieve the commits which list "NEWFILE" as an added file :
$.payload.commits[?(#.changed_files.added.indexOf("NEWFILE") != -1)]

Parsing JSON data using GSON api

I have a JSON data and able to get the JSON data into a string.
I wish to get a json array from the root object one by one (using a loop if many are there)
The returned object must be of type "com.google.gson.JsonObject" as another method takes this object as input to get me the attribute values.
my json looks like this:
I would be needing the json array in "Resources" -
{
"totalResults": 2,
"startIndex": 1,
"schemas": [
"urn:ietf:params:scim:api:messages:2.0:ListResponse"
],
"Resources": [
{
"owner": {},
"application": {
"displayName": "Non Auth App",
"value": "4028b88168fa3d38016900d1759405a6",
"$ref": "http://localhost:8080/identityiq/scim/v2/Applications/4028b88168fa3d38016900d1759405a6"
},
"meta": {
"created": "2019-03-22T18:40:30.505+05:30",
"location": "http://localhost:8080/identityiq/scim/v2/Entitlements/297eff8a699f8b3a0169a5863769008b",
"version": "W/\"1553260230505\"",
"resourceType": "Entitlement"
},
"schemas": [
"urn:ietf:params:scim:schemas:sailpoint:1.0:Entitlement"
],
"displayableName": "reader",
"aggregated": false,
"id": "297eff8a699f8b3a0169a5863769008b",
"requestable": true,
"attribute": "Privileges",
"type": "Entitlement",
"descriptions": [],
"value": "reader"
},
{
"owner": {},
"application": {
"displayName": "Non Auth App",
"value": "4028b88168fa3d38016900d1759405a6",
"$ref": "http://localhost:8080/identityiq/scim/v2/Applications/4028b88168fa3d38016900d1759405a6"
},
"meta": {
"created": "2019-03-18T19:11:47.912+05:30",
"location": "http://localhost:8080/identityiq/scim/v2/Entitlements/297eff8a6990fc49016991096d08001e",
"version": "W/\"1552916507912\"",
"resourceType": "Entitlement"
},
"schemas": [
"urn:ietf:params:scim:schemas:sailpoint:1.0:Entitlement"
],
"displayableName": "admin",
"aggregated": false,
"id": "297eff8a6990fc49016991096d08001e",
"requestable": true,
"attribute": "Privileges",
"type": "Entitlement",
"descriptions": [],
"value": "admin"
}
]
}
Any help is appreciated.