Using Python, got the JSON below from an API response.
All I'm looking to do at this stage is count up the totals of each status code.
I've tried to implement the counting of ToDos from this Real Python article but this JSON I'm pulling is a little more complex than the sample provided there so I'm a little stuck.
Looking at the JSON, the stuff that I want is under "results".
I'd like to learn and use proper methods to parse the JSON data.
{
"results": [
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS1",
"status_updated": "2022-09-30T05:57:27.675000-04:00",
"task_id": "T178",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS2",
"status_updated": null,
"task_id": "T1213",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS1",
"status_updated": null,
"task_id": "T1188",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS2",
"status_updated": null,
"task_id": "T1177",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS1",
"status_updated": null,
"task_id": "T554",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS2",
"status_updated": null,
"task_id": "T1539",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS3",
"status_updated": null,
"task_id": "T106",
},
{
"accepted": true,
"relevant_via_survey": true,
"status": "TS2",
"status_updated": null,
"task_id": "T65",
}
],
"facets": {}
}
I definitely recommend reading more about:
JSON serialization/deserialization.
Python Dictionaries
Python Lists
When you bring in the API response, the object will be a dictionary. I've called it api_response below.
You can target the list of results like this: results = api_response['results'] (see below).
Now that you have the results list, you can iterate through it and implement the counting technique that you linked to from Real Python's JSON example of counting TODOs.
import json
api_response = {
"results": [
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS1",
"status_updated": "2022-09-30T05:57:27.675000-04:00",
"task_id": "T178",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS2",
"status_updated": None,
"task_id": "T1213",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS1",
"status_updated": None,
"task_id": "T1188",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS2",
"status_updated": None,
"task_id": "T1177",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS1",
"status_updated": None,
"task_id": "T554",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS2",
"status_updated": None,
"task_id": "T1539",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS3",
"status_updated": None,
"task_id": "T106",
},
{
"accepted": True,
"relevant_via_survey": True,
"status": "TS2",
"status_updated": None,
"task_id": "T65",
}
],
"facets": {}
}
results = api_response['results']
count_by_status = {}
for result in results:
try:
count_by_status[result['status']] += 1
except KeyError:
count_by_status[result['status']] = 1
print(count_by_status)
The output will be:
{'TS1': 3, 'TS2': 4, 'TS3': 1}
Related
I have create a custom skill for my Azure Cognitive Service. After creating the datasouce, index, indexer and running the indexer I can see that my function is being called and it is outputting the correct information, but when I search the index the field that is connected to the custom skill is empty.
This is my indexer definition:
{
"name": "idxdocs",
"description": null,
"dataSourceName": "dsdocs",
"skillsetName": "skillset-procuradores",
"targetIndexName": "customer-documents",
"disabled": null,
"schedule": null,
"parameters": {
"batchSize": null,
"maxFailedItems": -1,
"maxFailedItemsPerBatch": null,
"base64EncodeKeys": null,
"configuration": {
"indexedFileNameExtensions": ".pdf,.docx,.doc",
"dataToExtract": "contentAndMetadata",
"failOnUnprocessableDocument": false,
"failOnUnsupportedContentType": false,
"indexStorageMetadataOnlyForOversizedDocuments": true,
"allowSkillsetToReadFileData": true
}
},
"fieldMappings": [],
"outputFieldMappings": [
{
"sourceFieldName": "/document/content/procuradores",
"targetFieldName": "procuradores"
}
],
"cache": null,
"encryptionKey": null
}
This is my index:
{
"name": "customer-documents",
"fields": [
{
"name": "key",
"type": "Edm.String",
"facetable": true,
"filterable": true,
"key": true,
"retrievable": true,
"searchable": false,
"sortable": true,
"analyzer": null,
"indexAnalyzer": null,
"searchAnalyzer": null,
"synonymMaps": [],
"fields": []
},
{
"name": "content",
"type": "Edm.String",
"facetable": false,
"filterable": false,
"key": false,
"retrievable": true,
"searchable": true,
"sortable": false,
"analyzer": "es.lucene",
"indexAnalyzer": null,
"searchAnalyzer": null,
"synonymMaps": [],
"fields": []
},
{
"name": "procuradores",
"type": "Edm.String",
"facetable": false,
"filterable": true,
"key": false,
"retrievable": true,
"searchable": false,
"sortable": false,
"analyzer": null,
"indexAnalyzer": null,
"searchAnalyzer": null,
"synonymMaps": [],
"fields": []
}
],
"suggesters": [],
"scoringProfiles": [],
"defaultScoringProfile": null,
"corsOptions": null,
"analyzers": [],
"charFilters": [],
"tokenFilters": [],
"tokenizers": [],
"similarity": {
"#odata.type": "#Microsoft.Azure.Search.BM25Similarity",
"k1": null,
"b": null
},
"encryptionKey": null,
"#odata.etag": "\"0x8D98BC85E9F6996\""
}
My skill set definition:
{
"name": "skillset-procuradores",
"description": "",
"skills": [
{
"#odata.type": "#Microsoft.Skills.Custom.WebApiSkill",
"name": "procuradores",
"description": "",
"context": "/document",
"uri": "my url ommited for secure reasons",
"httpMethod": "POST",
"timeout": "PT3M50S",
"batchSize": 1,
"degreeOfParallelism": null,
"inputs": [
{
"name": "text",
"source": "/document/content"
}
],
"outputs": [
{
"name": "procuradores",
"targetName": "procuradores"
}
],
"httpHeaders": {}
}
],
"cognitiveServices": null,
"knowledgeStore": null,
"encryptionKey": null
}
And finally my function output:
{
"values": [
{
"recordId": "1",
"data": {
"procuradores": "people's names"
}
}
]
}
What am I missing?
Rafael, can you give debug sessions a try? I suspect there is something happening in your custom skill that means the data is not coming back. By using this, you can test the actual input and output.
https://learn.microsoft.com/en-us/azure/search/cognitive-search-debug-session
This is just a hunch, but in the indexer definition, try changing your output field mapping sourceFieldName from "/document/content/procuradores" to "/document/procuradores". The context that you are giving in the skill is just "/document" so it should append the output onto that.
but it should retain the fields in other array elements which did not match the criteria.
I have json response like below
{
"content": [
{
"someId": 1498,
"someKey": {
"anotherKey": "Not Specified",
"keyOne": "",
"keyTwo": null,
"keyThree": null
},
"date": "2009-09-11",
"time": "17:14",
"location": "Not Specified"
},
{
"someId": 1498,
"someKey": {
"anotherKey": "criteria",
"keyOne": "some data",
"keyTwo": "some data for key two",
"keyThree": "some data for key three"
},
"date": "2009-09-12",
"time": "17:15",
"location": "Not Specified"
},...
],
"pageable": {
"sort": {
"sorted": false,
"unsorted": true,
"empty": true
},
"offset": 0,
"pageSize": 10,
"pageNumber": 0,
"unpaged": false,
"paged": true
},
"totalElements": 10630,
"totalPages": 1063,
"last": false,
"size": 10,
"number": 0,
"sort": {
"sorted": false,
"unsorted": true,
"empty": true
},
"first": true,
"numberOfElements": 10,
"empty": false
}
when "anotherKey" value is "criteria", the "keyOne","keyTwo","keyThree" should be omitted, else it should be retained. any solution to this? thanks in advance
Hashicorp Vault provides me the following JSON:
{
"somekey1/": {
"accessor": "generic_123456,
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"seal_wrap": false,
"type": "generic",
"uuid": "1111111-2222-3333-4444-55555555555"
},
"somekey2/": {
"accessor": "aws_123456",
"config": {
"default_lease_ttl": 3600,
"force_no_cache": false,
"max_lease_ttl": 86400
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": null,
"seal_wrap": false,
"type": "aws",
"uuid": "1111111-2222-3333-4444-55555555555"
},
"somekey3/": {
"accessor": "generic_1234567",
"config": {
"default_lease_ttl": 0,
"force_no_cache": false,
"max_lease_ttl": 0
},
"description": "",
"external_entropy_access": false,
"local": false,
"options": {},
"seal_wrap": false,
"type": "generic",
"uuid": "1111111-2222-3333-4444-55555555555"
}
}
I want to determine which top level keys have a sub-object K/V .type="generic".
In this case, "somekey1/" and "somekey3/"
Or "somekey2/" with a sub-object K/V .type="aws".
Can this be accomplished with JQ?
in this case, I use the following:
jq 'to_entries[] | select (.value.type == "generic") | .key' vault.json
which produces:
"somekey1/"
"somekey3/"
Which is exactly what I am looking for.
I have the following json response. I want to filter using json path as follows:
Get the first date for person ec7e231e-1fed-4860-a1ab-c2e3248a183e where 1. the date is not null and 2. the exam array is empty and 3. the absent flag is false
[
{
"Person": {
"ReferenceKey": "ec7e231e-1fed-4860-a1ab-c2e3248a183e",
"fname": "john",
"lname": "anderson",
"adress": null
},
"days": [
{
"date": null,
"absent": false,
"holiday": false,
"exam": [],
"blockingtimes": []
},
{
"date": "2021-02-19",
"absent": false,
"holiday": false,
"exam": [],
"blockingtimes": []
},
{
"date": "2021-02-20",
"absent": false,
"holiday": false,
"exam": [],
"blockingtimes": []
}
]
},
{
"Person": {
"ReferenceKey": "ec7e231e-1fed-4860-a1ab-c2e3248a1900",
"fname": "manny",
"lname": "panny",
"adress": null
},
"days": [
{
"date": null,
"absent": false,
"holiday": false,
"exam": [],
"blockingtimes": []
},
{
"date": "2021-02-19",
"absent": false,
"holiday": false,
"exam": [],
"blockingtimes": []
},
{
"date": "2021-02-20",
"absent": true,
"holiday": false,
"exam": [],
"blockingtimes": []
}
]
}
]
so far i was only able to retrieve all person uuids $..person.mitarbeiterReferenceKey
Something like
$.[?(#.Person.ReferenceKey == 'ec7e231e-1fed-4860-a1ab-c2e3248a183e')].days[?(#.date != null && #.absent == false && #.exam.length() == 0)])
More information:
Jayway JsonPath
JMeter's JSON Path Extractor Plugin - Advanced Usage Scenarios
How can I remove object from session which has 'pk':50 and store it for all pages. I added SESSION_SAVE_EVERY_REQUEST=True to Settings.py and request.session.modified = True to just above line when I modify request session, but there was no effect.
Json looks like
[
{
"pk": 50,
"model": "notifications.notification",
"fields": {
"recipient": 81,
"verb": "commented",
"emailed": false,
"action_object_object_id": "",
"level": "info",
"deleted": false,
"timestamp": "2017-01-25T11:18:53.197Z",
"target_content_type": null,
"actor_object_id": "1",
"action_object_content_type": null,
"target_object_id": "790",
"actor_content_type": 3,
"unread": true,
"data": "\"\"",
"public": true,
"description": "commented on your request"
}
},
{
"pk": 38,
"model": "notifications.notification",
"fields": {
"recipient": 81,
"verb": "commented",
"emailed": false,
"action_object_object_id": "",
"level": "info",
"deleted": false,
"timestamp": "2017-01-24T12:23:08Z",
"target_content_type": null,
"actor_object_id": "1",
"action_object_content_type": null,
"target_object_id": "790",
"actor_content_type": 3,
"unread": true,
"data": "\"\"",
"public": true,
"description": "commented on your request"
}
}
]
code
def setNotifRead(request, notif_id):
notifObject = Notification.objects.filter(pk=notif_id)
notifObject.update(unread=0)
notifications = request.session['notifications']
request.session.modified = True
del notifications[notif_id] # Something should be done here
return
You will need to iterate through the data and filter the items that don't have the relevant ID.
notifications = [item for item in request.session['notifications'] if item['id'] != notif_id
request.session['notifications'] = notifications