In a log I’d like to match all json objects which type is "sync.out.notify.job.status" and print them. I’m on a Mac (zsh).
I tried: grep -Eo \{.+"sync\.out\.notify\.job\.status".+\}
Mon Jan 11 2021 13:08:46 GMT+0100 (Central European Standard Time) CoreSyncNotifications: sync.out.notify.job.status:
{
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "syncing",
"minorstate": "transferring",
"requestid": "45768456-bd58-4cb5-9dff-dfgjfgdj456",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
Mon Jan 11 2021 13:08:46 GMT+0100 (Central European Standard Time) Vulcan: <- COSY 4.3: sync.out.notify.job.status
Mon Jan 11 2021 13:08:46 GMT+0100 (Central European Standard Time) CoreSyncNotifications: sync.out.notify.job.status:
{
"errors": "",
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "idle",
"minorstate": "complete",
"requestid": "4856658fdgh-efb3-4da9-b5f5-4856658fgj",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
Mon Jan 11 2021 13:08:46 GMT+0100 (Central European Standard Time) WebSocketServer: client_7: SEND syncstate creative_cloud: ok
grep usually searches pattern in single line only unless you use -z option in gnu grep.
You may try this gnu awk command:
awk -v RS='(^|\n){[^}]+?sync.out.notify.job.status[^}]+}(\n|$)' 'RT {printf "%s", RT}' file
{
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "syncing",
"minorstate": "transferring",
"requestid": "45768456-bd58-4cb5-9dff-dfgjfgdj456",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
{
"errors": "",
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "idle",
"minorstate": "complete",
"requestid": "4856658fdgh-efb3-4da9-b5f5-4856658fgj",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
You can easily and robustly use a JSON-aware tool like jq for this, assuming you want to print the matching blocks:
$ grep -v '^[[:alpha:]]' file | jq 'select(.type=="sync.out.notify.job.status")'
{
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "syncing",
"minorstate": "transferring",
"requestid": "45768456-bd58-4cb5-9dff-dfgjfgdj456",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
{
"errors": "",
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "idle",
"minorstate": "complete",
"requestid": "4856658fdgh-efb3-4da9-b5f5-4856658fgj",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
Here's another way to get from { to } to pipe to jq:
$ sed -n '/^{/,/^}/p' file
{
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "syncing",
"minorstate": "transferring",
"requestid": "45768456-bd58-4cb5-9dff-dfgjfgdj456",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
{
"errors": "",
"cloudid": "00000000-0000-0000-0000-000000000000",
"job": "files",
"majorstate": "idle",
"minorstate": "complete",
"requestid": "4856658fdgh-efb3-4da9-b5f5-4856658fgj",
"type": "sync.out.notify.job.status",
"userid": "XXXXXXXXXXXXXXXXXXXXXXXX#AdobeID",
"version": "1.1.43711"
}
I normally wouldn't suggest using a range expression over using a flag but in this case since it's ALL you're going to do with it, it's probably fine.
Related
i can't parse this json with logstash... someone could help me?
seems like the way it is parsed can't be readed by logstash.
there is a ruby code to parse this?
I cannot extract the fields nested in the square brackets
[
{
"capacity": 0,
"created_at": "2021-04-06T16:18:34+02:00",
"decisions": [
{
"duration": "22h16m4.141220361s",
"id": 842,
"origin": "CAPI",
"scenario": "crowdsecurity/http-bad-user-agent",
"scope": "ip",
"simulated": false,
"type": "ban",
"value": "3.214.184.223/32"
},
.
.
.
{
"duration": "22h16m4.195897491s",
"id": 904,
"origin": "CAPI",
"scenario": "crowdsecurity/http-backdoors-attempts",
"scope": "ip",
"simulated": false,
"type": "ban",
"value": "51.68.11.195/32"
}
],
"events": null,
"events_count": 0,
"id": 12,
"labels": null,
"leakspeed": "",
"machine_id": "N/A",
"message": "",
"scenario": "update : +63/-0 IPs",
"scenario_hash": "",
"scenario_version": "",
"simulated": false,
"source": {
"scope": "Community blocklist",
"value": ""
},
"start_at": "2021-04-06 16:18:34.750588276 +0200 +0200",
"stop_at": "2021-04-06 16:18:34.750588717 +0200 +0200"
}
]
Require JSON
JSON.parse(yourString)
Would likely be what you're looking for.
The module is described here
I'm new in the community and I'm not a dev, but I have a task I need to find a solution for. I hope I can get your ideas.
I have a set of JSON files. I want to be able to use jq or a command line that can help me unify the files into one single file.
For example:
File 1, has the following format:
{
"Interactions": [
{
"ID": "ispring://presentations/F7385CB7-DFDC-4D05-90FA-B927DB3D170D/quizzes/",
"Type": "2",
"TimestampUtc": "8/27/2020 12:09:54 PM",
"Timestamp": "8/27/2020 12:09:54 PM",
"Weighting": "",
"Result": "1",
"Latency": "1000",
"Description": "What is the purpose of Summarizing next steps?\n\nSelect the correct box or boxes",
"LearnerResponse": "0_correct_answer[,]1_Rectangle_2",
"ScormActivityId": "12392705",
"InteractionIndex": "3",
"AULMRID": "38093846"
},
]
}
File 2:
{
"Interactions": [
{
"ID": "ispring://presentations/CAA34147-7B48-40C6-84FD-5CE8077DB2BF/quizzes/",
"Type": "2",
"TimestampUtc": "12/8/2020 6:19:12 PM",
"Timestamp": "12/8/2020 6:19:12 PM",
"Weighting": "",
"Result": "1",
"Latency": "1300",
"Description": "'Can't do' language tends to relay this impression...\n\nSelect one.",
"LearnerResponse": "4_All_of_the_above",
"ScormActivityId": "13334358",
"InteractionIndex": "3",
"AULMRID": "40715598"
},
]
}
And this is my expected result in a third file:
{
"Interactions": [
{
"ID": "ispring://presentations/F7385CB7-DFDC-4D05-90FA-B927DB3D170D/quizzes/",
"Type": "2",
"TimestampUtc": "8/27/2020 12:09:54 PM",
"Timestamp": "8/27/2020 12:09:54 PM",
"Weighting": "",
"Result": "1",
"Latency": "1000",
"Description": "What is the purpose of Summarizing next steps?\n\nSelect the correct box or boxes",
"LearnerResponse": "0_correct_answer[,]1_Rectangle_2",
"ScormActivityId": "12392705",
"InteractionIndex": "3",
"AULMRID": "38093846"
},
{
"ID": "ispring://presentations/CAA34147-7B48-40C6-84FD-5CE8077DB2BF/quizzes/",
"Type": "2",
"TimestampUtc": "12/8/2020 6:19:12 PM",
"Timestamp": "12/8/2020 6:19:12 PM",
"Weighting": "",
"Result": "1",
"Latency": "1300",
"Description": "'Can't do' language tends to relay this impression...\n\nSelect one.",
"LearnerResponse": "4_All_of_the_above",
"ScormActivityId": "13334358",
"InteractionIndex": "3",
"AULMRID": "40715598"
},
]
}
Any ideas on how to unify them ?
Thank you!
RG
Try something like the following:
jq -n '{ Interactions: [ inputs.Interactions ] | add }' file1.json file2.json
This assumes that you have made both input files valid JSON by stripping that trailing comma at the end of each object in the Interactions array.
For input file1.json:
{
"Interactions": [
{
"ID": "file1",
"Type": "1",
"Timestamp": "8/27/2020 11:11:11 PM"
}
]
}
and input file2.json:
{
"Interactions": [
{
"ID": "file2",
"Type": "2",
"Timestamp": "8/27/2020 22:22:22 PM"
}
]
}
this results in:
{
"Interactions": [
{
"ID": "file1",
"Type": "1",
"Timestamp": "8/27/2020 11:11:11 PM"
},
{
"ID": "file2",
"Type": "2",
"Timestamp": "8/27/2020 22:22:22 PM"
}
]
}
I ran curl command and then parsed the value ("id").
request:
curl "http://192.168.22.22/test/index/limit:1/page:1/sort:id/pag1.json" | jq -r '.[0].id'
curl response:
[
{
"id": "381",
"org_id": "9",
"date": "2018-10-10",
"info": "THIS IS TEST",
"uuid": "5bbd1b41bc",
"published": 1,
"an": "2",
"attribute_count": "4",
"orgc_id": "8",
"timestamp": "1",
"dEST": "0",
"sharing": "0",
"proposal": false,
"locked": false,
"level_id": "1",
"publish_timestamp": "0",
"disable_correlation": false,
"extends_uuid": "",
"Org": {
"id": "5",
"name": "test",
"uuid": "5b9bc"
},
"Orgc": {
"id": "1",
"name": "test",
"uuid": "5b9f93bdeac1b41bc"
},
"ETag": []
}
]
jq response:
381
Now I'm trying to get the "id" number 381, and then to create a new JSON file on the disk when I place the "id" number in the right place.
The new JSON file for example:
{
"request": {
"Event": {
"id": "381",
"task": "new"
}
}
}
Given your input, this works:
jq -r '{"request": {"Event": {"id": .[0].id, "task": "new"}}}' > file
Please bear with me I'm still learning about JSON and programming in general. So I have this JSON file:
{
"root_200888546292191": {
"fields": {
"buyerId": "31392191"
},
"id": "200718546292191",
"tag": "root",
"type": "biz"
},
"shippingInfo_#package#OF04472002179150#WAREHOUSE_ACCEPTED": {
"fields": {
"delivery": {
"createdAt": "Sen 09 Apr - Rab 11 Apr",
"desc": "Standar",
"email": null,
"method": "Standard",
"status": "info"
},
"statusMap": {
"active": "Dalam proses",
"all": ["Dalam proses", "Dalam pengiriman", "Telah diterima"]
},
"trackingList": [{
"info": "Status One",
"updatedAt": "05 Apr 2018 - 11:00"
}, {
"info": "Status Two",
"updatedAt": "05 Apr 2018 - 11:00"
}]
},
"id": "#package#OF04472002179150#WAREHOUSE_ACCEPTED",
"tag": "shippingInfo",
"type": "biz"
},
"shippingInfo_#package#AAAAAAAAAAAAA#NOT_WAREHOUSE_ACCEPTED": {
"fields": {
"delivery": {
"createdAt": "Sen 09 Apr - Rab 11 Apr",
"desc": "Standar",
"email": null,
"method": "Standard",
"status": "info"
},
"statusMap": {
"active": "Dalam proses",
"all": ["Dalam proses", "Dalam pengiriman", "Telah diterima"]
},
"trackingList": [{
"info": "Status Three",
"updatedAt": "05 Apr 2018 - 11:00"
}, {
"info": "Status Four",
"updatedAt": "05 Apr 2018 - 11:00"
}]
},
"id": "#package#AAAAAAAAAAAAA#NOT_WAREHOUSE_ACCEPTED",
"tag": "shippingInfo",
"type": "biz"
},
"login_200718577292191": {
"fields": {
"buyerEmail": "myemail#gmail.com",
"buyerName": "myname"
},
"id": "200718522292191",
"tag": "login",
"type": "biz"
}
}
And I want to extract Info in shippingInfo_ > fields > trackingList So the output that I want is like this:
Status One
Status Two
Status Three
Status Four
The string after shippingInfo_ is always random, how do I extract it with jq?
This is as far as I've got jq '.shippingInfo_*.fields.trackingList.info'
Direct approaches
There are many direct approaches, e.g.:
Using paths
paths as $p
| select( $p|length == 5 and
($p[0] | startswith("shippingInfo_")) and
$p[1:3] == ["fields", "trackingList"] and
$p[4] == "info")
| getpath($p)
Using to_entries
to_entries[]
| select(.key | startswith("shippingInfo_"))
| .value
| .fields.trackingList[]
| .info
Indirect approaches
There are also some indirect approaches that are worth mentioning, e.g.
Using a helper function
def dot(s):
to_entries[] | select(.key|test(s)) | .value ;
dot("^shippingInfo_")
| .fields.trackingList[]
| .info
The last-mentioned filter can be abbreviated to:
dot("^shippingInfo_").fields.trackingList[].info
Relaxed requirements
If it is acceptable to ignore the "^shippingInfo_" requirement, the following may be worth considering as well:
.[].fields.trackingList[]?.info
or even:
.. | objects.fields.trackingList[]?.info
I want to find and delete 'indicent' object in my json using sublime. here's the regex
\s*+,\s*(.)+"\s*indices+"\s*:\s*+(.)\s*\s*(.)+\s*(.)+\s*+]
but, it's ran out of stack space. i think that regex isn't effiecient.
json example :
"created": "Fri Nov 27 11:12:43 +0000 2015",
"text": "https://t.co/8r5dQ7zRYG #johnl3375 Kim Gi Jung",
"source": "NOMOR1",
"hashtags": [
{
"text": "johnl3375",
"indices": [
24,
34
]
}
],
"url": [
{
"url": "https://t.co/8r5dQ7zRYG",
"expanded_url": "https://play.google.com/store/apps/details?id=com.aturkeuangan.nomor1#refid=johnl3375",
"display_url": "play.google.com/store/apps/det…",
"indices": [
0,
23
]
}
]
i want json like this
"created": "Fri Nov 27 11:12:43 +0000 2015",
"text": "https://t.co/8r5dQ7zRYG #johnl3375 Kim Gi Jung",
"source": "NOMOR1",
"hashtags": [
{
"text": "johnl3375"
}
],
"url": [
{
"url": "https://t.co/8r5dQ7zRYG",
"expanded_url": "https://play.google.com/store/apps/details?id=com.aturkeuangan.nomor1#refid=johnl3375",
"display_url": "play.google.com/store/apps/det…"
}
]
what should my regex be?
This works for me in ST3 using your JSON example: [\s,]+"indices":[^]]+]