I ran curl command and then parsed the value ("id").
request:
curl "http://192.168.22.22/test/index/limit:1/page:1/sort:id/pag1.json" | jq -r '.[0].id'
curl response:
[
{
"id": "381",
"org_id": "9",
"date": "2018-10-10",
"info": "THIS IS TEST",
"uuid": "5bbd1b41bc",
"published": 1,
"an": "2",
"attribute_count": "4",
"orgc_id": "8",
"timestamp": "1",
"dEST": "0",
"sharing": "0",
"proposal": false,
"locked": false,
"level_id": "1",
"publish_timestamp": "0",
"disable_correlation": false,
"extends_uuid": "",
"Org": {
"id": "5",
"name": "test",
"uuid": "5b9bc"
},
"Orgc": {
"id": "1",
"name": "test",
"uuid": "5b9f93bdeac1b41bc"
},
"ETag": []
}
]
jq response:
381
Now I'm trying to get the "id" number 381, and then to create a new JSON file on the disk when I place the "id" number in the right place.
The new JSON file for example:
{
"request": {
"Event": {
"id": "381",
"task": "new"
}
}
}
Given your input, this works:
jq -r '{"request": {"Event": {"id": .[0].id, "task": "new"}}}' > file
Related
I'm new in the community and I'm not a dev, but I have a task I need to find a solution for. I hope I can get your ideas.
I have a set of JSON files. I want to be able to use jq or a command line that can help me unify the files into one single file.
For example:
File 1, has the following format:
{
"Interactions": [
{
"ID": "ispring://presentations/F7385CB7-DFDC-4D05-90FA-B927DB3D170D/quizzes/",
"Type": "2",
"TimestampUtc": "8/27/2020 12:09:54 PM",
"Timestamp": "8/27/2020 12:09:54 PM",
"Weighting": "",
"Result": "1",
"Latency": "1000",
"Description": "What is the purpose of Summarizing next steps?\n\nSelect the correct box or boxes",
"LearnerResponse": "0_correct_answer[,]1_Rectangle_2",
"ScormActivityId": "12392705",
"InteractionIndex": "3",
"AULMRID": "38093846"
},
]
}
File 2:
{
"Interactions": [
{
"ID": "ispring://presentations/CAA34147-7B48-40C6-84FD-5CE8077DB2BF/quizzes/",
"Type": "2",
"TimestampUtc": "12/8/2020 6:19:12 PM",
"Timestamp": "12/8/2020 6:19:12 PM",
"Weighting": "",
"Result": "1",
"Latency": "1300",
"Description": "'Can't do' language tends to relay this impression...\n\nSelect one.",
"LearnerResponse": "4_All_of_the_above",
"ScormActivityId": "13334358",
"InteractionIndex": "3",
"AULMRID": "40715598"
},
]
}
And this is my expected result in a third file:
{
"Interactions": [
{
"ID": "ispring://presentations/F7385CB7-DFDC-4D05-90FA-B927DB3D170D/quizzes/",
"Type": "2",
"TimestampUtc": "8/27/2020 12:09:54 PM",
"Timestamp": "8/27/2020 12:09:54 PM",
"Weighting": "",
"Result": "1",
"Latency": "1000",
"Description": "What is the purpose of Summarizing next steps?\n\nSelect the correct box or boxes",
"LearnerResponse": "0_correct_answer[,]1_Rectangle_2",
"ScormActivityId": "12392705",
"InteractionIndex": "3",
"AULMRID": "38093846"
},
{
"ID": "ispring://presentations/CAA34147-7B48-40C6-84FD-5CE8077DB2BF/quizzes/",
"Type": "2",
"TimestampUtc": "12/8/2020 6:19:12 PM",
"Timestamp": "12/8/2020 6:19:12 PM",
"Weighting": "",
"Result": "1",
"Latency": "1300",
"Description": "'Can't do' language tends to relay this impression...\n\nSelect one.",
"LearnerResponse": "4_All_of_the_above",
"ScormActivityId": "13334358",
"InteractionIndex": "3",
"AULMRID": "40715598"
},
]
}
Any ideas on how to unify them ?
Thank you!
RG
Try something like the following:
jq -n '{ Interactions: [ inputs.Interactions ] | add }' file1.json file2.json
This assumes that you have made both input files valid JSON by stripping that trailing comma at the end of each object in the Interactions array.
For input file1.json:
{
"Interactions": [
{
"ID": "file1",
"Type": "1",
"Timestamp": "8/27/2020 11:11:11 PM"
}
]
}
and input file2.json:
{
"Interactions": [
{
"ID": "file2",
"Type": "2",
"Timestamp": "8/27/2020 22:22:22 PM"
}
]
}
this results in:
{
"Interactions": [
{
"ID": "file1",
"Type": "1",
"Timestamp": "8/27/2020 11:11:11 PM"
},
{
"ID": "file2",
"Type": "2",
"Timestamp": "8/27/2020 22:22:22 PM"
}
]
}
My original JSON is given below.
[
{
"id": "1",
"name": "AA_1",
"total": "100002",
"files": [
{
"filename": "8665b987ab48511eda9e458046fbc42e.csv",
"filename_original": "some.csv",
"status": "3",
"total": "100002",
"time": "2020-08-24 23:25:49"
}
],
"status": "3",
"created": "2020-08-24 23:25:49",
"filenames": "8665b987ab48511eda9e458046fbc42e.csv",
"is_append": "0",
"is_deleted": "0",
"comment": null
},
{
"id": "4",
"name": "AA_2",
"total": "43806503",
"files": [
{
"filename": "1b4812fe634938928953dd40db1f70b2.csv",
"filename_original": "other.csv",
"status": "3",
"total": "21903252",
"time": "2020-08-24 23:33:43"
},
{
"filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"filename_original": "some.csv",
"status": "2",
"total": 0,
"time": "2020-08-24 23:29:30"
}
],
"status": "2",
"created": "2020-08-24 23:35:51",
"filenames": "1b4812fe634938928953dd40db1f70b2.csv&&63ab85fef2412ce80ae8bd018497d8bf.csv",
"is_append": "0",
"is_deleted": "0",
"comment": null
}
]
From this JSON I want to create new objects by combining fields from objects which have status: 2 and their files which also have the same pair, status: 2.
So, I am expecting a JSON array as below.
[
{
"id": "4",
"name": "AA_2",
"file_filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"file_status": 2
}
]
So far I tried with this JQ filter:
.[]|select(.status=="2")|[{id:.id,file_filename:.files[].filename,file_status:.files[].status}]
But this produces some invalid data.
[
{
"id": "4", # want to remove this as file.status != 2
"file_filename": "1b4812fe634938928953dd40db1f70b2.csv",
"file_status": "3"
},
{
"id": "4",
"file_filename": "1b4812fe634938928953dd40db1f70b2.csv",
"file_status": "2"
},
{
"id": "4", # Repeat
"file_filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"file_status": "3"
},
{
"id": "4", # Repeat
"file_filename": "63ab85fef2412ce80ae8bd018497d8bf.csv",
"file_status": "2"
}
]
How do I filter the new JSON using JQ and remove these duplicate objects?
By applying [] operator to files twice, you're running into a combinatorial explosion. That needs to be avoided, for example:
[ .[] | select(.status == "2") | {id, name} + (.files[] | select(.status == "2") | {file_filename: .filename, file_status: .status}) ]
Online demo
I have a json file and I want to add some value from top in another place in json.
I am trying to use jq command line.
{
"channel": "mychannel",
"videos": [
{
"id": "10",
"url": "youtube.com"
},
{
"id": "20",
"url": "youtube.com"
}
]
}
The output would be:
{
"channel": "mychannel",
"videos": [
{
"channel": "mychannel",
"id": "10",
"url": "youtube.com"
},
{
"channel": "mychannel",
"id": "20",
"url": "youtube.com"
}
]
}
in my json the "channel" is static, same value always. I need a way to concatenate always in each video array.
Someone can help me?
jq .videos + channel
Use a variable to remember .channel in the later stages of the pipeline.
$ jq '.channel as $ch | .videos[].channel = $ch' tmp.json
{
"channel": "mychannel",
"videos": [
{
"id": "10",
"url": "youtube.com",
"channel": "mychannel"
},
{
"id": "20",
"url": "youtube.com",
"channel": "mychannel"
}
]
}
I am trying to remove the duplicates from the following json by id
Here is the json:
{
"Result": [
{
"name": "validation-of-art",
"id": "12",
"status": "passed",
"duration": 4740302
},
{
"name": "validation-of-art",
"id": "12",
"status": "passed",
"duration": 272320
},
{
"name": "validation-of-art",
"id": "13",
"status": "passed",
"duration": 272320
}
]
}
Here is what i have tried with:
jq -r 'unique_by(.Result.name)'
and also with jq 'unique_by(.Result[].name)'
I am getting an error - Cannot index array with string "Result"
Any help would be appreciated.
Here is an example which eliminates all but one of the .Result objects using unique_by(.name)
$ jq -M '.Result |= unique_by(.name)' data.json
{
"Result": [
{
"name": "validation-of-art",
"id": "12",
"status": "passed",
"duration": 4740302
}
]
}
If this isn't quite what you want you can generalize this easily. E.g. to keep one object for each unique {name,id} you could use
$ jq -M '.Result |= unique_by({name, id})' data.json
{
"Result": [
{
"name": "validation-of-art",
"id": "12",
"status": "passed",
"duration": 4740302
},
{
"name": "validation-of-art",
"id": "13",
"status": "passed",
"duration": 272320
}
]
}
I have a file with the return of a curl statement in it, in the form of json. Each object has a set of values, but the parameters for these values are all called the same names. See code below.
These objects are part of a larger object called workflow. The Cleaning up object is the last process that runs in our workflow. For every video that passes through the workflow, a json file in this format is created. (There are more than only these three objects, this is just for illustrative purposes)
I want to take the value of completed of the object with "description": "Cleaning up" and store it as a variable $end_time. Then I want to take the value of completed of the object with "description": "Ingest" and store it as a variable $start_time. These two values are then subtracted to give me an integer time in milliseconds so I can calculate the time it took for the video to go through this part of the process. With the maths part I am fine, and know how to do it. It is the extraction of the values that I am struggling with.
I hope this makes sense? ANY help would be appreciated. Thank you in advance!
EDIT: Had to delete original code in post, due to character limitations
Here is a proper example of the file that I have to work with:
{
"workflows": {
"count": "20",
"searchTime": "1",
"startPage": "0",
"totalCount": "1",
"workflow": {
"configurations": {
"configuration": [
{
"$": "1409750880000",
"key": "schedule.start"
},
{
"$": "1409755980000",
"key": "schedule.stop"
},
{
"$": "Capture_agent",
"key": "schedule.location"
},
{
"$": "false",
"key": "trimHold"
},
{
"$": "true",
"key": "archiveOp"
},
{
"$": "false",
"key": "captionHold"
},
{
"$": "false",
"key": "videoPreview"
}
]
},
"creator": {
"organization": "mh_default_org",
"roles": [
"76b1bdde-a080-40a4-b929-bde89af6a0a8_Instructor",
"ROLE_ADMIN",
"ROLE_ANONYMOUS",
"ROLE_USER"
],
"userName": user_name
},
"description": "This workflow definition defines the steps involved in scheduling a recording, capturing it, and\n ingesting it, after which processing operations may be added.\n ",
"errors": "",
"id": "15518",
"mediapackage": {
"attachments": "",
"creators": {
"creator": "Name"
},
"id": "2d25ed19-2978-458d-a4a0-c9c56d791c68",
"license": "Creative Commons 3.0: Attribution-NonCommercial-NoDerivs",
"media": "",
"metadata": "",
"publications": {
"publication": {
"channel": "engage-player",
"id": "b7b68f91-2c33-4673-ba7c-2e9b891788f9",
"mimetype": "text/html",
"tags": "",
"url": "http://some.url.com:80/engage/ui/watch.html?id=2d25ed19-2978-458d-a4a0-c9c56d791c68"
}
},
"series": "76b1bdde-a080-40a4-b929-bde89af6a0a8",
"seriestitle": "Recording_Title_user_name",
"start": "2014-09-03T13:28:00Z",
"title": "Recording_Title"
},
"operations": {
"operation": [
{
"abortable": "false",
"completed": 1409750882092,
"configurations": {
"configuration": [
{
"$": "1409750880000",
"key": "schedule.start"
},
{
"$": "1409755980000",
"key": "schedule.stop"
},
{
"$": "Capture_agent",
"key": "schedule.location"
}
]
},
"continuable": "false",
"description": "Scheduled",
"execution-history": "",
"execution-host": "http://some.url.com:8080",
"fail-on-error": "true",
"failed-attempts": "0",
"hold-action-title": "View schedule",
"holdurl": "/workflow/hold/org.opencastproject.workflow.handler.scheduleworkflowoperationhandler",
"id": "schedule",
"job": "15519",
"max-attempts": "1",
"retry-strategy": "none",
"started": 1409750881745,
"state": "SUCCEEDED",
"time-in-queue": 0
},
{
"abortable": "false",
"configurations": "",
"continuable": "false",
"description": "Capture",
"execution-history": "",
"execution-host": "http://some.url.com:8080",
"fail-on-error": "true",
"failed-attempts": "0",
"hold-action-title": "Monitor capture",
"holdurl": "/workflow/hold/org.opencastproject.workflow.handler.captureworkflowoperationhandler",
"id": "capture",
"job": "42894",
"max-attempts": "1",
"retry-strategy": "none",
"started": 1409750884085,
"state": "SKIPPED",
"time-in-queue": 0
},
{
"completed": 1409756171224,
"configurations": "",
"description": "Ingest",
"execution-history": "",
"fail-on-error": "true",
"failed-attempts": "0",
"id": "ingest",
"max-attempts": "1",
"retry-strategy": "none",
"state": "SUCCEEDED"
},
{
"completed": 1409854379552,
"configurations": {
"configuration": {
"key": "preserve-flavors"
}
},
"description": "Cleaning up",
"execution-history": "",
"execution-host": "http://some.url.com:8080",
"fail-on-error": "false",
"failed-attempts": "0",
"id": "cleanup",
"job": "45113",
"max-attempts": "1",
"retry-strategy": "none",
"started": 1409854378128,
"state": "SUCCEEDED",
"time-in-queue": 0
}
]
},
"organization": {
"adminRole": "ROLE_ADMIN",
"anonymousRole": "ROLE_ANONYMOUS",
"id": "mh_default_org",
"name": "Opencast Project",
"properties": {
"property": [
{
"$": "true",
"key": "adminui.i18n_tab_episode.enable"
},
{
"$": "false",
"key": "adminui.i18n_tab_users.enable"
},
{
"$": "/engage/ui/img/mh_logos/OpencastLogo.png",
"key": "logo_small"
},
{
"$": "http://opencast.org/matterhorn/",
"key": "engageui.link_mobile_redirect.url"
},
{
"$": "false",
"key": "engageui.annotations.enable"
},
{
"$": "true",
"key": "engageui.links_media_module.enable"
},
{
"$": "2024",
"key": "adminui.chunksize"
},
{
"$": "false",
"key": "adminui.series_prepopulate.enable"
},
{
"$": "true",
"key": "engageui.link_download.enable"
},
{
"$": "false",
"key": "engageui.link_mobile_redirect.enable"
},
{
"$": "For more information have a look at the official site.",
"key": "engageui.link_mobile_redirect.description"
},
{
"$": "/engage/ui/img/mh_logos/MatterhornLogo_large.png",
"key": "logo_large"
}
]
},
"servers": {
"server": {
"name": "localhost",
"port": "8080"
}
}
},
"parent": {
"nil": "true"
},
"state": "SUCCEEDED",
"template": "full",
"title": "Scheduled Workflow"
}
}
}
Here is a jq example that should point you to getting what you want:
#!/bin/bash
# Assuming the json is in a file workflow.json
end_time=$( jq '.workflows.workflow.operations.operation[] | select(.description == "Cleaning up") | .completed' < workflow.json )
start_time=$( jq '.workflows.workflow.operations.operation[] | select(.description == "Ingest") | .completed' < workflow.json )
This is assuming the input you have is in an JSON array called workflow at the top level. Here's this on the command line:
$ jq '.workflows.workflow.operations.operation[] | select(.description == "Ingest") | .completed' < workflow.json
1406051539118
$ jq '.workflows.workflow.operations.operation[] | select(.description == "Cleaning up") | .completed' < workflow.json
1406051695440