Remove values of a string in a JSON file - json

I have a JSON file containing the following:
[
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
}
]
I would like to remove the values after date, number and ID. I would like to do this using a powershell script.
My expected outcome is:
[
{
"Date": "",
"Number": "",
"ID": ""
},
{
"Date": "",
"Number": "",
"ID": ""
},
{
"Date": "",
"Number": "",
"ID": ""
},
}
]
Can someone help me to achieve this? Kind Regards!


I would suggest iterating over the array and just set the properties to an empty string:
#'
[
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
{
"Date": "05082021",
"Number": "123456789",
"ID": "123456789"
},
]
'# | ConvertFrom-Json | ForEach-Object {
$_.Date = [string]::Empty
$_.Number = [string]::Empty
$_.ID = [string]::Empty
$_
} | ConvertTo-Json

Related

Converting array to objects within an object

I have been battling with this for sometime, ive looked a few articles in regards to this topic. I can't seem to get exactly what I'm aiming for. I have this original json data that I would like to transform a bit.
Group the objects based on the label.env (for now only using one env), then inserting those objects from the original json into an object named projects.
orginial_json
{
"createTime": "2020-06-25T14:16:45.720Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "infra",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname",
"parent": {
"id": "123456577",
"type": "folder"
},
"projectId": "projectname-324234",
"projectNumber": "962363417856"
}
{
"createTime": "2020-06-24T19:45:42.851Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "ad",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname2",
"parent": {
"id": "4352564765437",
"type": "folder"
},
"projectId": "projectname2-4567",
"projectNumber": "3243456324"
}
Im using jq -s 'group_by(.labels.env) | .[] | { (.[0].labels.env) : { projects: .[] | . } }'
to nest the original json data into objects grouped by env\projects, which works but seems to duplicate it instead of putting each object within projects.
{
"prod": {
"projects": {
"createTime": "2020-06-25T14:16:45.720Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "infra",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname",
"parent": {
"id": "770593713153",
"type": "folder"
},
"projectId": "projectname-324234",
"projectNumber": "962363417856"
}
}
}
{
"prod": {
"projects": {
"createTime": "2020-06-24T19:45:42.851Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "ad",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname2",
"parent": {
"id": "4352564765437",
"type": "folder"
},
"projectId": "projectname2-4567",
"projectNumber": "3243456324"
}
}
}
What I'm aiming for is
{
"prod": {
"projects": {
"createTime": "2020-06-25T14:16:45.720Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "infra",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname",
"parent": {
"id": "770593713153",
"type": "folder"
},
"projectId": "projectname-324234",
"projectNumber": "962363417856"
},
{
"createTime": "2020-06-24T19:45:42.851Z",
"labels": {
"cb_domain": "cloud-services",
"cb_product": "ad",
"env": "prod",
"owner": "cloud-server"
},
"lifecycleState": "ACTIVE",
"name": "projectname2",
"parent": {
"id": "4352564765437",
"type": "folder"
},
"projectId": "projectname2-4567",
"projectNumber": "3243456324"
}
}
}

Allowing for some minor discrepancies in the Q, the following variant of your jq program seems to do what you're trying to achieve:
group_by(.labels.env)
| .[]
| .[0].labels.env as $key
| { ($key) : { projects: . } }

Ansible group by a json attribute

I've the below json data.
{
"images": [
{
"id": "79334504-3e48-4411-b602-afe920e32e14",
"name": "rhel-7-factory-os-ready",
"updated": "2019-05-07T15:51:00Z"
},
{
"id": "05569058-0b47-4c05-8ce7-f09b2e32076a",
"name": "sles-12-factory-os-ready",
"updated": "2019-05-07T15:48:59Z"
},
{
"id": "77c131bc-4dd6-4290-9dfb-89ddebc609f4",
"name": "sles-11-factory-os-ready",
"updated": "2019-05-07T15:45:10Z"
},
{
"id": "d42b7a07-e329-46ac-82c1-28875498b145",
"name": "rhel-6-factory-os-ready",
"updated": "2019-05-07T15:44:54Z"
},
{
"id": "d282ae55-ecc6-47a3-bb2f-4354bebf33e1",
"name": "rhel-7-factory-base-vg00",
"updated": "2019-05-07T15:37:25Z"
},
{
"id": "0c0fb244-7553-4f20-bf3b-8e32d0527292",
"name": "sles-12-factory-base-vg00",
"updated": "2019-05-07T15:32:14Z"
},
{
"id": "14a34117-91d5-4cbf-b039-613fd8b5ea65",
"name": "rhel-6-factory-base-vg00",
"updated": "2019-05-07T15:29:28Z"
},
{
"id": "bf465c2d-9f4c-44a9-9cc8-ff488b79e9f6",
"name": "rhel-7-factory-base",
"updated": "2019-05-07T15:28:00Z"
},
{
"id": "90d4e12d-6d7f-4f59-a4e6-28b1757f95bc",
"name": "sles-11-factory-base-vg00",
"updated": "2019-05-07T15:24:19Z"
},
{
"id": "30109a53-4aab-4fb5-87ff-7650e75731b0",
"name": "rhel-6-factory-base",
"updated": "2019-05-07T15:20:39Z"
},
{
"id": "80cd641b-bb53-46f7-b8f6-7f287507fc3c",
"name": "sles-12-factory-base",
"updated": "2019-05-07T15:20:10Z"
},
{
"id": "dc2deb91-2ff0-4d9e-a44d-fa2d7a8e1c62",
"name": "sles-11-factory-base",
"updated": "2019-05-07T15:18:05Z"
},
{
"id": "b6646eab-5908-4db4-a93a-1ca0ce52b76b",
"name": "sles-11-factory-os-ready",
"updated": "2019-05-07T14:27:41Z"
},
{
"id": "850cdd08-a5a7-4e6b-a906-4fda157257b6",
"name": "sles-12-factory-base",
"updated": "2019-05-07T14:23:40Z"
},
{
"id": "30325ffd-f17a-4db8-a2ef-a2e529163f03",
"name": "sles-11-factory-base-vg00",
"updated": "2019-05-07T14:08:22Z"
},
{
"id": "70f49bc1-f793-4d1d-8317-e2d1d35993f7",
"name": "rhel-6-factory-base-vg00",
"updated": "2019-05-07T13:42:01Z"
},
{
"id": "6bb77f3d-335b-459d-8aa1-dd3effd60672",
"name": "rhel-6-factory-os-ready",
"updated": "2019-05-07T13:40:17Z"
},
{
"id": "1cbfe793-5ca5-4310-b121-e5065ffa0cf8",
"name": "rhel-6-factory-base-vg00",
"updated": "2019-05-07T13:25:46Z"
},
{
"id": "17d0b51f-1c19-472a-9104-31624435bd63",
"name": "rhel-7-factory-os-ready",
"updated": "2019-05-07T13:18:55Z"
},
{
"id": "4e1f6f7a-90fc-4055-9ee7-d284656fcd58",
"name": "rhel-7-factory-base-vg00",
"updated": "2019-05-07T12:41:21Z"
},
{
"id": "7129df12-ae5b-47ff-905d-1e7bf79069cb",
"name": "rhel-7-factory-base",
"updated": "2019-05-07T12:31:32Z"
},
{
"id": "a37ce4d3-fc3e-4ae9-8d24-038c638be629",
"name": "rhel-6-factory-base-vg00",
"updated": "2019-05-07T12:30:08Z"
},
{
"id": "294821b3-e9df-4b12-ae39-8deb8f9c72ff",
"name": "rhel-6-factory-base",
"updated": "2019-05-07T12:21:36Z"
},
{
"id": "520ae1ca-bb2f-4ccb-bed3-886ee39ccd5e",
"name": "sles-11-factory-base",
"updated": "2019-05-07T12:21:00Z"
},
{
"id": "e041c0ed-c30f-4c61-a695-682e5a80026a",
"name": "rhel-6-factory-os-ready",
"updated": "2019-04-29T14:17:52Z"
},
{
"id": "068ae5f0-d7a0-4419-829e-18eb25551c94",
"name": "rhel-6-factory-base",
"updated": "2019-04-29T14:04:20Z"
},
{
"id": "2dfa8476-54dd-4dc5-80cc-5033466f2732",
"name": "rhel-7-factory-base",
"updated": "2019-04-26T13:40:59Z"
},
{
"id": "61a5a77a-0e8a-47d0-938e-6202c5cc5177",
"name": "rhel-7-factory-os-ready",
"updated": "2019-04-26T13:40:11Z"
},
{
"id": "b877f241-caa1-43c4-9465-fb5019d44f9b",
"name": "sles-12-factory-base-vg00",
"updated": "2019-04-26T13:45:40Z"
},
{
"id": "baf4c87d-8363-48cf-ae02-be9caef4f60d",
"name": "sles-12-factory-base",
"updated": "2019-04-26T13:41:31Z"
},
{
"id": "67a2ffa1-6591-42cb-95db-d2b1a854a78d",
"name": "sles-11-factory-os-ready",
"updated": "2019-04-26T10:17:26Z"
}
]
}
I would like to group by above json by the name attribute like below.
{
"rhel-7-factory-os-ready" : [
{
"id": "79334504-3e48-4411-b602-afe920e32e14",
"updated": "2019-05-07T15:51:00Z"
},
{
"id": "17d0b51f-1c19-472a-9104-31624435bd63",
"updated": "2019-05-07T13:18:55Z"
},
{
"id": "61a5a77a-0e8a-47d0-938e-6202c5cc5177",
"updated": "2019-04-26T13:40:11Z"
}
],
"rhel-6-factory-os-ready" : [
{
"id": "d42b7a07-e329-46ac-82c1-28875498b145",
"updated": "2019-05-07T15:44:54Z"
},
{
"id": "6bb77f3d-335b-459d-8aa1-dd3effd60672",
"updated": "2019-05-07T13:40:17Z"
},
{
"id": "e041c0ed-c30f-4c61-a695-682e5a80026a",
"updated": "2019-04-29T14:17:52Z"
}
]
}
I somehow couldn't figure out how I could achieve this with ansible. It would be great if someone could help me with this.
Thanks in advance,
Harsha

It turns out that Ansible has a groupby filter already:
---
- hosts: localhost
gather_facts: false
tasks:
- include_vars:
file: data.json
name: data
- debug:
var: data.images|groupby('name')

Create ODBC wrapper/driver for REST API

just checking before i build the wheel
I need a hackjob to present an api endpoint in a database. It doesn't need to do anything fancy, just convert what the rest api spits out into a single column of json. A new row at each iteration at the root/top level would be nice but a single varchar or whatever would be ok too
Does an ODBC wrapper exist out there anywhere? Googling just brings up hits for doing the opposite (exposing databases as an api). I'm not interested in the simba etc paid stuff. The consumer is SQL server so i can just use xp_cmdshell with curl as a last resort
so for instance the output of this : http://jsonapiplayground.reyesoft.com/v2/authors could come out as a table like so (a row for each author)
|data |
---------------------------------------------------
|{
"type": "authors",
"id": "1",
"attributes": {
"name": "Madge Mohr DVM 2",
"date_of_birth": "1977-08-21",
"date_of_death": "2009-09-14"
},
"relationships": {
"photos": {
"data": []
},
"books": {
"data": [
{
"type": "books",
"id": "41"
}
]
}
},
"links": {
"self": "/v2/authors/1"
}
} |
---------------------------------------------------
|{
"type": "authors",
"id": "3",
"attributes": {
"name": "Zelma Ortiz DDS",
"date_of_birth": "1992-09-06",
"date_of_death": "2000-12-19"
},
"relationships": {
"photos": {
"data": [
{
"type": "photos",
"id": "3"
}
]
},
"books": {
"data": [
{
"type": "books",
"id": "36"
},
{
"type": "books",
"id": "48"
}
]
}
},
"links": {
"self": "/v2/authors/3"
}
}|
----------
|{
"type": "authors",
"id": "4",
"attributes": {
"name": "Fermin Barrows Sr.",
"date_of_birth": "1991-03-18",
"date_of_death": "1975-11-07"
},
"relationships": {
"photos": {
"data": [
{
"type": "photos",
"id": "4"
}
]
},
"books": {
"data": [
{
"type": "books",
"id": "1"
},
{
"type": "books",
"id": "26"
},
{
"type": "books",
"id": "44"
},
{
"type": "books",
"id": "46"
}
]
}
},
"links": {
"self": "/v2/authors/4"
}
}|
----------
|{
"type": "authors",
"id": "5",
"attributes": {
"name": "Terry Durgan",
"date_of_birth": "2011-03-06",
"date_of_death": "2017-04-13"
},
"relationships": {
"photos": {
"data": [
{
"type": "photos",
"id": "5"
}
]
},
"books": {
"data": [
{
"type": "books",
"id": "6"
},
{
"type": "books",
"id": "16"
},
{
"type": "books",
"id": "50"
}
]
}
},
"links": {
"self": "/v2/authors/5"
}
}|
----------
|{
"type": "authors",
"id": "6",
"attributes": {
"name": "Annalise Walsh",
"date_of_birth": "2004-11-27",
"date_of_death": "1997-07-20"
},
"relationships": {
"photos": {
"data": [
{
"type": "photos",
"id": "6"
}
]
},
"books": {
"data": [
{
"type": "books",
"id": "4"
},
{
"type": "books",
"id": "5"
},
{
"type": "books",
"id": "21"
}
]
}
},
"links": {
"self": "/v2/authors/6"
}
}|
---------

json - query with conditions

I have following json:
{
"id": "1",
"name": "profile1",
"userId": "0",
"groupId": "3",
"attributes": [
{
"id": "104",
"name": "Enable",
"value": "1"
},
{
"id": "105",
"name": "TargetNode",
"value": "system1"
},
{
"id": "106",
"name": "Timeout",
"value": "30"
}
],
"xconns": [
{
"id": "1",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "1",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "1",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
}
]
}
{
"id": "2",
"name": "profile2",
"userId": "7",
"groupId": "0",
"attributes": [
{
"id": "104",
"name": "Enable",
"value": "1"
},
{
"id": "105",
"name": "TargetNode",
"value": "system2"
},
{
"id": "106",
"name": "Timeout",
"value": "30"
}
],
"xconns": [
{
"id": "2",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "2",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
},
{
"id": "2",
"locked": false,
"attributeList": [
{
"id": "101",
"name": "Lgrp",
"value": "1"
},
{
"id": "102",
"name": "IsRem",
"value": "1"
},
{
"id": "103",
"name": "Media",
"value": "1"
}
]
}
]
}
I can filter following:
$ jq -r 'select([.attributes[] | .name == "TargetNode" ] | any ) | [{userId, groupId, id, name}] | .[] | if (.userId == "0") then del(.userId) else . end | if (.groupId == "0") then del(.groupId) else . end | to_entries | map("\(.key | ascii_upcase):\(.value)") | #tsv' file.json
GROUPID:3 ID:1 NAME:profile1
USERID:7 ID:2 NAME:profile2
I need to add also value of TargetNode:
GROUPID:3 ID:1 NAME:profile1 TARGETNODE:system1
USERID:7 ID:2 NAME:profile2 TARGETNODE:system2
is there a way to include it in
[{userId, groupId, id, name, TargetNode}]
to get the value of TargetNode and not null?
GROUPID:3 ID:1 NAME:profile1 TARGETNODE:null
USERID:7 ID:2 NAME:profile2 TARGETNODE:null
Update:
the solution provided by RomanPerekhrest is nearly ok, but there is one issue because the json file in real is much bigger, there are more attrobutes in "main secttion", for example:
{
"id": "1",
"name": "profile1",
"userId": "0",
"groupId": "3",
"attrib101": "A",
"attrib102": "B",
"attributes": [
...
...
it is cousing that RomanPerekhrest's jq filter returns too much...
how to rid of them too?
ID:1 NAME:profile1 GROUPID:3 ATTRIB101:A ATTRIB102:B TARGETNODE:system1
ID:2 NAME:profile2 USERID:7 ATTRIB101:C ATTRIB102:D TARGETNODE:system2

jq solution:
jq -r '.attributes |= map(select(.name == "TargetNode"))
| if (.attributes | length != 0) then .targetNode = .attributes[0].value else . end
| if (.userId == "0") then del(.userId) else . end
| if (.groupId == "0") then del(.groupId) else . end
| del(.attributes, .xconns) | to_entries
| map("\(.key | ascii_upcase):\(.value)") | #tsv' file.json
If an object with "name": "TargetNode" pair not exists - TARGETNODE won't be added into resulting structure
The output:
ID:1 NAME:profile1 GROUPID:3 TARGETNODE:system1
ID:2 NAME:profile2 USERID:7 TARGETNODE:system2

Convert json to include name and children values to feed D3

I am trying to get json converted from:
{
"Devices": [
{
"Udid": "7a2b0e6c928f2321a75e423ba23ae93d",
"SerialNumber": "RF1D232ZLEE",
"MacAddress": "40F232726FC8",
"Imei": "3576342323280150",
"EasId": "SEC1BC252327E92B",
"AssetNumber": "7a2b0e23223928f2321a75e423ba23ae93d",
"DeviceFriendlyName": "gel1 Android Android 5.0.1 ZLEE ",
"LocationGroupId": {
"Id": {
"Value": 19529
},
"Name": "Group Express"
},
"LocationGroupName": "Group Express",
"UserId": {
"Name": ""
},
"UserName": "",
"UserEmailAddress": "",
"Ownership": "S",
"PlatformId": {
"Id": {
"Value": 5
},
"Name": "Android"
},
"Platform": "Android",
"ModelId": {
"Id": {
"Value": 5
},
"Name": "samsung GT-I9505"
},
"Model": "samsung GT-I9505",
"OperatingSystem": "5.0.1",
"PhoneNumber": "+447881867010",
"LastSeen": "2016-07-06T14:01:03.590",
"EnrollmentStatus": "Unenrolled",
"ComplianceStatus": "NotAvailable",
"CompromisedStatus": false,
"LastEnrolledOn": "2016-06-15T16:01:38.763",
"LastComplianceCheckOn": "0001-01-01T00:00:00.000",
"LastCompromisedCheckOn": "2016-07-06T13:58:26.183",
"IsSupervised": false,
"DeviceMCC": {
"SIMMCC": "234",
"CurrentMCC": "234"
},
"AcLineStatus": 0,
"VirtualMemory": 0,
"Id": {
"Value": 23459
}
},
{
"Udid": "c5f94db71d406dae7f881d3edf059e",
"SerialNumber": "",
"MacAddress": "000C300F9108",
"Imei": "",
"EasId": "D80DB85EC411C8E9B28BC292A603F05C2C0EEEC8",
"AssetNumber": "c592f93db71d406dae7f881d3edf059e",
"DeviceFriendlyName": "user Windows 10 WinRT 10.0.10240 ",
"LocationGroupId": {
"Id": {
"Value": 18498
},
"Name": "Business Solutions"
},
"LocationGroupName": "Business Solutions",
"UserId": {
"Name": ""
},
"UserName": "",
"UserEmailAddress": "",
"Ownership": "C",
"PlatformId": {
"Id": {
"Value": 12
},
"Name": "WinRT"
},
"Platform": "WinRT",
"ModelId": {
"Id": {
"Value": 50
},
"Name": "Windows 10"
},
"Model": "Windows 10",
"OperatingSystem": "10.0.10240",
"PhoneNumber": "",
"LastSeen": "2016-05-03T10:54:07.650",
"EnrollmentStatus": "Unenrolled",
"ComplianceStatus": "NotAvailable",
"CompromisedStatus": false,
"LastEnrolledOn": "2016-01-29T16:41:57.760",
"LastComplianceCheckOn": "0001-01-01T00:00:00.000",
"LastCompromisedCheckOn": "0001-01-01T00:00:00.000",
"IsSupervised": false,
"DeviceMCC": {
"SIMMCC": "",
"CurrentMCC": ""
},
"AcLineStatus": 0,
"VirtualMemory": 0,
"Id": {
"Value": 23545
}
}
],
"Page": 0,
"PageSize": 500,
"Total": 13}
To something like:
{"name": "Devices",
"children": [
{"name":"Udid", "size":"7f0dsda63274692ea4f0b66fec67a020158"},
{"name":"SerialNumber", "size":"P988KJSPQF938"},
{"name":"MacAddress", "size":"1HJUSUD031C4"},
{"name":"Imei", "size":""},
{"name":"EasId", "size":"ApKJSPQF193"},
{"name":"AssetNumber", "size":"7f0cda636b3305fea4f0b66fec9997267a020158"},
{"name":"DeviceFriendlyName", "size":"TMcKenz iPad iOS 7.1.4 F193 "},
{"name":"LocationGroupId",
"children": [
{"name":"Id","size":7488},
{"name":"Name","size":"MCM"}
]
},
{"name":"UserId",
"children": [
{"name":"Id","size":6418},
{"name":"Name","size":"Tom McKenz"}
]
},
{"name":"UserName", "size":"TMcKenz"},
{"name":"UserEmailAddress", "size":"TMcKenz#awaw.com"}
]
}
Not sure what is the best practice here: is it possible to use D3.nest or do you need to iterate through all the nodes and change to 'name' and 'children' accordingly.