Folks,
I do run in PowerShell some ReST api call and do get back a
$response
page content
---- -------
#{size=20; number=0; totalPages=1; totalElements=1} {#{id=ZGR2ZS0yLnZsYWIubG9j…
When I convert to JSON
PS C:\Users\Administrator\Documents> $response | ConvertTo-Json
{
"page": {
"size": 20,
"number": 0,
"totalPages": 1,
"totalElements": 1
},
"content": [
{
"id": "ZGR2ZS0yLnZsYWIubG9jYWw6MzAwOTpob3N0",
"host": "ddve-2.vlab.local",
"port": "3009",
"notValidBefore": "Fri Mar 29 21:32:19 PDT 2019",
"notValidAfter": "Sat Mar 29 04:32:19 PDT 2025",
"fingerprint": "E1BB40B0284595297071177FE02BC9C76E85CD66",
"subjectName": "CN=ddve-2.vlab.local, O=Valued DataDomain customer, OU=Hos
t Certificate, ST=CA, C=US",
"issuerName": "CN=ddve-2.vlab.local, OU=Root CA, O=Valued Datadomain Custo
mer, L=Santa Clara, ST=CA, C=US",
"state": "ACCEPTED",
"type": "HOST"
}
]
}
an idividual value can be seen
PS C:\Users\Administrator\Documents> $response.content.state
ACCEPTED
But setting a new value fails
PS C:\Users\Administrator\Documents> $response.content.state = 123
InvalidOperation: The property 'state' cannot be found on this object. Verify th
at the property exists and can be set.
Because your content property is a (single-element) array (as evidenced by its value being enclosed in [ ... ] in the JSON representation), you must use an index to specify which element's .state property to set:
$response.content[0].state = 123
Note that an index is not required for getting the value (that is, $response.content.state works and returns "ACCEPTED", as you state), because PowerShell then applies member-access enumeration, which means that it enumerates the state property values of all elements of the array (which with a single-element array returns just the single element's value, and with a multiple-element array returns an array of values).
On setting a property value, member-access enumeration is by design not supported, although the error message could certainly be more helpful - see this answer.
Your object is a PSCustomObject and the value you're trying to change is a NoteProperty on that object, you can use Add-Member with the -Force switch to overwrite it.
$Response = #"
{
"page": {
"size": 20,
"number": 0,
"totalPages": 1,
"totalElements": 1
},
"content": [
{
"id": "ZGR2ZS0yLnZsYWIubG9jYWw6MzAwOTpob3N0",
"host": "ddve-2.vlab.local",
"port": "3009",
"notValidBefore": "Fri Mar 29 21:32:19 PDT 2019",
"notValidAfter": "Sat Mar 29 04:32:19 PDT 2025",
"fingerprint": "E1BB40B0284595297071177FE02BC9C76E85CD66",
"subjectName": "CN=ddve-2.vlab.local, O=Valued DataDomain customer, OU=Hos
t Certificate, ST=CA, C=US",
"issuerName": "CN=ddve-2.vlab.local, OU=Root CA, O=Valued Datadomain Custo
mer, L=Santa Clara, ST=CA, C=US",
"state": "ACCEPTED",
"type": "HOST"
}
]
}
"# | ConvertFrom-Json
$Response.Content.State
$Response.Content | Add-Member -MemberType NoteProperty -Name State -Value 123 -Force
$Response.Content.State
Output
ACCEPTED
123
Related
Folks,
I do run in PowerShell some ReST api call and do get back a
$response
page content
---- -------
#{size=20; number=0; totalPages=1; totalElements=1} {#{id=ZGR2ZS0yLnZsYWIubG9j…
When I convert to JSON
PS C:\Users\Administrator\Documents> $response | ConvertTo-Json
{
"page": {
"size": 20,
"number": 0,
"totalPages": 1,
"totalElements": 1
},
"content": [
{
"id": "ZGR2ZS0yLnZsYWIubG9jYWw6MzAwOTpob3N0",
"host": "ddve-2.vlab.local",
"port": "3009",
"notValidBefore": "Fri Mar 29 21:32:19 PDT 2019",
"notValidAfter": "Sat Mar 29 04:32:19 PDT 2025",
"fingerprint": "E1BB40B0284595297071177FE02BC9C76E85CD66",
"subjectName": "CN=ddve-2.vlab.local, O=Valued DataDomain customer, OU=Hos
t Certificate, ST=CA, C=US",
"issuerName": "CN=ddve-2.vlab.local, OU=Root CA, O=Valued Datadomain Custo
mer, L=Santa Clara, ST=CA, C=US",
"state": "ACCEPTED",
"type": "HOST"
}
]
}
an idividual value can be seen
PS C:\Users\Administrator\Documents> $response.content.state
ACCEPTED
But setting a new value fails
PS C:\Users\Administrator\Documents> $response.content.state = 123
InvalidOperation: The property 'state' cannot be found on this object. Verify th
at the property exists and can be set.
Because your content property is a (single-element) array (as evidenced by its value being enclosed in [ ... ] in the JSON representation), you must use an index to specify which element's .state property to set:
$response.content[0].state = 123
Note that an index is not required for getting the value (that is, $response.content.state works and returns "ACCEPTED", as you state), because PowerShell then applies member-access enumeration, which means that it enumerates the state property values of all elements of the array (which with a single-element array returns just the single element's value, and with a multiple-element array returns an array of values).
On setting a property value, member-access enumeration is by design not supported, although the error message could certainly be more helpful - see this answer.
Your object is a PSCustomObject and the value you're trying to change is a NoteProperty on that object, you can use Add-Member with the -Force switch to overwrite it.
$Response = #"
{
"page": {
"size": 20,
"number": 0,
"totalPages": 1,
"totalElements": 1
},
"content": [
{
"id": "ZGR2ZS0yLnZsYWIubG9jYWw6MzAwOTpob3N0",
"host": "ddve-2.vlab.local",
"port": "3009",
"notValidBefore": "Fri Mar 29 21:32:19 PDT 2019",
"notValidAfter": "Sat Mar 29 04:32:19 PDT 2025",
"fingerprint": "E1BB40B0284595297071177FE02BC9C76E85CD66",
"subjectName": "CN=ddve-2.vlab.local, O=Valued DataDomain customer, OU=Hos
t Certificate, ST=CA, C=US",
"issuerName": "CN=ddve-2.vlab.local, OU=Root CA, O=Valued Datadomain Custo
mer, L=Santa Clara, ST=CA, C=US",
"state": "ACCEPTED",
"type": "HOST"
}
]
}
"# | ConvertFrom-Json
$Response.Content.State
$Response.Content | Add-Member -MemberType NoteProperty -Name State -Value 123 -Force
$Response.Content.State
Output
ACCEPTED
123
My PowerShell ConvertFrom-Json for the below throws the "Invalid JSON primitive" error.
{
"ScriptEndTime": "19 April 2022 10:26:32",
"AbortReason": "Server Offline",
"Server": "Server3",
"ScriptStartTime": "19 April 2022 10:25:55",
"Notes": "",
"NumberOfReboots": 0,
"Abort": true,
"UserAccount": "Contoso\\ContosoAdmin"
} {
"OS": "Microsoft Windows Server 2012 R2 Standard",
"Abort": false,
"KB5011560": "Succeeded",
"Server": "Server4.contoso.co.uk",
"IPAddress": "3.3.3.3"
}
I even tried to validate the above using jsonlint.com and it shows Expecting 'EOF', '}', ',', ']', got '{' error
What am I missing? Thanks.
Update: 20/04/2022
Below is what was happening internally in my case:
$MyHt1 = #{}
$MyHt1.Server = "Server1"
$MyHt1.StartTime = '1/1/2021 09:30:30 AM'
$MyHt1."" = "Succeeded" #my script was setting empty key here which is the source of the issue
$MyHt2 = #{}
$MyHt2.Server = "Server2"
$MyHt2.StartTime = '2/2/2022 09:30:30 PM'
$MyHt2."KB54231" = "Pending"
$JsonOutputArray = #($MyHt1, $MyHt2 | Foreach-Object { ConvertTo-Json -InputObject $_ })
$JsonOutputArray | Out-String | ConvertFrom-Json #throws runtime error here due to empty key
Looks like PowerShell hash table allows an empty key. That was the root cause of this issue which later resulted in runtime error during json conversion. Interestingly, ConvertTo-Json allows empty hash table key but fails with ConvertFrom-Json
Those are two separate JSON files to add them in one JSON file you should ad them in one bracket {} and the other items are separated by ","
as the following :
{
name1:{
"ScriptEndTime": "19 April 2022 10:26:32",
"AbortReason": "Server Offline",
"Server": "Server3",
"ScriptStartTime": "19 April 2022 10:25:55",
"Notes": "",
"NumberOfReboots": 0,
"Abort": true,
"UserAccount": "Contoso\\ContosoAdmin"
},
name2:{
"OS": "Microsoft Windows Server 2012 R2 Standard",
"Abort": false,
"KB5011560": "Succeeded",
"Server": "Server4.contoso.co.uk",
"IPAddress": "3.3.3.3"
}
}
I have following json and want to remove keys if the "value" matches specific pattern. Name of the keys are not fixed or pre-determined.
{
"resources": [{
"tags": null,
"properties": {
"customerId": "1234-cbd9-42bc-9193-f6432a6ef0d4",
"provisioningState": "Succeeded",
"sku": {
"maxCapacityReservationLevel": 3000,
"lastSkuUpdate": "Fri, 19 Mar 2021 16:38:12 GMT"
},
"createdDate": "Fri, 19 Mar 2021 16:38:12 GMT",
"modifiedDate": "Fri, 19 Mar 2021 17:27:54 GMT",
"status": {
"events": [{
"count": 1,
"firstTimestamp": "2021-03-19T16:40:59Z",
"lastTimestamp": "2021-03-19T16:40:59Z",
"name": "Pulling",
"type": "Normal"
}]
}
}
}]
}
Expected output
After removal of the following keys as values matched timestamp format.
lastSkuUpdate
createdDate
modifiedDate
firstTimestamp
lastTimestamp
{
"resources": [{
"tags": null,
"properties": {
"customerId": "1234-cbd9-42bc-9193-f6432a6ef0d4",
"provisioningState": "Succeeded",
"sku": {
"maxCapacityReservationLevel": 3000,
},
"status": {
"events": [{
"count": 1,
"name": "Pulling",
"type": "Normal"
}]
}
}
}]
}
"(?x:
^
(?: \\d{4}-\\d{2}-\\d{2}T
| \\w{3},[ ][\\d ]\\d[ ]\\w{3}[ ]\\d{4}
)
)" as $date_pattern |
( .. | select(type == "object") ) |= del(.[
. as $o |
keys_unsorted[] |
select( $o[.] | type == "string" and test($date_pattern) )
])
jqplay
Adjust the pattern to your liking.
A straightforward way to delete key-value pairs from a JSON object is using with_entries, e.g. like so:
def matches:
type == "string"
and test("^[1-9]\\d{3}-\\d+-\\d|[1-9]\\d{3} \\d{2}:\\d{2}:\\d{2}");
walk(if type=="object"
then with_entries(if .value|matches then empty else . end)
else . end)
You may of course wish to adjust def matches: according to your requirements, or to shorten the filter-argument of with_entries to:
select(.value|matches|not)
Here's a superficially complicated but very efficient and actually quite simple solution. It uses jq's streaming parser and would thus be especially suitable for very large JSON input(s). Of course the matches filter as defined here should be taken as illustrative rather than definitive.
jq --stream -n '
def matches:
type == "string"
and test("^[1-9]\\d{3}-\\d+-\\d|[1-9]\\d{3} \\d{2}:\\d{2}:\\d{2}");
fromstream(inputs
| select((length==2
and (.[0][-1]|type)=="string"
and (.[-1]|matches))
| not) )
' input.json
I'm trying to parse the results of a cURL command and the information I need is in a structure.
I tried getting to the data unsuccessfully and tried converting to PS Object but not sure how to access the structure as I'm new to PS.
Below is a sample of our cURL response.
I have a git commit hash ('c64a568399a572e82c223d55cb650b87ea1c22b8' matches latestCommit in fromRef for entry id 1101) and I need to find the corresponding displayId ('develop' in toRef)
I've done this in Linux using jq but need to replicate this in PS.
jq '.values | map(select(.fromRef.latestCommit=="'"$HASH"'")) | .[0].toRef.displayId'
I'm having 2 issues.
I can get to fromRef but it looks like #{id=refs/heads/feature/add-support; displayId=feature/add-support; latestCommit=c64a568399a572e82c223d55cb650b87ea1c22b8; repository=} and I cannot figure out how to parse
I'm not sure how to get the id so I can find the correct corresponding toRef
Any help would be greatly appreciated.
{
"size": 15,
"limit": 20,
"isLastPage": true,
"values": [
{
"id": 1101,
"version": 0,
"title": "Added header",
"description": "Added notes in header",
"state": "OPEN",
"open": true,
"closed": false,
"createdDate": 1595161367863,
"updatedDate": 1595161367863,
"fromRef": "#{id=refs/heads/feature/add-support; displayId=feature/add-support; latestCommit=c64a568399a572e82c223d55cb650b87ea1c22b8; repository=}",
"toRef": "#{id=refs/heads/develop; displayId=develop; latestCommit=58b3e3482bb35f3a735048849c2474cc676fbd9b; repository=}",
"locked": false,
"author": "#{user=; role=AUTHOR; approved=False; status=UNAPPROVED}",
"reviewers": " ",
"participants": "",
"properties": "#{mergeResult=; resolvedTaskCount=0; openTaskCount=0}",
"links": "#{self=System.Object[]}"
},
{
"id": 1053,
"version": 4,
"title": "Help with checking,",
"description": "fixed up code.",
"state": "OPEN",
"open": true,
"closed": false,
"createdDate": 1591826401310,
"updatedDate": 1595018917357,
"fromRef": "#{id=refs/heads/bugfix/checking-2.7; displayId=bugfix/checking-2.7; latestCommit=cf7d8860262c6a46b0b65ef5b6d66ae8cd698b75; repository=}",
"toRef": "#{id=refs/heads/hotfix/2.7_Improvements; displayId=hotfix/2.7_Improvements; latestCommit=01f1100c559ba41ec317421399c3bfb9a0aea91f; repository=}",
"locked": false,
"author": "#{user=; role=AUTHOR; approved=False; status=UNAPPROVED}",
"reviewers": " ",
"participants": "",
"properties": "#{mergeResult=; resolvedTaskCount=0; commentCount=4; openTaskCount=0}",
"links": "#{self=System.Object[]}"
}
],
"start": 0
}
Once you have converted the result with ConvertTo-Json and the correct -Depth parameter, you can get the values of the returned object quite easily in PowerShell.
Let's say you have used something like $json = $curlResult | ConvertTo-Json -Depth 100, then finding the displayId from the corresponding toRef can be done like this:
# this is the known hashvalue of the `fromRef` value to look for
$latestCommitHash = "c64a568399a572e82c223d55cb650b87ea1c22b8"
# get the value item. from here you can get all other properties belonging to that item
$valueItem = $json.values | Where-Object { $_.fromRef.latestCommit -eq $latestCommitHash }
# get the displayId value of the corresponding 'toRef' element:
$displayId = $valueItem.toRef.displayId
Returns
develop
I would like to add new path into existing document
./jq < test.json
{
"correlationId": "6298865a73b477106c98d021",
"leg": 0,
"tag": "sent",
"offset": 322858,
"len": 178,
"prev": {
"page": {
"file": 10352,
"page": 2
},
"record": 911
},
"data": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied"
}
I am using filter setpath described in jq manual.
But even if I copied the documented string
./jq 'setpath([0,"a"]; 1)' < test.json
still getting error:
jq: error (at <stdin>:1): Cannot index object with number
I do not see there any syntax issue. Did I overlooked something?
Regards and thanks, Reddy
You cannot use integer indices as keys in the JSON object. For JSON objects, the key must be a string, so you could write:
jq 'setpath(["0","a"]; 1)' < test.json
Output:
{
"correlationId": "6298865a73b477106c98d021",
"leg": 0,
"tag": "sent",
"offset": 322858,
"len": 178,
"prev": {
"page": {
"file": 10352,
"page": 2
},
"record": 911
},
"data": "HTTP/1.1 403 Forbidden\r\nDate: Fri, 16 Feb 2018 08:37:54 GMT\r\nServer: \r\nConnection: close\r\nX-CorrelationID: Id-6298865a73b477106c98d021 0\r\nContent-Type: text/html\r\n\r\nAccess Denied",
"0": {
"a": 1
}
}