Is there a best practice for handling JSON documents with duplicated keys in PowerShell?
Ideally would like to compile the values of said keys into a single array mapped to the key.
For example:
{
"column01" : "id1",
"column02" : "id2",
"column03: : "id3",
"column03" : "id4"
}
Transformed to:
{
"column01" : "id1",
"column02" : "id2",
"column03: : [
"id3",
"id4"
]
}
I have been exploring options with the ConvertTo-Json cmdlet, but have not found a solution.
Appreciate the help!
While JSON does allow duplicate keys, this is not recommended and I suggest handling this by normalizing the JSON. Otherwise you can certainly pass your JSON with the duplicate key to ConvertFrom-Json but this won't result in your desired output.
It should be...
$obj = #"
{ "column01" : "id1",
"column02" : "id2",
"column03" : ["id3", "id4"]
}
"#
Then using $json = $obj | ConvertFrom-Json to convert into a powershell obj.
Then you can do the same with a PowerShell object and convert to JSON.
$obj = #{
"column01" = "id1";
"column02" = "id2";
"column03" = ("id3", "id4")
}
$json = $obj | ConvertTo-Json
$json
If you want to know how to normalize the data, I suggest you either edit your question or ask a new one.
Related
I have this JSON. Trying to retrieve tactics array by using below code in PowerShell.
Why the actual output is weird even though tactics contains comma separated values, it is considering as one value with space separated values.
How to get the expected output?
My JSON:
{
"displayName": "travel with mailbox permission",
"tactics": [
"InitialAccess",
"PrivilegeEscalation"
],
"techniques": [
"T1078",
"T1548"
]
}
My code:
param (
[Parameter(Mandatory = $true)]
# The resourceVariable which holds the resource details
[string] $jsonvariable
)
$json = Get-AutomationVariable -Name $jsonvariable
$jsonObject = ConvertFrom-Json -InputObject $json
echo $jsonObject.tactics
Output:
Expected o/p:
InitialAccess,PrivilegeEscalation
Actual O/p :
InitialAccess PrivilegeEscalation
Try using
$json.tactics -join ','
Powershell is outputting each value in the array separately, rather than as a comma-separated list.
I am new to Powershell and am having trouble doing something that I imagine may be pretty simple, but not having luck so far.
I have an array with two objects in it. It looks like this basically:
[
{
"name":"John",
"age":30,
...
...
},
{
"score":null,
"vehicle":"Camaro",
"engine":"V8",
...
...
}
]
My goal is to update the score value in the second object. I have had luck doing so when the key's value is already present as a String, but am not understanding why I am unable to get this to work when the key is present but the value is null (as shown above).
I have tried using a function which I learned about when doing a search for a previously posted question trying to do something similar:
Set Value of Nested Object Property by Name in PowerShell
The function from that question looks like this:
function SetValue($object, $key, $Value)
{
$p1,$p2 = $key.Split(".")
if($p2) { SetValue -object $object.$p1 -key $p2 -Value $Value }
else { $object.$p1 = $Value }
}
and can be called like this:
SetValue -object $Obj -key $Key -Value $Value
I am not sure why it matters if the value is NULL or not. It can find the key, but just not doing anything if its value is NULL.
My apologies if this already out there. Powershell is just a little different than anything I have worked with before! Any help is greatly appreciated! :)
The object generated by your JSON string is an array of two objects:
$json = #'
[
{
"name":"John",
"age":30,
},
{
"score":null,
"vehicle":"Camaro",
"engine":"V8"
},
]
'# | ConvertFrom-Json
$json[0] # => Element 0 of the Array
name age
---- ---
John 30
$json[1] # => Element 1 of the Array
score vehicle engine
----- ------- ------
Camaro V8
Updating the Value of the score property considering the JSON will always be the same, meaning, the element 1 of the array will always have that property would be as simple as:
$json[1].score = 'hello' # => Updating the property value
$json | ConvertTo-Json # => Converting the array back to Json
[
{
"name": "John",
"age": 30
},
{
"score": "hello",
"vehicle": "Camaro",
"engine": "V8"
}
]
On the other hand, if the arrangement of the objects wouldn't be always the same, you could loop over all elements of the array searching for that property and, once found, update it. For example:
# For each element of the array
foreach($object in $json) {
# if this object has a property with name `score`
# assign that `NoteProperty` to the variable `$prop`
if($prop = $object.PSObject.Properties.Item('score')) {
# update the Value of the `NoteProperty`
$prop.Value = 'hello'
}
}
$json | ConvertTo-Json # Convert it back
I am a newbie to Powershell and I couldn't find this on googling but how do I build a json structure with an array inside? Is it through a custom object? Have been a bit confused by the syntax that I have seen online and not sure what is the recommended way to do it. Eventually I need to be able to save it to a JSON file too.
json_ret = {
"a": 4,
"b": [ {"c" : 5, "d": "text", "e": "foo"}]
}
Yes you can build a json object through a PSCustomObject:
[PSCustomObject]#{
a = 4
b = #([ordered]#{
c = 5
d = "text"
e = "foo"
})
} | ConvertTo-Json
First we create PSObject using its type accelerator PSCustomObject.
Then we define the root key and value "a", and we have to create an array inside "b".
The #() statement creates an array, but we can't we create key-value pairs in array. So we use #{} to create hashtable. Before it [ordered] flag says the hashtable to keep the exact structure as we have created it.
Then we define the array values, and after that close the internal array-hashtable.
Now we end the PSCustomObject and pipe it ConvertTo-Json. Now you get a converted json.
Footnotes
If you want to dump the json to a file, then use this:
[PSCustomObject]#{
a = 4
b = #([ordered]#{
c = 5
d = "text"
e = "foo"
})
} | ConvertTo-Json | Out-File "Filepath"
If you want to save json to a variable:
$variable = ([PSCustomObject]#{
a = 4
b = #([ordered]#{
c = 5
d = "text"
e = "foo"
})
} | ConvertTo-Json)
If you want to create the JSON document directly, as a string, it's simplest to use a verbatim here-string:
$json_ret = #'
{
"a": 4,
"b": [ {"c" : 5, "d": "text", "e": "foo"}]
}
'#
You can easily save that to a file with $json_ret | Set-Content file.json, for instance.
By contrast, if you want to construct your data as an object graph first, to be converted to JSON with ConvertTo-Json later, see Wasif_Hasan's helpful answer.
As for what you tried:
An unquoted { ... } construct is a script block, which is a piece of PowerShell code for later invocation on demand - and the contents of your JSON document happen not to constitute valid PowerShell code, causing construction of the script block to fail.
If using variables, then can create body including array like shown below, where $text is the variable. No need to user ConvertTo-Json and can easily copy the body from postman directly.
$text = "ABC"
# Post Body
$body = #"
{
"name" = "$text",
"description" = "$text",
"myArray": [
{
"id": "2c91808680d3c34b0180dc81d78c21e9",
"type": "myType",
"name": "myName"
}
]
}
"#
$body
Currently, I'm attempting to call upon an API to run a POST, with JSON data as the body. So I was wondering if anyone would be able to tell me how I need to format the text below inside the variable $postParams. I'm pretty new at working with JSON so I'm having so trouble with this.
Currently, I only have the following and don't know what to do about the second line on.
$postParams = #{name='Example'}
Here's is the entire data I was hoping to add to $postParams. So if you could help me with the 2nd, 4th, and 8th that'd be awesome. Thanks!
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
You'll need a here-string and ConvertFrom-Json.
here-string:
Quotation marks are also used to create a here-string. A here-string is a single-quoted or double-quoted string in which quotation marks are interpreted literally. A here-string can span multiple lines. All the lines in a here-string are interpreted as strings, even though they are not enclosed in quotation marks.
The resulting code:
# Use a PowerShell here string to take JSON as it is
$jsonString = #"
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
"#
# Pipe the string to create a new JSON object
$jsonObject = $jsonString | ConvertFrom-Json
# The resulting JSON object has properties matching the properties in the orig. JSON
$jsonObject.name
$jsonObject.url
# Nested property
$jsonObject.template.name
# Nested property in array
$jsonObject.groups[0].name
I've posted an online version of the above code at tio.run, so you can play around with it.
If you want to update several properties of the $jsonObject you can do the following:
$jsonObject.name = "NEW NAME"
$jsonObject.url = "NEW URL"
$jsonObject | ConvertTo-Json
ConvertTo-Json will take your object and create an appropriate JSON string:
{
"name": "NEW NAME",
"template": {
"name": "Template"
},
"url": "NEW URL",
"page": {
"name": "Landing Page"
},
"smtp": {
"name": "Sending Profile"
},
"launch_date": "2019-10-08T17:20:00+00:00",
"send_by_date": null,
"groups": [
{
"name": "test group"
}
]
}
If you $jsonObject has more than two levels of depth, use the -Depth parameter, otherwise not all object information will be included in the JSON string.
ConvertTo-Json:
-Depth
Specifies how many levels of contained objects are included in the JSON representation. The default value is 2.
Here is a tio.run link to a ConvertTo-Json example.
Hope that helps.
I can't test it currently, but try this.
$postParams = #'
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
'#
Make a hashtable, then convert to JSON:
$Hashtable = #{
Key1 = "Value1"
Key2 = "Value2"
}
$Json = $Hashtable | ConvertTo-Json
The following JSON is getting returned from OData API service:
{
"d": {
"results": [
{
"FileSystemObjectType": 0,
"Id": 1,
"ContentTypeId": "0x0100BC97B2F575CB0C42B79549F3BABD32A8",
"Title": "Nokia California",
"Address": "200 South Matilda Avenue\nW Washington Ave\n94086 Sunnyvale, California\nUnited States of America",
"ID": 1,
"Modified": "2014-02-24T10:06:39Z",
"Created": "2014-02-24T10:06:39Z",
"AuthorId": 12,
"EditorId": 12,
"OData__UIVersionString": "1.0",
"Attachments": false,
"GUID": "d12aafad-502a-4968-a69e-36a7ea05ec80"
}
]
}
}
and saved as a string into variable named $data
An attempt to convert a JSON-formatted string to a custom object using ConvertFrom-Json cmdlet:
$results = $data | ConvertFrom-Json
gives the following error:
ConvertFrom-Json : Cannot convert the JSON string because a dictionary
that was converted from the string contains the duplicated keys 'Id'
and 'ID'.
Is there any way to convert the specified JSON-formatted string in PowerShell?
This is how I have done with it:
$results = $data.ToString().Replace("ID", "_ID") | ConvertFrom-Json
Note, both examples assume the JSON is stored in the $jsonstring variable.
In PowerShell Core, ConvertFrom-Json -AsHashtable is the easiest alternative:
$json = $jsonstring | ConvertFrom-Json -AsHashtable
$json['d']['results']
Name Value
---- -----
Modified 2/24/2014 10:06:39 AM
Title Nokia California
Attachments False
ID 1
ContentTypeId 0x0100BC97B2F575CB0C42B79549F3BABD32A8
GUID d12aafad-502a-4968-a69e-36a7ea05ec80
Created 2/24/2014 10:06:39 AM
EditorId 12
AuthorId 12
Address 200 South Matilda Avenue…
Id 1
OData__UIVersionString 1.0
FileSystemObjectType 0
In Windows PowerShell, you can use the Deserialize(String, Type) method from the JavaScriptSerializer Class.
Add-Type -AssemblyName System.Web.Extensions
$serializer = [Web.Script.Serialization.JavaScriptSerializer]::new()
$json = $serializer.Deserialize($jsonstring, [hashtable])
$json['d']['results']
In PowerShell V1.0, or in PowerShell V2.0 when the JSON is too big, I still use a convertion to XML :
Add-Type -AssemblyName System.ServiceModel.Web, System.Runtime.Serialization
function Convert-JsonToXml
{
PARAM([Parameter(ValueFromPipeline=$true)][string[]]$json)
BEGIN
{
$mStream = New-Object System.IO.MemoryStream
}
PROCESS
{
$json | Write-String -stream $mStream
}
END
{
$mStream.Position = 0
try
{
$jsonReader = [System.Runtime.Serialization.Json.JsonReaderWriterFactory]::CreateJsonReader($mStream,[System.Xml.XmlDictionaryReaderQuotas]::Max)
$xml = New-Object Xml.XmlDocument
$xml.Load($jsonReader)
$xml
}
finally
{
$jsonReader.Close()
$mStream.Dispose()
}
}
}
Using this code you can loop thru your items you can test :
$a = Get-Content C:\temp\jsontest.txt
$b.root.d.results.Item
$b.root.d.results.Item[7].Id[0].InnerText
(Edited)
In you case I would only replace the expected duplicate ID/Id
$data = Get-Content C:\temp\jsontest.txt -Raw
$datacorrected = $a -creplace '"Id":','"Id-minus":'
$psJsonIn = $datacorrected | ConvertFrom-Json
If really you've got unexpected duplicate you can write a function that trap the convertion error due to duplicated key and replace one.
ConvertFrom-JSON it going to try to create a PS Custom Object from the JSON string. PowerShell object property names are case-insensitive, so "ID" and "id" represent the same property name. You're going to have to do something with those duplicate property names in your JSON before you try to do that conversion, or it's going to fail.
I used the ToLower() before converting the json to object, resolved my issue.
$sdf = $data.ToLower() | ConvertFrom-Json