I have a Json object
{
"ProjectDirectory": "C:\\Main",
"SiteName": "RemoteOrder",
"ParentPath": "/Areas//Views",
"VirtualDirectories": [
{
"Name": "Alerts",
"Path": "\\Areas\\RemoteOrder\\Views\\Alerts"
},
{
"Name": "Analytics",
"Path": "\\Areas\\RemoteOrder\\Views\\Analytics"
},
{
"Name": "Auth",
"Path": "\\Areas\\RemoteOrder\\Views\\Auth"
}
]
}
that I created by
$config = [Newtonsoft.Json.Linq.JObject]::Parse($file)
I can access things like
$config["ProjectDirectory"]
$config["VirtualDirectories"]
But I can not get to the element inside the VirtualDirectories JArray
I confirmed
$config["VirtualDirectories"][0].GetType() // JObject
$config["VirtualDirectories"].GetType() // JArray
$config // JObject
I have tried
$config["VirtualDirectories"][0]["Name"]
$config["VirtualDirectories"][0]["Path"]
$config["VirtualDirectories"][0][0]
$config["VirtualDirectories"][0].GetValue("Name")
When I try
$config["VirtualDirectories"][0].ToString()
I get
{
"Name": "Alerts",
"Path": "\\Areas\\RemoteOrder\\Views\\Alerts"
}
What I am really trying to do is access it a loop but again I can not seem to access the JObject Elements
You are close. $config["VirtualDirectories"][0]["Name"] will give you a JValue containing the text. You just need to use the Value property from there to get the actual string. Here is how you would do it in a ForEach loop:
$config = [Newtonsoft.Json.Linq.JObject]::Parse($file)
ForEach ($dir in $config["VirtualDirectories"])
{
$name = $dir["Name"].Value
$path = $dir["Path"].Value
...
}
To complement Brian Rogers' helpful answer:
As a more convenient alternative to index syntax (["<name>"]) you can use property syntax
(.<name>), because the JObject instances returned have dynamic properties named for their keys:
$config = [Newtonsoft.Json.Linq.JObject]::Parse($file)
foreach ($dir in $config.VirtualDirectories) {
$name = $dir.Name.Value # as in Brian's answer: note the need for .Value
$path = $dir.Path.Value # ditto
# Sample output
"$name=$path" # outputs 'Alerts=\Areas\RemoteOrder\Views\Alerts', ...
}
I presume that the reason you chose to work with Json.NET types directly is performance compared to PowerShell's built-in ConvertFrom-Json cmdlet.
As an aside: There is a PowerShell wrapper for Json.NET that you can install with Install-Module -Scope CurrentUser newtonsoft.json , for instance, which implicitly gives you access to the [Newtonsoft.Json.Linq.JObject] type. However, this wrapper - which represents objects as ordered hashtables - is even slower than ConvertFrom-Json.
Aside from performance, the following limitations of ConvertFrom-Json may make it necessary to use a third-party library such as Json.Net anyway:
Empty-string keys are not supported.
Keys that differ in case only (e.g., foo vs. Foo) are not supported.
For contrast, here's the equivalent - but generally slower - ConvertFrom-Json solution:
ConvertFrom-Json represents the JSON objects as [pscustomobject] instances whose properties are named for the keys, allowing for a more natural syntax without the need to access .Value:
$config = ConvertFrom-Json $json
foreach ($dir in $config.VirtualDirectories) {
$name = $dir.Name # no .Value needed
$path = $dir.Path # ditto
# Sample output
"$name=$path" # outputs 'Alerts=\Areas\RemoteOrder\Views\Alerts', ...
}
Related
Say I have JSON like:
{
"a" : {
"b" : 1,
"c" : 2
}
}
Now ConvertTo-Json will happily create PSObjects out of that. I want to access an item I could do $json.a.b and get 1 - nicely nested properties.
Now if I have the string "a.b" the question is how to use that string to access the same item in that structure? Seems like there should be some special syntax I'm missing like & for dynamic function calls because otherwise you have to interpret the string yourself using Get-Member repeatedly I expect.
No, there is no special syntax, but there is a simple workaround, using iex, the built-in alias[1] for the Invoke-Expression cmdlet:
$propertyPath = 'a.b'
# Note the ` (backtick) before $json, to prevent premature expansion.
iex "`$json.$propertyPath" # Same as: $json.a.b
# You can use the same approach for *setting* a property value:
$newValue = 'foo'
iex "`$json.$propertyPath = `$newValue" # Same as: $json.a.b = $newValue
Caveat: Do this only if you fully control or implicitly trust the value of $propertyPath.
Only in rare situation is Invoke-Expression truly needed, and it should generally be avoided, because it can be a security risk.
Note that if the target property contains an instance of a specific collection type and you want to preserve it as-is (which is not common) (e.g., if the property value is a strongly typed array such as [int[]], or an instance of a list type such as [System.Collections.Generic.List`1]), use the following:
# "," constructs an aux., transient array that is enumerated by
# Invoke-Expression and therefore returns the original property value as-is.
iex ", `$json.$propertyPath"
Without the , technique, Invoke-Expression enumerates the elements of a collection-valued property and you'll end up with a regular PowerShell array, which is of type [object[]] - typically, however, this distinction won't matter.
Note: If you were to send the result of the , technique directly through the pipeline, a collection-valued property value would be sent as a single object instead of getting enumerated, as usual. (By contrast, if you save the result in a variable first and the send it through the pipeline, the usual enumeration occurs). While you can force enumeration simply by enclosing the Invoke-Expression call in (...), there is no reason to use the , technique to begin with in this case, given that enumeration invariably entails loss of the information about the type of the collection whose elements are being enumerated.
Read on for packaged solutions.
Note:
The following packaged solutions originally used Invoke-Expression combined with sanitizing the specified property paths in order to prevent inadvertent/malicious injection of commands. However, the solutions now use a different approach, namely splitting the property path into individual property names and iteratively drilling down into the object, as shown in Gyula Kokas's helpful answer. This not only obviates the need for sanitizing, but turns out to be faster than use of Invoke-Expression (the latter is still worth considering for one-off use).
The no-frills, get-only, always-enumerate version of this technique would be the following function:
# Sample call: propByPath $json 'a.b'
function propByPath { param($obj, $propPath) foreach ($prop in $propPath.Split('.')) { $obj = $obj.$prop }; $obj }
What the more elaborate solutions below offer: parameter validation, the ability to also set a property value by path, and - in the case of the propByPath function - the option to prevent enumeration of property values that are collections (see next point).
The propByPath function offers a -NoEnumerate switch to optionally request preserving a property value's specific collection type.
By contrast, this feature is omitted from the .PropByPath() method, because there is no syntactically convenient way to request it (methods only support positional arguments). A possible solution is to create a second method, say .PropByPathNoEnumerate(), that applies the , technique discussed above.
Helper function propByPath:
function propByPath {
param(
[Parameter(Mandatory)] $Object,
[Parameter(Mandatory)] [string] $PropertyPath,
$Value, # optional value to SET
[switch] $NoEnumerate # only applies to GET
)
Set-StrictMode -Version 1
# Note: Iteratively drilling down into the object turns out to be *faster*
# than using Invoke-Expression; it also obviates the need to sanitize
# the property-path string.
$props = $PropertyPath.Split('.') # Split the path into an array of property names.
if ($PSBoundParameters.ContainsKey('Value')) { # SET
$parentObject = $Object
if ($props.Count -gt 1) {
foreach ($prop in $props[0..($props.Count-2)]) { $parentObject = $parentObject.$prop }
}
$parentObject.($props[-1]) = $Value
}
else { # GET
$value = $Object
foreach ($prop in $props) { $value = $value.$prop }
if ($NoEnumerate) {
, $value
} else {
$value
}
}
}
Instead of the Invoke-Expression call you would then use:
# GET
propByPath $obj $propertyPath
# GET, with preservation of the property value's specific collection type.
propByPath $obj $propertyPath -NoEnumerate
# SET
propByPath $obj $propertyPath 'new value'
You could even use PowerShell's ETS (extended type system) to attach a .PropByPath() method to all [pscustomobject] instances (PSv3+ syntax; in PSv2 you'd have to create a *.types.ps1xml file and load it with Update-TypeData -PrependPath):
'System.Management.Automation.PSCustomObject',
'Deserialized.System.Management.Automation.PSCustomObject' |
Update-TypeData -TypeName { $_ } `
-MemberType ScriptMethod -MemberName PropByPath -Value { #`
param(
[Parameter(Mandatory)] [string] $PropertyPath,
$Value
)
Set-StrictMode -Version 1
$props = $PropertyPath.Split('.') # Split the path into an array of property names.
if ($PSBoundParameters.ContainsKey('Value')) { # SET
$parentObject = $this
if ($props.Count -gt 1) {
foreach ($prop in $props[0..($props.Count-2)]) { $parentObject = $parentObject.$prop }
}
$parentObject.($props[-1]) = $Value
}
else { # GET
# Note: Iteratively drilling down into the object turns out to be *faster*
# than using Invoke-Expression; it also obviates the need to sanitize
# the property-path string.
$value = $this
foreach ($prop in $PropertyPath.Split('.')) { $value = $value.$prop }
$value
}
}
You could then call $obj.PropByPath('a.b') or $obj.PropByPath('a.b', 'new value')
Note: Type Deserialized.System.Management.Automation.PSCustomObject is targeted in addition to System.Management.Automation.PSCustomObject in order to also cover deserialized custom objects, which are returned in a number of scenarios, such as using Import-CliXml, receiving output from background jobs, and using remoting.
.PropByPath() will be available on any [pscustomobject] instance in the remainder of the session (even on instances created prior to the Update-TypeData call [2]); place the Update-TypeData call in your $PROFILE (profile file) to make the method available by default.
[1] Note: While it is generally advisable to limit aliases to interactive use and use full cmdlet names in scripts, use of iex to me is acceptable, because it is a built-in alias and enables a concise solution.
[2] Verify with (all on one line) $co = New-Object PSCustomObject; Update-TypeData -TypeName System.Management.Automation.PSCustomObject -MemberType ScriptMethod -MemberName GetFoo -Value { 'foo' }; $co.GetFoo(), which outputs foo even though $co was created before Update-TypeData was called.
This workaround is maybe useful to somebody.
The result goes always deeper, until it hits the right object.
$json=(Get-Content ./json.json | ConvertFrom-Json)
$result=$json
$search="a.c"
$search.split(".")|% {$result=$result.($_) }
$result
You can have 2 variables.
$json = '{
"a" : {
"b" : 1,
"c" : 2
}
}' | convertfrom-json
$a,$b = 'a','b'
$json.$a.$b
1
Prolog
It turns out that in my case it is important to understand the source of the objects - it is a JSON payload from a REST API response. Unfortunately, JSON -> Object conversion produces different results on PS desktop vs PS core. On desktop the numbers are deserialized into Int32 types, but on core - to Int64 types. From that it follows that I cannot use Export-CliXml, because the binary layout of the objects is different.
Main question
I have a unit test that needs to compare the actual result with an expected. The expected result is saved in a json file, so the procedure is:
Convert the actual result to json string
Read the expected result from disk to string
Compare the actual and the expected as strings
Unfortunately, this scheme does not work because PS desktop ConvertTo-Json and PS core ConvertTo-Json do not produce identical results. So, if the expected result was saved on desktop and the test runs on core - boom, failure. And vice versa.
One way is to keep two versions of jsons. Another way is to use a library to create the json.
First I tried the Newtonsoft-Json powershell module, but it just does not work. I think the problem is that whatever C# library we use, it must be aware of PSCustomObject and alike and treat them specially. So, we cannot just take any C# JSON library.
At this point I am left with having two jsons - one per PS edition, which is kind of sad.
Are there better options?
EDIT 1
I guess I can always read the json, convert to object and then back to json again. That sucks.
EDIT 2
I tried to use ConvertTo-Json -Compress. This eliminates the difference in spacing, but the problem is that for some reason the desktop version translates all the non characters to \u000... representation. The core version does not do it.
Please, observe:
Desktop
C:\> #{ x = "'a'" } |ConvertTo-Json -Compress
{"x":"\u0027a\u0027"}
C:\>
Core
C:\> #{ x = "'a'" } |ConvertTo-Json -Compress
{"x":"'a'"}
C:\>
Now the core version has the flag -EscapeHandling, so:
C:\> #{ x = "'a'" } |ConvertTo-Json -Compress -EscapeHandling EscapeHtml
{"x":"\u0027a\u0027"}
C:\>
Bingo! Same result. But now this code does not run on the desktop version, which does not have this flag. More massaging is needed. I will check if that is the only problem.
EDIT 3
It is impossible to reconcile the differences between the core and the desktop versions without expensive post processing. Please, observe:
Desktop
C:\> #{ x = '"a"';y = "'b'" } |ConvertTo-Json -Compress
{"y":"\u0027b\u0027","x":"\"a\""}
C:\>
Core
C:\> #{ x = '"a"';y = "'b'" } |ConvertTo-Json -Compress -EscapeHandling EscapeHtml
{"y":"\u0027b\u0027","x":"\u0022a\u0022"}
C:\> #{ x = '"a"';y = "'b'" } |ConvertTo-Json -Compress
{"y":"'b'","x":"\"a\""}
C:\>
Any suggestions on how to salvage the json approach?
EDIT 4
The Export-CliXml approach does not work too, because of the differences between the PS versions.
Desktop
C:\> ('{a:1}' | ConvertFrom-Json).a.gettype()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Int32 System.ValueType
C:\>
Core
C:\> ('{a:1}' | ConvertFrom-Json).a.gettype()
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Int64 System.ValueType
C:\>
So the same JSON is represented using different numeric types - Int32 in desktop and Int64 in core. That puts to bed the option of using Export-CliXml.
Unless I am missing something.
I believe there is no other choice, but do the double conversion - json -> object -> json and then I will have two jsons created on the same PS edition. That sucks big time.
On converting from the original JSON, use the third-party Newtonsoft.Json PowerShell wrapper's ConvertFrom-JsonNewtonsoft cmdlet - this should ensure cross-edition compatibility (which the built-in ConvertFrom-Json does not guarantee across PowerShell editions, because Windows PowerShell uses a custom parser, whereas PowerShell [Core] v6+ uses Newtonsoft.json up to at least v7.1, though a move to the new(ish) .NET Core System.Text.Json API is coming).
Important: ConvertFrom-JsonNewtonsoft returns (arrays of) nested ordered hashtables ([ordered] #{ ... }, System.Collections.Specialized.OrderedDictionary), unlike the nested [pscustomobject] graphs that the built-in ConvertFrom-Json outputs. Similarly, ConvertTo-JsonNewtonsoft expects only (arrays of) hashtables (dictionaries) as input, and notably does not support [pscustomobject] instances, as you've learned yourself.
Caveat: As of this writing, the wrapper module was last updated in May 2019, and the version of the underlying bundled Newtonsoft.Json.dll assembly is quite old (8.0, whereas 12.0 is current as of this writing). See the module's source code.
Note that in order to parse JSON obtained from a RESTful web service manually, you mustn't use Invoke-RestMethod, as it implicitly parses and returns [pscustomobject] object graphs. Instead, use Invoke-WebRequest and access the returned response's .Content property.
On converting to a format suitable for storing on disk, you have two options:
(A) If you do need the serialized format to be JSON also, you must convert all [pscustomobject] graphs to (ordered) hashtables before passing them to ConvertTo-JsonNewtonsoft.
See below for function ConvertTo-OrderedHashTable, which does just that.
(B) If the specific serialization format isn't important, i.e. if all that matters is that the formats are identical across PowerShell editions for the purpose of comparison, no extra works is needed: use the built-in Export-Clixml cmdlet, which can handle any type and produces PowerShell's native, XML-based serialization format called CLIXML (as notably used in PowerShell remoting), which should be cross-edition-compatible (at least with v5.1 on the Windows PowerShell side and as of PowerShell [Core] v7.1, both of which use the same version of the serialization protocol, 1.1.0.1, as reported by $PSVersionTable.SerializationVersion).
While you could re-convert such a persisted file to objects with Import-Clixml, the potential loss of type fidelity on deserialization makes comparing the serialized (CLIXML) representations advisable.
Also note that as of PowerShell v7.1 there is no cmdlet-based way to create an in-memory CLIXML representation, so you'll have to use the PowerShell API directly for now: System.Management.Automation.PSSerializer.Serialize. However, providing in-memory counterparts to Import-CliXml / Export-CliXml in the form of ConvertFrom-CliXml / ConvertTo-CliXml cmdlets has been green-lighted as a future enhancement.
Re (A): Here's function ConvertTo-OrderedHashtable, which converts (potentially nested) [pscustomobject] objects to ordered hashtables while passing other types through, so you should be able to simply insert it into a pipeline as follows:
# CAVEAT: ConvertTo-JsonNewtonSoft only accepts a *single* input object.
[pscustomobject] #{ foo = 1 }, [pscustomobject] #{ foo = 2 } |
ConvertTo-OrderedHashtable |
ForEach-Object { ConvertTo-JsonNewtonSoft $_ }
function ConvertTo-OrderedHashtable {
<#
.SYNOPSIS
Converts custom objects to ordered hashtables.
.DESCRIPTION
Converts PowerShell custom objects (instances of [pscustomobject]) to
ordered hashtables (instances of [System.Collections.Specialized.OrderedDictionary]),
which is useful for to-JSON serialization via the Newtonsoft.JSON library.
Note:
* Custom objects are processed recursively.
* Any scalar non-custom objects are passed through as-is.
* Any (non-dictionary) collections in property values are converted to
[object[]] arrays.
.EXAMPLE
1, [pscustomobject] #{ foo = [pscustomobject] #{ bar = 'none' }; other = 2 } | ConvertTo-OrderedHashtable
Passes integer 1 through, and converts the custom object to a nested ordered
hashtable.
#>
[CmdletBinding()]
param(
[Parameter(ValueFromPipeline)] $InputObject
)
begin {
# Recursive helper function
function convert($obj) {
if ($obj -is [System.Management.Automation.PSCustomObject]) {
# a custom object: recurse on its properties
$oht = [ordered] #{ }
foreach ($prop in $obj.psobject.Properties) {
$oht.Add($prop.Name, (convert $prop.Value))
}
return $oht
}
elseif ($obj -isnot [string] -and $obj -is [System.Collections.IEnumerable] -and $obj -isnot [System.Collections.IDictionary]) {
# A collection of sorts (other than a string or dictionary (hash table)), recurse on its elements.
return #(foreach ($el in $obj) { convert $el })
}
else {
# a non-custom object, including .NET primitives and strings: use as-is.
return $obj
}
}
}
process {
convert $InputObject
}
}
Re (B): A demonstration of the Export-CliXml approach (you can run this code from either PS edition):
$sb = {
Install-Module -Scope CurrentUser Newtonsoft.json
if (-not $IsCoreClr) {
# Workaround for PS Core's $env:PSModulePath overriding WinPS'
Import-Module $HOME\Documents\WindowsPowerShell\Modules\newtonsoft.json
}
#'
{
"results": {
"users": [
{
"userId": 1,
"emailAddress": "jane.doe#example.com",
"date": "2020-10-05T08:08:43.743741-04:00",
"attributes": {
"height": 165,
"weight": 60
}
},
{
"userId": 2,
"emailAddress": "john.doe#example.com",
"date": "2020-10-06T08:08:43.743741-04:00",
"attributes": {
"height": 180,
"weight": 72
}
}
]
}
}
'# | ConvertFrom-JsonNewtonsoft | Export-CliXml "temp-$($PSVersionTable.PSEdition).xml"
}
# Execute the script block in both editions
Write-Verbose -vb 'Running in Windows PowerShell...'
powershell -noprofile $sb
Write-Verbose -vb 'Running in PowerShell Core...'
pwsh -noprofile $sb
# Compare the resulting CLIXML files.
Write-Verbose -vb "Comparing the resulting files: This should produce NO output,`n indicating that the files have identical content."
Compare-Object (Get-Content 'temp-Core.xml') (Get-Content 'temp-Desktop.xml')
Write-Verbose -vb 'Cleaning up...'
Remove-Item 'temp-Core.xml', 'temp-Desktop.xml'
You should see the following verbose output:
VERBOSE: Running in Windows PowerShell...
VERBOSE: Running in PowerShell Core...
VERBOSE: Comparing the resulting files: This should produce NO output,
indicating that the files have identical content.
VERBOSE: Cleaning up...
I am not able to twist my head into understanding how to get Powershell to loop the entire JSON Structure, it wont' loop the System.Object[]
$x = ConvertFrom-Json '{
"Car companies": {
"Name of Company": "Ford",
"Cars": [{
"Name of car": "Ranger",
"Config": "Pickup"
},
{
"Name of car": "Puma",
"Config": "Hatchback"
}]
}
}'
foreach( $rootProperty in #($x.psobject.properties | where-object {$_.MemberType -eq "NoteProperty"}) ) {
write-host " - '$($rootProperty.Name)' = '$($rootProperty.Value)'"
foreach( $childProperty in #($rootProperty.Value.psobject.properties ) ) {
write-host "'$($childProperty.Name)' = '$($childProperty.Value)'"
}
}
Outut I get now is just
- 'Brand' = '#{Name of Brand=Ford; Cars=System.Object[]}'
Name of Brand' = 'Ford'
Cars' = ' '
...as a follop How to iterate through a unknown JSON data/object?
tl;dr
You're seeing a bug that unexpectedly string-expands the Cars property value's array elements to the empty string.
A simple workaround - for display purposes only - is to pipe the property value to Out-String to get the usual display representation:
"'$($childProperty.Name)' = '$($childProperty.Value | Out-String)'"
You're seeing a bug in how arrays of [pscustomobject] instances are stringified (as of PowerShell Core 7.0.0-preview.6):
Generally, PowerShell arrays are stringified by joining the stringified element representations with the separator specified in the $OFS preference variable, which defaults to a space char.
Normally, [pscustomobject] instances have a string representation that resembles a hashtable literal (but isn't one); e.g.:
PS> $custObj = [pscustomobject] #{ foo = 'bar' }; "$custObj"
#{foo=bar} # string representation that *resembles* a hashtable literal
Unexpectedly - and this is the bug - when custom objects are the elements of an array, they stringify to the empty string, which is what you saw:
PS> $custObj = [pscustomobject] #{ foo = 'bar' }; $arr = $custObj, $custObj; "[$arr]"
[ ] # !! Bug: custom objects stringified to empty strings, joined with a space
This is an indirect manifestation of a long-standing bug reported in this GitHub issue: that is, elements of an array being stringified are stringified by calls to their .ToString() method, and calling .ToString() on custom objects unexpectedly yields the empty string (unlike the string representation you get when you directly reference a single custom object in an expandable string, as shown above).
Currently, I'm attempting to call upon an API to run a POST, with JSON data as the body. So I was wondering if anyone would be able to tell me how I need to format the text below inside the variable $postParams. I'm pretty new at working with JSON so I'm having so trouble with this.
Currently, I only have the following and don't know what to do about the second line on.
$postParams = #{name='Example'}
Here's is the entire data I was hoping to add to $postParams. So if you could help me with the 2nd, 4th, and 8th that'd be awesome. Thanks!
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
You'll need a here-string and ConvertFrom-Json.
here-string:
Quotation marks are also used to create a here-string. A here-string is a single-quoted or double-quoted string in which quotation marks are interpreted literally. A here-string can span multiple lines. All the lines in a here-string are interpreted as strings, even though they are not enclosed in quotation marks.
The resulting code:
# Use a PowerShell here string to take JSON as it is
$jsonString = #"
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
"#
# Pipe the string to create a new JSON object
$jsonObject = $jsonString | ConvertFrom-Json
# The resulting JSON object has properties matching the properties in the orig. JSON
$jsonObject.name
$jsonObject.url
# Nested property
$jsonObject.template.name
# Nested property in array
$jsonObject.groups[0].name
I've posted an online version of the above code at tio.run, so you can play around with it.
If you want to update several properties of the $jsonObject you can do the following:
$jsonObject.name = "NEW NAME"
$jsonObject.url = "NEW URL"
$jsonObject | ConvertTo-Json
ConvertTo-Json will take your object and create an appropriate JSON string:
{
"name": "NEW NAME",
"template": {
"name": "Template"
},
"url": "NEW URL",
"page": {
"name": "Landing Page"
},
"smtp": {
"name": "Sending Profile"
},
"launch_date": "2019-10-08T17:20:00+00:00",
"send_by_date": null,
"groups": [
{
"name": "test group"
}
]
}
If you $jsonObject has more than two levels of depth, use the -Depth parameter, otherwise not all object information will be included in the JSON string.
ConvertTo-Json:
-Depth
Specifies how many levels of contained objects are included in the JSON representation. The default value is 2.
Here is a tio.run link to a ConvertTo-Json example.
Hope that helps.
I can't test it currently, but try this.
$postParams = #'
{
"name":"Example",
"template":{"name":"Template"},
"url":"http://localhost",
"page":{"name":"Landing Page"},
"smtp":{"name":"Sending Profile"},
"launch_date":"2019-10-08T17:20:00+00:00",
"send_by_date":null,
"groups":[{"name":"test group"}]
}
'#
Make a hashtable, then convert to JSON:
$Hashtable = #{
Key1 = "Value1"
Key2 = "Value2"
}
$Json = $Hashtable | ConvertTo-Json
Apologies for my lack of powershell knowledge, have been searching far and wide for a solution as i am not much of a programmer.
Background:
I am currently trying to standardise some site settings in Incapsula. To do this i want to maintain a local XML with rules and use some powershell to pull down the existing rules and compare them with what is there to ensure im not doubling up. I am taking this approach of trying to only apply the deltas as:
For most settings incapsula is not smart enough to know it already exists
What can be posted to the API is different varies from what is returned by the API
Examples:
Below is an example of what the API will return on request, this is in a JSON format.
JSON FROM WEBSITE
{
"security": {
"waf": {
"rules": [{
"id": "api.threats.sql_injection",
"exceptions": [{
"values": [{
"urls": [{
"value": "google.com/thisurl",
"pattern": "EQUALS"
}],
"id": "api.rule_exception_type.url",
"name": "URL"
}],
"id": 256354634
}]
}, {
"id": "api.threats.cross_site_scripting",
"action": "api.threats.action.block_request",
"exceptions": [{
"values": [{
"urls": [{
"value": "google.com/anotherurl",
"pattern": "EQUALS"
}],
"id": "api.rule_exception_type.url",
"name": "URL"
}],
"id": 78908790780
}]
}]
}
}
}
And this is the format of the XML with our specific site settings in it
OUR XML RULES
<waf>
<ruleset>
<rule>
<id>api.threats.sql_injection</id>
<exceptions>
<exception>
<type>api.rule_exception_type.url</type>
<url>google.com/thisurl</url>
</exception>
<exception>
<type>api.rule_exception_type.url</type>
<url>google.com/thisanotherurl</url>
</exception>
</exceptions>
</rule>
<rule>
<id>api.threats.cross_site_scripting</id>
<exceptions>
<exception>
<type>api.rule_exception_type.url</type>
<url>google.com/anotherurl</url>
</exception>
<exception>
<type>api.rule_exception_type.url</type>
<url>google.com/anotherurl2</url>
</exception>
</exceptions>
</rule>
</ruleset>
</waf>
I have successfully been able to compare other settings from the site against the XML using the compare-object command, however they had a bit simpler nesting and didn't give me as much trouble. I'm stuck to whether it is a logic problem or a limitation with compare object. An example code is below, it will require the supplied json and xml saved as stack.json/xml in the same directory and should produce the mentioned result :
$existingWaf = Get-Content -Path stack.json | ConvertFrom-Json
[xml]$xmlFile = Get-Content -Path stack.xml
foreach ($rule in $xmlFile)
{
$ruleSet = $rule.waf.ruleset
}
foreach ($siteRule in $ExistingWaf.security.waf.rules)
{
foreach ($xmlRule in $ruleSet)
{
if ($xmlRule.rule.id -eq $siteRule.id)
{
write-output "yes"
$delta = Compare-Object -ReferenceObject #($siteRule.exceptions.values.urls.value | Select-Object) -DifferenceObject #($xmlRule.rule.exceptions.exception.url | Select-Object) -IncludeEqual | where {$xmlRule.rule.id -eq $siteRule.id}
$delta
}
}
}
This is kind of working but not quite what i wanted. I do get a compare between the objects but not for the specific id's, it shows me the results below:
InputObject SideIndicator
----------- -------------
google.com/thisurl ==
google.com/thisanotherurl =>
google.com/anotherurl =>
google.com/anotherurl2 =>
google.com/anotherurl ==
google.com/thisurl =>
google.com/thisanotherurl =>
google.com/anotherurl2 =>
Where as i am more after
InputObject SideIndicator
----------- -------------
google.com/thisurl ==
google.com/thisanotherurl =>
google.com/anotherurl ==
google.com/anotherurl2 =>
Hopefully that makes sense.
Is it possible to only do the compares only on the values where the ids match?
Please let me know if you have any further questions.
Thanks.
The problem was your iteration logic, which mistakenly processed multiple rules from the XML document in a single iteration:
foreach ($xmlRule in $ruleSet) didn't enumerate anything - instead it processed the single <ruleset> element; to enumerate the child <rule> elements, you must use $ruleSet.rule.
$xmlRule.rule.exceptions.exception.url then implicitly iterated over all <rule> children and therefore reported the URLs across all of them, which explains the extra lines in your Compare-Object output.
Here's a streamlined, annotated version of your code:
$existingWaf = Get-Content -LiteralPath stack.json | ConvertFrom-Json
$xmlFile = [xml] (Get-Content -raw -LiteralPath stack.xml )
# No need for a loop; $xmlFile is a single [System.Xml.XmlDocument] instance.
$ruleSet = $xmlFile.waf.ruleset
foreach ($siteRule in $ExistingWaf.security.waf.rules)
{
# !! Note the addition of `.rule`, which ensures that the rules
# !! are enumerated *one by one*.
foreach ($xmlRule in $ruleSet.rule)
{
if ($xmlRule.id -eq $siteRule.id)
{
# !! Note: `$xmlRule` is now a single, rule, therefore:
# `$xmlRule.rule.[...]-> `$xmlRule.[...]`
# Also note that neither #(...) nor Select-Object are needed, and
# the `| where ...` (Where-Object) is not needed.
Compare-Object -ReferenceObject $siteRule.exceptions.values.urls.value `
-DifferenceObject $xmlRule.exceptions.exception.url -IncludeEqual
}
}
}
Additional observations regarding your code:
There is no need to ensure that operands passed to Compare-Object are arrays, so there's no need to wrap them in array sub-expression operator #(...). Compare-Object handles scalar operands fine.
... | Select-Object is a virtual no-op - the input object is passed through[1]
... | Where-Object {$xmlRule.rule.id -eq $siteRule.id} is pointless, because it duplicates the enclosing foreach loop's condition.
Generally speaking, because you're not referencing the pipeline input object at hand via automatic variable $_, your Where-Object filter is static and will either match all input objects (as in your case) or none.
[1] There is a subtle, invisible side effect that typically won't make a difference: Select-Object adds an invisible [psobject] wrapper around the input object, which on rare occasions does cause different behavior later -
see this GitHub issue.