I've recently been working on a project that needs me to compare two JSON files and return the differences between both in PowerShell. I recently started to learn the language, and I've come across multiple issues.
I've tried checking all over the internet and forums from what I could understand of my issue, but nothing really worked.
I tried using Compare-Object but it returned me the whole JSON as a difference, transforming them into PsObjects, it kinda worked but I couldn't get the full path and broke at some point, etc...
My final attempt was to iterate over each property of my object and when I would encounter an object, I would simply "open" the object / "redirect" myself in it and so on... (basically, iterating in object & sub-objects).
It's almost working, thus I'm not sure why I've got an issue. Here's the code sample :
(note that $obj is one single PowerShell Object coming from a foreach loop (foreach $obj in $Obj) and $template is another PS object)
function Compare-Templates($obj, $template, $current_path)
{
ForEach ($prop in $obj.PsObject.Properties) {
Write-Host "$($prop.Name):$($prop.value)" -ForegroundColor Magenta
Write-Host "$($current_path)$($prop.Name)" -ForegroundColor DarkBlue
Write-Host ""
if ($ObjectTypesName -contains $prop.TypeNameOfValue) {
Write-Host "$($prop.Value) is a PsObject!" -ForegroundColor DarkRed
Write-Host ""
Write-Host "-------- Nested in $($prop.Name) --------" -ForegroundColor Yellow
Compare-Templates $prop.value $template ($current_path + "$($prop.Name)/")
Write-Host "-------- Out Nested --------" -ForegroundColor Yellow
Write-Host ""
}
}
}
Output:
-------- Nested in oneObject --------
firstValue:1
oneObject/firstvalue
It would give me the value of the variables found and its path (I need it later on).
The issue :
In my JSON file, I can have an array inside a variable such as :
"Object": [ <-------- the array that gives the weird `SyncRoot` variables over and over
{
"anotherObject": {
"firstValue": false,
"secondValue": false
},
[...]
The program understands that the array can be accessed (it even gives me the content in "preview", e.g. object:#(anotherObject=))
In my debug, it would say :
-------- Nested in OBJ--------
Count:1
OBJ/Count
Length:1
OBJ/Length
LongLength:1
OBJ/LongLength
Rank:1
OBJ/Rank
SyncRoot:#{object1=; object2=; object3=System.Object[]; object4=; object5=; object6=; object7=; object8=System.Object[]; object9=System.Object[]}
OBJ/SyncRoot
(Note that for security reasons I changed all variable names, but there is still a cohesion between them)
Everything found in the object is something which doesn't exist in my object, and I'm not sure where it's from (probably the array's "default" variable).
My question is : How could I access the array which is nested in my object, without looping indefinitely over the SyncRoot object?
Or should I use this hand-made method or is there an actual function / simple way to iterate over a PowerShell object, nest with other PowerShell objects OR comparing JSONs that would give me the full "path" of a variable with different values?
Note that it can iterate indefinitely such as OBJ_which_has_an_array_like_the_example_above/SyncRoot/SyncRoot/SyncRoot/SyncRoot/SyncRoot/SyncRoot/Length and so on...
Related
I have the JSON block below that I am trying to parse with PowerShell. What the JSON is for is to denote permissions that an AD group should be given on an Azure App Registration. An application will have multiple groups assigned to it and each group will have a unique set of roles (permissions). The full JSON file has several of these application blocks.
Basically, what I want to be able to do with PowerShell is to take the application, query Azure AD to get the application resource and then modify the resource to assign each group to the app with its roles using New-AzureADGroupAppRoleAssignment.
Conventional thinking would be the I need a loop nested inside of a loop nested inside of a loop where the first loop grabs the application block, then the second loop grabs the groupname block and then the third loop grabs the array of roles for that group.
"application":
[
{
"groupname": "adgroup1",
"roles": [
"Permission1",
"Permission2",
"Permission3",
"Permission4",
"Permission5",
"Permission6"
]
},
{
"groupname": "adgroup2",
"roles": [
"Permission1",
"Permission2",
"Permission4",
"Permission5",
"Permission6"
]
}
Something like this should get you started
$json = SomethingThatGetsJson | ConvertFrom-Json
$json.application | % {
$data = $_
Write-Host $data.groupname
$data.roles | % {
$role = $_
Write-Host $role
}
}
% is shorthand for ForEach-Object if you want to get some more details on that. $_ is the element value in the current iteration of the enumeration.
So long as the JSON returned is valid you should use the ConvertFrom-Json cmdlet, It will convert the JSON string in to a proper object you can dot walk through and run your other standard cmdlets against like select, where, foreach etc
For some reason, my program is refusing to work in this scenario:
I have a custom function that is meant to replace a cmdlet due to lack of flexibility
I am passing this function a file or folder object, through the pipeline
This is the function:
function Get-ChildItemCustom {
Param(
[Parameter(ValueFromPipeline=$true)]
[System.IO.FileSystemInfo] $item,
[Parameter(ValueFromPipeline=$false)]
[string] $archive_path
)
Process {
Get-ChildItem $item
}
}
I want to be able to use this function just like Get-ChildItem: input a [System.IO.FileSystemInfo] object, and get all the children (sorted with some extra criteria that I didn't include here) as output.
This is how I call the function:
Get-ChildItem $parentfolder_path |
Get-ChildItemCustom |
Do-SomethingElse
The error returns explains that the results of Get-ChildItem (which are verifiably of type [System.IO.FileSystemInfo]) are being treated as strings.
Cannot convert the "E:\Data\VHG-ITC-Test\New folder\archive" value of type "System.String" to type "System.IO.FileSystemInfo".
The type preceding the parameter was not always there. When $item did not explicitly have a type, the function would misread the input (supposedly only taking the Name property as input):
Get-ChildItem : Cannot find path 'C:\Windows\system32\New folder' because it does not exist.
So the function does not seem to be able to accept the object input properly. I want to avoid using strings at all costs, and just move objects around. Have I setup the parameters wrong? What can I do?
The problem isn't exactly with your function, but with the way Get-ChildItem handles arguments (as Moerwald already suspected in his answer).
When you call Get-ChildItem with a FileInfo object as an unnamed argument that argument is passed to the first positional parameter (-Path), which expects a string array as input, so the object is cast to a string. However, in some situations casting FileInfo objects to a string expands the FullName property, while in others it expands just the Name property (I can't explain how PowerShell decides when to pick which, though). The latter is what's happening in your case. And since Get-ChildItem sees just a name, not a full path, it's looking for the item in the current working directory, which fails.
There are a number of ways to avoid this issue, one of which Moerwald has already shown. Others are:
using a pipeline for passing $item to Get-ChildItem:
function Get-ChildItemCustom {
Param(
[Parameter(ValueFromPipeline=$true)]
[IO.FileSystemInfo]$item,
[Parameter(ValueFromPipeline=$false)]
[string]$archive_path
)
Process {
$item | Get-ChildItem
}
}
passing the full path by mapping the property by name:
function Get-ChildItemCustom {
Param(
[Parameter(
ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true
)]
[string[]]$FullName,
[Parameter(ValueFromPipeline=$false)]
[string]$archive_path
)
Process {
Get-ChildItem $FullName
}
}
Personally, I'd prefer the last variant (passing by property name).
First off, I am bound to PowerShell v2 because that is what is installed by default on Windows 7. What I want to do works out of the box with PowerShell v4, and, possibly, PowerShell v3.
I want to read a JSON object from a file and use it as an object in my script. PowerShell v2 does not have the ConvertFrom-Json method, so I wrote my own implementation that I want to have the same functionality:
function ConvertFrom-Json
{
param(
[Parameter(ValueFromPipeline=$true)]
[string]$json
)
[System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
$ser = New-Object System.Web.Script.Serialization.JavaScriptSerializer
$obj = $ser.DeserializeObject($json)
Write-Host $obj.GetType()
return $obj
}
I call this function like this:
$configObj = (Get-Content $configFile) -join "`n" | ConvertFrom-Json
Write-Host $configObj.GetType()
The output of this:
System.Collections.Generic.Dictionary`2[System.String,System.Object]
System.Object[]
And I am not able to interact with $configObj like an object and get it's properties. In fact, I can't figure out how to get any information out of it. When I print the object, the output looks like:
AC Version Location
--- ------- --------
True v2.0.50727 C:\Windows\assembly\GAC_MSIL\System.Web.Extensions\3.5.0.0__31bf3856ad364e35\System.Web.Extensions.dll
Key : DownloadURL
Value : https://example.com
Key : dir
Value :
When I expect it to look like:
Key Value
--- -----
DownloadURL https://example.com
dir
Is there a way to work around this and get the same object back from my ConvertFrom-Json method as exists inside my method?
Thanks in advance,
Andy
Result of every statement, with exception for assignment and increment/decrement, considered part of your function return. You does not have to use return statement to return something from function. So your function actually return two objects: loaded Assembly object from LoadWithPartialName method and dictionary from $obj variable. When you assign multiple function results to variable, PowerShell have to pack results into array. As you does not actually need Assembly object, you can cast result of LoadWithPartialName method to [void]:
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Web.Extensions")
Also LoadWithPartialName method is Obsolete and you should not use it at all. For loading assembly, you can use Add-Type build-in cmdlet:
Add-Type -AssemblyName System.Web.Extensions
As you does not use named blocks in your function, all code considered to be an end block, thus you only process last pipeline input in your function. You should use process block to process every pipeline input object:
process{
$ser.DeserializeObject($json)
}
When statement return collection, than PowerShell enumerate this collection and write each individual collection's item instead of collection as single element. So if your JSON contains array at top level, you function will return array elements instead of array itself, as build-in ConvertFrom-Json does. You can use unary array operator to prevent enumeration of returned array:
,$ser.DeserializeObject($json)
The total:
function ConvertFrom-Json {
param(
[Parameter(ValueFromPipeline=$true)]
[string]$json
)
begin {
Add-Type -AssemblyName System.Web.Extensions
$ser = New-Object System.Web.Script.Serialization.JavaScriptSerializer
}
process {
,$ser.DeserializeObject($json)
}
}
I am using PowerShell v3 and the Windows PowerShell ISE. I have the following function that works fine:
function Get-XmlNode([xml]$XmlDocument, [string]$NodePath, [string]$NamespaceURI = "", [string]$NodeSeparatorCharacter = '.')
{
# If a Namespace URI was not given, use the Xml document's default namespace.
if ([string]::IsNullOrEmpty($NamespaceURI)) { $NamespaceURI = $XmlDocument.DocumentElement.NamespaceURI }
# In order for SelectSingleNode() to actually work, we need to use the fully qualified node path along with an Xml Namespace Manager, so set them up.
[System.Xml.XmlNamespaceManager]$xmlNsManager = New-Object System.Xml.XmlNamespaceManager($XmlDocument.NameTable)
$xmlNsManager.AddNamespace("ns", $NamespaceURI)
[string]$fullyQualifiedNodePath = Get-FullyQualifiedXmlNodePath -NodePath $NodePath -NodeSeparatorCharacter $NodeSeparatorCharacter
# Try and get the node, then return it. Returns $null if the node was not found.
$node = $XmlDocument.SelectSingleNode($fullyQualifiedNodePath, $xmlNsManager)
return $node
}
Now, I will be creating a few similar functions, so I want to break the first 3 lines out into a new function so that I don't have to copy-paste them everywhere, so I have done this:
function Get-XmlNamespaceManager([xml]$XmlDocument, [string]$NamespaceURI = "")
{
# If a Namespace URI was not given, use the Xml document's default namespace.
if ([string]::IsNullOrEmpty($NamespaceURI)) { $NamespaceURI = $XmlDocument.DocumentElement.NamespaceURI }
# In order for SelectSingleNode() to actually work, we need to use the fully qualified node path along with an Xml Namespace Manager, so set them up.
[System.Xml.XmlNamespaceManager]$xmlNsManager = New-Object System.Xml.XmlNamespaceManager($XmlDocument.NameTable)
$xmlNsManager.AddNamespace("ns", $NamespaceURI)
return $xmlNsManager
}
function Get-XmlNode([xml]$XmlDocument, [string]$NodePath, [string]$NamespaceURI = "", [string]$NodeSeparatorCharacter = '.')
{
[System.Xml.XmlNamespaceManager]$xmlNsManager = Get-XmlNamespaceManager -XmlDocument $XmlDocument -NamespaceURI $NamespaceURI
[string]$fullyQualifiedNodePath = Get-FullyQualifiedXmlNodePath -NodePath $NodePath -NodeSeparatorCharacter $NodeSeparatorCharacter
# Try and get the node, then return it. Returns $null if the node was not found.
$node = $XmlDocument.SelectSingleNode($fullyQualifiedNodePath, $xmlNsManager)
return $node
}
The problem is that when "return $xmlNsManager" executes the following error is thrown:
Cannot convert the "System.Object[]" value of type "System.Object[]" to type "System.Xml.XmlNamespaceManager".
So even though I have explicitly cast my $xmlNsManager variables to be of type System.Xml.XmlNamespaceManager, when it gets returned from the Get-XmlNamespaceManager function PowerShell is converting it to an Object array.
If I don't explicitly cast the value returned from the Get-XmlNamespaceManager function to System.Xml.XmlNamespaceManager, then the following error is thrown from the .SelectSingleNode() function because the wrong data type is being passed into the function's 2nd parameter.
Cannot find an overload for "SelectSingleNode" and the argument count: "2".
So for some reason PowerShell is not maintaining the data type of the return variable. I would really like to get this working from a function so that I don't have to copy-paste those 3 lines all over the place. Any suggestions are appreciated. Thanks.
What's happening is PowerShell is converting your namespace manager object to a string array.
I think it has to do with PowerShell's nature of "unrolling" collections when sending objects down the pipeline. I think PowerShell will do this for any type implementing IEnumerable (has a GetEnumerator method).
As a work around you can use the comma trick to prevent this behavior and send the object as a whole collection.
function Get-XmlNamespaceManager([xml]$XmlDocument, [string]$NamespaceURI = "")
{
...
$xmlNsManager.AddNamespace("ns", $NamespaceURI)
return ,$xmlNsManager
}
More specifically, what is happening here is that your coding habit of strongly typing $fullyQualifiedModePath is trying to turn the result of the Get (which is a list of objects) into a string.
[string]$foo
will constrain the variable $foo to only be a string, no matter what came back. In this case, your type constraint is what is subtly screwing up the return and making it Object[]
Also, looking at your code, I would personally recommend you use Select-Xml (built into V2 and later), rather than do a lot of hand-coded XML unrolling. You can do namespace queries in Select-Xml with -Namespace #{x="..."}.
I'm writing a Powershell script that will extract a set of data files from a ZIP file and will then attach them to a server. I've written a function that takes care of the unzip and since I need to grab all of the files so that I know what I'm attaching I return that from the function:
function Unzip-Files
{
param([string]$zip_path, [string]$zip_filename, [string]$target_path, [string]$filename_pattern)
# Append a \ if the path doesn't already end with one
if (!$zip_path.EndsWith("\")) {$zip_path = $zip_path + "\"}
if (!$target_path.EndsWith("\")) {$target_path = $target_path + "\"}
# We'll need a string collection to return the files that were extracted
$extracted_file_names = New-Object System.Collections.Specialized.StringCollection
# We'll need a Shell Application for some file movement
$shell_application = New-Object -com shell.Application
# Get a handle for the target folder
$target_folder = $shell_application.NameSpace($target_path)
$zip_full_path = $zip_path + $zip_filename
if (Test-Path($zip_full_path))
{
$target_folder = $shell_application.NameSpace($target_path)
$zip_folder = $shell_application.NameSpace($zip_full_path)
foreach ($zipped_file in $zip_folder.Items() | Where {$_.Name -like $filename_pattern})
{
$extracted_file_names.Add($zipped_file.Name) | Out-Null
$target_folder.CopyHere($zipped_file, 16)
}
}
$extracted_file_names
}
I then call another function to actually attach the database (I've removed some code that checks for existence of the database, but that shouldn't affect things here):
function Attach-Database
{
param([object]$server, [string]$database_name, [object]$datafile_names)
$database = $server.Databases[$database_name]
$server.AttachDatabase($database_name, $datafile_names)
$database = $server.Databases[$database_name]
Return $database
}
I keep getting an error though, "Cannot convert argument "1", with value: "System.Object[]", for "AttachDatabase" to type "System.Collections.Specialized.StringCollection"".
I've tried declaring the data types explicitly at various points, but that just changes the location where I get the error (or one similar to it). I've also changed the parameter declaration to use the string collection instead of object with no luck.
I'm starting with a string collection and ultimately want to consume a string collection. I just don't seem to be able to get Powershell to stop trying to convert it to a generic Object at some point.
Any suggestions?
Thanks!
It looks like you should return the names using the comma operator:
...
, $extracted_file_names
}
to avoid "unrolling" the collection to its items and to preserve the original collection object.
There were several questions like this, here is just a couple:
Strange behavior in PowerShell function returning DataSet/DataTable
Loading a serialized DataTable in PowerShell - Gives back array of DataRows not a DataTable
UPDATE:
This similar code works:
Add-Type #'
using System;
using System.Collections.Specialized;
public static class TestClass
{
public static string TestMethod(StringCollection data)
{
string result = "";
foreach (string s in data)
result += s;
return result;
}
}
'#
function Unzip-Files
{
$extracted_file_names = New-Object System.Collections.Specialized.StringCollection
foreach ($zipped_file in 'xxx', 'yyy', 'zzz')
{
$extracted_file_names.Add($zipped_file) | Out-Null
}
, $extracted_file_names
}
function Attach-Database
{
param([object]$datafile_names)
# write the result
Write-Host ([TestClass]::TestMethod($datafile_names))
}
# get the collection
$names = Unzip-Files
# write its type
Write-Host $names.GetType()
# pass it in the function
Attach-Database $names
As expected, its output is:
System.Collections.Specialized.StringCollection
xxxyyyzzz
If I remove the suggested comma, then we get:
System.Object[]
Cannot convert argument "0", with value: "System.Object[]",
for "TestMethod" to type "System.Collections.Specialized.StringCollection"...
The symptoms look the same, so the solution presumably should work, too, if there are no other unwanted conversions/unrolling in the omitted code between Unzip-Files and Attach-Database calls.