powershell if statement for null condition [duplicate] - json

I have a piece of code that works but I want to know if there is a better way to do it. I could not find anything related so far. Here are the facts:
I have an object with n properties.
I want to convert this object to JSON using (ConvertTo-Json).
I don't want to include in the JSON those object properties that are not valued.
Building the object (not really important):
$object = New-Object PSObject
Add-Member -InputObject $object -MemberType NoteProperty -Name TableName -Value "MyTable"
Add-Member -InputObject $object -MemberType NoteProperty -Name Description -Value "Lorem ipsum dolor.."
Add-Member -InputObject $object -MemberType NoteProperty -Name AppArea -Value "UserMgmt"
Add-Member -InputObject $object -MemberType NoteProperty -Name InitialVersionCode -Value ""
The line that I need improvements (to filter out the non-valued properties and not include them in the JSON)
# So I want to 'keep' and deliver to the JSON only the properties that are valued (first 3).
$object | select -Property TableName, Description, AppArea, InitialVersion | ConvertTo-Json
What this line delivers:
Results:
{
"TableName": "MyTable",
"Description": "Lorem ipsum dolor..",
"AppArea": "UserMgmt",
"InitialVersion": null
}
What I want to obtain:
{
"TableName": "MyTable",
"Description": "Lorem ipsum dolor..",
"AppArea": "UserMgmt"
}
What I've tried and works, but I don't like it since I have much more properties to handle:
$JSON = New-Object PSObject
if ($object.TableName){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name TableName -Value $object.TableName
}
if ($object.Description){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name Description -Value $object.Description
}
if ($object.AppArea){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name AppArea -Value $object.AppArea
}
if ($object.InitialVersionCode){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name InitialVersionCode -Value $object.InitialVersionCode
}
$JSON | ConvertTo-Json

Something like this?
$object = New-Object PSObject
Add-Member -InputObject $object -MemberType NoteProperty -Name TableName -Value "MyTable"
Add-Member -InputObject $object -MemberType NoteProperty -Name Description -Value "Lorem ipsum dolor.."
Add-Member -InputObject $object -MemberType NoteProperty -Name AppArea -Value "UserMgmt"
Add-Member -InputObject $object -MemberType NoteProperty -Name InitialVersionCode -Value ""
# Iterate over objects
$object | ForEach-Object {
# Get array of names of object properties that can be cast to boolean TRUE
# PSObject.Properties - https://msdn.microsoft.com/en-us/library/system.management.automation.psobject.properties.aspx
$NonEmptyProperties = $_.psobject.Properties | Where-Object {$_.Value} | Select-Object -ExpandProperty Name
# Convert object to JSON with only non-empty properties
$_ | Select-Object -Property $NonEmptyProperties | ConvertTo-Json
}
Result:
{
"TableName": "MyTable",
"Description": "Lorem ipsum dolor..",
"AppArea": "UserMgmt"
}

I have the following function in my profile for this purpose. Advantage: I can pipe a collection of objects to it and remove nulls from all the objects on the pipeline.
Function Remove-Null {
[cmdletbinding()]
param(
# Object to remove null values from
[parameter(ValueFromPipeline,Mandatory)]
[object[]]$InputObject,
#By default, remove empty strings (""), specify -LeaveEmptyStrings to leave them.
[switch]$LeaveEmptyStrings
)
process {
foreach ($obj in $InputObject) {
$AllProperties = $obj.psobject.properties.Name
$NonNulls = $AllProperties |
where-object {$null -ne $obj.$PSItem} |
where-object {$LeaveEmptyStrings.IsPresent -or -not [string]::IsNullOrEmpty($obj.$PSItem)}
$obj | Select-Object -Property $NonNulls
}
}
}
Some examples of usage:
$AnObject = [pscustomobject]#{
prop1="data"
prop2="moredata"
prop5=3
propblnk=""
propnll=$null
}
$AnObject | Remove-Null
prop1 prop2 prop5
----- ----- -----
data moredata 3
$ObjList =#(
[PSCustomObject]#{
notnull = "data"
more = "sure!"
done = $null
another = ""
},
[PSCustomObject]#{
notnull = "data"
more = $null
done = $false
another = $true
}
)
$objList | Remove-Null | fl #format-list because the default table is misleading
notnull : data
more : sure!
notnull : data
done : False
another : True

beatcracker's helpful answer offers an effective solution; let me complement it with a streamlined version that takes advantage of PSv4+ features:
# Sample input object
$object = [pscustomobject] #{
TableName = 'MyTable'
Description = 'Lorem ipsum dolor...'
AppArea = 'UserMgmt'
InitialVersionCode = $null
}
# Start with the list of candidate properties.
# For simplicity we target *all* properties of input object $obj
# but you could start with an explicit list as wellL
# $candidateProps = 'TableName', 'Description', 'AppArea', 'InitialVersionCode'
$candidateProps = $object.psobject.properties.Name
# Create the filtered list of those properties whose value is non-$null
# The .Where() method is a PSv4+ feature.
$nonNullProps = $candidateProps.Where({ $null -ne $object.$_ })
# Extract the list of non-null properties directly from the input object
# and convert to JSON.
$object | Select-Object $nonNullProps | ConvertTo-Json

I made my own modified version of batmanama's answer that accepts an additional parameter, letting you remove elements that are also present in the list present in that parameter.
For example:
Get-CimInstance -ClassName Win32_UserProfile |
Remove-Null -AlsoRemove 'Win32_FolderRedirectionHealth' | Format-Table
I've posted a gist version including PowerShell documentation as well.
Function Remove-Null {
[CmdletBinding()]
Param(
# Object from which to remove the null values.
[Parameter(ValueFromPipeline,Mandatory)]
$InputObject,
# Instead of also removing values that are empty strings, include them
# in the output.
[Switch]$LeaveEmptyStrings,
# Additional entries to remove, which are either present in the
# properties list as an object or as a string representation of the
# object.
# I.e. $item.ToString().
[Object[]]$AlsoRemove = #()
)
Process {
# Iterate InputObject in case input was passed as an array
ForEach ($obj in $InputObject) {
$obj | Select-Object -Property (
$obj.PSObject.Properties.Name | Where-Object {
-not (
# If prop is null, remove it
$null -eq $obj.$_ -or
# If -LeaveEmptyStrings is not specified and the property
# is an empty string, remove it
(-not $LeaveEmptyStrings.IsPresent -and
[string]::IsNullOrEmpty($obj.$_)) -or
# If AlsoRemove contains the property, remove it
$AlsoRemove.Contains($obj.$_) -or
# If AlsoRemove contains the string representation of
# the property, remove it
$AlsoRemove.Contains($obj.$_.ToString())
)
}
)
}
}
}
Note that the process block here automatically iterates a pipeline object, so the ForEach will only iterate more than once when an item is either explicitly passed in an array—such as by wrapping it in a single element array ,$array—or when provided as a direct argument, such as Remove-Null -InputObject $(Get-ChildItem).
It's also worth mentioning that both mine and batmanama's functions will remove these properties from each individual object. That is how it can properly utilize the PowerShell pipeline. Furthermore, that means that if any of the objects in the InputObject have a property that does not match (e.g. they are not null), an output table will still show that property, even though it has removed those properties from other items that did match.
Here's a simple example showing that behavior:
#([pscustomobject]#{Number=1;Bool=$true};
[pscustomobject]#{Number=2;Bool=$false},
[pscustomobject]#{Number=3;Bool=$true},
[pscustomobject]#{Number=4;Bool=$false}) | Remove-Null -AlsoRemove $false
Number Bool
------ ----
1 True
2
3 True
4

Related

Returning string of a function as an Add-Member value parameter in PowerShell

I'm trying to create a Powershell script that shows the folder permissions and the members of the permission groups. I have a function called "Get-Members" that returns (as a comma separated string) the members of the group that has sent to the function as an argument.
Now I'd like to know how i can use the returning string with the Add-Member's value parameter. How can i use the function with that? I tried
Add-Member -MemberType NoteProperty -Name "Members" -Value Get-Members($_.IdentityReference) -PassThru
but it doesn't seem to be working.
Here's the whole thing:
($root | get-acl).Access | Add-Member -MemberType NoteProperty -Name "Members" -Value Get-Members($_.IdentityReference) -PassThru | Add-Member -MemberType NoteProperty -Name "Folder" -Value $($root.fullname).ToString() -PassThru | select -Property Path, IdentityReference, FileSystemRights
And here's the function:
Function Get-Members {
param( [string]$group )
$xyz=$group
if ($group -match '\\')
{
$xyz=$group -creplace '^[^\\]*\\', ''
}
$Group = [ADSI]"LDAP://cn=$xyz,ou=SecurityGroups,ou=Accounting,ou=Services,dc=CONTOSO,dc=ny,dc=local"
$Members = $Group.Member | ForEach-Object {[ADSI]"LDAP://$_"}
$combined = $Members | select -ExpandProperty name
$result= $combined -join ","
return $result
}
How can I get this to work?
Your brackets are wrong. Try this:
Add-Member -MemberType NoteProperty -Name "Members" -Value (Get-Members $_.IdentityReference) -PassThru
Which would make the Get-Members part execute first and return its values to the -Value property due to the brackets.
As an aside, i'd recommend you choose a different name over Get-Members because its too close to the very well known Get-Member cmdlet.

PowerShell function not accepting array of objects

In PowerShell, I have an array of objects that I need to pass to a function. The function is to then loop through all of the objects in the array, but it seems that it is not accepting the parameter value correctly.
Take the following example, where I pass an array containing two objects. I would expect the count of the array to be 2 both before the function and within the function, but as soon as it hits the function the count is 1, and my input is not as expected; only the last object is discovered.
Am I missing something here, or is this a bug in PowerShell?
Example code
### I've also tries '[object]', '[array]' and '[array[]]' as the type for '$testArr'.
function Test-PassArrayOfObjects
{
param(
[parameter(Mandatory,ValueFromPipeline)]
[object[]]$testArr
)
Write-Host "In function count: $($testArr.Count)"
$testArr | ForEach-Object { $_ }
}
$test1 = New-Object –TypeName PSObject
$test1 | Add-Member -MemberType NoteProperty -Name Test1 -Value Value1
$test2 = New-Object –TypeName PSObject
$test2 | Add-Member -MemberType NoteProperty -Name Test2 -Value Value2
$testArr = #($test1, $test2)
$testArr.GetType() | Format-Table
Write-Host "Before function count: $($testArr.Count)"
$testArr | Test-PassArrayOfObjects
Output from example code
IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True True Object[] System.Array
Before function count: 2
In function count: 1
Test2
-----
Value2
Working fix
Based on the answer below, I have this working example, which I've been able to apply to my real life scenario.
function Test-PassArrayOfObjects
{
param(
[parameter(Mandatory,ValueFromPipeline)]
[object]$testArr
)
Process {
Write-Host "In function count: $($testArr.Count)"
$testArr
}
}
$test1 = New-Object –TypeName PSObject
$test1 | Add-Member -MemberType NoteProperty -Name Test1 -Value Value1
$test1 | Add-Member -MemberType NoteProperty -Name Test2 -Value Value2
$test2 = New-Object –TypeName PSObject
$test2 | Add-Member -MemberType NoteProperty -Name Test1 -Value Value1
$test2 | Add-Member -MemberType NoteProperty -Name Test2 -Value Value2
$testArr = #($test1, $test2)
$testArr.GetType() | Format-Table
Write-Host "Before function count: $($testArr.Count)"
$testArr | ForEach-Object { $_ | Test-PassArrayOfObjects }
When sending input to a function via the pipeline, your function should include a Process block:
function Test-PassArrayOfObjects
{
param(
[parameter(Mandatory,ValueFromPipeline)]
[object[]]$testArr
)
Process {
Write-Host "In function count: $($testArr.Count)"
$testArr | ForEach-Object { $_ }
}
}
This is necessary because of the way the pipeline handles collections I believe. It automatically unrolls them and handles them one item at a time, so your ForEach-Object isn't getting the whole collection in the $testArr variable.
You often see functions like this still incorporate a ForEach though, in case the input is sent via a parameter in which case it is received all at once. For example: Test-PassArrayOfObjects -TestArr #(1,2,3).
Your issue is further conflated by the fact that your array has two objects with different properties. This is creating confusion in the output because PowerShell decides how to format the output based on the first object and uses the same formatting when it outputs the second object, but then you don't see it because it doesn't share any of the same properties (I think this is what is occurring anyway..).
You can see that both objects get processed by putting | Format-List on the $_ which forces both outputs to be formatted as list output independently. Note that this isn't good practice in a real function scenario of course. An alternative is to make the property name on both objects Test1. Then you will see the output you probably expected without using Format-List.

Add nested properties to a PowerShell object

I'm working with a script in PowerShell that updates JSON data, accessing an modifying data that's 3 layers deep. The general flow is:
$obj = Get-Content -Raw -Path $pathstring | ConvertFrom-Json
$obj.prop1.prop2.prop3.prop4 = "test"
$outjson = ConvertTo-Json -InputObject $obj -Depth 5
Set-Content -Path $pathstring -Value $outjson
This works when the property already exists. However, in some cases $obj.prop1.prop2.prop3.prop4 does not exist. I want to add a series of nested properties to a PowerShell object, and then convert that to JSON to create it.
Is that possible/how is that done/is there a better way to add JSON values to something in PowerShell?
Edit: I'm currently running
if(Get-Member -inputobject $js.prop1 -name "prop2" -Membertype Properties)
to test if the property exists, and if prop2 doesn't exist then I need to create all the properties.
#J.Peter I really liked your solution, but I needed to be able to provide a way to add properties in the name with periods. so I made a slight mod to yours. I added an escape character to the parameters, that gets replaced in the string with a period.
Edit: another rewrite. Got rid of the recursion, and now it can handle creating very complex objects as well as having "." in the property names. Was so happy with the changes I made a gist for it https://gist.github.com/tcartwright/72cac052e1f8058abed1f7028f674a10 with credits.
function Add-NoteProperty {
param(
$InputObject,
$Property,
$Value,
[switch]$Force,
[char]$escapeChar = '#'
)
process {
$path = $Property -split "\."
$obj = $InputObject
# loop all but the very last property
for ($x = 0; $x -lt $path.count -1; $x ++) {
$propName = $path[$x] -replace $escapeChar, '.'
if (!($obj | Get-Member -MemberType NoteProperty -Name $propName)) {
$obj | Add-Member NoteProperty -Name $propName -Value (New-Object PSCustomObject) -Force:$Force.IsPresent
}
$obj = $obj.$propName
}
$propName = ($path | Select-Object -Last 1) -replace $escapeChar, '.'
if (!($obj | Get-Member -MemberType NoteProperty -Name $propName)) {
$obj | Add-Member NoteProperty -Name $propName -Value $Value -Force:$Force.IsPresent
}
}
}
$obj = [PSCustomObject]#{}
Add-NoteProperty -InputObject $obj -Property "Person.Name.First" -Value "Tim"
Add-NoteProperty -InputObject $obj -Property "Person.Name.Last" -Value "C"
Add-NoteProperty -InputObject $obj -Property "Person.Age" -Value "Old"
Add-NoteProperty -InputObject $obj -Property "Person.Address.City" -Value "Houston"
Add-NoteProperty -InputObject $obj -Property "Person.Address.State" -Value "Texas"
$obj | ConvertTo-JSON
Which results in:
{
"Person": {
"Name": {
"First": "Tim",
"Last": "C"
},
"Age": "Old",
"Address": {
"City": "Houston",
"State": "Texas"
}
}
}
If a property doesn't exist you need to add it, otherwise you can't assign a value to it:
$obj.prop1.prop2.prop3 | Add-Member -Type NoteProperty -Name 'prop4' -Value 'test'
I recently encountered a similiar problem where I needed to add nested properties to objects so I wrote a recursive function for it.
function Add-NoteProperty {
param(
$InputObject,
$Property,
$Value,
[switch]$Force
)
process {
[array]$path = $Property -split "\."
If ($Path.Count -gt 1) {
#go in to recursive mode
$Obj = New-Object PSCustomObject
Add-NoteProperty -InputObject $Obj -Property ($path[1..($path.count - 1)] -join ".") -Value $Value
}
else {
#last node
$Obj = $Value
}
$InputObject | Add-Member NoteProperty -Name $path[0] -Value $Obj -Force:$Force
}
}
Usage example:
$obj = [PSCustomObject]#{
prop1 = "1"
prop2 = "2"
}
Add-NoteProperty -InputObject $obj -Property "prop3.nestedprop31.nestedprop311" -Value "somevalue"
$obj | ConvertTo-JSON
<#Should give you this
{
"prop1": "1",
"prop2": "2",
"prop3": {
"subprop": {
"asdf": "3"
}
}
}
#>

Powershell: Get-Content from the large file (server list) [duplicate]

This question already has answers here:
Can Powershell Run Commands in Parallel?
(10 answers)
Closed 6 years ago.
I have 100,000 list of servers from the text file (serverlist.txt)
When I run in one shot it will burst my memory and cpu and the time took longer (about 3 days)to complete the scanning for DNSlookup.
I tried to split the file that contain 20k list of servers below and can be completed to scan up to 10mins for each file.
serverlist1.txt
serverlist2.txt
serverlist3.txt
serverlist4.txt
serverlist5.txt
$objContainer = #()
$values = #()
$domains = Get-Content -path "serverlist1.txt"
$named = 0
$timestamp= get-date
$domains | ForEach-Object {
$domain = $_
nslookup $domain 2>&1 | ForEach-Object {
if ($_ -match '^Name:\s*(.*)$') {
$values += $matches[1]
$named = 1;
} elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) {
$values += $matches[1]
} elseif ($_ -match '^Aliases:\s*(.*)$') {
$values += $matches[1]
}
}
$obj = New-Object -TypeName PSObject
#$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain
$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0]
$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1]
$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2]
$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp
$objContainer += $obj
$values = #()
$named = 0
}
Write-Output $objContainer
$objContainer | Export-csv "dnslog_$((Get-Date).ToString('MM-dd-yyyy_hh-mm-ss')).csv" -NoTypeInformation
My question is, how to execute at once and looping the input from the text file after generate the dnslog(datetime).csv
e.g:
run the powershell script .\filename.ps1
input from serverlist1.txt
output dnslog(datetime).csv
input from serverlist2.txt
output dnslog(datetime).csv
input from serverlist3.txt
output dnslog(datetime).csv
input from serverlist4.txt
output dnslog(datetime).csv
input from serverlist5.txt
output dnslog(datetime).csv
Finish!
If i have more then 5 list of text file, it will continue to loop from the input file until completed.
Adding to Chris's answer I would also add a ReadCount flag to the Get-Content like so:
Get-Content -path "serverlist1.txt" -ReadCount 1 | % {
This will save having to read the entire file into memory.
You should consider running this a parallel batching job. Have you already tried doing so?
You can deal with the RAM busting problem by removing all those commits to memory (variable assignments and array rewriting with +=).
$timestamp = get-date
Get-Content -path "serverlist1.txt" | ForEach-Object {
$domain = $_
# You can clear this here.
$values = #()
$named = 0
# There are potentially better options than nslookup.
# Needs a bit of care to understand what's an alias here though.
# [System.Net.Dns]::GetHostEntry($domain)
# And if you don't like that, quite a few of us have written equivalent tools in PowerShell.
nslookup $domain 2>&1 | ForEach-Object {
if ($_ -match '^Name:\s*(.*)$') {
$values += $matches[1]
$named = 1;
} elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) {
$values += $matches[1]
} elseif ($_ -match '^Aliases:\s*(.*)$') {
$values += $matches[1]
}
}
# Leave the output object in the output pipeline
# If you're running PowerShell 3 or better:
[PSCustomObject]#{
Domain = $domain
Name = $values[0]
'IP Address' = $values[1]
Alias = $values[2]
TimeStamp = $timestamp
}
# PowerShell 2 is less flexible. This or Select-Object.
#$obj = New-Object -TypeName PSObject
##$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain
#$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0]
#$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1]
#$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2]
#$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp
# To leave this in the output pipeline, uncomment this
# $obj
# No version of PowerShell needs you to do this. It's a good way to ramp up memory usage
# for large data sets.
# $objContainer += $obj
} | Export-Csv "dnslog_$(Get-Date -Format 'MM-dd-yyyy_hh-mm-ss').csv" -NoTypeInformation

ConvertTo-Json truncating object

I have a simple object with 1 parameter being an ArrayList of objects. I am using ConvertTo-Json to output this to Json. However even if I set -Depth 1000 I still see truncation of data.
Structure is:
Object
Property
Property - ArrayList of Object2.
Object 2 is a simple collection of properties.
The output I see is:
{
"CheckDate": "03 February 2016 10:12:30",
"Versions": [
{
},
{
}
]
}
Calling convert on the ArrayList directly all the data is shown. It would appear as if the -Depth argument is not being honored and is stuck at 2.
edit: Code to create object
$returnValue = New-Object System.Object
$returnValue | Add-Member -type NoteProperty -name CheckDate -value (Get-Date).DateTime
$versions = New-Object System.Collections.ArrayList
# This bit is in a loop.
$app = New-Object System.Object
$app | Add-Member -type NoteProperty -Name Name -Value $name
$app | Add-Member -type NoteProperty -Name Version -Value $version
$versions.Add($app)
# Back out of the loop.
$returnValue | Add-Member -type NoteProperty -name Versions -value $versions
Use PSObject instead of System.Object. Unfortunately, I cannot provide any details, it is some internal "magic" of ConvertTo-Json. Interestingly, it is enough to use PSObject instead of the second System.Object.