When exporting data to a CSV file, the first column always are without quotes and I do not understand why. I can move and change so that the secound value becomes the first and the same thing happends there since it's the first column.
foreach ($item in $item_array)
{
$global:f_server_s = "SERVER123"
$global:f_share = "$item"
$global:f_path = "\\$global:f_server_s\$global:f_share"
$obj_fileshare_list = New-Object -TypeName PSObject
$obj_fileshare_list | Add-Member -Type NoteProperty -Name Name -Value "$global:f_share"
$obj_fileshare_list | Add-Member -Type NoteProperty -Name Server -Value "$global:f_server"
$obj_fileshare_list | Add-Member -Type NoteProperty -Name Path -Value "$global:f_path"
$Fileshare_list += $obj_fileshare_list
}
CSV output:
Name,"Server","Path"
item1,"SERVER123","\\SERVER123\item1"
item2,"SERVER123","\\SERVER123\item2"
Save file:
$Fileshare_list | Export-Csv ".\file.csv" -NoTypeInformation -Encoding unicode
In this case, why is Name and all the items without double quotes?
Related
I'm trying to create a Powershell script that shows the folder permissions and the members of the permission groups. I have a function called "Get-Members" that returns (as a comma separated string) the members of the group that has sent to the function as an argument.
Now I'd like to know how i can use the returning string with the Add-Member's value parameter. How can i use the function with that? I tried
Add-Member -MemberType NoteProperty -Name "Members" -Value Get-Members($_.IdentityReference) -PassThru
but it doesn't seem to be working.
Here's the whole thing:
($root | get-acl).Access | Add-Member -MemberType NoteProperty -Name "Members" -Value Get-Members($_.IdentityReference) -PassThru | Add-Member -MemberType NoteProperty -Name "Folder" -Value $($root.fullname).ToString() -PassThru | select -Property Path, IdentityReference, FileSystemRights
And here's the function:
Function Get-Members {
param( [string]$group )
$xyz=$group
if ($group -match '\\')
{
$xyz=$group -creplace '^[^\\]*\\', ''
}
$Group = [ADSI]"LDAP://cn=$xyz,ou=SecurityGroups,ou=Accounting,ou=Services,dc=CONTOSO,dc=ny,dc=local"
$Members = $Group.Member | ForEach-Object {[ADSI]"LDAP://$_"}
$combined = $Members | select -ExpandProperty name
$result= $combined -join ","
return $result
}
How can I get this to work?
Your brackets are wrong. Try this:
Add-Member -MemberType NoteProperty -Name "Members" -Value (Get-Members $_.IdentityReference) -PassThru
Which would make the Get-Members part execute first and return its values to the -Value property due to the brackets.
As an aside, i'd recommend you choose a different name over Get-Members because its too close to the very well known Get-Member cmdlet.
I have a piece of code that works but I want to know if there is a better way to do it. I could not find anything related so far. Here are the facts:
I have an object with n properties.
I want to convert this object to JSON using (ConvertTo-Json).
I don't want to include in the JSON those object properties that are not valued.
Building the object (not really important):
$object = New-Object PSObject
Add-Member -InputObject $object -MemberType NoteProperty -Name TableName -Value "MyTable"
Add-Member -InputObject $object -MemberType NoteProperty -Name Description -Value "Lorem ipsum dolor.."
Add-Member -InputObject $object -MemberType NoteProperty -Name AppArea -Value "UserMgmt"
Add-Member -InputObject $object -MemberType NoteProperty -Name InitialVersionCode -Value ""
The line that I need improvements (to filter out the non-valued properties and not include them in the JSON)
# So I want to 'keep' and deliver to the JSON only the properties that are valued (first 3).
$object | select -Property TableName, Description, AppArea, InitialVersion | ConvertTo-Json
What this line delivers:
Results:
{
"TableName": "MyTable",
"Description": "Lorem ipsum dolor..",
"AppArea": "UserMgmt",
"InitialVersion": null
}
What I want to obtain:
{
"TableName": "MyTable",
"Description": "Lorem ipsum dolor..",
"AppArea": "UserMgmt"
}
What I've tried and works, but I don't like it since I have much more properties to handle:
$JSON = New-Object PSObject
if ($object.TableName){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name TableName -Value $object.TableName
}
if ($object.Description){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name Description -Value $object.Description
}
if ($object.AppArea){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name AppArea -Value $object.AppArea
}
if ($object.InitialVersionCode){
Add-Member -InputObject $JSON -MemberType NoteProperty -Name InitialVersionCode -Value $object.InitialVersionCode
}
$JSON | ConvertTo-Json
Something like this?
$object = New-Object PSObject
Add-Member -InputObject $object -MemberType NoteProperty -Name TableName -Value "MyTable"
Add-Member -InputObject $object -MemberType NoteProperty -Name Description -Value "Lorem ipsum dolor.."
Add-Member -InputObject $object -MemberType NoteProperty -Name AppArea -Value "UserMgmt"
Add-Member -InputObject $object -MemberType NoteProperty -Name InitialVersionCode -Value ""
# Iterate over objects
$object | ForEach-Object {
# Get array of names of object properties that can be cast to boolean TRUE
# PSObject.Properties - https://msdn.microsoft.com/en-us/library/system.management.automation.psobject.properties.aspx
$NonEmptyProperties = $_.psobject.Properties | Where-Object {$_.Value} | Select-Object -ExpandProperty Name
# Convert object to JSON with only non-empty properties
$_ | Select-Object -Property $NonEmptyProperties | ConvertTo-Json
}
Result:
{
"TableName": "MyTable",
"Description": "Lorem ipsum dolor..",
"AppArea": "UserMgmt"
}
I have the following function in my profile for this purpose. Advantage: I can pipe a collection of objects to it and remove nulls from all the objects on the pipeline.
Function Remove-Null {
[cmdletbinding()]
param(
# Object to remove null values from
[parameter(ValueFromPipeline,Mandatory)]
[object[]]$InputObject,
#By default, remove empty strings (""), specify -LeaveEmptyStrings to leave them.
[switch]$LeaveEmptyStrings
)
process {
foreach ($obj in $InputObject) {
$AllProperties = $obj.psobject.properties.Name
$NonNulls = $AllProperties |
where-object {$null -ne $obj.$PSItem} |
where-object {$LeaveEmptyStrings.IsPresent -or -not [string]::IsNullOrEmpty($obj.$PSItem)}
$obj | Select-Object -Property $NonNulls
}
}
}
Some examples of usage:
$AnObject = [pscustomobject]#{
prop1="data"
prop2="moredata"
prop5=3
propblnk=""
propnll=$null
}
$AnObject | Remove-Null
prop1 prop2 prop5
----- ----- -----
data moredata 3
$ObjList =#(
[PSCustomObject]#{
notnull = "data"
more = "sure!"
done = $null
another = ""
},
[PSCustomObject]#{
notnull = "data"
more = $null
done = $false
another = $true
}
)
$objList | Remove-Null | fl #format-list because the default table is misleading
notnull : data
more : sure!
notnull : data
done : False
another : True
beatcracker's helpful answer offers an effective solution; let me complement it with a streamlined version that takes advantage of PSv4+ features:
# Sample input object
$object = [pscustomobject] #{
TableName = 'MyTable'
Description = 'Lorem ipsum dolor...'
AppArea = 'UserMgmt'
InitialVersionCode = $null
}
# Start with the list of candidate properties.
# For simplicity we target *all* properties of input object $obj
# but you could start with an explicit list as wellL
# $candidateProps = 'TableName', 'Description', 'AppArea', 'InitialVersionCode'
$candidateProps = $object.psobject.properties.Name
# Create the filtered list of those properties whose value is non-$null
# The .Where() method is a PSv4+ feature.
$nonNullProps = $candidateProps.Where({ $null -ne $object.$_ })
# Extract the list of non-null properties directly from the input object
# and convert to JSON.
$object | Select-Object $nonNullProps | ConvertTo-Json
I made my own modified version of batmanama's answer that accepts an additional parameter, letting you remove elements that are also present in the list present in that parameter.
For example:
Get-CimInstance -ClassName Win32_UserProfile |
Remove-Null -AlsoRemove 'Win32_FolderRedirectionHealth' | Format-Table
I've posted a gist version including PowerShell documentation as well.
Function Remove-Null {
[CmdletBinding()]
Param(
# Object from which to remove the null values.
[Parameter(ValueFromPipeline,Mandatory)]
$InputObject,
# Instead of also removing values that are empty strings, include them
# in the output.
[Switch]$LeaveEmptyStrings,
# Additional entries to remove, which are either present in the
# properties list as an object or as a string representation of the
# object.
# I.e. $item.ToString().
[Object[]]$AlsoRemove = #()
)
Process {
# Iterate InputObject in case input was passed as an array
ForEach ($obj in $InputObject) {
$obj | Select-Object -Property (
$obj.PSObject.Properties.Name | Where-Object {
-not (
# If prop is null, remove it
$null -eq $obj.$_ -or
# If -LeaveEmptyStrings is not specified and the property
# is an empty string, remove it
(-not $LeaveEmptyStrings.IsPresent -and
[string]::IsNullOrEmpty($obj.$_)) -or
# If AlsoRemove contains the property, remove it
$AlsoRemove.Contains($obj.$_) -or
# If AlsoRemove contains the string representation of
# the property, remove it
$AlsoRemove.Contains($obj.$_.ToString())
)
}
)
}
}
}
Note that the process block here automatically iterates a pipeline object, so the ForEach will only iterate more than once when an item is either explicitly passed in an array—such as by wrapping it in a single element array ,$array—or when provided as a direct argument, such as Remove-Null -InputObject $(Get-ChildItem).
It's also worth mentioning that both mine and batmanama's functions will remove these properties from each individual object. That is how it can properly utilize the PowerShell pipeline. Furthermore, that means that if any of the objects in the InputObject have a property that does not match (e.g. they are not null), an output table will still show that property, even though it has removed those properties from other items that did match.
Here's a simple example showing that behavior:
#([pscustomobject]#{Number=1;Bool=$true};
[pscustomobject]#{Number=2;Bool=$false},
[pscustomobject]#{Number=3;Bool=$true},
[pscustomobject]#{Number=4;Bool=$false}) | Remove-Null -AlsoRemove $false
Number Bool
------ ----
1 True
2
3 True
4
This question already has answers here:
Can Powershell Run Commands in Parallel?
(10 answers)
Closed 6 years ago.
I have 100,000 list of servers from the text file (serverlist.txt)
When I run in one shot it will burst my memory and cpu and the time took longer (about 3 days)to complete the scanning for DNSlookup.
I tried to split the file that contain 20k list of servers below and can be completed to scan up to 10mins for each file.
serverlist1.txt
serverlist2.txt
serverlist3.txt
serverlist4.txt
serverlist5.txt
$objContainer = #()
$values = #()
$domains = Get-Content -path "serverlist1.txt"
$named = 0
$timestamp= get-date
$domains | ForEach-Object {
$domain = $_
nslookup $domain 2>&1 | ForEach-Object {
if ($_ -match '^Name:\s*(.*)$') {
$values += $matches[1]
$named = 1;
} elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) {
$values += $matches[1]
} elseif ($_ -match '^Aliases:\s*(.*)$') {
$values += $matches[1]
}
}
$obj = New-Object -TypeName PSObject
#$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain
$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0]
$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1]
$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2]
$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp
$objContainer += $obj
$values = #()
$named = 0
}
Write-Output $objContainer
$objContainer | Export-csv "dnslog_$((Get-Date).ToString('MM-dd-yyyy_hh-mm-ss')).csv" -NoTypeInformation
My question is, how to execute at once and looping the input from the text file after generate the dnslog(datetime).csv
e.g:
run the powershell script .\filename.ps1
input from serverlist1.txt
output dnslog(datetime).csv
input from serverlist2.txt
output dnslog(datetime).csv
input from serverlist3.txt
output dnslog(datetime).csv
input from serverlist4.txt
output dnslog(datetime).csv
input from serverlist5.txt
output dnslog(datetime).csv
Finish!
If i have more then 5 list of text file, it will continue to loop from the input file until completed.
Adding to Chris's answer I would also add a ReadCount flag to the Get-Content like so:
Get-Content -path "serverlist1.txt" -ReadCount 1 | % {
This will save having to read the entire file into memory.
You should consider running this a parallel batching job. Have you already tried doing so?
You can deal with the RAM busting problem by removing all those commits to memory (variable assignments and array rewriting with +=).
$timestamp = get-date
Get-Content -path "serverlist1.txt" | ForEach-Object {
$domain = $_
# You can clear this here.
$values = #()
$named = 0
# There are potentially better options than nslookup.
# Needs a bit of care to understand what's an alias here though.
# [System.Net.Dns]::GetHostEntry($domain)
# And if you don't like that, quite a few of us have written equivalent tools in PowerShell.
nslookup $domain 2>&1 | ForEach-Object {
if ($_ -match '^Name:\s*(.*)$') {
$values += $matches[1]
$named = 1;
} elseif (($_ -match '^.*?(\d*\.\d*\.\d*\.\d*)$') -and ($named -eq 1)) {
$values += $matches[1]
} elseif ($_ -match '^Aliases:\s*(.*)$') {
$values += $matches[1]
}
}
# Leave the output object in the output pipeline
# If you're running PowerShell 3 or better:
[PSCustomObject]#{
Domain = $domain
Name = $values[0]
'IP Address' = $values[1]
Alias = $values[2]
TimeStamp = $timestamp
}
# PowerShell 2 is less flexible. This or Select-Object.
#$obj = New-Object -TypeName PSObject
##$obj | Add-Member -MemberType NoteProperty -name 'Domain' -value $domain
#$obj | Add-Member -MemberType NoteProperty -name 'Name' -value $values[0]
#$obj | Add-Member -MemberType NoteProperty -name 'IP Address' -value $values[1]
#$obj | Add-Member -MemberType NoteProperty -name 'Alias' -value $values[2]
#$obj | Add-Member -MemberType NoteProperty -name 'Timestamp' -value $timestamp
# To leave this in the output pipeline, uncomment this
# $obj
# No version of PowerShell needs you to do this. It's a good way to ramp up memory usage
# for large data sets.
# $objContainer += $obj
} | Export-Csv "dnslog_$(Get-Date -Format 'MM-dd-yyyy_hh-mm-ss').csv" -NoTypeInformation
I have a simple object with 1 parameter being an ArrayList of objects. I am using ConvertTo-Json to output this to Json. However even if I set -Depth 1000 I still see truncation of data.
Structure is:
Object
Property
Property - ArrayList of Object2.
Object 2 is a simple collection of properties.
The output I see is:
{
"CheckDate": "03 February 2016 10:12:30",
"Versions": [
{
},
{
}
]
}
Calling convert on the ArrayList directly all the data is shown. It would appear as if the -Depth argument is not being honored and is stuck at 2.
edit: Code to create object
$returnValue = New-Object System.Object
$returnValue | Add-Member -type NoteProperty -name CheckDate -value (Get-Date).DateTime
$versions = New-Object System.Collections.ArrayList
# This bit is in a loop.
$app = New-Object System.Object
$app | Add-Member -type NoteProperty -Name Name -Value $name
$app | Add-Member -type NoteProperty -Name Version -Value $version
$versions.Add($app)
# Back out of the loop.
$returnValue | Add-Member -type NoteProperty -name Versions -value $versions
Use PSObject instead of System.Object. Unfortunately, I cannot provide any details, it is some internal "magic" of ConvertTo-Json. Interestingly, it is enough to use PSObject instead of the second System.Object.
I have powershell version 2.0.0.1082 on my server. I want to import data from a text file to SQL server, and add two new fields (character and current datetime) to the input to SQL 2008 database. I have been using bulk import or SSIS to do it but I want to use powershell for the ease of maintaining the process. The file has column name at top and each field is seperated by "|".
Code so far:
clear
import-csv "Disk.txt" -Delimiter "|" |
foreach { add-member -in $_ -membertype noteproperty DateRecorded $((Get-Date).ToString("yyyy-MM-dd"))
add-member -in $_ -membertype noteproperty SystemName 'System Name'
add-member -in $_ -membertype noteproperty Drive 'Drive Letter'
add-member -in $_ -membertype noteproperty TotalSizeGB 'Total Size'
add-member -in $_ -membertype noteproperty UsedGB Used
add-member -in $_ -membertype noteproperty FreeGB Free
add-member -in $_ -membertype noteproperty PercentFree '% Free'}|
select DateRecorded,SystemName,Drive,TotalSizeGB,UsedGB,FreeGB,PercentFree |Format-Table
Any ideas please?
Thanks!
Manjot
You'll either need to add the -PassThru switch to the Add-Member cmdlet to forward the objects on in the pipeline or put $_ all by itself at the end of the ForEach block.
Here is an example using -PassThru:
Import-Csv "Disk.txt" -Delimiter "|" | ForEach {
Add-Member -InputObject $_ -MemberType NoteProperty -name "DateRecorded" -value $((Get-Date).ToString("yyyy-MM-dd"))
Add-Member -InputObject $_ -MemberType NoteProperty -name "SystemName" -value 'System Name' -PassThru
} | Select DateRecorded,SystemName,Drive,TotalSizeGB,UsedGB,FreeGB,PercentFree | Format-Table
Another example without -PassThru:
Import-Csv "Disk.txt" -Delimiter "|" | ForEach {
Add-Member -InputObject $_ -MemberType NoteProperty -name "DateRecorded" -value $((Get-Date).ToString("yyyy-MM-dd"))
Add-Member -InputObject $_ -MemberType NoteProperty -name "SystemName" -value 'System Name'
$_
} | Select DateRecorded,SystemName,Drive,TotalSizeGB,UsedGB,FreeGB,PercentFree | Format-Table
Once you've prepared your CSV data for input into the database you can use the Invoke-SqlCmd cmdlet that comes with SQL Server 2008 to perform Inserts. There is a wealth of information on MSDN here to show you how.