PowerShell how to present function parameters in the form of CSV - powershell-5.1

I'm new to PowerShell and I have very simple task: to output three function parameters as CSV string. Here what I get:
function ToCsv($v1, $v2, $v3) {
Write-Host $v1, $v2, $v3
Write-Host "$v1,$v2,$v3"
Write-Host $v1 + "," + $v2 + "," + $v3
}
ToCsv("One", "Two", "Three")
One Two Three
One Two Three,,
One Two Three + , + + , +
What would be the right way to output a bunch of function parameters as comma-separated string?
I'm using PowerShell 5.1

Mark,
It takes a little work but not biggie.
Function ToCsv($v1, $v2, $v3) {
#Convert arguments to an Ordered Hash Table
$MyArgs = [Ordered]#{
Arg1 = "$v1"
Arg2 = "$v2"
Arg3 = "$v3"
}
#Convert the Hash table to a PS Custom Object.
$object = new-object psobject -Property $MyArgs
#Export the PS Custom Object as a CSV file.
Export-csv -InputObject $object -Path G:\BEKDocs\Test.csv -NoTypeInformation
}
ToCsv "One" "Two" "Three"
Results: (as seen in NP++)
"Arg1","Arg2","Arg3"
"One","Two","Three"
HTH
Update:
Here's a better solution that will handle any number of arguments.
Function ToCsv {
$ArgCnt = $Args.Count
$object = new-object psobject
For ($Cntr = 0; $Cntr -lt $ArgCnt; $Cntr++) {
$AMArgs = #{InputObject = $object
MemberType = "NoteProperty"
Name = $("Arg" + $($Cntr + 1))
Value = "$($Args[$($Cntr)])"
}
Add-Member #AMArgs
}
Export-CSV -InputObject $object -Path G:\BEKDocs\Test.csv -NoTypeInformation
} #End Function ToCSV
ToCsv "One" "Two" "Three" "Four"
Results: (as seen in NP++)
"Arg1","Arg2","Arg3","Arg4"
"One","Two","Three","Four"

Related

What is the good way to read data from CSV and converting them to JSON?

I am trying to read the data from CSV file which has 2200000 records using PowerShell and storing each record in JSON file, but this takes almost 12 hours.
Sample CSV Data:
We will only concern about the 1st column value's.
Code:
function Read-IPData
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object{ $_.FullName }
Write-Host "file path - $dbFilePath"
Write-Host "Reading..."
$data = Get-Content -Path $dbFilePath | Select-Object -Skip 1
Write-Host "Reading data finished"
$count = $data.Count
Write-host "Total $count records found"
return $data
}
function Convert-NumbetToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
try
{
$w = [int64]($number/16777216)%256
$x = [int64]($number/65536)%256
$y = [int64]($number/256)%256
$z = [int64]$number%256
$ipAddress = "$w.$x.$y.$z"
Write-Host "IP Address - $ipAddress"
return $ipAddress
}
catch
{
Write-Host "$_"
continue
}
}
Write-Host "Getting IP Addresses from $dbFileName"
$data = Read-IPData
Write-Host "Checking whether output.json file exist, if not create"
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
if(!(Test-Path $outputFile))
{
Write-Host "$outputFile doestnot exist, creating..."
New-Item -Path $outputFile -type "file"
}
foreach($item in $data)
{
$row = $item -split ","
$ipNumber = $row[0].trim('"')
Write-Host "Converting $ipNumber to ipaddress"
$toIpAddress = Convert-NumbetToIP -number $ipNumber
Write-Host "Preparing document JSON"
$object = [PSCustomObject]#{
"ip-address" = $toIpAddress
"is-vpn" = "true"
"#timestamp" = (Get-Date).ToString("o")
}
$document = $object | ConvertTo-Json -Compress -Depth 100
Write-Host "Adding document - $document"
Add-Content -Path $outputFile $document
}
Could you please help optimize the code or is there a better way to do it. or is there a way like multi-threading.
Here is a possible optimization:
function Get-IPDataPath
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object FullName | Select-Object -First 1
Write-Host "file path - $dbFilePath"
$dbFilePath # implicit output
}
function Convert-NumberToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
[Int64] $numberInt = 0
if( [Int64]::TryParse( $number, [ref] $numberInt ) ) {
if( ($numberInt -ge 0) -and ($numberInt -le 0xFFFFFFFFl) ) {
# Convert to IP address like '192.168.23.42'
([IPAddress] $numberInt).ToString()
}
}
# In case TryParse() returns $false or the number is out of range for an IPv4 address,
# the output of this function will be empty, which converts to $false in a boolean context.
}
$dbFilePath = Get-IPDataPath
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
Write-Host "Converting CSV file $dbFilePath to $outputFile"
$object = [PSCustomObject]#{
'ip-address' = ''
'is-vpn' = 'true'
'#timestamp' = ''
}
# Enclose foreach loop in a script block to be able to pipe its output to Set-Content
& {
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
# Implicit output
$object | ConvertTo-Json -Compress -Depth 100
}
}
} | Set-Content -Path $outputFile
Remarks for improving performance:
Avoid Get-Content, especially for line-by-line processing it tends to be slow. A much faster alternative is the File.ReadLines method. To skip the header line, use the Linq.Enumerable.Skip() method.
There is no need to read the whole CSV into memory first. Using ReadLines in a foreach loop does lazy enumeration, i. e. it reads only one line per loop iteration. This works because it returns an enumerator instead of a collection of lines.
Avoid try and catch if exceptions occur often, because the "exceptional" code path is very slow. Instead use Int64.TryParse() which returns a boolean indicating successful conversion.
Instead of "manually" converting the IP number to bytes, use the IPAddress class which has a constructor that takes an integer number. Use its method .GetAddressBytes() to get an array of bytes in network (big-endian) order. Finally use the PowerShell -join operator to create a string of the expected format.
Don't allocate a [pscustomobject] for each row, which has some overhead. Create it once before the loop and inside the loop only assign the values.
Avoid Write-Host (or any output to the console) within inner loops.
Unrelated to performance:
I've removed the New-Item call to create the output file, which isn't necessary because Set-Content automatically creates the file if it doesn't exist.
Note that the output is in NDJSON format, where each line is like a JSON file. In case you actually want this to be a regular JSON file, enclose the output in [ ] and insert a comma , between each row.
Modified processing loop to write a regular JSON file instead of NDJSON file:
& {
'[' # begin array
$first = $true
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
$row = $object | ConvertTo-Json -Compress -Depth 100
# write array element delimiter if necessary
if( $first ) { $row; $first = $false } else { ",$row" }
}
}
']' # end array
} | Set-Content -Path $outputFile
You can optimize the function Convert-NumberToIP like below:
function Convert-NumberToIP {
param(
[Parameter(Mandatory=$true)][uint32]$number
)
# either do the math yourself like this:
# $w = ($number -shr 24) -band 255
# $x = ($number -shr 16) -band 255
# $y = ($number -shr 8) -band 255
# $z = $number -band 255
# '{0}.{1}.{2}.{3}' -f $w, $x, $y, $z # output the dotted IP string
# or use .Net:
$n = ([IPAddress]$number).GetAddressBytes()
[array]::Reverse($n)
([IPAddress]$n).IPAddressToString
}

Powershell Parse Swagger Json

The following works to parse a Swagger json into resource, method, httptype but probably... the $path.Definition part is weirdly, how can i get $path.Definition to be an array not a string that i need to parse for the array symbol.
$json = Get-Content -Path "$PSScriptRoot/Test/example_swagger.json" | ConvertFrom-Json
$paths = Get-Member -InputObject $json.paths -MemberType NoteProperty
$result = ""
foreach($path in $paths) {
$elements = $path.Name.Substring(5).split("/") -join ","
$httpmethods = $path.Definition.Substring($path.Definition.IndexOf("#{"))
if ($httpmethods.Contains("get")) {
$result += $elements + ", GET" + "`n"
}
if ($httpmethods.Contains("post")) {
$result += $elements + ", POST" + "`n" #same methodnames different http methods
}
}
$result
As detailed in my answer to Iterating through a JSON file PowerShell, the output of ConvertFrom-Json is hard to iterate over. This makes "for each key in object" and "keys of object not known ahead of time" kinds of situations more difficult to handle, but not impossible.
You need a helper function:
# helper to turn PSCustomObject into a list of key/value pairs
function Get-ObjectMember {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$True, ValueFromPipeline=$True)]
[PSCustomObject]$obj
)
$obj | Get-Member -MemberType NoteProperty | ForEach-Object {
$key = $_.Name
[PSCustomObject]#{Key = $key; Value = $obj."$key"}
}
}
with this, the approach gets a whole lot simpler:
$swagger = Get-Content -Path "example_swagger.json" -Encoding UTF8 -Raw | ConvertFrom-Json
$swagger.paths | Get-ObjectMember | ForEach-Object {
[pscustomobject]#{
path = $_.Key
methods = $_.Value | Get-ObjectMember | ForEach-Object Key
}
}
Applied to the default Swagger file from https://editor.swagger.io/ as a sample, this is printed
path methods
---- -------
/pet {post, put}
/pet/findByStatus get
/pet/findByTags get
/pet/{petId} {delete, get, post}
/pet/{petId}/uploadImage post
/store/inventory get
/store/order post
/store/order/{orderId} {delete, get}
/user post
/user/createWithArray post
/user/createWithList post
/user/login get
/user/logout get
/user/{username} {delete, get, put}

Easy way to reference JSON arrays [duplicate]

I want to get a JSON representation of a Hashtable such as this:
#{Path="C:\temp"; Filter="*.js"}
ConvertTo-Json results in:
{
"Path": "C:\\temp",
"Filter": "*.js"
}
However, if you convert that JSON string back with ConvertFrom-Json you don't get a HashTable but a PSCustomObject.
So how can one reliably serialize the above Hashmap?
$json = #{Path="C:\temp"; Filter="*.js"} | ConvertTo-Json
$hashtable = #{}
(ConvertFrom-Json $json).psobject.properties | Foreach { $hashtable[$_.Name] = $_.Value }
Adapted from PSCustomObject to Hashtable
A little late to the discussion here, but in PowerShell 6 (Core) there is a -AsHashtable parameter in ConvertFrom-Json.
JavaScriptSerializer is available since .NET3.5 (may be installed on XP, included in Win7 and newer), it's several times faster than Convert-FromJSON and it properly parses nested objects, arrays etc.
function Parse-JsonFile([string]$file) {
$text = [IO.File]::ReadAllText($file)
$parser = New-Object Web.Script.Serialization.JavaScriptSerializer
$parser.MaxJsonLength = $text.length
Write-Output -NoEnumerate $parser.DeserializeObject($text)
}
The answer for this post is a great start, but is a bit naive when you start getting more complex json representations.
The code below will parse nested json arrays and json objects.
[CmdletBinding]
function Get-FromJson
{
param(
[Parameter(Mandatory=$true, Position=1)]
[string]$Path
)
function Get-Value {
param( $value )
$result = $null
if ( $value -is [System.Management.Automation.PSCustomObject] )
{
Write-Verbose "Get-Value: value is PSCustomObject"
$result = #{}
$value.psobject.properties | ForEach-Object {
$result[$_.Name] = Get-Value -value $_.Value
}
}
elseif ($value -is [System.Object[]])
{
$list = New-Object System.Collections.ArrayList
Write-Verbose "Get-Value: value is Array"
$value | ForEach-Object {
$list.Add((Get-Value -value $_)) | Out-Null
}
$result = $list
}
else
{
Write-Verbose "Get-Value: value is type: $($value.GetType())"
$result = $value
}
return $result
}
if (Test-Path $Path)
{
$json = Get-Content $Path -Raw
}
else
{
$json = '{}'
}
$hashtable = Get-Value -value (ConvertFrom-Json $json)
return $hashtable
}
I believe the solution presented in Converting JSON to a hashtable is closer to the PowerShell 6.0 implementation of ConvertFrom-Json
I tried with several JSON sources and I always got the right hashtable.
$mappings = #{
Letters = (
"A",
"B")
Numbers = (
"1",
"2",
"3")
Yes = 1
False = "0"
}
# TO JSON
$jsonMappings = $mappings | ConvertTo-JSON
$jsonMappings
# Back to hashtable
# In PowerShell 6.0 would be:
# | ConvertFrom-Json -AsHashtable
$jsonMappings | ConvertFrom-Json -As hashtable
you can write a function convert psobject to hashtable.
I wrote a answer here:enter link description here

Importing a CDATA string literal from a JSON array into PowerShell

So I have a XML value extractor for a large rule database.
I stored the values in a Json file with sections for each file and then the values tied to their names.
However some of the values are a CDATA Snippet.
I can pull the CDATA as a string literal to store in the Json but I CANNOT seem to find a way to get power shell to let me put it back into the XML.
Cannot set "Value" because only strings can be used as values to set XmlNode properties.
At C:\Users\vagrant\Desktop\RuleSetter.ps1:24 char:21
+ $rule.value = $ruleFileSet. ($singleRuleXmlFile.directory.nam ...
+
+ CategoryInfo : NotSpecified: (:) [], SetValueException
+ FullyQualifiedErrorId : XmlNodeSetShouldBeAString`
The Getter looks like this
$directories = Get-ChildItem -dir -path C:\inetpub\wwwroot\
foreach($instName in $directories){
if($instName -notlike 'client' -and $instName -notlike 'bin'-and $instName -notlike 'aspnet_client'-and $instName -notlike 'AccessLibertyLogs'){
$obj = [hashtable]#{}
$head = [hashtable]#{Institution = $instName}
$obj.Add('head',$head)
$files = Get-ChildItem C:\inetpub\wwwroot\$instName\filepath -Include Rules.XML -Recurse
$files |
ForEach-Object {
[xml]$temp = Get-Content -path $_.FullName
[string]$title = $_.Directory.name
[hashtable]$output = #{}
foreach($rule in $temp.RuleCollection.Rules.Rule){
$t = ""
if($rule.value -isnot [string]){
$t = $rule.value.innerXml
}else{
$t = $rule.value
}
$output.Add($rule.name,$t)
}
$obj.Add($title,$output)
}
$transfer = New-Object -TypeName PSObject -Property $obj
$location = "C:\Users\vagrant\desktop\json\" + $instName + ".Json"
$transfer | ConvertTo-Json -Compress | Out-file -FilePath $location
}
}
And the Setter like this
Param(
[string]$location,
[string]$ruleConfigFileLocation)
$allInstConfigFiles = Get-ChildItem -Path C:\Users\vagrant\Desktop\json\
foreach($singleInstFile in $allInstConfigFiles) {
$instaName = $singleInstFile.BaseName
$allRuleXmlFileList = Get-ChildItem C:\inetpub\wwwroot\$instaName\Liberty \Applications\Origination\Configuration -Include Rules.XML -Recurse
ForEach($singleRuleXmlFile in $allRuleXmlFileList) {
[xml]$singleRuleXmlFileContents = Get-Content -path $singleRuleXmlFile.FullName
$singleInstFileContent = Get-Content $singleInstFile.FullName | ConvertFrom-Json
foreach($ruleFileSet in $singleInstFileContent){
foreach($rule in $singleRuleXmlFileContents.RuleCollection.Rules.Rule){
#write-host $ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name)
if($ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name) -as [xml])
{
$rule.value = $ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name).OuterXml
write-host 'skipped'
}else{
write-host 'not skipped'
$rule.value = $ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name)
}
}
}
}
}

Create Hashtable from JSON

I want to get a JSON representation of a Hashtable such as this:
#{Path="C:\temp"; Filter="*.js"}
ConvertTo-Json results in:
{
"Path": "C:\\temp",
"Filter": "*.js"
}
However, if you convert that JSON string back with ConvertFrom-Json you don't get a HashTable but a PSCustomObject.
So how can one reliably serialize the above Hashmap?
$json = #{Path="C:\temp"; Filter="*.js"} | ConvertTo-Json
$hashtable = #{}
(ConvertFrom-Json $json).psobject.properties | Foreach { $hashtable[$_.Name] = $_.Value }
Adapted from PSCustomObject to Hashtable
A little late to the discussion here, but in PowerShell 6 (Core) there is a -AsHashtable parameter in ConvertFrom-Json.
JavaScriptSerializer is available since .NET3.5 (may be installed on XP, included in Win7 and newer), it's several times faster than Convert-FromJSON and it properly parses nested objects, arrays etc.
function Parse-JsonFile([string]$file) {
$text = [IO.File]::ReadAllText($file)
$parser = New-Object Web.Script.Serialization.JavaScriptSerializer
$parser.MaxJsonLength = $text.length
Write-Output -NoEnumerate $parser.DeserializeObject($text)
}
The answer for this post is a great start, but is a bit naive when you start getting more complex json representations.
The code below will parse nested json arrays and json objects.
[CmdletBinding]
function Get-FromJson
{
param(
[Parameter(Mandatory=$true, Position=1)]
[string]$Path
)
function Get-Value {
param( $value )
$result = $null
if ( $value -is [System.Management.Automation.PSCustomObject] )
{
Write-Verbose "Get-Value: value is PSCustomObject"
$result = #{}
$value.psobject.properties | ForEach-Object {
$result[$_.Name] = Get-Value -value $_.Value
}
}
elseif ($value -is [System.Object[]])
{
$list = New-Object System.Collections.ArrayList
Write-Verbose "Get-Value: value is Array"
$value | ForEach-Object {
$list.Add((Get-Value -value $_)) | Out-Null
}
$result = $list
}
else
{
Write-Verbose "Get-Value: value is type: $($value.GetType())"
$result = $value
}
return $result
}
if (Test-Path $Path)
{
$json = Get-Content $Path -Raw
}
else
{
$json = '{}'
}
$hashtable = Get-Value -value (ConvertFrom-Json $json)
return $hashtable
}
I believe the solution presented in Converting JSON to a hashtable is closer to the PowerShell 6.0 implementation of ConvertFrom-Json
I tried with several JSON sources and I always got the right hashtable.
$mappings = #{
Letters = (
"A",
"B")
Numbers = (
"1",
"2",
"3")
Yes = 1
False = "0"
}
# TO JSON
$jsonMappings = $mappings | ConvertTo-JSON
$jsonMappings
# Back to hashtable
# In PowerShell 6.0 would be:
# | ConvertFrom-Json -AsHashtable
$jsonMappings | ConvertFrom-Json -As hashtable
you can write a function convert psobject to hashtable.
I wrote a answer here:enter link description here