I'm currently stuck on finding information on how to easily output the function results to a file.
$Balls = 5
$Plays = 1..26
function PowerBall {
foreach ($Trys in $Plays) {
$random = Get-Random -Input #(1..69) -Count $Balls
Write-Host $random -NoNewline
Write-Host " $Trys" -ForegroundColor Red
}
}
PowerBall | Out-File C:\Windows\Temp\numbers2.txt
Your function is not returning anything because the last line is a Write-Host command. Change your function to this:
function PowerBall {
Foreach($Trys in $Plays) {
$random = get-random -input #(1..69) -count $Balls
Write-Host $random -nonewline
write-host " $Trys" -ForegroundColor Red
"$random $Trys"
}
}
Also if you want to output as a text file then you should use Set-Content instead of Export-Csv.
Related
I am trying to read the data from CSV file which has 2200000 records using PowerShell and storing each record in JSON file, but this takes almost 12 hours.
Sample CSV Data:
We will only concern about the 1st column value's.
Code:
function Read-IPData
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object{ $_.FullName }
Write-Host "file path - $dbFilePath"
Write-Host "Reading..."
$data = Get-Content -Path $dbFilePath | Select-Object -Skip 1
Write-Host "Reading data finished"
$count = $data.Count
Write-host "Total $count records found"
return $data
}
function Convert-NumbetToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
try
{
$w = [int64]($number/16777216)%256
$x = [int64]($number/65536)%256
$y = [int64]($number/256)%256
$z = [int64]$number%256
$ipAddress = "$w.$x.$y.$z"
Write-Host "IP Address - $ipAddress"
return $ipAddress
}
catch
{
Write-Host "$_"
continue
}
}
Write-Host "Getting IP Addresses from $dbFileName"
$data = Read-IPData
Write-Host "Checking whether output.json file exist, if not create"
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
if(!(Test-Path $outputFile))
{
Write-Host "$outputFile doestnot exist, creating..."
New-Item -Path $outputFile -type "file"
}
foreach($item in $data)
{
$row = $item -split ","
$ipNumber = $row[0].trim('"')
Write-Host "Converting $ipNumber to ipaddress"
$toIpAddress = Convert-NumbetToIP -number $ipNumber
Write-Host "Preparing document JSON"
$object = [PSCustomObject]#{
"ip-address" = $toIpAddress
"is-vpn" = "true"
"#timestamp" = (Get-Date).ToString("o")
}
$document = $object | ConvertTo-Json -Compress -Depth 100
Write-Host "Adding document - $document"
Add-Content -Path $outputFile $document
}
Could you please help optimize the code or is there a better way to do it. or is there a way like multi-threading.
Here is a possible optimization:
function Get-IPDataPath
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object FullName | Select-Object -First 1
Write-Host "file path - $dbFilePath"
$dbFilePath # implicit output
}
function Convert-NumberToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
[Int64] $numberInt = 0
if( [Int64]::TryParse( $number, [ref] $numberInt ) ) {
if( ($numberInt -ge 0) -and ($numberInt -le 0xFFFFFFFFl) ) {
# Convert to IP address like '192.168.23.42'
([IPAddress] $numberInt).ToString()
}
}
# In case TryParse() returns $false or the number is out of range for an IPv4 address,
# the output of this function will be empty, which converts to $false in a boolean context.
}
$dbFilePath = Get-IPDataPath
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
Write-Host "Converting CSV file $dbFilePath to $outputFile"
$object = [PSCustomObject]#{
'ip-address' = ''
'is-vpn' = 'true'
'#timestamp' = ''
}
# Enclose foreach loop in a script block to be able to pipe its output to Set-Content
& {
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
# Implicit output
$object | ConvertTo-Json -Compress -Depth 100
}
}
} | Set-Content -Path $outputFile
Remarks for improving performance:
Avoid Get-Content, especially for line-by-line processing it tends to be slow. A much faster alternative is the File.ReadLines method. To skip the header line, use the Linq.Enumerable.Skip() method.
There is no need to read the whole CSV into memory first. Using ReadLines in a foreach loop does lazy enumeration, i. e. it reads only one line per loop iteration. This works because it returns an enumerator instead of a collection of lines.
Avoid try and catch if exceptions occur often, because the "exceptional" code path is very slow. Instead use Int64.TryParse() which returns a boolean indicating successful conversion.
Instead of "manually" converting the IP number to bytes, use the IPAddress class which has a constructor that takes an integer number. Use its method .GetAddressBytes() to get an array of bytes in network (big-endian) order. Finally use the PowerShell -join operator to create a string of the expected format.
Don't allocate a [pscustomobject] for each row, which has some overhead. Create it once before the loop and inside the loop only assign the values.
Avoid Write-Host (or any output to the console) within inner loops.
Unrelated to performance:
I've removed the New-Item call to create the output file, which isn't necessary because Set-Content automatically creates the file if it doesn't exist.
Note that the output is in NDJSON format, where each line is like a JSON file. In case you actually want this to be a regular JSON file, enclose the output in [ ] and insert a comma , between each row.
Modified processing loop to write a regular JSON file instead of NDJSON file:
& {
'[' # begin array
$first = $true
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
$row = $object | ConvertTo-Json -Compress -Depth 100
# write array element delimiter if necessary
if( $first ) { $row; $first = $false } else { ",$row" }
}
}
']' # end array
} | Set-Content -Path $outputFile
You can optimize the function Convert-NumberToIP like below:
function Convert-NumberToIP {
param(
[Parameter(Mandatory=$true)][uint32]$number
)
# either do the math yourself like this:
# $w = ($number -shr 24) -band 255
# $x = ($number -shr 16) -band 255
# $y = ($number -shr 8) -band 255
# $z = $number -band 255
# '{0}.{1}.{2}.{3}' -f $w, $x, $y, $z # output the dotted IP string
# or use .Net:
$n = ([IPAddress]$number).GetAddressBytes()
[array]::Reverse($n)
([IPAddress]$n).IPAddressToString
}
Apologize for the length..
Trying to modify a script that currently uses robocopy to report a folder path's size, child folder count, and size. This script currently takes about 24 hrs to run as it goes one by one to each folder. I'm trying to implement invoke-command to set-up jobs so that it will run at least 10 instances of robocopy in the hopes if drastically reducing run time.
I've tried multiple variances given multiple resources to set parameters, call a function and get the results back with no success.
with current code (below) getting error;
Invoke-Command : Parameter set cannot be resolved using the specified named parameters.
ERROR: + Invoke-Command -ScriptBlock { param ($item,$Filter,$params) $ ...
ERROR: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: + CategoryInfo : InvalidArgument: (:) [Invoke-Command], ParameterBindingException
ERROR: + FullyQualifiedErrorId : AmbiguousParameterSet,Microsoft.PowerShell.Commands.InvokeCommandCommand
Function ListFolder ($Folder, $FilterSet, $Roboparam)
{
write-host $Folder
Try
{
$Folder = (Resolve-Path -LiteralPath $Folder -ErrorAction Stop).ProviderPath
If (-Not (Test-Path -LiteralPath $Folder -Type Container -ErrorAction Stop))
{Write-Warning ("{0} is not a directory and will be skipped" -f $Folder)
Return
}
$Script = robocopy $Folder NULL $FilterSet $FRoboparam
$exit_code = $LASTEXITCODE
16, 8, 4, 2, 1 | % {
Switch ($exit_code -band $_)
{
16 { $exit_reason = "Usage error or insufficient access privileges" }
8 { $exit_reason = "Retry limit exceeded" }
4 { $exit_reason = "Some Mismatched files or directories were detected" }
2 { $exit_reason = "Some Extra files or directories were detected. No files were copied" }
1 { $exit_reason = " " }
}
}
If ($exit_code -eq 0)
{$exit_reason = 'No Change'}
If ($Script[-6] -match $dirPattern)
{$Dir = $matches.Dir}
Else
{$Dir = 0}
If ($Script[-5] -match $countPattern)
{$Count = $matches.Count}
Else
{$Count = 0}
If ($Count -gt 0)
{If ($Script[-4] -match $sizePattern)
{$FSize = $matches.Size}}
Else
{$FSize = 0}
$Script:TotSize += $FSize
$Script:Report += New-Object PSObject -Property #{
'Folder Name' = $Folder
Files = "{0:N0}" -f [int]$Count
Folders = "{0:N0}" -f [int]$Dir
Size = "{0:N0}" -f [long]$FSize
Comment = $exit_reason}
Clear-Variable -Name Dir
Clear-Variable -Name Count
Clear-Variable -Name FSize
If ($exit_reason) { Clear-Variable -Name exit_reason }}
Catch
{$Script:Report += New-Object PSObject -Property #{
'Folder Name' = $Folder
Files = "{0:N0}" -f [int]$Count
Folders = "{0:N0}" -f [int]$Dir
Size = [long]$FSize
Comment = [string]$_.Exception.Message}
Continue
}
Return #$Script
}
}
$params = New-Object System.Collections.Arraylist
$params.AddRange(#("/L","/S","/NJH","/BYTES","/FP","/NFL","/NC","/NDL","/TS","/XJ","/R:0","/W:0"))
$dirPattern = "^\s{4}Dirs\s:\s+(?<Dir>\d+).*"
$countPattern = "^\s{3}Files\s:\s+(?<Count>\d+).*"
$sizePattern = "^\s{3}Bytes\s:\s+(?<Size>\d+(?:\.?\d+)).*"
If ($PSBoundParameters['Force']) {$FileDate = 10}
If ($FileDate -lt 6){Exit}
Else
{$Paths = Get-Content $InputFile
ForEach ($item in $Paths)
{
$MaxThreads = 10
While (#(Get-Job | where { $_.State -eq "Running" }).Count -ge $MaxThreads)
{
Write-Host "Waiting for open thread...($MaxThreads Maximum)"
Start-Sleep -Seconds 3
}
Invoke-Command -Computer . -ScriptBlock { param ($item,$Filter,$params) ${function:ListFolder} } -ArgumentList $item, $Filter, $params -AsJob
}
}
When all said and done I have a collection of $Report to show folder name, count of files, count of folders, size in bytes, and comments tho show any errors to Excel
$Report | Sort-Object {[long]$_.Size} -descending | Select 'Folder Name', Files, Folders, Size, Comment | Export-Csv -Path $ReportPath\$(Get-Date -uformat "%Y_%m_%d")-FileSizes.csv -Encoding ascii -NoTypeInformation
I am writing a code to check if Blue Prism process is running and if it is not to start its listener. I want to try twice and if it fails then stop. I have this
$time = Get-Date -Format "dddd MM/dd/yyyy HH:mm K"
$status = Get-Process Automate -ErrorAction SilentlyContinue
If (-not $?)
{
for ($count = 1; $count -le 2; $count++)
{
startListener
}
}
Else {
#exit
Write-Host "Up"
}
function startListener {
$time | Add-Content -Path "C:\Program Files\Blue Prism Limited\BluePrismDown.txt"
Start-Process -FilePath "C:\Program Files\Blue Prism Limited\Blue Prism Automate\Automate.exe" -ArgumentList "/resourcepc", "/public", "/port 9188", "/dbconname PROD"
Start-Sleep -Seconds 30
}
But I am not doing it correctly because there is no additional checking to see if the first try was successful.
I thought about adding another If(-not $?) but I cannot get that to work
$time = Get-Date -Format "dddd MM/dd/yyyy HH:mm K"
$status = Get-Process Automate -ErrorAction SilentlyContinue
If (-not $?)
{
for ($count = 1; $count -le 2; $count++)
{
If (-not $?)
{
startListener
}
Else
{
Break
}
}
}
Else {
#exit
Write-Host "Up"
}
function startListener {
$time | Add-Content -Path "C:\Program Files\Blue Prism Limited\BluePrismDown.txt"
Start-Process -FilePath "C:\Program Files\Blue Prism Limited\Blue Prism Automate\Automate.exe" -ArgumentList "/resourcepc", "/public", "/port 9188", "/dbconname PROD"
Start-Sleep -Seconds 30
}
I know there has got to be a better way to structure this so that I can check if it worked, if not try again and if I tried twice quit.
Augment your function to return a boolean indicating whether or not the process stops again within the 30 seconds:
function Start-BluePrismListener {
Get-Date -Format "dddd MM/dd/yyyy HH:mm K" | Add-Content -Path "C:\Program Files\Blue Prism Limited\BluePrismDown.txt"
$process = Start-Process -FilePath "C:\Program Files\Blue Prism Limited\Blue Prism Automate\Automate.exe" -ArgumentList "/resourcepc", "/public", "/port 9188", "/dbconname PROD" -PassThru
return -not $process.WaitForExit(30000)
}
The Process.WaitForExit(int milliseconds) method overload will return true if the process exits within the timeout, hence the -not. Now we can use the output of the function to decide whether to break out of the loop:
if( -not (Get-Process Automate -ErrorAction SilentlyContinue) ){
for ($count = 1; $count -le 2; $count++){
if(Start-BluePrismListener){
# didn't exit, looks good, let's break out
break
}
}
}
here's a less spiffy way to do what you asked. [grin]
swap the comment markers on lines 3 & 4 to allow testing with "failed to start" and "started properly".
# the next two allow testing for not started OR started
# comment out the one you _don't_ want to work with
$TargetProcess = 'BetterNotBeThere'
#$TargetProcess = 'notepad'
$ProcessFilePath = 'C:\windows\notepad.exe'
$RetryLimit = 2
function Get-IsProcessRunning {
[CmdletBinding ()]
Param
(
[Parameter (
Mandatory,
Position = 0
)]
[string]
$TargetProcess
)
begin {}
process
{
#($False, $True)[[bool](Get-Process -Name $TargetProcess -ErrorAction SilentlyContinue)]
}
end {}
} # end >>> function Get-IsProcessRunning
$RetryCount = 0
while (
-not (Get-IsProcessRunning -TargetProcess $TargetProcess) -and
$RetryCount -lt $RetryLimit
)
{
'Trying to start the [ {0} ] process ...' -f $TargetProcess
Start-Process -FilePath $ProcessFilePath
Start-Sleep -Seconds 5
$RetryCount ++
}
if (Get-IsProcessRunning -TargetProcess $TargetProcess)
{
'The [ {0} ] process it running at this time.' -f $TargetProcess
}
else
{
Write-Warning (' Unable to start the [ {0} ] process.' -f $TargetProcess)
}
output with the "not there" process ...
Trying to start the [ BetterNotBeThere ] process ...
Trying to start the [ BetterNotBeThere ] process ...
WARNING: Unable to start the [ BetterNotBeThere ] process.
output with notepad as the target process ...
Trying to start the [ notepad ] process ...
The [ notepad ] process it running at this time.
the 1st started two copies of Notepad.exe - i stopped them before running the 2nd. [grin]
I have a script that works fine but I want to improve my powershell knowledge and would like to know if there is an easier way to do this.....
Part of my script connects to a sever and pulls bak a list of files and sizes on it and exports it to a csv.
I have found a function (Exportcsv) that allows me to append to the csv with earlier versions of powershell.
At the moment I use the invoke-command to remote to each server and the run the script in the script block but this means adding the function each time. So I have the function in my script but then have to repeat it for each server I connect to so it will run remotely
Is there any way to pass the local function to the remote server so I don't have to add to each invoke command.
Invoke-Command –ComputerName server –ScriptBlock {
$wfile = "d:\folder\directorysize_H.csv"
$xfile = "d:\folder\directorysize_F.csv"
function ExportCSV {
[CmdletBinding(DefaultParameterSetName='Delimiter',
SupportsShouldProcess=$true, ConfirmImpact='Medium')]
param(
[Parameter(Mandatory=$true, ValueFromPipeline=$true,
ValueFromPipelineByPropertyName=$true)]
[System.Management.Automation.PSObject]
${InputObject},
[Parameter(Mandatory=$true, Position=0)]
[Alias('PSPath')]
[System.String]
${Path},
#region -Append
[Switch]
${Append},
#endregion
[Switch]
${Force},
[Switch]
${NoClobber},
[ValidateSet('Unicode','UTF7','UTF8','ASCII','UTF32',
'BigEndianUnicode','Default','OEM')]
[System.String]
${Encoding},
[Parameter(ParameterSetName='Delimiter', Position=1)]
[ValidateNotNull()]
[System.Char]
${Delimiter},
[Parameter(ParameterSetName='UseCulture')]
[Switch]
${UseCulture},
[Alias('NTI')]
[Switch]
${NoTypeInformation})
begin
{
# This variable will tell us whether we actually need to append
# to existing file
$AppendMode = $false
try {
$outBuffer = $null
if ($PSBoundParameters.TryGetValue('OutBuffer', [ref]$outBuffer))
{
$PSBoundParameters['OutBuffer'] = 1
}
$wrappedCmd = $ExecutionContext.InvokeCommand.GetCommand('Export-Csv',
[System.Management.Automation.CommandTypes]::Cmdlet)
#String variable to become the target command line
$scriptCmdPipeline = ''
# Add new parameter handling
#region Process and remove the Append parameter if it is present
if ($Append) {
$PSBoundParameters.Remove('Append') | Out-Null
if ($Path) {
if (Test-Path $Path) {
# Need to construct new command line
$AppendMode = $true
if ($Encoding.Length -eq 0) {
# ASCII is default encoding for Export-CSV
$Encoding = 'ASCII'
}
# For Append we use ConvertTo-CSV instead of Export
$scriptCmdPipeline += 'ConvertTo-Csv -NoTypeInformation '
# Inherit other CSV convertion parameters
if ( $UseCulture ) {
$scriptCmdPipeline += ' -UseCulture '
}
if ( $Delimiter ) {
$scriptCmdPipeline += " -Delimiter '$Delimiter' "
}
# Skip the first line (the one with the property names)
$scriptCmdPipeline += ' | Foreach-Object {$start=$true}'
$scriptCmdPipeline += '{if ($start) {$start=$false} else {$_}} '
# Add file output
$scriptCmdPipeline += " | Out-File -FilePath '$Path'"
$scriptCmdPipeline += " -Encoding '$Encoding' -Append "
if ($Force) {
$scriptCmdPipeline += ' -Force'
}
if ($NoClobber) {
$scriptCmdPipeline += ' -NoClobber'
}
}
}
}
$scriptCmd = {& $wrappedCmd #PSBoundParameters }
if ( $AppendMode ) {
# redefine command line
$scriptCmd = $ExecutionContext.InvokeCommand.NewScriptBlock(
$scriptCmdPipeline
)
} else {
# execute Export-CSV as we got it because
# either -Append is missing or file does not exist
$scriptCmd = $ExecutionContext.InvokeCommand.NewScriptBlock(
[string]$scriptCmd
)
}
# standard pipeline initialization
$steppablePipeline = $scriptCmd.GetSteppablePipeline(
$myInvocation.CommandOrigin)
$steppablePipeline.Begin($PSCmdlet)
} catch {
throw
}
}
process
{
try {
$steppablePipeline.Process($_)
} catch {
throw
}
}
end
{
try {
$steppablePipeline.End()
} catch {
throw
}
}
}
Write-Host "Removing old files from xxx"
If (Test-Path $wfile){
Remove-Item $wfile
}
If (Test-Path $xfile){
Remove-Item $xfile
}
write-host "Getting _f details"
get-childitem \\server\F$ -recurse |select directory, name, length|exportcsv $xfile -append -noclobber -notypeinformation
write-host "Getting _H details"
get-childitem \\server\H$ -recurse |select directory, name, length|exportcsv $wfile -append -noclobber -notypeinformation
}
TIA
Andy
No there isn't a straightforward way to pass the function to the remote computers, well other than what you are already doing. :-) However you can put all that script in a file (dirsize.ps1) and pass that to Invoke-Command using the FilePath parameter:
Invoke-Command –ComputerName server –FilePath .\dirsize.ps1
The file will be copied to the remote computers and executed.
try this:
#Local Function
function Get-Bits
{
Get-Service -Name BITS
}
Invoke-Command -ComputerName RemoteServer -ScriptBlock ${function:Get-Bits}
personally i put all my functions in a psm1 file in other words a module on a network share and import the module while in the remote session.
I am attempting to write a function to compress files using 7zip, but I am having issues passing multiple parameters to the function.
$In = "C:\test\gateways_25357_20140407000204.pcap"
$Out = "C:\test\gateways_25357_20140407000204.zip"
function CompressFile([string]$Output,[string]$Input) {
Write-Host $Output
write-host $Input
$7zipPath = "C:\Program Files\7-Zip\7z.exe"
$Arguments = "a","-tzip",$Output,$Input
& $7zipPath $Arguments
}
CompressFile $Out $In
My results of this code is the compressing of the files in the working directory of this script and the output goes to the correct location c:\test.
What exactly am I doing wrong here with passing in the $Input parameter?
$Input is a powershell automatic variable, try changing the name.
see
$In = "C:\test\gateways_25357_20140407000204.pcap"
$Out = "C:\test\gateways_25357_20140407000204.zip"
function CompressFile([string]$Outputz, [String]$Inputz) {
Write-Host $Outputz
write-host $Inputz
}
Write-Host $Out
write-host $In
CompressFile $Out $In
http://technet.microsoft.com/en-us/library/hh847768.aspx