Returning data calling function from invoke-command with parameters - function

Apologize for the length..
Trying to modify a script that currently uses robocopy to report a folder path's size, child folder count, and size. This script currently takes about 24 hrs to run as it goes one by one to each folder. I'm trying to implement invoke-command to set-up jobs so that it will run at least 10 instances of robocopy in the hopes if drastically reducing run time.
I've tried multiple variances given multiple resources to set parameters, call a function and get the results back with no success.
with current code (below) getting error;
Invoke-Command : Parameter set cannot be resolved using the specified named parameters.
ERROR: + Invoke-Command -ScriptBlock { param ($item,$Filter,$params) $ ...
ERROR: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: + CategoryInfo : InvalidArgument: (:) [Invoke-Command], ParameterBindingException
ERROR: + FullyQualifiedErrorId : AmbiguousParameterSet,Microsoft.PowerShell.Commands.InvokeCommandCommand
Function ListFolder ($Folder, $FilterSet, $Roboparam)
{
write-host $Folder
Try
{
$Folder = (Resolve-Path -LiteralPath $Folder -ErrorAction Stop).ProviderPath
If (-Not (Test-Path -LiteralPath $Folder -Type Container -ErrorAction Stop))
{Write-Warning ("{0} is not a directory and will be skipped" -f $Folder)
Return
}
$Script = robocopy $Folder NULL $FilterSet $FRoboparam
$exit_code = $LASTEXITCODE
16, 8, 4, 2, 1 | % {
Switch ($exit_code -band $_)
{
16 { $exit_reason = "Usage error or insufficient access privileges" }
8 { $exit_reason = "Retry limit exceeded" }
4 { $exit_reason = "Some Mismatched files or directories were detected" }
2 { $exit_reason = "Some Extra files or directories were detected. No files were copied" }
1 { $exit_reason = " " }
}
}
If ($exit_code -eq 0)
{$exit_reason = 'No Change'}
If ($Script[-6] -match $dirPattern)
{$Dir = $matches.Dir}
Else
{$Dir = 0}
If ($Script[-5] -match $countPattern)
{$Count = $matches.Count}
Else
{$Count = 0}
If ($Count -gt 0)
{If ($Script[-4] -match $sizePattern)
{$FSize = $matches.Size}}
Else
{$FSize = 0}
$Script:TotSize += $FSize
$Script:Report += New-Object PSObject -Property #{
'Folder Name' = $Folder
Files = "{0:N0}" -f [int]$Count
Folders = "{0:N0}" -f [int]$Dir
Size = "{0:N0}" -f [long]$FSize
Comment = $exit_reason}
Clear-Variable -Name Dir
Clear-Variable -Name Count
Clear-Variable -Name FSize
If ($exit_reason) { Clear-Variable -Name exit_reason }}
Catch
{$Script:Report += New-Object PSObject -Property #{
'Folder Name' = $Folder
Files = "{0:N0}" -f [int]$Count
Folders = "{0:N0}" -f [int]$Dir
Size = [long]$FSize
Comment = [string]$_.Exception.Message}
Continue
}
Return #$Script
}
}
$params = New-Object System.Collections.Arraylist
$params.AddRange(#("/L","/S","/NJH","/BYTES","/FP","/NFL","/NC","/NDL","/TS","/XJ","/R:0","/W:0"))
$dirPattern = "^\s{4}Dirs\s:\s+(?<Dir>\d+).*"
$countPattern = "^\s{3}Files\s:\s+(?<Count>\d+).*"
$sizePattern = "^\s{3}Bytes\s:\s+(?<Size>\d+(?:\.?\d+)).*"
If ($PSBoundParameters['Force']) {$FileDate = 10}
If ($FileDate -lt 6){Exit}
Else
{$Paths = Get-Content $InputFile
ForEach ($item in $Paths)
{
$MaxThreads = 10
While (#(Get-Job | where { $_.State -eq "Running" }).Count -ge $MaxThreads)
{
Write-Host "Waiting for open thread...($MaxThreads Maximum)"
Start-Sleep -Seconds 3
}
Invoke-Command -Computer . -ScriptBlock { param ($item,$Filter,$params) ${function:ListFolder} } -ArgumentList $item, $Filter, $params -AsJob
}
}
When all said and done I have a collection of $Report to show folder name, count of files, count of folders, size in bytes, and comments tho show any errors to Excel
$Report | Sort-Object {[long]$_.Size} -descending | Select 'Folder Name', Files, Folders, Size, Comment | Export-Csv -Path $ReportPath\$(Get-Date -uformat "%Y_%m_%d")-FileSizes.csv -Encoding ascii -NoTypeInformation

Related

What is the good way to read data from CSV and converting them to JSON?

I am trying to read the data from CSV file which has 2200000 records using PowerShell and storing each record in JSON file, but this takes almost 12 hours.
Sample CSV Data:
We will only concern about the 1st column value's.
Code:
function Read-IPData
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object{ $_.FullName }
Write-Host "file path - $dbFilePath"
Write-Host "Reading..."
$data = Get-Content -Path $dbFilePath | Select-Object -Skip 1
Write-Host "Reading data finished"
$count = $data.Count
Write-host "Total $count records found"
return $data
}
function Convert-NumbetToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
try
{
$w = [int64]($number/16777216)%256
$x = [int64]($number/65536)%256
$y = [int64]($number/256)%256
$z = [int64]$number%256
$ipAddress = "$w.$x.$y.$z"
Write-Host "IP Address - $ipAddress"
return $ipAddress
}
catch
{
Write-Host "$_"
continue
}
}
Write-Host "Getting IP Addresses from $dbFileName"
$data = Read-IPData
Write-Host "Checking whether output.json file exist, if not create"
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
if(!(Test-Path $outputFile))
{
Write-Host "$outputFile doestnot exist, creating..."
New-Item -Path $outputFile -type "file"
}
foreach($item in $data)
{
$row = $item -split ","
$ipNumber = $row[0].trim('"')
Write-Host "Converting $ipNumber to ipaddress"
$toIpAddress = Convert-NumbetToIP -number $ipNumber
Write-Host "Preparing document JSON"
$object = [PSCustomObject]#{
"ip-address" = $toIpAddress
"is-vpn" = "true"
"#timestamp" = (Get-Date).ToString("o")
}
$document = $object | ConvertTo-Json -Compress -Depth 100
Write-Host "Adding document - $document"
Add-Content -Path $outputFile $document
}
Could you please help optimize the code or is there a better way to do it. or is there a way like multi-threading.
Here is a possible optimization:
function Get-IPDataPath
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object FullName | Select-Object -First 1
Write-Host "file path - $dbFilePath"
$dbFilePath # implicit output
}
function Convert-NumberToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
[Int64] $numberInt = 0
if( [Int64]::TryParse( $number, [ref] $numberInt ) ) {
if( ($numberInt -ge 0) -and ($numberInt -le 0xFFFFFFFFl) ) {
# Convert to IP address like '192.168.23.42'
([IPAddress] $numberInt).ToString()
}
}
# In case TryParse() returns $false or the number is out of range for an IPv4 address,
# the output of this function will be empty, which converts to $false in a boolean context.
}
$dbFilePath = Get-IPDataPath
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
Write-Host "Converting CSV file $dbFilePath to $outputFile"
$object = [PSCustomObject]#{
'ip-address' = ''
'is-vpn' = 'true'
'#timestamp' = ''
}
# Enclose foreach loop in a script block to be able to pipe its output to Set-Content
& {
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
# Implicit output
$object | ConvertTo-Json -Compress -Depth 100
}
}
} | Set-Content -Path $outputFile
Remarks for improving performance:
Avoid Get-Content, especially for line-by-line processing it tends to be slow. A much faster alternative is the File.ReadLines method. To skip the header line, use the Linq.Enumerable.Skip() method.
There is no need to read the whole CSV into memory first. Using ReadLines in a foreach loop does lazy enumeration, i. e. it reads only one line per loop iteration. This works because it returns an enumerator instead of a collection of lines.
Avoid try and catch if exceptions occur often, because the "exceptional" code path is very slow. Instead use Int64.TryParse() which returns a boolean indicating successful conversion.
Instead of "manually" converting the IP number to bytes, use the IPAddress class which has a constructor that takes an integer number. Use its method .GetAddressBytes() to get an array of bytes in network (big-endian) order. Finally use the PowerShell -join operator to create a string of the expected format.
Don't allocate a [pscustomobject] for each row, which has some overhead. Create it once before the loop and inside the loop only assign the values.
Avoid Write-Host (or any output to the console) within inner loops.
Unrelated to performance:
I've removed the New-Item call to create the output file, which isn't necessary because Set-Content automatically creates the file if it doesn't exist.
Note that the output is in NDJSON format, where each line is like a JSON file. In case you actually want this to be a regular JSON file, enclose the output in [ ] and insert a comma , between each row.
Modified processing loop to write a regular JSON file instead of NDJSON file:
& {
'[' # begin array
$first = $true
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
$row = $object | ConvertTo-Json -Compress -Depth 100
# write array element delimiter if necessary
if( $first ) { $row; $first = $false } else { ",$row" }
}
}
']' # end array
} | Set-Content -Path $outputFile
You can optimize the function Convert-NumberToIP like below:
function Convert-NumberToIP {
param(
[Parameter(Mandatory=$true)][uint32]$number
)
# either do the math yourself like this:
# $w = ($number -shr 24) -band 255
# $x = ($number -shr 16) -band 255
# $y = ($number -shr 8) -band 255
# $z = $number -band 255
# '{0}.{1}.{2}.{3}' -f $w, $x, $y, $z # output the dotted IP string
# or use .Net:
$n = ([IPAddress]$number).GetAddressBytes()
[array]::Reverse($n)
([IPAddress]$n).IPAddressToString
}

Function to check file in 2 file locations not working

I am trying to create a program that will copy files from our backup to archive directory using Powershell. I have two criteria in order for this program to run smoothly. One is that we have files from both current year and past years so only the files from this year must be copied over. Another is that we have to check to make sure that we are not copying over files of the same file name in case if the data in the file is accidentally modified. Whenever I have this program not in a function, it works. But in a function, it gives me errors that it "cannot find the path" of the folder that I am copying from and the folder that I'm pasting the files to. I am going to use this for more than sixty locations, so it would be better that I don't have to rewrite the code in the function sixty times. I thought about using Robocopy, but I am still getting the same issues regardless with files not being copied over.
Function Copy-Data {
param (
[system.object]$copyFolder,
[system.object]$pasteFolder,
[int]$currentYear,
[int]$lastYear,
[int]$nextYear)
$copyItem = Get-ChildItem -Path $copyFolder
$pasteItem = Get-ChildItem -Path $pasteFolder
$copyCount = $copyItem.count
for ($i = 0; $i -lt $copyCount; $i++)
{
$copyName = $copyItem.Name
$testPath = Test-Path "$pasteFolder$copyName"
if ($copyItem[$i].LastWriteTime -gt $firstDate -and $copyItem[$i].LastWriteTime -lt $lastDate)
{
if ($testPath -eq $false)
{
Copy-Item -Path $copyFolder$copyName -Destination $pasteFolder
#Robocopy "$copyFolder$copyItem[$i]" "$pasteFolder"
Write-Host $pasteFolder$copyName
}
}
}
}
$currentYear = Get-Date -Format "yyyy"
$lastYear = [int]$currentYear - 1
$nextYear = [int]$currentYear + 1
$firstDate = "12/31/$lastYear"
$lastDate = "01/01/$nextYear"
$copyFolder = "\\fileshare\test\copy\"
$pasteFolder = "\\fileshare\test\$currentYear\paste\"
Copy-Data ($copyFolder, $pasteFolder, $currentYear, $lastYear, $nextYear)
I feel like you are making this more complicated than it needs to be.
Core code can be something like:
Get-ChildItem -Path $copyFolder |
ForEach-Object{
If( !(Test-Path $pasteFolder -Name $_.Name ) )
{
Copy-Item $_.FullName -Destination $pasteFolder
}
}
You can use just the destination path you don't have to give the full path of the destination file.
As a function it may look something like:
Function CopyFoldercontents
{
Param(
[Parameter( Mandatory = $true, Position = 0)]
[String]$copyFolder,
[Parameter( Mandatory = $true, Position = 1)]
[String]$pasteFolder
) # End Parameter Block.
Get-ChildItem -Path $copyFolder |
ForEach-Object{
If( !(Test-Path $pasteFolder -Name $_.Name ) )
{
Copy-Item $_.FullName -Destination $pasteFolder
}
}
} # End Function CopyFolderContents
This could be more robust though, depends on what direction you want to take it.
Continuing from my comment.
You could just do this...
(validate what you are after on both sides, the construct the final function)
Function Start-FolderMirror
{
[CmdletBinding()]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles'
# Results
<#
InputObject SideIndicator
----------- -------------
D:\temp\TestFiles\abc - Copy - Copy.bat =>
D:\temp\TestFiles\abc - Copy.bat =>
D:\temp\TestFiles\abc.bat =>
D:\temp\(MSINFO32) command-line tool switches.pdf <=
D:\temp\23694d1213305764-revision-number-in-excel-book1.xls <=
#>
Function Start-FolderMirror
{
[CmdletBinding(SupportsShouldProcess)]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5 |
Where-Object -Property SideIndicator -Match '<='
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles' -WhatIf
# Results
<#
InputObject SideIndicator
----------- -------------
D:\temp\(MSINFO32) command-line tool switches.pdf <=
D:\temp\23694d1213305764-revision-number-in-excel-book1.xls <=
#>
Function Start-FolderMirror
{
[CmdletBinding(SupportsShouldProcess)]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5 |
Where-Object -Property SideIndicator -Match '<=' |
ForEach {Copy-Item -Path $PSItem.InputObject -Destination $DestinationPath}
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles' -WhatIf
# Results
<#
What if: Performing the operation "Copy File" on target "Item: D:\temp\(MSINFO32) command-line tool switches.pdf Destination: D:\temp\TestFiles\(MSINFO32) command-line tool switches.pdf".
What if: Performing the operation "Copy File" on target "Item: D:\temp\23694d1213305764-revision-number-in-excel-book1.xls Destination: D:\temp\TestFiles\23694d1213305764-revision-number-in-excel-book1.xls".
#>

Is it possible to load all functions of a script before it runs?

I have a rather complex script, which follows the following steps;
-->Login (Ask user to enter admin details)
--->Start (Queries Ad for user creditals)
--->Progress (Creates a progress bar)
--->Search (Carries out the search for the data)
--->Question1 -(Yes - Select-Folder No - Create)
--->Select-Folder (Asks the user to create a file path for the document to be stored)
--->Go (Creates a csv from the results of the search)
--->Question2 (CSV - Result XLSX - Excel) (Asks the user if they wish to create a Xlsx file from the Csv)
--->Create (Checks to see if file path exists C:\temp\Server_shares if not creates it)
--->Done (creates csv at default location C:\temp\Server_shares)
--->Question2 (Asks the user if they wish to create a Xlsx file from the Csv)
--->Question2 -(CSV - Result XLSX - Excel) (Asks the user if they wish to create a Xlsx file from the Csv)
--->Result (User has chosen not to create a Xlsx notifys the user of the file path)
--->End (Closes script)
--->Excel (creates Xlsx from csv and stores it in either default or user defined location)
--->Delete (Deletes all remaingin Csv's from file path)
--->End (Closes script)
it seems that I keep get unhandelled exception errors everytime that I run the code.
I have found out that it runs then loads the functions, I need to run them all in memory so that when called each one will run without error.
I have tried creating varibles for each function, but this exactly the same as what I have already.
[void][System.Reflection.Assembly]::LoadWithPartialName("System.windows.forms")
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Drawing")
[void][System.Reflection.Assembly]::LoadWithPartialName ("Microsoft.VisualBasic")
$Script:ErrorActionPreference = "Stop"
$Script:Ma3 = "C:\Temp\Server_Shares\"
Function Get-Event{
Function Get-Login {
Clear-Host
#Write-Host "Please provide admin credentials (for example DOMAIN\admin.user and your password)"
$Global:Credential = Get-Credential
}
Function Get-Start{
#Get user credentials
$Cred = Get-Credential -Message "Enter Your Credentials (Domain\Username)"
if ($Cred -eq $Null)
{
Write-Host "Please enter your username in the form of Domain\UserName and try again" -BackgroundColor Black -ForegroundColor Yellow
Rerun
Break
}
#Parse provided user credentials
$DomainNetBIOS = $Cred.username.Split("{\}")[0]
$UserName = $Cred.username.Split("{\}")[1]
$Password = $Cred.GetNetworkCredential().password
Write-Host "`n"
Write-Host "Checking Credentials for $DomainNetBIOS\$UserName" -BackgroundColor Black -ForegroundColor White
Write-Host "***************************************"
If ($DomainNetBIOS -eq $Null -or $UserName -eq $Null)
{
Write-Host "Missing domain please type in the following format: Domain\Username" -BackgroundColor Black -ForegroundColor Yellow
Rerun
Break
}
# Checks if the domain in question is reachable, and get the domain FQDN.
Try
{
$DomainFQDN = (Get-ADDomain $DomainNetBIOS).DNSRoot
}
Catch
{
Write-Host "Error: Domain was not found: " $_.Exception.Message -BackgroundColor Black -ForegroundColor Red
Write-Host "Please make sure the domain NetBios name is correct, and is reachable from this computer" -BackgroundColor Black -ForegroundColor Red
Rerun
Break
}
#Checks user credentials against the domain
$DomainObj = "LDAP://" + $DomainFQDN
$DomainBind = New-Object System.DirectoryServices.DirectoryEntry($DomainObj,$UserName,$Password)
$DomainName = $DomainBind.distinguishedName
If ($DomainName -eq $Null)
{
Write-Host "Domain $DomainFQDN was found: True" -BackgroundColor Black -ForegroundColor Green
$UserExist = Get-ADUser -Server $DomainFQDN -Properties LockedOut -Filter {sAMAccountName -eq $UserName}
If ($UserExist -eq $Null)
{
Write-Host "Error: Username $Username does not exist in $DomainFQDN Domain." -BackgroundColor Black -ForegroundColor Red
Rerun
Break
}
Else
{
Write-Host "User exists in the domain: True" -BackgroundColor Black -ForegroundColor Green
If ($UserExist.Enabled -eq "True")
{
Write-Host "User Enabled: "$UserExist.Enabled -BackgroundColor Black -ForegroundColor Green
}
Else
{
Write-Host "User Enabled: "$UserExist.Enabled -BackgroundColor Black -ForegroundColor RED
Write-Host "Enable the user account in Active Directory, Then check again" -BackgroundColor Black -ForegroundColor RED
Rerun
Break
}
If ($UserExist.LockedOut -eq "True")
{
Write-Host "User Locked: " $UserExist.LockedOut -BackgroundColor Black -ForegroundColor Red
Write-Host "Unlock the User Account in Active Directory, Then check again..." -BackgroundColor Black -ForegroundColor RED
Rerun
Break
}
Else
{
Write-Host "User Locked: " $UserExist.LockedOut -BackgroundColor Black -ForegroundColor Green
}
}
Write-Host "Authentication failed for $DomainNetBIOS\$UserName with the provided password." -BackgroundColor Black -ForegroundColor Red
Write-Host "Please confirm the password, and try again..." -BackgroundColor Black -ForegroundColor Red
Rerun
Break
}
Else
{
Write-Host "SUCCESS: The account $Username successfully authenticated against the domain: $DomainFQDN" -BackgroundColor Black -ForegroundColor Green
}
Search
}
Function Rerun {
$Title = "Enter another set of Credentials?"
$Message = "Do you want to try another set of credentials?"
$Yes = New-Object System.Management.Automation.Host.ChoiceDescription "&Yes", "Try Again?"
$No = New-Object System.Management.Automation.Host.ChoiceDescription "&No", "End Script."
$Options = [System.Management.Automation.Host.ChoiceDescription[]]($Yes, $No)
$Result = $host.ui.PromptForChoice($Title, $Message, $Options, 0)
Switch ($Result)
{
0 {Get-Start}
1 {"End Script."}
}
}
Function Get-Progress{
Try{
{If (Test-Path $PC -ErrorAction Stop) {
Add-Type -assembly System.Windows.Forms
## -- Create The Progress-Bar
$ObjForm = New-Object System.Windows.Forms.Form
$ObjForm.Text = "Progress-Bar of searched folders"
$ObjForm.Height = 100
$ObjForm.Width = 500
$ObjForm.BackColor = "White"
$ObjForm.FormBorderStyle = [System.Windows.Forms.FormBorderStyle]::FixedSingle
$ObjForm.StartPosition = [System.Windows.Forms.FormStartPosition]::CenterScreen
## -- Create The Label
$ObjLabel = New-Object System.Windows.Forms.Label
$ObjLabel.Text = "Starting. Please wait ... "
$ObjLabel.Left = 5
$ObjLabel.Top = 10
$ObjLabel.Width = 500 - 20
$ObjLabel.Height = 15
$ObjLabel.Font = "Tahoma"
## -- Add the label to the Form
$ObjForm.Controls.Add($ObjLabel)
$PB = New-Object System.Windows.Forms.ProgressBar
$PB.Name = "PowerShellProgressBar"
$PB.Value = 0
$PB.Style="Continuous"
$System_Drawing_Size = New-Object System.Drawing.Size
$System_Drawing_Size.Width = 500 - 40
$System_Drawing_Size.Height = 20
$PB.Size = $System_Drawing_Size
$PB.Left = 5
$PB.Top = 40
$ObjForm.Controls.Add($PB)
## -- Show the Progress-Bar and Start The PowerShell Script
$ObjForm.Show() | Out-Null
$ObjForm.Focus() | Out-NUll
$ObjLabel.Text = "Starting. Please wait ... "
$ObjForm.Refresh()
Start-Sleep -Seconds 1
Out-Null
## -- Execute The PowerShell Code and Update the Status of the Progress-Bar
$result = (get-acl $pc).Access
$Counter = 0
ForEach ($Item In $Result) {
## -- Calculate The Percentage Completed
$Counter++
[Int]$Percentage = ($Counter/$Result.Count)*100
$PB.Value = $Percentage
$ObjLabel.Text = "Scanning $Folders For $criteria in $PC"
#$ObjLabel.Text = "Found $counter Pst's in $Search"
$ObjForm.Refresh()
Start-Sleep -Milliseconds 150
# -- $Item.Name
#"`t" + $Item.Path
$ObjForm.Close()
#Write-Host "`n"
Else {
#Write-Host
#Write-Host "`t Cannot Execute The Script." -ForegroundColor "Yellow"
#Write-Host "`t $Search Does Not Exist in the System." -ForegroundColor "Yellow"
#Write-Host
}
}
}
}
}
Catch{
Write-Host "Please enter a vaild path" -ForegroundColor Cyan
Search
}
}
Function Script:Get-Question {
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Drawing")
$Form1 = New-Object System.Windows.Forms.Form
$Form1.ClientSize = New-Object System.Drawing.Size(200, 100)
$form1.topmost = $true
$Text = New-Object System.Windows.Forms.Label
$Text.Location = New-Object System.Drawing.Point(15, 15)
$Text.Size = New-Object System.Drawing.Size(200, 40)
$Text.Text = "Would you like to save the file to a custom location?"
$Form1.Controls.Add($Text)
#$ErrorActionPreference = "SilentlyContinue"
Function Button1
{
$Button1 = New-Object System.Windows.Forms.Button
$Button1.Location = New-Object System.Drawing.Point(20, 55)
$Button1.Size = New-Object System.Drawing.Size(55, 20)
$Button1.Text = "Yes"
$Button1.add_Click({Get-Go -ErrorAction SilentlyContinue
$Form1.Close()})
$Form1.Controls.Add($Button1)
}
Function Button2
{
$Button2 = New-Object System.Windows.Forms.Button
$Button2.Location = New-Object System.Drawing.Point(80, 55)
$Button2.Size = New-Object System.Drawing.Size(55, 20)
$Button2.Text = "No"
$Button2.add_Click({Get-Create -ErrorAction SilentlyContinue
$Form1.Close()})
$Form1.Controls.Add($Button2)
}
Button1
Button2
[void]$form1.showdialog()
}
Function Select-FolderDialog{
param([string]$Description="Select Folder",[string] $RootFolder="Desktop")
[System.Reflection.Assembly]::LoadWithPartialName ("System.windows.forms") | Out-Null
Write-host "Please minimize the console to select a folder in which to save the results"
$objForm = New-Object System.Windows.Forms.FolderBrowserDialog
$objForm.Rootfolder = $RootFolder
$objForm.Description = $Description
$objForm.ShowNewFolderButton = $false
$Show = $objForm.ShowDialog()
If ($Show -eq "OK")
{
Return $objForm.SelectedPath
}
Else
{
Write-Error "Operation cancelled by user."
Exit
}
}
Function Get-Search{
Write-host "Please Minimize the console and enter the full folder path that you require permissions for" -ForegroundColor Green
$Script:PC = [Microsoft.VisualBasic.Interaction]::InputBox("Please enter the full path of the folder you wish to search", "Folder choice")
If ($PC -eq "")
{
Exit
}
Get-Progress
$Res = (get-acl $pc).Access
$Script:Gold = $Res| Select-object #{label = "User Groups";Expression = {$_.IdentityReference}},
#{label = "Rights";Expression = {$_.FileSystemRights}},
#{label = "Access";Expression = {$_.AccessControlType}}
Get-Question
}
Function Script:Get-Go{
$Form1.Close()
$FPath = Select-FolderDialog
$Folder = $FPath + "\" + [Microsoft.VisualBasic.Interaction]::InputBox ("Please select a folder to save the data to", "Path Choice") + "\"
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic') | Out-Null
"Please minimize the console to select a folder in which to save the results"
$Name = [Microsoft.VisualBasic.Interaction]::InputBox("Please choose a filename", "File Name Choice")
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic') | Out-Null
$cfgOutpath = $Folder + "$Name"
if( -Not (Test-Path -Path $Folder ) )
{
New-Item -ItemType directory -Path $Folder |out-null
}
Else{
[System.Windows.MessageBox]::Show('The directory already exists','Error','Ok','Error')
}
$Gold | Export-Csv "$cfgOutpath.csv" -NoClobber -NoTypeInformation
Write-Host "File has been saved to $cfgOutpath.csv" -ForegroundColor Yellow
Get-Q2
}
##############################################
## Testing Phases ##
## Get-Start ##
## ##
##############################################
Search
Function Script:Get-Create {
$Form1.Close()
if( -Not (Test-Path -Path $Ma3 ) )
{
New-Item -ItemType directory -Path $Ma3 |out-null
}
Done
}
Function Script:Get-Done{
$PC2 = ($PC -split '\\')[-1]
$CSV = "C:\Temp\Server_Shares\User access for $PC2"
$cfgOutpath = $CSV
$Gold | Export-Csv "$cfgOutpath.csv" -NoClobber -NoTypeInformation
Get-Q2
}
Function Script:Get-Q2 {
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Drawing")
$Form2 = New-Object System.Windows.Forms.Form
$Form2.ClientSize = New-Object System.Drawing.Size(200, 100)
$form2.topmost = $true
$Text = New-Object System.Windows.Forms.Label
$Text.Location = New-Object System.Drawing.Point(15, 15)
$Text.Size = New-Object System.Drawing.Size(200, 40)
$Text.Text = "Would you like to create an Xlsx document or leave it as csv?"
$Form2.Controls.Add($Text)
$ErrorActionPreference = "SilentlyContinue"
Function Button1
{
$Button1 = New-Object System.Windows.Forms.Button
$Button1.Location = New-Object System.Drawing.Point(20, 55)
$Button1.Size = New-Object System.Drawing.Size(55, 20)
$Button1.Text = "CSV"
$Button1.add_Click({Get-Result
$Form2.Close()})
$Form2.Controls.Add($Button1)}
Function Button2
{
$Button2 = New-Object System.Windows.Forms.Button
$Button2.Location = New-Object System.Drawing.Point(80, 55)
$Button2.Size = New-Object System.Drawing.Size(55, 20)
$Button2.Text = "XLSX"
$Button2.add_Click({Get-Excel
$Form2.Close()})
$Form2.Controls.Add($Button2)
}
Button1
Button2
[void]$form2.showdialog()
}
Function Script:Get-Ans{
$Form2.Close()
Try{
Get-Excel
}
Catch{
Write-Host "Unable to create XSLX please check full path." -ForegroundColor Red
}
}
Function Script:Get-Result{
$Form2.Close()
Write-Host "File has been saved to $CSV.csv" -ForegroundColor Yellow
}
Function Script:Get-Excel{
$RD = $Ma3 + "*.csv"
$CsvDir = $RD
$csvs = dir -path $CsvDir # Collects all the .csv's from the driectory
$outputxls = "$Ma4.Xlsx"
$Excel = New-Object -ComObject excel.application
$Excel.displayAlerts = $false
$workbook = $excel.Workbooks.add()
# Loops through each CVS, pulling all the data from each one
foreach($iCsv in $csvs){
$iCsv
$WN = ($iCsv -Split "\\")[5]
$wn = ($WN -Split " ")[3]
$WN = $WN -replace ".{5}$"
$Excel = New-Object -ComObject excel.application
$Excel.displayAlerts = $false
$Worksheet = $workbook.worksheets.add()
$Worksheet.name = $WN
$TxtConnector = ("TEXT;" + $iCsv)
$Connector = $worksheet.Querytables.add($txtconnector,$worksheet.Range("A1"))
$query = $Worksheet.QueryTables.item($Connector.name)
$query.TextfileOtherDelimiter = $Excel.Application.International(5)
$Query.TextfileParseType =1
$Query.TextFileColumnDataTypes = ,2 * $worksheet.cells.column.count
$query.AdjustColumnWidth =1
$Query.Refresh()
$Query.Delete()
$Worksheet.Cells.EntireColumn.AutoFit()
$Worksheet.Rows.Item(1).Font.Bold = $true
$Worksheet.Rows.Item(1).HorizontalAlignment = -4108
$Worksheet.Rows.Item(1).Font.Underline = $true
$Workbook.save()
}
$Empty = $workbook.worksheets.item("Sheet1")
$Empty.Delete()
$Workbook.SaveAs($outputxls,51)
$Workbook.close()
$Excel.quit()
Write-Host "File has been saved to $outputxls" -ForegroundColor Yellow
Delete
}
Function Script:Delete{
get-childitem $MA3 -recurse -force -include *.txt | remove-item -force #Removes all txt files from final directory
get-childitem $MA3 -recurse -force -include *.csv | remove-item -force #Removes all CSV files from final directory
}
Write-Host "Finished"
}
Get-Event
#Excel-Write'
I know it's a lot of code, but in-order for anyone to replicate the probelm, you will need it all.
I want it to run, first time, everytime, at the moment it take 3-4 tries before its loaded each function into memory.
Put all of your functions into a PowerShell module file.
C:\Module.psm1
And then import that module before anything else in the script:
Import-Module C:\Module.psm1
For larger scripts this makes things more manageable as you can keep your long list of functions separate from the script that calls them.

Powershell script not recognising Functions

I've written the following PS script to delete log files from specific server paths. I'm a novice to PS but I'm getting some errors with a few of the functions that I have written in this script:
#* FileName: FileCleaner.ps1
#Clear the screen
Clear
#Read XML Config File to get settings
[xml]$configfile = Get-Content "C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell Script\FileCleaner.config.xml"
#Declare and set variables from Config values
$hostServer = $configfile.Settings.HostServer
$dirs = #($configfile.Settings.DirectoryName.Split(",").Trim())
$scanSubDirectories = $configfile.Settings.ScanSubDirectories
$deleteAllFiles = $configfile.Settings.deleteAllFiles
$fileTypesToDelete = #($configfile.Settings.FileTypesToDelete.Split(";").Trim())
$liveSiteLogs = $configfile.Settings.LiveSiteLogs
$fileExclusions = #($configfile.Settings.FileExclusions.Split(";").Trim())
$retentionPeriod = $configfile.Settings.RetentionPeriod
$AICLogs = $configfile.Settings.AICLogs
$AICLogsRententionPeriod = $configfile.Settings.AICLogsRententionPeriod
$fileCleanerLogs = $configfile.Settings.FileCleanerLogs
$fileCleanerLogsRententionPeriod = $configfile.Settings.FileCleanerLogsRententionPeriod
#Setup FileCleaner output success logfiles
$successLogfile = $configfile.Settings.SuccessOutputLogfile
$dirName = [io.path]::GetDirectoryName($successLogfile)
$filename = [io.path]::GetFileNameWithoutExtension($successLogfile)
$ext = [io.path]::GetExtension($successLogfile)
$successLogfile = "$dirName\$filename$(get-date -Format yyyy-MM-dd)$ext"
#Setup FileCleaner output error logfiles
$errorLogfile = $configfile.Settings.ErrorOutputLogfile
$dirName = [io.path]::GetDirectoryName($errorLogfile)
$filename = [io.path]::GetFileNameWithoutExtension($errorLogfile)
$ext = [io.path]::GetExtension($errorLogfile)
$errorLogfile = "$dirName\$filename$(get-date -Format yyyy-MM-dd)$ext"
#Setup Retention Period
$LastWrite = (Get-Date).AddDays(-$retentionPeriod)#.ToString("d")
$AICLastWrite = (Get-Date).AddDays(-$AICLogsRententionPeriod)#.ToString("d")
$fileCleanerLastWrite = (Get-Date).AddDays(-$fileCleanerLogsRententionPeriod)
#EMAIL SETTINGS
$smtpServer = $configfile.Settings.SMTPServer
$emailFrom = $configfile.Settings.EmailFrom
$emailTo = $configfile.Settings.EmailTo
$emailSubject = $configfile.Settings.EmailSubject
#Update the email subject to display the Host Server value
$emailSubject -replace "HostServer", $hostServer
$countUnaccessibleUNCPaths = 0
#Check Logfiles exists, if not create them
if(!(Test-Path -Path $successLogfile))
{
New-Item -Path $successLogfile –itemtype file
}
if(!(Test-Path -Path $errorLogfile))
{
New-Item -Path $errorLogfile –itemtype file
}
foreach ($dir in $dirs)
{
#needs a check to determine if server/the UNC Path is accessible. If it fails to connect, it needs to move on to the next UNC share but a flag needs to
#be generate to alert us to investigate why the UNC share was not accessible during the job run.
If(Test-Path -Path $dir)
{
#write to output logfile Directory info
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $dir"
$Msg | out-file $successLogfile
If ($scanSubDirectories -eq "True")
{
If ($deleteAllFiles -eq "True")
{
#ScanSubDirectories and delete all files older than the $retentionPeriod, include Sub-Directories / also forces the deletion of any hidden files
$logFiles = Get-ChildItem -Path $dir -Force -Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$LastWrite" }
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
Else
{
#"ScanSubDirectories but only delete specified file types."
$logFiles = Get-Childitem $dir -Include $fileTypesToDelete[0],$fileTypesToDelete[1],$fileTypesToDelete[2], $liveSiteLogs -Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where {$_.LastWriteTime -le "$LastWrite"}
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
}
Else
{
#Only delete files in top level Directory
If ($deleteAllFiles -eq "True")
{
$logFiles = Get-ChildItem -Path $dir -Force -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$LastWrite" }
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
Else
{
$logFiles = Get-Childitem $dir -Include $fileTypesToDelete[0],$fileTypesToDelete[1],$fileTypesToDelete[2], $liveSiteLogs -Exclude $fileExclusions[0],$fileExclusions[1] | Where {$_.LastWriteTime -le "$LastWrite"}
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
}
}
Else
{
$countUnaccessibleUNCPaths++
#server/the UNC Path is unaccessible
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") Unable to access $dir."
$Msg | out-file $errorLogfile -append
}
# Call the function to Delete the AIC XML Logfiles
DeleteAICXMLLogs $dir
}
#If any of the directories were unaccessible send an email to alert the team
if($countUnaccessibleUNCPaths.count -gt 0)
{
# Call the function to send the email
SendEmail $emailSubject $emailFrom $emailTo
}
#Only keep 2 weeks worth of the FileCleaner App logs for reference purposes
If(Test-Path -Path $fileCleanerLogs)
{
#write to output logfile Directory info
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $fileCleanerLogs"
$Msg | out-file $successLogfile
$fileCleanerLogs = Get-Childitem $fileCleanerLogs -Recurse | Where {$_.LastWriteTime -le "$fileCleanerLastWrite"}
DeleteLogFiles($fileCleanerLogs)
#foreach($fileCleanerLog in $fileCleanerLogs)
#{
# if($fileCleanerLog -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $fileCleanerLog")"
# $Msg | out-file $successLogfile -append
# Remove-Item $fileCleanerLog.FullName -Force
# }
#}
}
Function DeleteLogFiles($logFiles)
{
foreach($logFile in $logFiles)
{
if($logFile -ne $null)
{
$Msg = Write-Output "$("Deleting File $logFile")"
$Msg | out-file $successLogfile -append
Remove-Item $logFile.FullName -Force
}
}
}
Function DeleteAICXMLLogs($dir)
{
#Split the UNC path $dir to retrieve the server value
$parentpath = "\\" + [string]::join("\",$dir.Split("\")[2])
#test access to the \\server\D$\DebugXML path
If(Test-Path -Path $parentpath$AICLogs)
{
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $parentpath$AICLogs"
$Msg | out-file $successLogfile
#Concantenate server value to $AICLogs to delete all xml logs in \\server\D$\DebugXML with a retention period of 30Days
$XMLlogFiles = Get-ChildItem -Path $parentpath$AICLogs -Force -Include $fileTypesToDelete[3]-Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$AICLastWrite" }
#get each file and add the filename to be deleted to the successLogfile before deleting the file
DeleteLogFiles($XMLlogFiles)
#foreach($XMLlogFile in $XMLlogFiles)
#{
# if($XMLlogFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $XMLlogFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $XMLlogFile.FullName -Force
# }
#}
}
Else
{
$Msg = Write-Output "$("$parentpath$AICLogs does not exist.")"
$Msg | out-file $successLogfile -append
}
}
Function SendEmail($emailSubject, $emailFrom, $emailTo)
{
$MailMessage = New-Object System.Net.Mail.MailMessage
$SMTPClient = New-Object System.Net.Mail.smtpClient
$SMTPClient.host = $smtpServer
$Recipient = New-Object System.Net.Mail.MailAddress($emailTo, "Recipient")
$Sender = New-Object System.Net.Mail.MailAddress($emailFrom, "Sender")
$MailMessage.Sender = $Sender
$MailMessage.From = $Sender
$MailMessage.Subject = $emailSubject
$MailMessage.Body = #"
This email was generated because the FileCleaner script was unable to access some UNC Paths, please refer to $errorLogfile for more information.
Please inform the Team if you plan to resolve this.
This is an automated email please do not respond.
"#
$SMTPClient.Send($MailMessage)
}
when debugging I'm getting these errors:
DeleteAICXMLLogs : The term 'DeleteAICXMLLogs' is not recognized as
the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify
that the path is correct and try again. At
C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell
Script\FileCleaner.ps1:158 char:5
+ DeleteAICXMLLogs $dir
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (DeleteAICXMLLogs:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
SendEmail : The term 'SendEmail' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the spelling
of the name, or if a path was included, verify that the path is
correct and try again. At C:\Users\pmcma\Documents\Projects\Replace
FileCleaner with PowerShell Script\FileCleaner.ps1:164 char:5
+ SendEmail $emailSubject $emailFrom $emailTo
+ ~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (SendEmail:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
DeleteLogFiles : The term 'DeleteLogFiles' is not recognized as the
name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the
path is correct and try again. At
C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell
Script\FileCleaner.ps1:175 char:5
+ DeleteLogFiles($fileCleanerLogs)
+ ~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (DeleteLogFiles:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I don't see anything wrong with how I'm declaring the functions or calling them. Any ideas why this script is failing?
PowerShell Scripts are read from the top to the bottom, so you can't use any references before they are defined, most probably that is why you are receiving errors.
Try adding your function definition blocks above the point where you call them.
Alternatively you can make a function having global scope. Just preface the function name with the keyword global: like,
function global:test ($x, $y)
{
$x * $y
}
I've had this happen as well. Try placing the functions before the business logic. This is a script, not compiled code. So the functions are yet to be declared before you are calling them.

How can I make a win32 command (sqlcommand) print its output to the console in PowerShell

I have a script that eventually calls this line
& sqlcmd.exe -S $DbHost -d $DbSchema -Q "do some crazy db change here"
Where "do some crazy db change here" will eventually be replaced by a dynamic bit of SQL/script.
When I run this whether it succeeds or not I see no output from sqlcmd.exe in my console. For user feedback I'd like to pipe this in real time to the console. How could I do that?
Here's how I have done it in the past, it makes use of events from StandardOut and StandardError. As these are executed asynchronously you are not entirely in control of the output (in terms of when it happens), but it should be close to what you need.
$SqlCommandArguments = #()
$SqlCommandArguments += "-S $DbHost"
$SqlCommandArguments += "-d $DbSchema"
$SqlCommandArguments += "-Q `"do some crazy db change here`""
ExecuteProcess -FileName "SqlCmd.exe" -CommandArguments $SqlCommandArguments -Verbose:$VerbosePreference
function ExecuteProcess
{
[cmdletbinding()]
param
(
[string]$FileName,
[string[]]$CommandArguments
)
Write-Verbose "$FileName $CommandArguments"
$startInfo = New-Object System.Diagnostics.ProcessStartInfo
$startInfo.FileName = $FileName
$startInfo.Arguments = $CommandArguments
$startInfo.RedirectStandardError = $true
$startInfo.RedirectStandardOutput = $true
$startInfo.UseShellExecute = $false
$startInfo.CreateNoWindow = $true
$process = New-Object System.Diagnostics.Process
$process.StartInfo = $startInfo
$eventOutputDataReceived = Register-ObjectEvent -InputObject $process -EventName OutputDataReceived -MessageData $VerbosePreference -Action {
if ($($EventArgs.data))
{
Write-Verbose $EventArgs.data -verbose:$event.MessageData
}
}
$global:standardError = New-Object System.Text.StringBuilder
$eventErrorDataReceived = Register-ObjectEvent -InputObject $process -EventName ErrorDataReceived -Action {
if ($($EventArgs.data))
{
$global:standardError.Append("$($EventArgs.data)`r`n")
Write-Warning -message $EventArgs.data
}
}
$process.Start() | Out-Null
$process.BeginOutputReadLine()
$process.BeginErrorReadLine()
$process.WaitForExit()
Unregister-Event -SourceIdentifier $eventOutputDataReceived.Name
Unregister-Event -SourceIdentifier $eventErrorDataReceived.Name
$exitCode = $process.ExitCode
if ($exitCode -ne 0)
{
Write-Error $global:standardError.ToString()
throw "$FileName Failed!"
}
}