I am trying to create a program that will copy files from our backup to archive directory using Powershell. I have two criteria in order for this program to run smoothly. One is that we have files from both current year and past years so only the files from this year must be copied over. Another is that we have to check to make sure that we are not copying over files of the same file name in case if the data in the file is accidentally modified. Whenever I have this program not in a function, it works. But in a function, it gives me errors that it "cannot find the path" of the folder that I am copying from and the folder that I'm pasting the files to. I am going to use this for more than sixty locations, so it would be better that I don't have to rewrite the code in the function sixty times. I thought about using Robocopy, but I am still getting the same issues regardless with files not being copied over.
Function Copy-Data {
param (
[system.object]$copyFolder,
[system.object]$pasteFolder,
[int]$currentYear,
[int]$lastYear,
[int]$nextYear)
$copyItem = Get-ChildItem -Path $copyFolder
$pasteItem = Get-ChildItem -Path $pasteFolder
$copyCount = $copyItem.count
for ($i = 0; $i -lt $copyCount; $i++)
{
$copyName = $copyItem.Name
$testPath = Test-Path "$pasteFolder$copyName"
if ($copyItem[$i].LastWriteTime -gt $firstDate -and $copyItem[$i].LastWriteTime -lt $lastDate)
{
if ($testPath -eq $false)
{
Copy-Item -Path $copyFolder$copyName -Destination $pasteFolder
#Robocopy "$copyFolder$copyItem[$i]" "$pasteFolder"
Write-Host $pasteFolder$copyName
}
}
}
}
$currentYear = Get-Date -Format "yyyy"
$lastYear = [int]$currentYear - 1
$nextYear = [int]$currentYear + 1
$firstDate = "12/31/$lastYear"
$lastDate = "01/01/$nextYear"
$copyFolder = "\\fileshare\test\copy\"
$pasteFolder = "\\fileshare\test\$currentYear\paste\"
Copy-Data ($copyFolder, $pasteFolder, $currentYear, $lastYear, $nextYear)
I feel like you are making this more complicated than it needs to be.
Core code can be something like:
Get-ChildItem -Path $copyFolder |
ForEach-Object{
If( !(Test-Path $pasteFolder -Name $_.Name ) )
{
Copy-Item $_.FullName -Destination $pasteFolder
}
}
You can use just the destination path you don't have to give the full path of the destination file.
As a function it may look something like:
Function CopyFoldercontents
{
Param(
[Parameter( Mandatory = $true, Position = 0)]
[String]$copyFolder,
[Parameter( Mandatory = $true, Position = 1)]
[String]$pasteFolder
) # End Parameter Block.
Get-ChildItem -Path $copyFolder |
ForEach-Object{
If( !(Test-Path $pasteFolder -Name $_.Name ) )
{
Copy-Item $_.FullName -Destination $pasteFolder
}
}
} # End Function CopyFolderContents
This could be more robust though, depends on what direction you want to take it.
Continuing from my comment.
You could just do this...
(validate what you are after on both sides, the construct the final function)
Function Start-FolderMirror
{
[CmdletBinding()]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles'
# Results
<#
InputObject SideIndicator
----------- -------------
D:\temp\TestFiles\abc - Copy - Copy.bat =>
D:\temp\TestFiles\abc - Copy.bat =>
D:\temp\TestFiles\abc.bat =>
D:\temp\(MSINFO32) command-line tool switches.pdf <=
D:\temp\23694d1213305764-revision-number-in-excel-book1.xls <=
#>
Function Start-FolderMirror
{
[CmdletBinding(SupportsShouldProcess)]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5 |
Where-Object -Property SideIndicator -Match '<='
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles' -WhatIf
# Results
<#
InputObject SideIndicator
----------- -------------
D:\temp\(MSINFO32) command-line tool switches.pdf <=
D:\temp\23694d1213305764-revision-number-in-excel-book1.xls <=
#>
Function Start-FolderMirror
{
[CmdletBinding(SupportsShouldProcess)]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5 |
Where-Object -Property SideIndicator -Match '<=' |
ForEach {Copy-Item -Path $PSItem.InputObject -Destination $DestinationPath}
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles' -WhatIf
# Results
<#
What if: Performing the operation "Copy File" on target "Item: D:\temp\(MSINFO32) command-line tool switches.pdf Destination: D:\temp\TestFiles\(MSINFO32) command-line tool switches.pdf".
What if: Performing the operation "Copy File" on target "Item: D:\temp\23694d1213305764-revision-number-in-excel-book1.xls Destination: D:\temp\TestFiles\23694d1213305764-revision-number-in-excel-book1.xls".
#>
Related
I am trying to read the data from CSV file which has 2200000 records using PowerShell and storing each record in JSON file, but this takes almost 12 hours.
Sample CSV Data:
We will only concern about the 1st column value's.
Code:
function Read-IPData
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object{ $_.FullName }
Write-Host "file path - $dbFilePath"
Write-Host "Reading..."
$data = Get-Content -Path $dbFilePath | Select-Object -Skip 1
Write-Host "Reading data finished"
$count = $data.Count
Write-host "Total $count records found"
return $data
}
function Convert-NumbetToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
try
{
$w = [int64]($number/16777216)%256
$x = [int64]($number/65536)%256
$y = [int64]($number/256)%256
$z = [int64]$number%256
$ipAddress = "$w.$x.$y.$z"
Write-Host "IP Address - $ipAddress"
return $ipAddress
}
catch
{
Write-Host "$_"
continue
}
}
Write-Host "Getting IP Addresses from $dbFileName"
$data = Read-IPData
Write-Host "Checking whether output.json file exist, if not create"
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
if(!(Test-Path $outputFile))
{
Write-Host "$outputFile doestnot exist, creating..."
New-Item -Path $outputFile -type "file"
}
foreach($item in $data)
{
$row = $item -split ","
$ipNumber = $row[0].trim('"')
Write-Host "Converting $ipNumber to ipaddress"
$toIpAddress = Convert-NumbetToIP -number $ipNumber
Write-Host "Preparing document JSON"
$object = [PSCustomObject]#{
"ip-address" = $toIpAddress
"is-vpn" = "true"
"#timestamp" = (Get-Date).ToString("o")
}
$document = $object | ConvertTo-Json -Compress -Depth 100
Write-Host "Adding document - $document"
Add-Content -Path $outputFile $document
}
Could you please help optimize the code or is there a better way to do it. or is there a way like multi-threading.
Here is a possible optimization:
function Get-IPDataPath
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object FullName | Select-Object -First 1
Write-Host "file path - $dbFilePath"
$dbFilePath # implicit output
}
function Convert-NumberToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
[Int64] $numberInt = 0
if( [Int64]::TryParse( $number, [ref] $numberInt ) ) {
if( ($numberInt -ge 0) -and ($numberInt -le 0xFFFFFFFFl) ) {
# Convert to IP address like '192.168.23.42'
([IPAddress] $numberInt).ToString()
}
}
# In case TryParse() returns $false or the number is out of range for an IPv4 address,
# the output of this function will be empty, which converts to $false in a boolean context.
}
$dbFilePath = Get-IPDataPath
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
Write-Host "Converting CSV file $dbFilePath to $outputFile"
$object = [PSCustomObject]#{
'ip-address' = ''
'is-vpn' = 'true'
'#timestamp' = ''
}
# Enclose foreach loop in a script block to be able to pipe its output to Set-Content
& {
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
# Implicit output
$object | ConvertTo-Json -Compress -Depth 100
}
}
} | Set-Content -Path $outputFile
Remarks for improving performance:
Avoid Get-Content, especially for line-by-line processing it tends to be slow. A much faster alternative is the File.ReadLines method. To skip the header line, use the Linq.Enumerable.Skip() method.
There is no need to read the whole CSV into memory first. Using ReadLines in a foreach loop does lazy enumeration, i. e. it reads only one line per loop iteration. This works because it returns an enumerator instead of a collection of lines.
Avoid try and catch if exceptions occur often, because the "exceptional" code path is very slow. Instead use Int64.TryParse() which returns a boolean indicating successful conversion.
Instead of "manually" converting the IP number to bytes, use the IPAddress class which has a constructor that takes an integer number. Use its method .GetAddressBytes() to get an array of bytes in network (big-endian) order. Finally use the PowerShell -join operator to create a string of the expected format.
Don't allocate a [pscustomobject] for each row, which has some overhead. Create it once before the loop and inside the loop only assign the values.
Avoid Write-Host (or any output to the console) within inner loops.
Unrelated to performance:
I've removed the New-Item call to create the output file, which isn't necessary because Set-Content automatically creates the file if it doesn't exist.
Note that the output is in NDJSON format, where each line is like a JSON file. In case you actually want this to be a regular JSON file, enclose the output in [ ] and insert a comma , between each row.
Modified processing loop to write a regular JSON file instead of NDJSON file:
& {
'[' # begin array
$first = $true
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
$row = $object | ConvertTo-Json -Compress -Depth 100
# write array element delimiter if necessary
if( $first ) { $row; $first = $false } else { ",$row" }
}
}
']' # end array
} | Set-Content -Path $outputFile
You can optimize the function Convert-NumberToIP like below:
function Convert-NumberToIP {
param(
[Parameter(Mandatory=$true)][uint32]$number
)
# either do the math yourself like this:
# $w = ($number -shr 24) -band 255
# $x = ($number -shr 16) -band 255
# $y = ($number -shr 8) -band 255
# $z = $number -band 255
# '{0}.{1}.{2}.{3}' -f $w, $x, $y, $z # output the dotted IP string
# or use .Net:
$n = ([IPAddress]$number).GetAddressBytes()
[array]::Reverse($n)
([IPAddress]$n).IPAddressToString
}
Good evening.
I'm trying to use parallelism for the first time but I don't understand how to call a function within foreach loop.
I get a series of this error: Cannot bind argument to parameter 'Path' because it is null.
This is what I've done so far:
$FolderPath = "C:\myfolder\"
function AppendLog ($client) {
$so = New-CimSessionOption -Protocol 'DCOM'
$s = New-CimSession -ComputerName $client -SessionOption $so
Add-Content -Path (join-path $folderpath "LOGS.txt") -Value ( (get-date -Format "[yyyy.MM.dd HH:mm:ss]").tostring() + $client + " -PING: OK")
$arch = Get-CimInstance –query "select * from win32_operatingsystem" -CimSession $s | select -expandproperty osarchitecture
Add-Content -Path (join-path $folderpath "LOGS.txt") -Value ( (get-date -Format "[yyyy.MM.dd HH:mm:ss]").tostring() + $client + " -ARCH:" + $arch )
$lastboot = Get-CimInstance –query "select * from win32_operatingsystem" -CimSession $s | select -expandproperty lastbootuptime
Add-Content -Path (join-path $folderpath "LOGS.txt") -Value ( (get-date -Format "[yyyy.MM.dd HH:mm:ss]").tostring() + $client + " -BOOT:" + $lastboot )
}
$funcDef = $function:AppendLog.ToString()
$clients = get-content -path (join-path $folderPath "client.txt")
$clients | ForEach-Object -parallel {
if (test-connection $_ -count 2 -Quiet)
{
$function:AppendLog = $using:funcDef
AppendLog ($_)
}
} -throttlelimit 3
Could you explain me how to pass my path?
My bad on the comment, the error you're getting is most likely coming from your function. The error is being thrown by Join-Path:
PS /> Join-Path $null 'Logs.txt'
Join-Path : Cannot bind argument to parameter 'Path' because it is null.
At line:1 char:11
+ Join-Path $null 'Logs.txt'
+ ~~~~~
+ CategoryInfo : InvalidData: (:) [Join-Path], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.JoinPathCommand
The reason is because $FolderPath doesn't exist in the scope of your parallel loop. $folderpath should be replace with $using:folderpath inside your function.
As a side note, adding information to the same file on a parallel execution doesn't seem to be a good idea.
Last edit, I understand if this is meant to test how ForEach-Object -Parallel works but again, if the cmdlet allows remote querying / remote execution with multiple hosts at the same time, let the cmdlet handle that for you, it is more efficient.
As for the code, this is what I would use with what you already have:
$FolderPath = "C:\myfolder\"
$sessionOption = New-CimSessionOption -Protocol 'DCOM'
$clients = Get-Content -Path (Join-Path $FolderPath -ChildPath "Client.txt")
$results = $clients | ForEach-Object -Parallel {
$out = #{
Time = [datetime]::Now.ToString('[yyyy.MM.dd HH:mm:ss]')
ComputerName = $_
}
if ($ping = Test-Connection $_ -Count 2 -Quiet)
{
$session = New-CimSession -ComputerName $_ -SessionOption $using:sessionOption
$OSInfo = Get-CimInstance -CimSession $session -ClassName win32_operatingSystem
Remove-CimSession $session
}
$out.Ping = $ping
$out.Arch = $OSInfo.OSArchitecture
$out.LastBoot = $OSInfo.LastBootUpTime
[pscustomobject]$out
} -ThrottleLimit 3
$results | Export-Csv "$myFolder\LOGS.csv" -NoTypeInformation
This will output an object like this below:
Time ComputerName Ping OSArchitecture LastBoot
---- ------------ ---- -------------- --------
[2021.06.19 20:06:00] ComputerName01 True 64-bit 6/16/2021 11:47:16 AM
[2021.06.19 20:07:00] ComputerName02 False
[2021.06.19 20:08:00] ComputerName03 True 64-bit 6/13/2021 11:47:16 AM
[2021.06.19 20:09:00] ComputerName04 True 64-bit 6/14/2021 11:47:16 AM
[2021.06.19 20:10:00] ComputerName05 True 64-bit 6/15/2021 11:47:16 AM
Which can be exported nicely to a CSV instead of a text file. P.D.: sorry for the syntax highlighting :(
Apologize for the length..
Trying to modify a script that currently uses robocopy to report a folder path's size, child folder count, and size. This script currently takes about 24 hrs to run as it goes one by one to each folder. I'm trying to implement invoke-command to set-up jobs so that it will run at least 10 instances of robocopy in the hopes if drastically reducing run time.
I've tried multiple variances given multiple resources to set parameters, call a function and get the results back with no success.
with current code (below) getting error;
Invoke-Command : Parameter set cannot be resolved using the specified named parameters.
ERROR: + Invoke-Command -ScriptBlock { param ($item,$Filter,$params) $ ...
ERROR: + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: + CategoryInfo : InvalidArgument: (:) [Invoke-Command], ParameterBindingException
ERROR: + FullyQualifiedErrorId : AmbiguousParameterSet,Microsoft.PowerShell.Commands.InvokeCommandCommand
Function ListFolder ($Folder, $FilterSet, $Roboparam)
{
write-host $Folder
Try
{
$Folder = (Resolve-Path -LiteralPath $Folder -ErrorAction Stop).ProviderPath
If (-Not (Test-Path -LiteralPath $Folder -Type Container -ErrorAction Stop))
{Write-Warning ("{0} is not a directory and will be skipped" -f $Folder)
Return
}
$Script = robocopy $Folder NULL $FilterSet $FRoboparam
$exit_code = $LASTEXITCODE
16, 8, 4, 2, 1 | % {
Switch ($exit_code -band $_)
{
16 { $exit_reason = "Usage error or insufficient access privileges" }
8 { $exit_reason = "Retry limit exceeded" }
4 { $exit_reason = "Some Mismatched files or directories were detected" }
2 { $exit_reason = "Some Extra files or directories were detected. No files were copied" }
1 { $exit_reason = " " }
}
}
If ($exit_code -eq 0)
{$exit_reason = 'No Change'}
If ($Script[-6] -match $dirPattern)
{$Dir = $matches.Dir}
Else
{$Dir = 0}
If ($Script[-5] -match $countPattern)
{$Count = $matches.Count}
Else
{$Count = 0}
If ($Count -gt 0)
{If ($Script[-4] -match $sizePattern)
{$FSize = $matches.Size}}
Else
{$FSize = 0}
$Script:TotSize += $FSize
$Script:Report += New-Object PSObject -Property #{
'Folder Name' = $Folder
Files = "{0:N0}" -f [int]$Count
Folders = "{0:N0}" -f [int]$Dir
Size = "{0:N0}" -f [long]$FSize
Comment = $exit_reason}
Clear-Variable -Name Dir
Clear-Variable -Name Count
Clear-Variable -Name FSize
If ($exit_reason) { Clear-Variable -Name exit_reason }}
Catch
{$Script:Report += New-Object PSObject -Property #{
'Folder Name' = $Folder
Files = "{0:N0}" -f [int]$Count
Folders = "{0:N0}" -f [int]$Dir
Size = [long]$FSize
Comment = [string]$_.Exception.Message}
Continue
}
Return #$Script
}
}
$params = New-Object System.Collections.Arraylist
$params.AddRange(#("/L","/S","/NJH","/BYTES","/FP","/NFL","/NC","/NDL","/TS","/XJ","/R:0","/W:0"))
$dirPattern = "^\s{4}Dirs\s:\s+(?<Dir>\d+).*"
$countPattern = "^\s{3}Files\s:\s+(?<Count>\d+).*"
$sizePattern = "^\s{3}Bytes\s:\s+(?<Size>\d+(?:\.?\d+)).*"
If ($PSBoundParameters['Force']) {$FileDate = 10}
If ($FileDate -lt 6){Exit}
Else
{$Paths = Get-Content $InputFile
ForEach ($item in $Paths)
{
$MaxThreads = 10
While (#(Get-Job | where { $_.State -eq "Running" }).Count -ge $MaxThreads)
{
Write-Host "Waiting for open thread...($MaxThreads Maximum)"
Start-Sleep -Seconds 3
}
Invoke-Command -Computer . -ScriptBlock { param ($item,$Filter,$params) ${function:ListFolder} } -ArgumentList $item, $Filter, $params -AsJob
}
}
When all said and done I have a collection of $Report to show folder name, count of files, count of folders, size in bytes, and comments tho show any errors to Excel
$Report | Sort-Object {[long]$_.Size} -descending | Select 'Folder Name', Files, Folders, Size, Comment | Export-Csv -Path $ReportPath\$(Get-Date -uformat "%Y_%m_%d")-FileSizes.csv -Encoding ascii -NoTypeInformation
I am using PowerShell to compare the file count and size (per file extension) in 2 separate directories.
$User = $env:username
$pwd = pwd
clear
write-host "`n"
write-host "`n"
write-host "`n"
write "The current user is: $User"
write-host "`n"
write "The current path is: $pwd"
write-host "`n"
write-host "`n"
write-host "`n"
write "We need to know the following information:"
write "`n"
write "`n"
$UserDesktopPath = Read-Host "New PC User Desktop Path" # This should be the new PC Desktop Path
$UserDocumentPath = Read-Host "New PC User Document Path" # This should be the new PC Document Path
$USBDesktopPathServer = Read-Host "USB User Desktop Path" # This should be the USB User Desktop Path
$USBDocumentPathServer = Read-Host "USB User Document Path" # This should be the USB User Document Path
clear
write-host "`n"
write-host "`n"
write-host "`n"
write "This is the results for your Desktop Folder Paths:"
write-host "`n"
$folder_new = Get-ChildItem -Recurse -path "$USBDesktopPathServer" # Recurses the New PC Desktop
$folder_old = Get-ChildItem -Recurse -path "$UserDesktopPath" # Recurses the USB Backup Desktop
Compare-Object -ReferenceObject "$folder_new" -DifferenceObject "$folder_old" # Compares the two folders for the path to identify discrepancies
write-host "`n"
write "This is the results for your Documents Folder Paths:"
write-host "`n"
write-host "`n"
write-host "`n"
$folder_new1 = Get-ChildItem -Recurse -path "$UserDocumentPath" # Recurses the New PC Documents
$folder_old1 = Get-ChildItem -Recurse -path "$USBDocumentPathServer" # Recurses the USB Backup Documents
Compare-Object -ReferenceObject "$folder_new1" -DifferenceObject "$folder_old1" # Compares the two folders for the path to identify discrepancies
write-host "`n"
write-host "`n"
write-host "`n"
write "Now we shall compare file sizes of your Documents:"
write-host "`n"
write-host "`n"
write-host "`n"
write-host "`n"
function doc{
$DirectoryDocuments = "$USBDocumentPathServer", "$UserDocumentPath"
foreach ($Directory in $DirectoryDocuments) {
Get-ChildItem -Path $Directory -Recurse |
Where-Object {-not $_.PSIsContainer} |
Tee-Object -Variable Files |
Group-Object -Property Extension |
Select-Object -Property #{
n = "Directory"
e = {$Directory}
},
#{
n = "Extension"
e = { $_.Name -replace '^\.' }
},
#{
n = "Size (MB)"
e={ [math]::Round( ( ( $_.Group | Measure-Object Length -Sum ).Sum / 1MB ), 2 ) }
},
Count
$Files |
Measure-Object -Sum -Property Length |
Select-Object -Property #{
n = 'Extension'
e = { 'Total' }
},
#{
n = 'Size (MB)'
e = { [math]::Round( ( $_.Sum / 1MB ), 2 ) }
},
Count
}
}
When using the ISE and calling dtop I get the correct return:
PS C:\Users\Michael Nancarrow> dtop
Directory Extension Size (MB) Count
--------- --------- --------- -----
D:\Deployment Kit\Test\Desktop2 txt 0 1
Total 0 1
D:\Deployment Kit\Test\Desktop1 txt 0 11
Total 0 11
Yet when run in the script, it does not return any value. I have attempted to call a function write $tst which runs dtop and that does the same (writes null).
Furthermore, I have removed the { so it does not run as a function, and it operates without an issue. My concern is perhaps the -Path file cannot be parsed at the same time as the input - meaning: when I call dtop from ISE it already has the $Directory variable stored in memory.
Are there any obvious errors here? I am rather new to PowerShell and am unsure where the mistake lies.
I've written the following PS script to delete log files from specific server paths. I'm a novice to PS but I'm getting some errors with a few of the functions that I have written in this script:
#* FileName: FileCleaner.ps1
#Clear the screen
Clear
#Read XML Config File to get settings
[xml]$configfile = Get-Content "C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell Script\FileCleaner.config.xml"
#Declare and set variables from Config values
$hostServer = $configfile.Settings.HostServer
$dirs = #($configfile.Settings.DirectoryName.Split(",").Trim())
$scanSubDirectories = $configfile.Settings.ScanSubDirectories
$deleteAllFiles = $configfile.Settings.deleteAllFiles
$fileTypesToDelete = #($configfile.Settings.FileTypesToDelete.Split(";").Trim())
$liveSiteLogs = $configfile.Settings.LiveSiteLogs
$fileExclusions = #($configfile.Settings.FileExclusions.Split(";").Trim())
$retentionPeriod = $configfile.Settings.RetentionPeriod
$AICLogs = $configfile.Settings.AICLogs
$AICLogsRententionPeriod = $configfile.Settings.AICLogsRententionPeriod
$fileCleanerLogs = $configfile.Settings.FileCleanerLogs
$fileCleanerLogsRententionPeriod = $configfile.Settings.FileCleanerLogsRententionPeriod
#Setup FileCleaner output success logfiles
$successLogfile = $configfile.Settings.SuccessOutputLogfile
$dirName = [io.path]::GetDirectoryName($successLogfile)
$filename = [io.path]::GetFileNameWithoutExtension($successLogfile)
$ext = [io.path]::GetExtension($successLogfile)
$successLogfile = "$dirName\$filename$(get-date -Format yyyy-MM-dd)$ext"
#Setup FileCleaner output error logfiles
$errorLogfile = $configfile.Settings.ErrorOutputLogfile
$dirName = [io.path]::GetDirectoryName($errorLogfile)
$filename = [io.path]::GetFileNameWithoutExtension($errorLogfile)
$ext = [io.path]::GetExtension($errorLogfile)
$errorLogfile = "$dirName\$filename$(get-date -Format yyyy-MM-dd)$ext"
#Setup Retention Period
$LastWrite = (Get-Date).AddDays(-$retentionPeriod)#.ToString("d")
$AICLastWrite = (Get-Date).AddDays(-$AICLogsRententionPeriod)#.ToString("d")
$fileCleanerLastWrite = (Get-Date).AddDays(-$fileCleanerLogsRententionPeriod)
#EMAIL SETTINGS
$smtpServer = $configfile.Settings.SMTPServer
$emailFrom = $configfile.Settings.EmailFrom
$emailTo = $configfile.Settings.EmailTo
$emailSubject = $configfile.Settings.EmailSubject
#Update the email subject to display the Host Server value
$emailSubject -replace "HostServer", $hostServer
$countUnaccessibleUNCPaths = 0
#Check Logfiles exists, if not create them
if(!(Test-Path -Path $successLogfile))
{
New-Item -Path $successLogfile –itemtype file
}
if(!(Test-Path -Path $errorLogfile))
{
New-Item -Path $errorLogfile –itemtype file
}
foreach ($dir in $dirs)
{
#needs a check to determine if server/the UNC Path is accessible. If it fails to connect, it needs to move on to the next UNC share but a flag needs to
#be generate to alert us to investigate why the UNC share was not accessible during the job run.
If(Test-Path -Path $dir)
{
#write to output logfile Directory info
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $dir"
$Msg | out-file $successLogfile
If ($scanSubDirectories -eq "True")
{
If ($deleteAllFiles -eq "True")
{
#ScanSubDirectories and delete all files older than the $retentionPeriod, include Sub-Directories / also forces the deletion of any hidden files
$logFiles = Get-ChildItem -Path $dir -Force -Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$LastWrite" }
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
Else
{
#"ScanSubDirectories but only delete specified file types."
$logFiles = Get-Childitem $dir -Include $fileTypesToDelete[0],$fileTypesToDelete[1],$fileTypesToDelete[2], $liveSiteLogs -Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where {$_.LastWriteTime -le "$LastWrite"}
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
}
Else
{
#Only delete files in top level Directory
If ($deleteAllFiles -eq "True")
{
$logFiles = Get-ChildItem -Path $dir -Force -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$LastWrite" }
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
Else
{
$logFiles = Get-Childitem $dir -Include $fileTypesToDelete[0],$fileTypesToDelete[1],$fileTypesToDelete[2], $liveSiteLogs -Exclude $fileExclusions[0],$fileExclusions[1] | Where {$_.LastWriteTime -le "$LastWrite"}
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
}
}
Else
{
$countUnaccessibleUNCPaths++
#server/the UNC Path is unaccessible
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") Unable to access $dir."
$Msg | out-file $errorLogfile -append
}
# Call the function to Delete the AIC XML Logfiles
DeleteAICXMLLogs $dir
}
#If any of the directories were unaccessible send an email to alert the team
if($countUnaccessibleUNCPaths.count -gt 0)
{
# Call the function to send the email
SendEmail $emailSubject $emailFrom $emailTo
}
#Only keep 2 weeks worth of the FileCleaner App logs for reference purposes
If(Test-Path -Path $fileCleanerLogs)
{
#write to output logfile Directory info
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $fileCleanerLogs"
$Msg | out-file $successLogfile
$fileCleanerLogs = Get-Childitem $fileCleanerLogs -Recurse | Where {$_.LastWriteTime -le "$fileCleanerLastWrite"}
DeleteLogFiles($fileCleanerLogs)
#foreach($fileCleanerLog in $fileCleanerLogs)
#{
# if($fileCleanerLog -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $fileCleanerLog")"
# $Msg | out-file $successLogfile -append
# Remove-Item $fileCleanerLog.FullName -Force
# }
#}
}
Function DeleteLogFiles($logFiles)
{
foreach($logFile in $logFiles)
{
if($logFile -ne $null)
{
$Msg = Write-Output "$("Deleting File $logFile")"
$Msg | out-file $successLogfile -append
Remove-Item $logFile.FullName -Force
}
}
}
Function DeleteAICXMLLogs($dir)
{
#Split the UNC path $dir to retrieve the server value
$parentpath = "\\" + [string]::join("\",$dir.Split("\")[2])
#test access to the \\server\D$\DebugXML path
If(Test-Path -Path $parentpath$AICLogs)
{
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $parentpath$AICLogs"
$Msg | out-file $successLogfile
#Concantenate server value to $AICLogs to delete all xml logs in \\server\D$\DebugXML with a retention period of 30Days
$XMLlogFiles = Get-ChildItem -Path $parentpath$AICLogs -Force -Include $fileTypesToDelete[3]-Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$AICLastWrite" }
#get each file and add the filename to be deleted to the successLogfile before deleting the file
DeleteLogFiles($XMLlogFiles)
#foreach($XMLlogFile in $XMLlogFiles)
#{
# if($XMLlogFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $XMLlogFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $XMLlogFile.FullName -Force
# }
#}
}
Else
{
$Msg = Write-Output "$("$parentpath$AICLogs does not exist.")"
$Msg | out-file $successLogfile -append
}
}
Function SendEmail($emailSubject, $emailFrom, $emailTo)
{
$MailMessage = New-Object System.Net.Mail.MailMessage
$SMTPClient = New-Object System.Net.Mail.smtpClient
$SMTPClient.host = $smtpServer
$Recipient = New-Object System.Net.Mail.MailAddress($emailTo, "Recipient")
$Sender = New-Object System.Net.Mail.MailAddress($emailFrom, "Sender")
$MailMessage.Sender = $Sender
$MailMessage.From = $Sender
$MailMessage.Subject = $emailSubject
$MailMessage.Body = #"
This email was generated because the FileCleaner script was unable to access some UNC Paths, please refer to $errorLogfile for more information.
Please inform the Team if you plan to resolve this.
This is an automated email please do not respond.
"#
$SMTPClient.Send($MailMessage)
}
when debugging I'm getting these errors:
DeleteAICXMLLogs : The term 'DeleteAICXMLLogs' is not recognized as
the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify
that the path is correct and try again. At
C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell
Script\FileCleaner.ps1:158 char:5
+ DeleteAICXMLLogs $dir
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (DeleteAICXMLLogs:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
SendEmail : The term 'SendEmail' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the spelling
of the name, or if a path was included, verify that the path is
correct and try again. At C:\Users\pmcma\Documents\Projects\Replace
FileCleaner with PowerShell Script\FileCleaner.ps1:164 char:5
+ SendEmail $emailSubject $emailFrom $emailTo
+ ~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (SendEmail:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
DeleteLogFiles : The term 'DeleteLogFiles' is not recognized as the
name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the
path is correct and try again. At
C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell
Script\FileCleaner.ps1:175 char:5
+ DeleteLogFiles($fileCleanerLogs)
+ ~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (DeleteLogFiles:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I don't see anything wrong with how I'm declaring the functions or calling them. Any ideas why this script is failing?
PowerShell Scripts are read from the top to the bottom, so you can't use any references before they are defined, most probably that is why you are receiving errors.
Try adding your function definition blocks above the point where you call them.
Alternatively you can make a function having global scope. Just preface the function name with the keyword global: like,
function global:test ($x, $y)
{
$x * $y
}
I've had this happen as well. Try placing the functions before the business logic. This is a script, not compiled code. So the functions are yet to be declared before you are calling them.