Good evening.
I'm trying to use parallelism for the first time but I don't understand how to call a function within foreach loop.
I get a series of this error: Cannot bind argument to parameter 'Path' because it is null.
This is what I've done so far:
$FolderPath = "C:\myfolder\"
function AppendLog ($client) {
$so = New-CimSessionOption -Protocol 'DCOM'
$s = New-CimSession -ComputerName $client -SessionOption $so
Add-Content -Path (join-path $folderpath "LOGS.txt") -Value ( (get-date -Format "[yyyy.MM.dd HH:mm:ss]").tostring() + $client + " -PING: OK")
$arch = Get-CimInstance –query "select * from win32_operatingsystem" -CimSession $s | select -expandproperty osarchitecture
Add-Content -Path (join-path $folderpath "LOGS.txt") -Value ( (get-date -Format "[yyyy.MM.dd HH:mm:ss]").tostring() + $client + " -ARCH:" + $arch )
$lastboot = Get-CimInstance –query "select * from win32_operatingsystem" -CimSession $s | select -expandproperty lastbootuptime
Add-Content -Path (join-path $folderpath "LOGS.txt") -Value ( (get-date -Format "[yyyy.MM.dd HH:mm:ss]").tostring() + $client + " -BOOT:" + $lastboot )
}
$funcDef = $function:AppendLog.ToString()
$clients = get-content -path (join-path $folderPath "client.txt")
$clients | ForEach-Object -parallel {
if (test-connection $_ -count 2 -Quiet)
{
$function:AppendLog = $using:funcDef
AppendLog ($_)
}
} -throttlelimit 3
Could you explain me how to pass my path?
My bad on the comment, the error you're getting is most likely coming from your function. The error is being thrown by Join-Path:
PS /> Join-Path $null 'Logs.txt'
Join-Path : Cannot bind argument to parameter 'Path' because it is null.
At line:1 char:11
+ Join-Path $null 'Logs.txt'
+ ~~~~~
+ CategoryInfo : InvalidData: (:) [Join-Path], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.JoinPathCommand
The reason is because $FolderPath doesn't exist in the scope of your parallel loop. $folderpath should be replace with $using:folderpath inside your function.
As a side note, adding information to the same file on a parallel execution doesn't seem to be a good idea.
Last edit, I understand if this is meant to test how ForEach-Object -Parallel works but again, if the cmdlet allows remote querying / remote execution with multiple hosts at the same time, let the cmdlet handle that for you, it is more efficient.
As for the code, this is what I would use with what you already have:
$FolderPath = "C:\myfolder\"
$sessionOption = New-CimSessionOption -Protocol 'DCOM'
$clients = Get-Content -Path (Join-Path $FolderPath -ChildPath "Client.txt")
$results = $clients | ForEach-Object -Parallel {
$out = #{
Time = [datetime]::Now.ToString('[yyyy.MM.dd HH:mm:ss]')
ComputerName = $_
}
if ($ping = Test-Connection $_ -Count 2 -Quiet)
{
$session = New-CimSession -ComputerName $_ -SessionOption $using:sessionOption
$OSInfo = Get-CimInstance -CimSession $session -ClassName win32_operatingSystem
Remove-CimSession $session
}
$out.Ping = $ping
$out.Arch = $OSInfo.OSArchitecture
$out.LastBoot = $OSInfo.LastBootUpTime
[pscustomobject]$out
} -ThrottleLimit 3
$results | Export-Csv "$myFolder\LOGS.csv" -NoTypeInformation
This will output an object like this below:
Time ComputerName Ping OSArchitecture LastBoot
---- ------------ ---- -------------- --------
[2021.06.19 20:06:00] ComputerName01 True 64-bit 6/16/2021 11:47:16 AM
[2021.06.19 20:07:00] ComputerName02 False
[2021.06.19 20:08:00] ComputerName03 True 64-bit 6/13/2021 11:47:16 AM
[2021.06.19 20:09:00] ComputerName04 True 64-bit 6/14/2021 11:47:16 AM
[2021.06.19 20:10:00] ComputerName05 True 64-bit 6/15/2021 11:47:16 AM
Which can be exported nicely to a CSV instead of a text file. P.D.: sorry for the syntax highlighting :(
Related
I am trying to read the data from CSV file which has 2200000 records using PowerShell and storing each record in JSON file, but this takes almost 12 hours.
Sample CSV Data:
We will only concern about the 1st column value's.
Code:
function Read-IPData
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object{ $_.FullName }
Write-Host "file path - $dbFilePath"
Write-Host "Reading..."
$data = Get-Content -Path $dbFilePath | Select-Object -Skip 1
Write-Host "Reading data finished"
$count = $data.Count
Write-host "Total $count records found"
return $data
}
function Convert-NumbetToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
try
{
$w = [int64]($number/16777216)%256
$x = [int64]($number/65536)%256
$y = [int64]($number/256)%256
$z = [int64]$number%256
$ipAddress = "$w.$x.$y.$z"
Write-Host "IP Address - $ipAddress"
return $ipAddress
}
catch
{
Write-Host "$_"
continue
}
}
Write-Host "Getting IP Addresses from $dbFileName"
$data = Read-IPData
Write-Host "Checking whether output.json file exist, if not create"
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
if(!(Test-Path $outputFile))
{
Write-Host "$outputFile doestnot exist, creating..."
New-Item -Path $outputFile -type "file"
}
foreach($item in $data)
{
$row = $item -split ","
$ipNumber = $row[0].trim('"')
Write-Host "Converting $ipNumber to ipaddress"
$toIpAddress = Convert-NumbetToIP -number $ipNumber
Write-Host "Preparing document JSON"
$object = [PSCustomObject]#{
"ip-address" = $toIpAddress
"is-vpn" = "true"
"#timestamp" = (Get-Date).ToString("o")
}
$document = $object | ConvertTo-Json -Compress -Depth 100
Write-Host "Adding document - $document"
Add-Content -Path $outputFile $document
}
Could you please help optimize the code or is there a better way to do it. or is there a way like multi-threading.
Here is a possible optimization:
function Get-IPDataPath
{
$dbFilePath = Get-ChildItem -Path $rootDir -Filter "IP2*.CSV" | ForEach-Object FullName | Select-Object -First 1
Write-Host "file path - $dbFilePath"
$dbFilePath # implicit output
}
function Convert-NumberToIP
{
param(
[Parameter(Mandatory=$true)][string]$number
)
[Int64] $numberInt = 0
if( [Int64]::TryParse( $number, [ref] $numberInt ) ) {
if( ($numberInt -ge 0) -and ($numberInt -le 0xFFFFFFFFl) ) {
# Convert to IP address like '192.168.23.42'
([IPAddress] $numberInt).ToString()
}
}
# In case TryParse() returns $false or the number is out of range for an IPv4 address,
# the output of this function will be empty, which converts to $false in a boolean context.
}
$dbFilePath = Get-IPDataPath
$outputFile = Join-Path -Path $rootDir -ChildPath "output.json"
Write-Host "Converting CSV file $dbFilePath to $outputFile"
$object = [PSCustomObject]#{
'ip-address' = ''
'is-vpn' = 'true'
'#timestamp' = ''
}
# Enclose foreach loop in a script block to be able to pipe its output to Set-Content
& {
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
# Implicit output
$object | ConvertTo-Json -Compress -Depth 100
}
}
} | Set-Content -Path $outputFile
Remarks for improving performance:
Avoid Get-Content, especially for line-by-line processing it tends to be slow. A much faster alternative is the File.ReadLines method. To skip the header line, use the Linq.Enumerable.Skip() method.
There is no need to read the whole CSV into memory first. Using ReadLines in a foreach loop does lazy enumeration, i. e. it reads only one line per loop iteration. This works because it returns an enumerator instead of a collection of lines.
Avoid try and catch if exceptions occur often, because the "exceptional" code path is very slow. Instead use Int64.TryParse() which returns a boolean indicating successful conversion.
Instead of "manually" converting the IP number to bytes, use the IPAddress class which has a constructor that takes an integer number. Use its method .GetAddressBytes() to get an array of bytes in network (big-endian) order. Finally use the PowerShell -join operator to create a string of the expected format.
Don't allocate a [pscustomobject] for each row, which has some overhead. Create it once before the loop and inside the loop only assign the values.
Avoid Write-Host (or any output to the console) within inner loops.
Unrelated to performance:
I've removed the New-Item call to create the output file, which isn't necessary because Set-Content automatically creates the file if it doesn't exist.
Note that the output is in NDJSON format, where each line is like a JSON file. In case you actually want this to be a regular JSON file, enclose the output in [ ] and insert a comma , between each row.
Modified processing loop to write a regular JSON file instead of NDJSON file:
& {
'[' # begin array
$first = $true
foreach( $item in [Linq.Enumerable]::Skip( [IO.File]::ReadLines( $dbFilePath ), 1 ) )
{
$row = $item -split ','
$ipNumber = $row[0].trim('"')
if( $ip = Convert-NumberToIP -number $ipNumber )
{
$object.'ip-address' = $ip
$object.'#timestamp' = (Get-Date).ToString('o')
$row = $object | ConvertTo-Json -Compress -Depth 100
# write array element delimiter if necessary
if( $first ) { $row; $first = $false } else { ",$row" }
}
}
']' # end array
} | Set-Content -Path $outputFile
You can optimize the function Convert-NumberToIP like below:
function Convert-NumberToIP {
param(
[Parameter(Mandatory=$true)][uint32]$number
)
# either do the math yourself like this:
# $w = ($number -shr 24) -band 255
# $x = ($number -shr 16) -band 255
# $y = ($number -shr 8) -band 255
# $z = $number -band 255
# '{0}.{1}.{2}.{3}' -f $w, $x, $y, $z # output the dotted IP string
# or use .Net:
$n = ([IPAddress]$number).GetAddressBytes()
[array]::Reverse($n)
([IPAddress]$n).IPAddressToString
}
I am trying to create a program that will copy files from our backup to archive directory using Powershell. I have two criteria in order for this program to run smoothly. One is that we have files from both current year and past years so only the files from this year must be copied over. Another is that we have to check to make sure that we are not copying over files of the same file name in case if the data in the file is accidentally modified. Whenever I have this program not in a function, it works. But in a function, it gives me errors that it "cannot find the path" of the folder that I am copying from and the folder that I'm pasting the files to. I am going to use this for more than sixty locations, so it would be better that I don't have to rewrite the code in the function sixty times. I thought about using Robocopy, but I am still getting the same issues regardless with files not being copied over.
Function Copy-Data {
param (
[system.object]$copyFolder,
[system.object]$pasteFolder,
[int]$currentYear,
[int]$lastYear,
[int]$nextYear)
$copyItem = Get-ChildItem -Path $copyFolder
$pasteItem = Get-ChildItem -Path $pasteFolder
$copyCount = $copyItem.count
for ($i = 0; $i -lt $copyCount; $i++)
{
$copyName = $copyItem.Name
$testPath = Test-Path "$pasteFolder$copyName"
if ($copyItem[$i].LastWriteTime -gt $firstDate -and $copyItem[$i].LastWriteTime -lt $lastDate)
{
if ($testPath -eq $false)
{
Copy-Item -Path $copyFolder$copyName -Destination $pasteFolder
#Robocopy "$copyFolder$copyItem[$i]" "$pasteFolder"
Write-Host $pasteFolder$copyName
}
}
}
}
$currentYear = Get-Date -Format "yyyy"
$lastYear = [int]$currentYear - 1
$nextYear = [int]$currentYear + 1
$firstDate = "12/31/$lastYear"
$lastDate = "01/01/$nextYear"
$copyFolder = "\\fileshare\test\copy\"
$pasteFolder = "\\fileshare\test\$currentYear\paste\"
Copy-Data ($copyFolder, $pasteFolder, $currentYear, $lastYear, $nextYear)
I feel like you are making this more complicated than it needs to be.
Core code can be something like:
Get-ChildItem -Path $copyFolder |
ForEach-Object{
If( !(Test-Path $pasteFolder -Name $_.Name ) )
{
Copy-Item $_.FullName -Destination $pasteFolder
}
}
You can use just the destination path you don't have to give the full path of the destination file.
As a function it may look something like:
Function CopyFoldercontents
{
Param(
[Parameter( Mandatory = $true, Position = 0)]
[String]$copyFolder,
[Parameter( Mandatory = $true, Position = 1)]
[String]$pasteFolder
) # End Parameter Block.
Get-ChildItem -Path $copyFolder |
ForEach-Object{
If( !(Test-Path $pasteFolder -Name $_.Name ) )
{
Copy-Item $_.FullName -Destination $pasteFolder
}
}
} # End Function CopyFolderContents
This could be more robust though, depends on what direction you want to take it.
Continuing from my comment.
You could just do this...
(validate what you are after on both sides, the construct the final function)
Function Start-FolderMirror
{
[CmdletBinding()]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles'
# Results
<#
InputObject SideIndicator
----------- -------------
D:\temp\TestFiles\abc - Copy - Copy.bat =>
D:\temp\TestFiles\abc - Copy.bat =>
D:\temp\TestFiles\abc.bat =>
D:\temp\(MSINFO32) command-line tool switches.pdf <=
D:\temp\23694d1213305764-revision-number-in-excel-book1.xls <=
#>
Function Start-FolderMirror
{
[CmdletBinding(SupportsShouldProcess)]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5 |
Where-Object -Property SideIndicator -Match '<='
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles' -WhatIf
# Results
<#
InputObject SideIndicator
----------- -------------
D:\temp\(MSINFO32) command-line tool switches.pdf <=
D:\temp\23694d1213305764-revision-number-in-excel-book1.xls <=
#>
Function Start-FolderMirror
{
[CmdletBinding(SupportsShouldProcess)]
[Alias('mir')]
Param
(
[string]$SourcePath = (Read-Host -Prompt 'Enter a source path'),
[string]$DestinationPath = (Read-Host -Prompt 'Enter a destination path')
)
$SourceFiles = (Get-ChildItem -Path $SourcePath -File).FullName
$DestinationFiles = (Get-ChildItem -Path $DestinationPath -File).FullName
Compare-Object -ReferenceObject $SourceFiles -DifferenceObject $DestinationFiles -IncludeEqual |
Select-Object -First 5 |
Where-Object -Property SideIndicator -Match '<=' |
ForEach {Copy-Item -Path $PSItem.InputObject -Destination $DestinationPath}
}
Start-FolderMirror -SourcePath 'd:\temp' -DestinationPath 'D:\temp\TestFiles' -WhatIf
# Results
<#
What if: Performing the operation "Copy File" on target "Item: D:\temp\(MSINFO32) command-line tool switches.pdf Destination: D:\temp\TestFiles\(MSINFO32) command-line tool switches.pdf".
What if: Performing the operation "Copy File" on target "Item: D:\temp\23694d1213305764-revision-number-in-excel-book1.xls Destination: D:\temp\TestFiles\23694d1213305764-revision-number-in-excel-book1.xls".
#>
So I have a XML value extractor for a large rule database.
I stored the values in a Json file with sections for each file and then the values tied to their names.
However some of the values are a CDATA Snippet.
I can pull the CDATA as a string literal to store in the Json but I CANNOT seem to find a way to get power shell to let me put it back into the XML.
Cannot set "Value" because only strings can be used as values to set XmlNode properties.
At C:\Users\vagrant\Desktop\RuleSetter.ps1:24 char:21
+ $rule.value = $ruleFileSet. ($singleRuleXmlFile.directory.nam ...
+
+ CategoryInfo : NotSpecified: (:) [], SetValueException
+ FullyQualifiedErrorId : XmlNodeSetShouldBeAString`
The Getter looks like this
$directories = Get-ChildItem -dir -path C:\inetpub\wwwroot\
foreach($instName in $directories){
if($instName -notlike 'client' -and $instName -notlike 'bin'-and $instName -notlike 'aspnet_client'-and $instName -notlike 'AccessLibertyLogs'){
$obj = [hashtable]#{}
$head = [hashtable]#{Institution = $instName}
$obj.Add('head',$head)
$files = Get-ChildItem C:\inetpub\wwwroot\$instName\filepath -Include Rules.XML -Recurse
$files |
ForEach-Object {
[xml]$temp = Get-Content -path $_.FullName
[string]$title = $_.Directory.name
[hashtable]$output = #{}
foreach($rule in $temp.RuleCollection.Rules.Rule){
$t = ""
if($rule.value -isnot [string]){
$t = $rule.value.innerXml
}else{
$t = $rule.value
}
$output.Add($rule.name,$t)
}
$obj.Add($title,$output)
}
$transfer = New-Object -TypeName PSObject -Property $obj
$location = "C:\Users\vagrant\desktop\json\" + $instName + ".Json"
$transfer | ConvertTo-Json -Compress | Out-file -FilePath $location
}
}
And the Setter like this
Param(
[string]$location,
[string]$ruleConfigFileLocation)
$allInstConfigFiles = Get-ChildItem -Path C:\Users\vagrant\Desktop\json\
foreach($singleInstFile in $allInstConfigFiles) {
$instaName = $singleInstFile.BaseName
$allRuleXmlFileList = Get-ChildItem C:\inetpub\wwwroot\$instaName\Liberty \Applications\Origination\Configuration -Include Rules.XML -Recurse
ForEach($singleRuleXmlFile in $allRuleXmlFileList) {
[xml]$singleRuleXmlFileContents = Get-Content -path $singleRuleXmlFile.FullName
$singleInstFileContent = Get-Content $singleInstFile.FullName | ConvertFrom-Json
foreach($ruleFileSet in $singleInstFileContent){
foreach($rule in $singleRuleXmlFileContents.RuleCollection.Rules.Rule){
#write-host $ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name)
if($ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name) -as [xml])
{
$rule.value = $ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name).OuterXml
write-host 'skipped'
}else{
write-host 'not skipped'
$rule.value = $ruleFileSet.($singleRuleXmlFile.directory.name).($rule.name)
}
}
}
}
}
I'm getting this error when I add to the return the -prefix "File_Name" to the end of Return $Results | Out-Excel -Prefix "File_Name" if I leave -prefix "File_Name" off of the script it works. What am I missing?
Out-Excel : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of
the parameters that take pipeline input.
At \\util01\cs\Scripts\PowerShell\BradleyDev\Google Chrome\Set-Chrome-Service-ON-V3.ps1:52 char:21
+ } Return $Results | Out-Excel -Prefix "File_Name"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (#{PC=CD-100TRAINER; Status=OFF-LINE}:PSObject) [Out-Excel], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectNotBound,Out-Excel
foreach ($PC in $Process){
$counter ++
$Service = "gupdate"
$State = "delayed-auto"
$command = "sc.exe \\$PC config $Service start= $State"
$count = $Process.Count
$command2 = "sc.exe \\$PC config gupdatem start= demand"
if (test-connection -computername $PC -BufferSize 16 -Count 1 -quiet) {
$Output = Invoke-Expression -Command $Command -ErrorAction Stop
$Output = Invoke-Expression -Command $Command2 -ErrorAction Stop
if($LASTEXITCODE -ne 0){
Write-Host "$counter of $count : $PC : Failed More details: $Output" -foregroundcolor White -BackgroundColor RED
$status = [string]$Output
} else {
Write-Host "$counter of $count : $PC : Success" -foregroundcolor DarkGreen
$status = "Success"
}
} else {
Write-Host "$counter of $count : $PC : OFF-LINE" -foregroundcolor black -BackgroundColor gray
$status = "OFF-LINE"
}
$PCobject = [PSCustomObject] #{
PC = $PC
Status = $status
}
$Results +=$PCobject
} Return $Results | Out-Excel -Prefix "File_Name"
function Out-Excel {
param(
[Parameter(Mandatory=$True,HelpMessage="Enter Prefix",ValueFromPipeline=$true, ValueFromPipelineByPropertyName=$true)]
[ValidateNotNullorEmpty()]
[string[]]$Prefix = $null,
$Path = "$env:temp\$Prefix`_$(Get-Date -Format yyMMdd_HHmmss).csv"
)
$input | Export-CSV -Path $Path -UseCulture -Encoding UTF8 -NoTypeInformation
Invoke-Item -Path $Path
} #end function
Updated but still failing:
function Out-Excel {
param(
[Parameter(Mandatory=$True,HelpMessage="Enter Prefix")]
[ValidateNotNullorEmpty()]
[string[]]$Prefix = $null,
$Path = "$env:temp\$Prefix`_$(Get-Date -Format yyMMdd_HHmmss).csv"
)
$input | Export-CSV -Path $Path -UseCulture -Encoding UTF8 -NoTypeInformation
Invoke-Item -Path $Path
} #end function
ERROR:
Out-Excel : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and
its properties do not match any of the parameters that take pipeline input.
At \\util01\cs\Scripts\PowerShell\BradleyDev\Google Chrome\Set-Chrome-Service-ON-V3.ps1:52 char:14
+ } $Results | Out-Excel -Prefix "Services_Results"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (#{PC=CD-100TRAINER; Status=OFF-LINE}:PSObject) [Out-Excel], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectNotBound,Out-Excel
This works:
I wanted to be able to use the function and set a file name prefix depending on name of the results. The code below works by adding another parameter "the object" I want to pass to the function, however I loose the ability to pipe the object to the function. Apparently if I use $input I can not take pipeline arguments?
function Out-Excel {
param(
[Parameter(Mandatory=$True,HelpMessage="Enter Prefix")]
[ValidateNotNullorEmpty()]
[string[]]$Prefix = $null,
[Parameter(Mandatory=$True,HelpMessage="Enter Prefix")]
[ValidateNotNullorEmpty()]
[object]$Object = $null,
$Path = "$env:temp\$Prefix`_$(Get-Date -Format yyMMdd_HHmmss).csv"
)
$Object | Export-CSV -Path $Path -UseCulture -Encoding UTF8 -NoTypeInformation
Invoke-Item -Path $Path
} #end function
$Results +=$PCobject
} Out-Excel -object $Results -Prefix "Services_Results"`
If you want to accept pipeline input from advanced function, then it have to have parameters, which declared as ValueFromPipeline or ValueFromPipelineByPropertyName and does not bound from command line.
function Out-Excel {
param(
[Parameter(Mandatory=$True,HelpMessage="Enter Prefix")]
[string]$Prefix = $null,
[Parameter(Mandatory=$True,ValueFromPipeline=$True)]
[psobject]$InputObject = $null,
$Path = "$env:temp\$Prefix`_$(Get-Date -Format yyMMdd_HHmmss).csv"
)
begin{
$List=New-Object System.Collections.Generic.List[PSObject]
}
process{
$List.Add($InputObject)
}
end{
$List | Export-CSV -Path $Path -UseCulture -Encoding UTF8 -NoTypeInformation
Invoke-Item -Path $Path
}
} #end function
I've written the following PS script to delete log files from specific server paths. I'm a novice to PS but I'm getting some errors with a few of the functions that I have written in this script:
#* FileName: FileCleaner.ps1
#Clear the screen
Clear
#Read XML Config File to get settings
[xml]$configfile = Get-Content "C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell Script\FileCleaner.config.xml"
#Declare and set variables from Config values
$hostServer = $configfile.Settings.HostServer
$dirs = #($configfile.Settings.DirectoryName.Split(",").Trim())
$scanSubDirectories = $configfile.Settings.ScanSubDirectories
$deleteAllFiles = $configfile.Settings.deleteAllFiles
$fileTypesToDelete = #($configfile.Settings.FileTypesToDelete.Split(";").Trim())
$liveSiteLogs = $configfile.Settings.LiveSiteLogs
$fileExclusions = #($configfile.Settings.FileExclusions.Split(";").Trim())
$retentionPeriod = $configfile.Settings.RetentionPeriod
$AICLogs = $configfile.Settings.AICLogs
$AICLogsRententionPeriod = $configfile.Settings.AICLogsRententionPeriod
$fileCleanerLogs = $configfile.Settings.FileCleanerLogs
$fileCleanerLogsRententionPeriod = $configfile.Settings.FileCleanerLogsRententionPeriod
#Setup FileCleaner output success logfiles
$successLogfile = $configfile.Settings.SuccessOutputLogfile
$dirName = [io.path]::GetDirectoryName($successLogfile)
$filename = [io.path]::GetFileNameWithoutExtension($successLogfile)
$ext = [io.path]::GetExtension($successLogfile)
$successLogfile = "$dirName\$filename$(get-date -Format yyyy-MM-dd)$ext"
#Setup FileCleaner output error logfiles
$errorLogfile = $configfile.Settings.ErrorOutputLogfile
$dirName = [io.path]::GetDirectoryName($errorLogfile)
$filename = [io.path]::GetFileNameWithoutExtension($errorLogfile)
$ext = [io.path]::GetExtension($errorLogfile)
$errorLogfile = "$dirName\$filename$(get-date -Format yyyy-MM-dd)$ext"
#Setup Retention Period
$LastWrite = (Get-Date).AddDays(-$retentionPeriod)#.ToString("d")
$AICLastWrite = (Get-Date).AddDays(-$AICLogsRententionPeriod)#.ToString("d")
$fileCleanerLastWrite = (Get-Date).AddDays(-$fileCleanerLogsRententionPeriod)
#EMAIL SETTINGS
$smtpServer = $configfile.Settings.SMTPServer
$emailFrom = $configfile.Settings.EmailFrom
$emailTo = $configfile.Settings.EmailTo
$emailSubject = $configfile.Settings.EmailSubject
#Update the email subject to display the Host Server value
$emailSubject -replace "HostServer", $hostServer
$countUnaccessibleUNCPaths = 0
#Check Logfiles exists, if not create them
if(!(Test-Path -Path $successLogfile))
{
New-Item -Path $successLogfile –itemtype file
}
if(!(Test-Path -Path $errorLogfile))
{
New-Item -Path $errorLogfile –itemtype file
}
foreach ($dir in $dirs)
{
#needs a check to determine if server/the UNC Path is accessible. If it fails to connect, it needs to move on to the next UNC share but a flag needs to
#be generate to alert us to investigate why the UNC share was not accessible during the job run.
If(Test-Path -Path $dir)
{
#write to output logfile Directory info
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $dir"
$Msg | out-file $successLogfile
If ($scanSubDirectories -eq "True")
{
If ($deleteAllFiles -eq "True")
{
#ScanSubDirectories and delete all files older than the $retentionPeriod, include Sub-Directories / also forces the deletion of any hidden files
$logFiles = Get-ChildItem -Path $dir -Force -Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$LastWrite" }
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
Else
{
#"ScanSubDirectories but only delete specified file types."
$logFiles = Get-Childitem $dir -Include $fileTypesToDelete[0],$fileTypesToDelete[1],$fileTypesToDelete[2], $liveSiteLogs -Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where {$_.LastWriteTime -le "$LastWrite"}
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
}
Else
{
#Only delete files in top level Directory
If ($deleteAllFiles -eq "True")
{
$logFiles = Get-ChildItem -Path $dir -Force -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$LastWrite" }
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
Else
{
$logFiles = Get-Childitem $dir -Include $fileTypesToDelete[0],$fileTypesToDelete[1],$fileTypesToDelete[2], $liveSiteLogs -Exclude $fileExclusions[0],$fileExclusions[1] | Where {$_.LastWriteTime -le "$LastWrite"}
DeleteLogFiles($logFiles)
#foreach($logFile in $logFiles)
#{
# if($logFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $logFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $logFile.FullName -Force
# }
#}
}
}
}
Else
{
$countUnaccessibleUNCPaths++
#server/the UNC Path is unaccessible
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") Unable to access $dir."
$Msg | out-file $errorLogfile -append
}
# Call the function to Delete the AIC XML Logfiles
DeleteAICXMLLogs $dir
}
#If any of the directories were unaccessible send an email to alert the team
if($countUnaccessibleUNCPaths.count -gt 0)
{
# Call the function to send the email
SendEmail $emailSubject $emailFrom $emailTo
}
#Only keep 2 weeks worth of the FileCleaner App logs for reference purposes
If(Test-Path -Path $fileCleanerLogs)
{
#write to output logfile Directory info
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $fileCleanerLogs"
$Msg | out-file $successLogfile
$fileCleanerLogs = Get-Childitem $fileCleanerLogs -Recurse | Where {$_.LastWriteTime -le "$fileCleanerLastWrite"}
DeleteLogFiles($fileCleanerLogs)
#foreach($fileCleanerLog in $fileCleanerLogs)
#{
# if($fileCleanerLog -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $fileCleanerLog")"
# $Msg | out-file $successLogfile -append
# Remove-Item $fileCleanerLog.FullName -Force
# }
#}
}
Function DeleteLogFiles($logFiles)
{
foreach($logFile in $logFiles)
{
if($logFile -ne $null)
{
$Msg = Write-Output "$("Deleting File $logFile")"
$Msg | out-file $successLogfile -append
Remove-Item $logFile.FullName -Force
}
}
}
Function DeleteAICXMLLogs($dir)
{
#Split the UNC path $dir to retrieve the server value
$parentpath = "\\" + [string]::join("\",$dir.Split("\")[2])
#test access to the \\server\D$\DebugXML path
If(Test-Path -Path $parentpath$AICLogs)
{
$Msg = Write-Output "$(Get-Date -UFormat "%D / %T") - Accessing: $parentpath$AICLogs"
$Msg | out-file $successLogfile
#Concantenate server value to $AICLogs to delete all xml logs in \\server\D$\DebugXML with a retention period of 30Days
$XMLlogFiles = Get-ChildItem -Path $parentpath$AICLogs -Force -Include $fileTypesToDelete[3]-Recurse -Exclude $fileExclusions[0],$fileExclusions[1] | Where { $_.LastWriteTime -le "$AICLastWrite" }
#get each file and add the filename to be deleted to the successLogfile before deleting the file
DeleteLogFiles($XMLlogFiles)
#foreach($XMLlogFile in $XMLlogFiles)
#{
# if($XMLlogFile -ne $null)
# {
# $Msg = Write-Output "$("Deleting File $XMLlogFile")"
# $Msg | out-file $successLogfile -append
# Remove-Item $XMLlogFile.FullName -Force
# }
#}
}
Else
{
$Msg = Write-Output "$("$parentpath$AICLogs does not exist.")"
$Msg | out-file $successLogfile -append
}
}
Function SendEmail($emailSubject, $emailFrom, $emailTo)
{
$MailMessage = New-Object System.Net.Mail.MailMessage
$SMTPClient = New-Object System.Net.Mail.smtpClient
$SMTPClient.host = $smtpServer
$Recipient = New-Object System.Net.Mail.MailAddress($emailTo, "Recipient")
$Sender = New-Object System.Net.Mail.MailAddress($emailFrom, "Sender")
$MailMessage.Sender = $Sender
$MailMessage.From = $Sender
$MailMessage.Subject = $emailSubject
$MailMessage.Body = #"
This email was generated because the FileCleaner script was unable to access some UNC Paths, please refer to $errorLogfile for more information.
Please inform the Team if you plan to resolve this.
This is an automated email please do not respond.
"#
$SMTPClient.Send($MailMessage)
}
when debugging I'm getting these errors:
DeleteAICXMLLogs : The term 'DeleteAICXMLLogs' is not recognized as
the name of a cmdlet, function, script file, or operable program.
Check the spelling of the name, or if a path was included, verify
that the path is correct and try again. At
C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell
Script\FileCleaner.ps1:158 char:5
+ DeleteAICXMLLogs $dir
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (DeleteAICXMLLogs:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
SendEmail : The term 'SendEmail' is not recognized as the name of a
cmdlet, function, script file, or operable program. Check the spelling
of the name, or if a path was included, verify that the path is
correct and try again. At C:\Users\pmcma\Documents\Projects\Replace
FileCleaner with PowerShell Script\FileCleaner.ps1:164 char:5
+ SendEmail $emailSubject $emailFrom $emailTo
+ ~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (SendEmail:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
DeleteLogFiles : The term 'DeleteLogFiles' is not recognized as the
name of a cmdlet, function, script file, or operable program. Check
the spelling of the name, or if a path was included, verify that the
path is correct and try again. At
C:\Users\pmcma\Documents\Projects\Replace FileCleaner with PowerShell
Script\FileCleaner.ps1:175 char:5
+ DeleteLogFiles($fileCleanerLogs)
+ ~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (DeleteLogFiles:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
I don't see anything wrong with how I'm declaring the functions or calling them. Any ideas why this script is failing?
PowerShell Scripts are read from the top to the bottom, so you can't use any references before they are defined, most probably that is why you are receiving errors.
Try adding your function definition blocks above the point where you call them.
Alternatively you can make a function having global scope. Just preface the function name with the keyword global: like,
function global:test ($x, $y)
{
$x * $y
}
I've had this happen as well. Try placing the functions before the business logic. This is a script, not compiled code. So the functions are yet to be declared before you are calling them.