I am trying to download zip files from an FTP site, based off retrieving a directory list to find file names.
Download Portion:
$folderPath='ftp://11.111.11.11/'
$target = "C:\Scripts\ps\ftpdl\"
Foreach ($file in ($array | where {$_ -like "data.zip"})) {
$Source = $folderPath+$file
$Path = $target+$file
#$Source = "ftp://11.111.11.11/data.zip"
#$Path = "C:\Scripts\ps\ftpdl\data.zip"
$source
Write-Verbose -Message $Source -verbose
$path
Write-Verbose -message $Path -verbose
$U = "User"
$P = "Pass"
$WebClient2 = New-Object System.Net.WebClient
$WebClient2.Credentials = New-Object System.Net.Networkcredential($U, $P)
$WebClient2.DownloadFile( $source, $path )
}
If I use the commented out and define the string it downloads correctly. But if I run it as shown I receive the exception error illegal characters in path. Interestingly enough, there is a difference between write-verbose and not.
Output when run as shown:
ftp://11.111.11.11/data.zip
data.zip
C:\Scripts\ps\ftpdl\data.zip
data.zip
Exception calling "DownloadFile" with "2" .........
Output when run with hard coded path & source
ftp://11.111.11.11/data.zip
VERBOSE: ftp://11.111.11.11/data.zip
C:\Scripts\ps\ftpdl\data.zip
VERBOSE: C:\Scripts\ps\ftpdl\data.zip
And the file downloads nicely.
Well, of course once I post the question I figured it out. My $array contained `n and `r characters. I needed to find and replace both of them out.
$array=$array -replace "`n",""
$array=$array -replace "`r",""
Related
I am attempting to index my movie collection and in doing so have run across an issue where at least one title is skipped in the import phase due to special characters. The code skips over "Æon Flux" due to it starting with Æ. Would anyone know how to correct this, please?
Clear-Host
# Variables:
$movie_dir = "K:\Movies"
# Because reasons...
$PSDefaultParameterValues['*:Encoding'] = 'utf8'
# Connect to the library MySQL.Data.dll
Add-Type -Path 'C:\Program Files (x86)\MySQL\Connector NET 8.0\Assemblies\v4.8\MySql.Data.dll'
# Create a MySQL Database connection variable that qualifies:
$Connection = [MySql.Data.MySqlClient.MySqlConnection]#{ConnectionString='server=127.0.0.1;uid=username;pwd=password;database=media'}
$Connection.Open()
# Drop the table to clear all entries.
$sql_drop_table = New-Object MySql.Data.MySqlClient.MySqlCommand
$sql_drop_table.Connection = $Connection
$sql_drop_table.CommandText = 'DROP TABLE Movies'
$sql_drop_table.ExecuteNonQuery() | Out-Null
# (Re)create the table.
$sql_create_table = New-Object MySql.Data.MySqlClient.MySqlCommand
$sql_create_table.Connection = $Connection
$sql_create_table.CommandText = 'create table Movies(movie_id INT NOT NULL AUTO_INCREMENT, movie_title VARCHAR(255) NOT NULL, movie_file_date INT, movie_IMDB_id INT, PRIMARY KEY (movie_id))'
$sql_create_table.ExecuteNonQuery() | Out-Null
$movies = Get-ChildItem $movie_dir -File -include *.mp4 -Recurse -Depth 1 |
Select-Object -ExpandProperty FullName |
Sort-Object |
Get-Unique |
where{$_ -ne ""}
foreach ($movie in $movies)
{
# .net function to get just the filename (movie title).
$title = [System.IO.Path]::GetFileNameWithoutExtension($movie)
# Get the creation date of the movie and reformat it to yearmonthday.
$add_date = (Get-ChildItem $movie).CreationTime.toString("yyyyMMdd")
$query = "INSERT INTO Movies(movie_id, movie_title, movie_file_date) VALUES(NULL, #title, $add_date)"
$command = $connection.CreateCommand()
$command.CommandText = $query
# Sanatize single quotes in filenames for input.
$command.Parameters.AddWithValue("#title", $title) | Out-Null
$command.ExecuteNonQuery() | Out-Null
}
# Close the MySQL connection.
$Connection.Close()
Write-Host
Write-Host("Added") $movies.Count ("movies.")
I don't think it is the Get-ChildItem that skips the file with that special character. More likely, you need to tell your MySql to use UTF-8.
For that, have a look at How to make MySQL handle UTF-8 properly
As for your code, I would change this:
$movies = Get-ChildItem $movie_dir -File -include *.mp4 -Recurse -Depth 1 |
Select-Object -ExpandProperty FullName |
Sort-Object |
Get-Unique |
where{$_ -ne ""}
into
$movies = Get-ChildItem -Path $movie_dir -File -Filter '*.mp4' -Recurse -Depth 1 | Sort-Object -Property FullName
and work with the FileInfo objects from there on:
foreach ($movie in $movies) {
$title = $movie.BaseName
# Get the creation date of the movie and reformat it to yearmonthday.
$add_date = '{0}:yyyyMMdd}' -f $movie.CreationTime
. . .
}
Though Æ is not an ASCII character it is not otherwise "special", so I edited the question title and tags to reflect that.
ExecuteNonQuery() returns the number of rows affected by the command; in the case of $command, it's the number of rows inserted. You are discarding this value, however...
$command.ExecuteNonQuery() | Out-Null
...which masks the problem in the event the INSERT fails. Instead, test the result and respond appropriately...
if ($command.ExecuteNonQuery() -eq 1)
{
Write-Host -Message "Successfully inserted movie ""$title""."
}
else
{
Write-Warning -Message "Failed to insert movie ""$title""."
}
This will make it clear if the issue lies in interacting with the filesystem or the database.
Some other notes:
MySqlCommand implements the IDisposable interface and so each instance should be disposed when you're done using it...
$query = "INSERT INTO Movies(movie_id, movie_title, movie_file_date) VALUES(NULL, #title, $add_date)"
$command = $connection.CreateCommand()
try
{
$command.CommandText = $query
# Sanatize single quotes in filenames for input.
$command.Parameters.AddWithValue("#title", $title) | Out-Null
if ($command.ExecuteNonQuery() -eq 1)
{
Write-Host -Message "Successfully inserted movie ""$title""."
}
else
{
Write-Warning -Message "Failed to insert movie ""$title""."
}
}
finally
{
$command.Dispose()
}
...and the same for $sql_drop_table and $sql_create_table. The code in the finally block will run even if an error is thrown from within the try block.
See Difference with Parameters.Add and Parameters.AddWithValue and its links for why AddWithValue() can be problematic.
Instead of...
Write-Host("Added") $movies.Count ("movies.")
...a more typical way to build this message would be with string interpolation...
Write-Host "Added $($movies.Count) movies."
...or the format operator...
Write-Host ('Added {0} movies.' -f $movies.Count)
You can also incorporate numeric format strings, so if $movies.Count is 1234 and $PSCulture is 'en-US' then...
Write-Host "Added $($movies.Count.ToString('N0')) movies."
...and...
Write-Host ('Added {0:N0} movies.' -f $movies.Count)
...will both write...
Added 1,234 movies.
As some background, this should take an excel file, and convert it to PDF (and place the PDF into a temporary folder).
E.g. 'C:\Users\gjacobs\Desktop\test\stock.xlsx'
becomes
'C:\Users\gjacobs\Desktop\test\pdf_merge_tmp\stock.pdf'
However, the new file path does not return correctly.
If I echo the string $export_name from within the function, I can see that it has the correct value: "C:\Users\gjacobs\Desktop\test\pdf_merge_tmp\stock.pdf".
But once $export_name is returned, it has a different (incorrect value): "C:\Users\gjacobs\Desktop\test\pdf_merge_tmp C:\Users\gjacobs\Desktop\test\pdf_merge_tmp\stock.pdf".
function excel_topdf{
param(
$file
)
#Get the parent path
$parent = Split-Path -Path $file
#Get the filename (no ext)
$leaf = (Get-Item $file).Basename
#Add them together.
$export_name = $parent + "\pdf_merge_tmp\" + $leaf + ".pdf"
echo ($export_name) #prints without issue.
#Create tmp dir
New-Item -Path $parent -Name "pdf_merge_tmp" -ItemType "Directory" -Force
$objExcel = New-Object -ComObject excel.application
$objExcel.visible = $false
$workbook = $objExcel.workbooks.open($file, 3)
$workbook.Saved = $true
$xlFixedFormat = “Microsoft.Office.Interop.Excel.xlFixedFormatType” -as [type]
$workbook.ExportAsFixedFormat($xlFixedFormat::xlTypePDF, $export_name)
$objExcel.Workbooks.close()
$objExcel.Quit()
return $export_name
}
$a = excel_topdf -file 'C:\Users\gjacobs\Desktop\test\stock.xlsx'
echo ($a)
The issue you're experiencing is caused by the way how PowerShell returns from functions. It's not something limited to New-Item cmdlet. Every cmdlet which returns anything would cause function output being altered with the value from that cmdlet.
As an example, let's take function with one cmdlet, which returns an object:
function a {
Get-Item -Path .
}
$outputA = a
$outputA
#### RESULT ####
Directory:
Mode LastWriteTime Length Name
---- ------------- ------ ----
d--hs- 12/01/2021 10:47 C:\
If you want to avoid that, these are most popular options (as pointed out by Lasse V. Karlsen in comments):
# Assignment to $null (or any other variable)
$null = Get-Item -Path .
# Piping to Out-Null
Get-Item -Path . | Out-Null
NOTE: The behavior described above doesn't apply to Write-Host:
function b {
Write-Host "bbbbbb"
}
$outputB = b
$outputB
# Nothing displayed
Interesting thread to check if you want to learn more.
I have a zip file which contains several CSV files inside it. How do I read the contents of those CSV files without extracting the zip files using PowerShell?
I having been using the Read-Archive Cmdlet which is included as part of the PowerShell Community Extensions (PSCX)
This is what I have tried so far.
$path = "$env:USERPROFILE\Downloads\"
$fullpath = Join-Path $path filename.zip
Read-Archive $fullpath | Foreach-Object {
Get-Content $_.Name
}
But when I run the code, I get this error message
Get-Content : An object at the specified path filename.csv does not exist, or has been filtered by the -Include or -Exclude parameter.
However, when I run Read-Archive $fullpath, it lists all the file inside the zip file
There are multiple ways of achieving this:
1. Here's an example using Ionic.zip dll:
clear
Add-Type -Path "E:\sw\NuGet\Packages\DotNetZip.1.9.7\lib\net20\Ionic.Zip.dll"
$zip = [Ionic.Zip.ZipFile]::Read("E:\E.zip")
$file = $zip | where-object { $_.FileName -eq "XMLSchema1.xsd"}
$stream = new-object IO.MemoryStream
$file.Extract($stream)
$stream.Position = 0
$reader = New-Object IO.StreamReader($stream)
$text = $reader.ReadToEnd()
$text
$reader.Close()
$stream.Close()
$zip.Dispose()
It's picking the file by name (XMLSchema1.xsd) and extracting it into the memory stream. You then need to read the memory stream into something that you like (string in my example).
2. In Powershell 5, you could use Expand-Archive, see: https://technet.microsoft.com/en-us/library/dn841359.aspx?f=255&MSPPError=-2147217396
It would extract entire archive into a folder:
Expand-Archive "E:\E.zip" "e:\t"
Keep in mind that extracting entire archive is taking time and you will then have to cleanup the temporary files
3. And one more way to extract just 1 file:
$shell = new-object -com shell.application
$zip = $shell.NameSpace("E:\E.zip")
$file = $zip.items() | Where-Object { $_.Name -eq "XMLSchema1.xsd"}
$shell.Namespace("E:\t").copyhere($file)
4. And one more way using native means:
Add-Type -assembly "system.io.compression.filesystem"
$zip = [io.compression.zipfile]::OpenRead("e:\E.zip")
$file = $zip.Entries | where-object { $_.Name -eq "XMLSchema1.xsd"}
$stream = $file.Open()
$reader = New-Object IO.StreamReader($stream)
$text = $reader.ReadToEnd()
$text
$reader.Close()
$stream.Close()
$zip.Dispose()
Based on 4. solution of Andrey, I propose the following function:
(keep in mind that "ZipFile" class exists starting at .NET Framework 4.5)
Add-Type -assembly "System.IO.Compression.FileSystem"
function Read-FileInZip($ZipFilePath, $FilePathInZip) {
try {
if (![System.IO.File]::Exists($ZipFilePath)) {
throw "Zip file ""$ZipFilePath"" not found."
}
$Zip = [System.IO.Compression.ZipFile]::OpenRead($ZipFilePath)
$ZipEntries = [array]($Zip.Entries | where-object {
return $_.FullName -eq $FilePathInZip
});
if (!$ZipEntries -or $ZipEntries.Length -lt 1) {
throw "File ""$FilePathInZip"" couldn't be found in zip ""$ZipFilePath""."
}
if (!$ZipEntries -or $ZipEntries.Length -gt 1) {
throw "More than one file ""$FilePathInZip"" found in zip ""$ZipFilePath""."
}
$ZipStream = $ZipEntries[0].Open()
$Reader = [System.IO.StreamReader]::new($ZipStream)
return $Reader.ReadToEnd()
}
finally {
if ($Reader) { $Reader.Dispose() }
if ($Zip) { $Zip.Dispose() }
}
}
I'm trying to take input from a CSV file, which has a list of group names (canonical names) and get the Distinguished Name from it, then output to another CSV file. The code:
#get input file if passed
Param($InputFile)
#Set global variable to null
$WasError = $null
#Prompt for file name if not already provided
If ($InputFile -eq $NULL) {
$InputFile = Read-Host "Enter the name of the input CSV file (file must have header of 'Group')"
}
#Import Active Directory module
Import-Module -Name ActiveDirectory -ErrorAction SilentlyContinue
$DistinguishedNames = Import-Csv -Path $InputFile -Header Group | foreach-Object {
$GN = $_.Group
$DN = Get-ADGroup -Identity $GN | Select DistinguishedName
}
$FileName = "RESULT_Get-DistinguishedNames" + ".csv"
#Export list to CSV
$DNarray | Export-Csv -Path $FileName -NoTypeInformation
I've tried multiple solutions, and none have seemed to work. Currently, it throws an error because
Cannot validate argument on parameter 'Identity'. The argument is null. Supply a non-null argument and try the command again.
I tried using -Filter also, and in a previous attempt I used this code:
Param($InputFile)
#Set global variable to null
$WasError = $null
#Prompt for file name if not already provided
If ($InputFile -eq $NULL) {
$InputFile = Read-Host "Enter the name of the input CSV file(file must have header of 'GroupName')"
}
#Import Active Directory module
Import-Module -Name ActiveDirectory -ErrorAction SilentlyContinue
$DistinguishedNames = Import-Csv -Path $InputFile | foreach {
$strFilter = "*"
$Root = [ADSI]"GC://$($objDomain.Name)"
$objSearcher = New-Object System.DirectoryServices.DirectorySearcher($root)
$objSearcher.Filter = $strFilter
$objSearcher.PageSize = 1000
$objsearcher.PropertiesToLoad.Add("distinguishedname") | Out-Null
$objcolresults = $objsearcher.FindAll()
$objitem = $objcolresults.Properties
[string]$objDomain = [System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()
[string]$DN = $objitem.distinguishedname
[string]$GN = $objitem.groupname
#Get group info and add mgr ID and Display Name
$props = #{'Group Name'= $GN;'Domain' = $objDomain;'Distinguished Name' = $DN;}
$DNS = New-Object psobject -Property $props
}
$FileName = "RESULT_Get-DistinguishedNames" + ".csv"
#Export list to CSV
$DistinguishedNames | Sort Name | Export-Csv $FileName -NoTypeInformation
The filter isn't the same one I was using here, I can't find the one I was using, the I currently have is a broken attempt.
Anyway, the main issue I was having is that it will get the group name, but search for it in the wrong domain (it wouldn't include Organizational Units) which caused none of them to be found. When I search for a group in PowerShell though (using Get-ADGroup ADMIN) they show up with the correct DN and everything. Any hints or code samples are appreciated.
You seemingly miss the point of $variable = cmdlet|foreach {script-block} assignment. The objects to assign to $variable should be returned (passed through the script block) in order to end up in $variable. Both your main loops contain the structure of the line $somevar=expectedOutput where expectedOutput is either a New-Object psobject or Get-ADGroup call. The assignment to $someVar suppresses the output, so that the script block does not have anything to return, and $variable remains null. To fix, do not prepend the call that should return an object into outside variable with an assignment.
$DistinguishedNames = Import-Csv -Path $InputFile -Header Group | foreach-Object {
$GN = $_.Group
Get-ADGroup -Identity $GN | Select DistinguishedName # drop '$DN=`
}
$DistinguishedNames | Export-CSV -Path $FileName -NoTypeInformation
The same issue with the second script.
I am trying to write a script that will take a text file containing URL links to documents and download them. I am having a hard time understanding how to pass the arguments and manipulate them in powershell. Here is what I got so far. I think I should be using the param method of taking an argument so I can require it for the script, but $args seemed easier on face value... A little help would be much appreciated.
**UPDATE
$script = ($MyInvocation.MyCommand.Name)
$scriptName = ($MyInvocation.MyCommand.Name -replace "(.ps1)" , "")
$scriptPath = ($MyInvocation.MyCommand.Definition)
$scriptDirectory = ($scriptPath.Replace("$script" , ""))
## ##################################
## begin code for directory creation.
## ##################################
## creates a direcory based on the name of the script.
do {
$scriptFolderTestPath = Test-Path $scriptDirectory\$scriptName -PathType container
$scriptDocumentFolderTestPath = Test-Path $scriptFolder\$scriptName"_Script_Documents" -PathType container
$scriptLogFolderTestPath = Test-Path $scriptFolder\$scriptName"_Script_Logs" -PathType container
if ($scriptFolderTestPath -match "False") {
$scriptFolder = New-Item $scriptDirectory\$scriptName -ItemType directory
}
elseif ($scriptDocumentFolderTestPath -match "False") {
New-Item $scriptFolder\$scriptName"_Script_Documents" -ItemType directory
}
elseif ($scriptLogFolderTestPath -match "False") {
New-Item $scriptFolder\$scriptName"_Script_Logs" -ItemType directory
}
} Until (($scriptFolderTestPath -match "True") -and ($scriptDocumentFolderTestPath -match "True") -and ($scriptLogFolderTestPath -match "True"))
## variables for downloading and renaming code.
$date = (Get-Date -Format yyyy-MM-dd)
## ################################
## begin code for link downloading.
## ################################
## gets contents of the arguement variable.
Get-Content $linkList
## downloads the linked file.
Invoke-WebRequest $linkList
Resulting Errors
PS C:\Windows\system32> C:\Users\Steve\Desktop\Website_Download.ps1
cmdlet Website_Download.ps1 at command pipeline position 1
Supply values for the following parameters:
linkList: C:\Users\Steve\Desktop\linkList.txt
Directory: C:\Users\Steve\Desktop\Website_Download
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 10/27/2012 3:59 PM Website_Download_Script_Documents
d---- 10/27/2012 3:59 PM Website_Download_Script_Logs
Get-Content : Cannot find path 'C:\Users\Steve\Desktop\linkList.txt' because it does not exist.
At C:\Users\Steve\Desktop\Website_Download.ps1:42 char:1
+ Get-Content $linkList
+ ~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\Users\Steve\Desktop\linkList.txt:String) [Get-Content], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Invoke-WebRequest : Could not find file 'C:\Users\Steve\Desktop\linkList.txt'.
At C:\Users\Steve\Desktop\Website_Download.ps1:45 char:1
+ Invoke-WebRequest $linkList
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.FileWebRequest:FileWebRequest) [Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
There is no difference in passing an array of arguments in Powershell, compared to any other type of arguments. See here for how it's done. Considering you have a text file, you don't need to pass an array of arguments, and only need to pass a file name, so just a string.
I don't have any experience with Powershell 3.0, which is what you are using (judging by the presence of Invoke-WebRequest in your code), but I would start with something like this:
$URLFile = #"
http://www.google.ca
http://www.google.com
http://www.google.co.uk/
"#
$URLs = $URLFile -split "`n";
$savedPages = #();
foreach ($url in $URLs) {
$savedPages += Invoke-WebRequest $url
}
That is, you have a single file, all in one place, and make sure you receive your content correctly. Not sure why you would need Start-BitsTransfer, since Invoke-WebRequest will already get you page contents. Note that I did not do anything with $savedPages, so my code is effectively useless.
After that, contents of $URLFile goes into a file and you replace a call to it with
gc "Path_To_Your_File"`
If still working, introduce a $Path parameter to your script like this:
param([string]$Path)
test again, and so on. If you are new to Powershell, always start with smaller code pieces and keep growing to include all functionality you need. If you start with a big piece, chances are you will never finish.
Figured this out with the link from Neolisk about handling params. Then changed some code at the end to create another variable and handle things as I normally would. Just some confusion with passing params.
## parameter passed to the script.
param (
[parameter(Position=0 , Mandatory=$true)]
[string]$linkList
)
## variables for dynamic naming.
$script = ($MyInvocation.MyCommand.Name)
$scriptName = ($MyInvocation.MyCommand.Name -replace "(.ps1)" , "")
$scriptPath = ($MyInvocation.MyCommand.Definition)
$scriptDirectory = ($scriptPath.Replace("$script" , ""))
## ##################################
## begin code for directory creation.
## ##################################
## creates a direcory based on the name of the script.
do {
$scriptFolderTestPath = Test-Path $scriptDirectory\$scriptName -PathType container
$scriptDocumentFolderTestPath = Test-Path $scriptFolder\$scriptName"_Script_Documents" -PathType container
$scriptLogFolderTestPath = Test-Path $scriptFolder\$scriptName"_Script_Logs" -PathType container
if ($scriptFolderTestPath -match "False") {
$scriptFolder = New-Item $scriptDirectory\$scriptName -ItemType directory
}
elseif ($scriptDocumentFolderTestPath -match "False") {
New-Item $scriptFolder\$scriptName"_Script_Documents" -ItemType directory
}
elseif ($scriptLogFolderTestPath -match "False") {
New-Item $scriptFolder\$scriptName"_Script_Logs" -ItemType directory
}
} Until (($scriptFolderTestPath -match "True") -and ($scriptDocumentFolderTestPath -match "True") -and ($scriptLogFolderTestPath -match "True"))
## variables for downloading and renaming code.
$date = (Get-Date -Format yyyy-MM-dd)
## ################################
## begin code for link downloading.
## ################################
## gets contents of the arguement variable.
$webTargets = Get-Content $linkList
## downloads the linked file.
Invoke-WebRequest $webTargets