I have extracted a load of files from a webapp, and all it gives me is a CSV file containing a load of blobs of the file, and the filename.
What is the best way to convert these to the actual files? I was thinking using a powershell script?
I created a PowerShell script, that extracts the blobs and filename from CSV and saves in a given directory as raw files.
$path = 'D:\TEMP\Attachments\extract.csv'
$exportPath = 'D:\TEMP\Files'
Import-Csv $path | Foreach-Object {
$b64 = $_.BODY
$bytes = [Convert]::FromBase64String($b64)
$filename = $exportPath + "\" + $_.NAME
[IO.File]::WriteAllBytes($filename, $bytes)
}
Related
I need to integrate a JSON file which contains paths of the different objects in a PS script that generates and compares the hash files of source and destination. The paths in the JSON file are written in the format that I have stated below. I want to use the paths in that manner and pipe them into Get-FileHash in PowerShell. I can't figure out how to integrate my current PowerShell script with the JSON file that contains the information (File Name, full path) etc.
I have two scripts that I have tested and they work fine. One generates the MD5 hashes of two directories (source and destination) and stores them in a csv file. The other compares the MD5 hashes from the two CSV files and generates a new one, showing the result(whether a file is absent from source or destination).
Now, I need to integrate these scripts in another one, which is basically a Powershell Installer. The installer saves the configurations (path, ports, new files to be made etc etc) in a JSON format. In my original scripts, the user would type the path of the source and destination which needed to be compared. However, I now need to take the path from the JSON configuration files. For example, the JSON file below is of the similar nature as the one I have.
{
"destinatiopath": "C:\\Destination\\Mobile Phones\\"
"sourcepath": "C:\\Source\\Mobile Phones\\"
"OnePlus" : {
"files": [
{
"source": "6T",
"destination: "Model\\6T",
]
}
"Samsung" : {
"files": [
{
"source": "S20",
"destination": "Galaxy\\S20",
}
]
This is just a snippet of the JSON code. It's supposed to have the destination and source files. So for instance if the destination path is: C:\\Destination\\Mobile Phones\\ and the source path is C:\\Source\\Mobile Phones\\ and OnePlus has 6T as source and Model\\6T as destination, that means that the Powershell Installer will have the full path C:\\Destination\\Mobile Phones\\Model\\6T as the destination, and C:\\Source\\Mobile Phones\\6T as the source. The same will happen for Samsung and others.
For now, the MD5 hash comparison PS script just generates the CSV files in the two desired directories and compares them. However, I need to check the source and destination of each object in this case. I can't figure out how I can integrate it here. I'm pasting my MD5 hash generation code below.
Generating hash
#$p is the path. In this case, I'm running the script twice in order to get the hashes of both source and destination.
#$csv is the path where the csv will be exported.
Get-ChildItem $p -Recurse | ForEach-Object{ Get-FileHash $_.FullName -Algorithm MD5 -ErrorAction SilentlyContinue} | Select-Object Hash,
#{
Name = "FileName";
Expression = { [string]::Join("\", ($_.Path -split "\\" | Select-Object -Skip ($number))) }
} | Export-Csv -Path $csv
I want to use the paths in that manner and pipe them into Get-FileHash in PowerShell.
As the first step I would reorganize the JSON to be easier to handle. This will make a big difference on the rest of the script.
{
"source": "C:\\Source\\Mobile Phones",
"destination": "C:\\Destination\\Mobile Phones",
"phones": [
{
"name": "OnePlus",
"source": "6T",
"destination": "Model\\6T"
},
{
"name": "Samsung",
"source": "S20",
"destination": "Galaxy\\S20"
}
]
}
Now it's very easy to get all the paths no matter how many "phone" entries there are. You don't even really need an intermediary CSV file.
$config = Get-Content config.json -Encoding UTF8 -Raw | ConvertFrom-Json
$config.phones | ForEach-Object {
$source_path = Join-Path $config.source $_.source
$destination_path = Join-Path $config.destination $_.destination
$source_hashes = Get-ChildItem $source_path -File -Recurse | Get-FileHash -Algorithm MD5
$destination_hashes = Get-ChildItem $destination_path -File -Recurse | Get-FileHash -Algorithm MD5
# the combination of relative path and file hash needs to be unique, so let's combine them
$source_relative = $source_hashes | ForEach-Object {
[pscustomobject]#{
Path = $_.Path
PathHash = $_.Path.Replace($source_path, "") + '|' + $_.Hash
}
}
$destination_relative = $destination_hashes | ForEach-Object {
[pscustomobject]#{
Path = $_.Path
PathHash = $_.Path.Replace($destination_path, "") + '|' + $_.Hash
}
}
# Compare-Object finds the difference between both lists
$diff = Compare-Object $source_relative $destination_relative -Property PathHash, Path
Write-Host $diff
$diff | ForEach-Object {
# work with $_.Path and $_.SideIndicator
}
}
I would like to rename files and folders based on keywords found in a CSV file.
The CSV holds the search and replace keywords that will make up file and folder names.
Search | Replace
Document | DTX
Processing | PRX
Implementation | IMX
...
Not all the file names include each word in the file name.
Not all the folders will include each word in the folder name
Powershell will have to search the child item ie the folder and file
names.
If it finds the word (match) - Substitute from the CSV
I have looked at these threads to help me:
Using Powershell to recursively rename directories using a lookup file
powershell script to rename all files in directory
http://code.adonline.id.au/batch-rename-files/
I have only managed below snippet
$folder = "C:\Folders" #target folder containing files
$csv = "C:\FileNameKeywords.csv" #path to CSV file
cd ($folder);
Import-Csv ($csv) | foreach {
Rename-Item -Path $_.Path -NewName $_.Filename
}
It only replaces one at a time.
Question:
How can I recursively search and replace in file and Folder Names using a CSV as a look up or reference file.
When you have the need to look up values by other values the usual go-to data structure is a dictionary, or in PowerShell terms a hashtable. Read your CSV into a dictionary like this:
$keywords = #{}
Import-Csv $csv | ForEach-Object {
$keywords[$_.Search] = $_.Replace
}
Then traverse your folder tree and build the new filenames by replacing each key with its respective value:
Get-ChildItem $folder -Recurse | ForEach-Object {
$newname = $_.Name
foreach ($word in $keywords.Keys) {
$newname = $newname.Replace($word, $keywords[$word])
}
if ($_.Name -ne $newname) {
Rename-Item -Path $_.FullName -NewName $newname
}
}
ill give it a shot. I'm assuming search and replace are your headers in this scenario. this in addition to your $folder and $csv variables.
$csvobject=import-csv $csv
Foreach($obj in $csvobject){
$search=$obj.search
$replace=$obj.replace
get-childitem path $folder |where{$_.name -like "$($obj.search)"} | rename-item -newname {$_.name -replace "$search", "$replace"} }
The replace handles regex so u will need to make sure Any special characters are properly escaped.
I am trying to Copy Directories - folders and Sub folders to another location using a CSV file that lists the source and destination of each directory or folder to be copied.
The Contents of the CSV are as such below:
I have referenced this thread:
https://serverfault.com/questions/399325/copying-list-of-files-through-powershell
Import-CSV C:\Users\WP\Desktop\a.csv | foreach{Copy-item "$_.Source" "$_.Destination"}
Error Received
CategoryInfo : ObjectNotFound: (#{Source=C:String) [Copy-Item], DriveNotFoundException
+ FullyQualifiedErrorId : DriveNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
The other question I have is if in the CSV I want to copy to a folder that does not exists in the destination - can I use the CSV to command powershell to create the folder?
Thank you for your advice.
PowerShell will not expand the variable and access the property of the object inside the variable if you have them placed in double quotes by default. Only the '$_' is being expanded and '.source' is being tacked on to the end of the string, so from the view of the shell, your command looks something like Copy-item "{source=C:\Users\WP\Desktop\a;Destination=C:\Users\WP\Desktop\a}.Source" "{source=C:\Users\WP\Desktop\a;Destination=C:\Users\WP\Desktop\a}.Destination", which is probably not what you mean.
Here is the syntax that should work (I also included -Recurse so that it will copy the items inside the directory as well)
Import-CSV C:\Users\WP\Desktop\a.csv | foreach{Copy-item -Path $_.Source -Destination $_.Destination -Recurse}
Note: if you want to access the properties on an object inside of double quotes, use this syntax "$($_.source)".
For a csv like this:
Source,Destination
D:\junk\test1,D:\junk\test3
D:\junk\test2,D:\junk\test4
You can use code like the following:
$csv = Import-Csv D:\junk\test.csv
$csv | ForEach-Object {
if (-not (Test-Path $_.Destination)) {
New-Item -Name $_.Destination -ItemType Directory -Force -WhatIf
}
Copy-Item $_.Source $_.Destination -Recurse -Force -WhatIf
}
Suggestions for learning more about PowerShell:
Use WhatIf to test things.
Research what each line of this code does.
Experiment with code to see what it does.
Learn and use the debugger (PowerShell ISE) to help you write better code.
Remove the WhatIf parameters from the code to have it execute for real...
If you have dozens of problems that all involve doing the same thing with each element of a list, you might want to consider getting or writing a generic CSV template expander tool, like Expand-csv. With this tool you start with a CSV file and a template, and generate a script that contains all the commands.
Sample.csv looks like this:
Source,Destination
C:\Users\WP\Desktop\a,C:\Users\WP\Desktop\c
C:\Users\WP\Desktop\b,C:\Users\WP\Desktop\d
Sample.tmplt looks like this:
Copy-Item -Path $Source -Destination $Destination -Recurse
And the command to invoke Expand-csv looks like this:
Expand-csv Sample.csv Sample.tmplt > Sample.ps1
The output file, Sample.ps1 contains one copy command for each entry in the CSV file
And here is the definition of Expand-csv:
<# This function is a table driven template tool.
It's a refinement of an earlier attempt.
It generates output from a template and
a driver table. The template file contains plain
text and embedded variables. The driver table
(in a csv file) has one column for each variable,
and one row for each expansion to be generated.
5/13/2015
#>
function Expand-csv {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true)]
[string] $driver,
[Parameter(Mandatory=$true)]
[string] $template
)
Process
{
$OFS = "`r`n"
$list = Import-Csv $driver
[string]$pattern = Get-Content $template
foreach ($item in $list) {
foreach ($key in $item.psobject.properties) {
Set-variable -name $key.name -value $key.value
}
$ExecutionContext.InvokeCommand.ExpandString($pattern)
}
}
}
This is a PowerShell question, not a SharePoint question.
I'm using a script to grab an inventory of SharePoint features, web parts, etc. It outputs each type of report in the same directory as csv files. So I'll end up with a directory on my computer with the csv files.
I'd like to run another PowerShell script after the first one that converts these csvs into html files for easily readable reports.
I'm getting stuck on the part where I would import-csv each file and create each html file with similarly named html files.
Here's what I have so far. Can anyone help me complete this to do what I want it to do? To use Import-CSV, I have to specify the file name as you can see in $dir. Is there another way?
$dir = "C:\Users\me\Desktop\output\TestInvSiteCollections.csv"
dir -LiteralPath $dir | % {Import-Csv $dir}
or use this somehow..
Import-Csv -LiteralPath $dir | ConvertTo-Html | Out-File "C:\Users\me\Desktop\output\myhtmlfile.html"
I would do it like this:
Get-ChildItem -Path 'C:\Users\me\Desktop\output\*.csv' | ForEach-Object {
Import-Csv $_ | ConvertTo-Html | Out-File -FilePath (Join-Path -Path $_.DirectoryName -ChildPath ($_.BaseName + '.html'));
}
I'm not entirely sure I find html tables easier to read than csv files. Excel's filtering and sorting is too useful.
Here's your code. It should split the name of the file and add the extension.
Import-Csv -LiteralPath $dir | ConvertTo-Html | Out-File ($dir.Split(".")[0]+".html")
This is a slightly more verbose method, but I generally prefer readable code over conciseness for maintainability:
#get the list of csv files
$csvFiles = Get-ChildItem $path -Filter *.csv
foreach ($file in $csvFiles)
{
#create FileInfo object
[System.IO.FileInfo]$fileInfo = "$path\$file"
#Get base name of file
$baseName = [System.IO.Path]::GetFileNameWithoutExtension($file.Name)
#do HTML conversion
Import-Csv $fileInfo.FullName | ConvertTo-Html | Out-File "$htmlPath\$baseName.html"
}
This is working code assuming you have $path defined somewhere and can obviously be modified to suite your needs.
I have a set of files (OldName) in a Windows directory that I would like to rename (NewName) based on the following CSV file:
OldName,NewName
Sources/texas play_PGC_Cpgc_entryPoint_Cbp_1.f4v,01 Texas Play.f4v
Sources/texas play_PGC_Cpgc_entryPoint_Dbp_1.f4v,02 First Song.f4v
Sources/texas play_PGC_Cpgc_entryPoint_Ebp_1.f4v,03 Yellow Rose.f4v
I'm not sure how to loop thru the CSV file... finding each file and replacing.
Any thoughts would be appreciated.
First Import Your CSV file into powershell
$AnyVariableName = Import-Csv "$env:USERPROFILE:\Desktop\directoryname.txt"
Note: In my example, the path to the CSV file is on my desktop, but it may be different in yours.
Then use a foreach loop rename the items
foreach ($objName in $AnyVariableName){
Rename-Item $objName.OldName $objName.NewName
}
One way to do it is to create two lists and loop though each of them. The CSV file will be a reference list, so we'll grab the contents and convert it from CSV then store it in a variable
$CSVRef = Get-Content "C:\Path\To\CSV.csv" | ConvertFrom-CSV
Then we'll get the list of files who's names you want to change, and loop through each file. From inside the loop you can run another loop to find the current name in your reference list, and then change it to the new name.
Get-ChildItem "C:\path\to\f4v\files" -Filter *.f4v | ForEach-Object {
#Grab the current item in a variable to access it within the second loop
$CurrentFile = $_
$CSVRef | ForEach-Object {
if ($CurrentFile.Name -ilike $_.OldName) {
Rename-Item $CurrentFile.FullPath $_.NewName
}
}
}
So during the second loop we try to compare the file name with every "OldName" item in the CSV file list. If the OldName matches somewhere in the current file we're looping through then we run Rename-Item and provide it the NewName. It should automatically rename the file.
Combining both examples works great
$CSVRef = Import-Csv "C:\Temp\Filename.txt"
Get-ChildItem "C:\Temp\FileFolder" -Filter *.pdf | ForEach-Object {
$CurrentFile = $_
ForEach ($objName in $CSVRef) {
if ($CurrentFile.Name -ilike $objName.OLDNAME) {
Rename-Item $CurrentFile.FullName $objName.NEWNAME
}
}
}