I have this code to remove return carriage (^M) characters to be absorbed by Unix. The below codes works, but I can't figure out how to:
loop through a number of CSVs (5), effectively using the for loop
replace the existing files with the new files
$csv = (Get-Content -Raw *.csv) -replace "`r`n","`n"
[io.file]::WriteAllText('C:\Powershell\test.csv', $csv)
The code you posted will take all CSV files and concatenate them to a single output file. You need enumerate and process the files individually. There's also no need to collect the content in a variable. Just pipe the modified content into Set-Content.
Get-ChildItem 'C:\some\folder' -Filter *.csv | ForEach-Object {
(Get-Content -Raw $_.FullName) -replace "`r`n", "`n" | Set-Content $_.FullName
}
Related
My goal is to prettify all the JSON files (about 70K of them) in one folder.
Say I have one folder named "juicero" and in there there are about 70K .JSON files with different names -
a.json, b.json
This is what I tried -
PS> $files = Get-ChildItem C:\users\gamer\desktop\juicero\*.json
PS> $json_test = (Get-Content $files -raw | ConvertFrom-Json)
PS> foreach ($file in $files) { ConvertTo-Json | Set-Content $files }
I thought it would iterate through the path and prettify these (pretty straight-forward logic) but for some reason, this is deleting the content of the files. If I don't iterate and just use this function on one .json file it works - so I'm guessing there is something wrong with the iteration logic?
You need to work on files inside the loop, like that:
foreach ($file in $files) {
$content = Get-Content $file -Raw | ConvertFrom-Json
$newFilePath = $file.FullName.Replace("OldFolder","NewFolder")
ConvertTo-Json -InputObject $content| Set-Content $newFilePath
}
Notice that I'm putting output files into new folder, for easier debugging in case any issues.
There's one more issue with your code. Here you're converting all the files at once:
$json_test = (Get-Content $files -raw | ConvertFrom-Json)
However, later on, PowerShell has no information about source file name (file name is not included in $json_test).
PowerShell newbie here,
I need to:
Get text files in recursive local directories that have a common string, students.txt in them.
Get another string, gc.student="name,name" in the resulting file set from #1 and get the name(s).
Put the filename from #1, and just the name,name from #2 (not gc.student="") into a hashtable where the filename is paired with its corresponding name,name.
Output the hashtable to an Excel spreadsheet with 2 columns: File and Name.
I've figured out, having searched and learned here and elsewhere, how to output #1 to the screen, but not how to put it into a hashtable with #2:
$documentsfolder = "C:\records\"
foreach ($file in Get-ChildItem $documentsfolder -recurse | Select String -pattern "students.txt" ) {$file}
I'm thinking to get name in #2 I'll need to use a RegEx since there might only be 1 name sometimes.
And for the output to Excel, this: | Export-Csv -NoType output.csv
Any help moving me on is appreciated.
I think this should get you started. The explanations are in the code comments.
# base directory
$documentsfolder = 'C:\records\'
# get files with names ending with students.txt
$files = Get-ChildItem $documentsfolder -recurse | Where-Object {$_.Name -like "*students.txt"}
# process each of the files
foreach ($file in $files)
{
$fileContents = Get-Content $file
$fileName = $file.Name
#series of matches to clean up different parts of the content
#first find the gc.... pattern
$fileContents = ($fileContents | Select-String -Pattern 'gc.student=".*"').Matches.Value
# then select the string with double quotes
$fileContents = ($fileContents | Select-String '".*"').Matches.Value
# then remove the leading and trailing double quotes
$fileContents = $fileContents -replace '^"','' -replace '"$',''
# drop the objects to the pipeline so that you can pipe it to export-csv
# I am creating custom objects so that your CSV headers will nave nice column names
Write-Output [pscustomobject]#{file=$fileName;name=$fileContents}
} | Export-Csv -NoType output.csv
So I recently have found the need to do a find and replace of mutliple items within a XML document. Currently I have found the code below which will allow me to do multiple find and replaces but these are hard coded within the powershell.
(get-content c:\temp\report2.xml) | foreach-object {$_ -replace "192.168.1.1", "Server1"} | foreach-object {$_ -replace "192.168.1.20", "RandomServername"} | set-content c:\temp\report3.xml
Ideally instead of hard coding the value I would like to find and replace from a list, ideally in a CSV or and XLSX. Maybe two txt file would be easier.
If it was from a CSV it could grab the value to find from A1 and the value to replace it with from B1 and keep looping down until the values are empty.
I understand I would have to use the get-content and the for each command I was just wondering if this was possible and how to go about it/ if anybody could help me.
Thanks in advance.
SG
#next line is to clear output file
$null > c:\temp\report3.xml
$replacers = Import-Csv c:\temp\replaceSource.csv
gc c:\temp\aip.xml | ForEach-Object {
$output = $_
foreach ($r in $replacers) {
$output = $output -replace $r.ReplaceWhat, $r.ReplaceTo
}
#the output has to be appended, not to rewrite everything
return $output | Out-File c:\temp\report3.xml -Append
}
Content of replaceSource.csv looks like:
ReplaceWhat,ReplaceTo
192.168.1.1,server1
192.168.1.20,SERVER2
Note the headers
I have updated a piece of software for our T&A system, this produces a CSV file in tab-delimited format. The payroll software needs this in the older format which was semicolon-delimited. I have been in touch with both vendors and neither one has a way to accommodate the other so I need to convert the CSV file to suit the payroll software. I have tried to do this with PowerShell with mixed results.
First I tried
Import-Csv ".\desktop\new version.csv" -Delimiter `t |
Export-Csv ".\converted.csv" -NoTypeInf
which removed the tab delimiter but didn't do the ;. So I then tried
Import-Csv ".\desktop\new version.csv" -Delimiter `t |
Export-Csv ".\desktop\converted.csv" -NoTypeInformation -Delimiter ";"
which did convert it from tabbed to ;, but only for the headers. It totally ignored the rest of the data. I then tried a different approach and used
$path = ".\desktop\new.csv"
$outPath = ".\desktop\converted.csv"
Get-Content -path $path |
ForEach-Object {$_ -replace "`t",";" } |
Out-File -filepath $outPath
which formatted the file correctly, but put an extra empty row between each row of data. I'm not sure what I'm doing wrong!
I'm pretty sure you are having an encoding issue with your last example. Get-Content reads in as Ascii whereas Out-File defaults to Unicode. Either set the -Encoding on Out-File or just use Set-Content.
Get-Content -path $path |
ForEach-Object {$_ -replace "`t",";" } |
Set-Content -filepath $outPath
You could even trim this down a bit if need be.
(Get-Content -path $path) -replace "`t",";" | Set-Content -filepath $outPath
However your 2nd code example...
Import-Csv ".\desktop\new version.csv" -Delimiter `t | Export-Csv ".\desktop\converted.csv" -NoTypeInformation -Delimiter ";"
should have worked just fine to replacing the tabs to semicolons. If it is not working then I would think your source data has an issue.
About the source file
Based on comments the code above is creating a trailing column. Most likely reason for that is trailing tabs on each row that are being converted. If that is the case then a little more manipulation would be required. Easier to use the foreach loop in this case.
Get-Content -path $path |
ForEach-Object {$_.Trim() -replace "`t",";" } |
Set-Content -filepath $outPath
That would remove the last tab/whitespace of each line. There is a potential enormous caveat doing it this way though. I think it has the potential to drop data if you have empty columns on the end. However if those columns were already empty it should not matter as long as the header is formed well and the input program can account for this. Else you are looking at reading in the file with Import-CSV and dropping the last column which can be done.
Here's a function I used to replace strings in text files like you're doing. This is assuming there's no tabs inside the text file other than those that are delimiting the columns. I'm assuming there's not. You can use it like this:
Find-InTextFile -FilePath C:\MyFile.csv -Find "`t" -Replace ';'
function Find-InTextFile
{
<#
.SYNOPSIS
Performs a find (or replace) on a string in a text file or files.
.EXAMPLE
PS> Find-InTextFile -FilePath 'C:\MyFile.txt' -Find 'water' -Replace 'wine'
Replaces all instances of the string 'water' into the string 'wine' in
'C:\MyFile.txt'.
.EXAMPLE
PS> Find-InTextFile -FilePath 'C:\MyFile.txt' -Find 'water'
Finds all instances of the string 'water' in the file 'C:\MyFile.txt'.
.PARAMETER FilePath
The file path of the text file you'd like to perform a find/replace on.
.PARAMETER Find
The string you'd like to replace.
.PARAMETER Replace
The string you'd like to replace your 'Find' string with.
.PARAMETER UseRegex
Use this switch parameter if you're finding strings using regex else the Find string will
be escaped from regex characters
.PARAMETER NewFilePath
If a new file with the replaced the string needs to be created instead of replacing
the contents of the existing file use this param to create a new file.
.PARAMETER Force
If the NewFilePath param is used using this param will overwrite any file that
exists in NewFilePath.
#>
[CmdletBinding(DefaultParameterSetName = 'NewFile')]
param (
[Parameter(Mandatory = $true)]
[ValidateScript({ Test-Path -Path $_ -PathType 'Leaf' })]
[string[]]$FilePath,
[Parameter(Mandatory = $true)]
[string]$Find,
[Parameter()]
[string]$Replace,
[Parameter()]
[switch]$UseRegex,
[Parameter(ParameterSetName = 'NewFile')]
[ValidateScript({ Test-Path -Path ($_ | Split-Path -Parent) -PathType 'Container' })]
[string]$NewFilePath,
[Parameter(ParameterSetName = 'NewFile')]
[switch]$Force
)
begin
{
if (!$UseRegex.IsPresent)
{
$Find = [regex]::Escape($Find)
}
}
process
{
try
{
foreach ($File in $FilePath)
{
if ($Replace)
{
if ($NewFilePath)
{
if ((Test-Path -Path $NewFilePath -PathType 'Leaf') -and $Force.IsPresent)
{
Remove-Item -Path $NewFilePath -Force
(Get-Content $File) -replace $Find, $Replace | Add-Content -Path $NewFilePath -Force
}
elseif ((Test-Path -Path $NewFilePath -PathType 'Leaf') -and !$Force.IsPresent)
{
Write-Warning "The file at '$NewFilePath' already exists and the -Force param was not used"
}
else
{
(Get-Content $File) -replace $Find, $Replace | Add-Content -Path $NewFilePath -Force
}
}
else
{
(Get-Content $File) -replace $Find, $Replace | Add-Content -Path "$File.tmp" -Force
Remove-Item -Path $File
Rename-Item -Path "$File.tmp" -NewName $File
}
}
else
{
Select-String -Path $File -Pattern $Find
}
}
}
catch
{
Write-Error -Message $_.Exception.Message
}
}
}
This is a PowerShell question, not a SharePoint question.
I'm using a script to grab an inventory of SharePoint features, web parts, etc. It outputs each type of report in the same directory as csv files. So I'll end up with a directory on my computer with the csv files.
I'd like to run another PowerShell script after the first one that converts these csvs into html files for easily readable reports.
I'm getting stuck on the part where I would import-csv each file and create each html file with similarly named html files.
Here's what I have so far. Can anyone help me complete this to do what I want it to do? To use Import-CSV, I have to specify the file name as you can see in $dir. Is there another way?
$dir = "C:\Users\me\Desktop\output\TestInvSiteCollections.csv"
dir -LiteralPath $dir | % {Import-Csv $dir}
or use this somehow..
Import-Csv -LiteralPath $dir | ConvertTo-Html | Out-File "C:\Users\me\Desktop\output\myhtmlfile.html"
I would do it like this:
Get-ChildItem -Path 'C:\Users\me\Desktop\output\*.csv' | ForEach-Object {
Import-Csv $_ | ConvertTo-Html | Out-File -FilePath (Join-Path -Path $_.DirectoryName -ChildPath ($_.BaseName + '.html'));
}
I'm not entirely sure I find html tables easier to read than csv files. Excel's filtering and sorting is too useful.
Here's your code. It should split the name of the file and add the extension.
Import-Csv -LiteralPath $dir | ConvertTo-Html | Out-File ($dir.Split(".")[0]+".html")
This is a slightly more verbose method, but I generally prefer readable code over conciseness for maintainability:
#get the list of csv files
$csvFiles = Get-ChildItem $path -Filter *.csv
foreach ($file in $csvFiles)
{
#create FileInfo object
[System.IO.FileInfo]$fileInfo = "$path\$file"
#Get base name of file
$baseName = [System.IO.Path]::GetFileNameWithoutExtension($file.Name)
#do HTML conversion
Import-Csv $fileInfo.FullName | ConvertTo-Html | Out-File "$htmlPath\$baseName.html"
}
This is working code assuming you have $path defined somewhere and can obviously be modified to suite your needs.