Kind of have a strange problem. I have a large JSON file that needs to be processed. Based on another question I need to stream the file since it will otherwise gets me problems because of memory: JSON Powershell memory issue
What I have is this:
get-content -Path largefile.json | ForEach-Object {
$row = $_ = $_.TrimStart('[').TrimEnd(']')
if ($_) { $_ | Out-String | ConvertFrom-Json }
New-Item -Path $($Row.Id).txt
Set-Content -Path $($Row.Id).txt -Value ($row.Body)
}
I can easily do $row to publish the last processed row in the Largefile.json. I want to create a file with the name of the Id in the row that is currently processed and add the body column to the file. But when I want to show a specific column using $row.Id, unfortunately this shows up empty.
The structure of the Largefile.json is as followed:
[{"Id":"1","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data1"}
{"Id":"2","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data2"}
{"Id":"3","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data3"}
{"Id":"4","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data4"}
{"Id":"5","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"data5"}
]
The end result should be that I have 5 files:
1.txt - Value inside the file should be: data1
2.txt - Value inside the file should be: data2
3.txt - Value inside the file should be: data3
4.txt - Value inside the file should be: data4
5.txt - Value inside the file should be: data5
I use Powershell 7.1.3
Is there any way that I can use $row.Id and $row.ParentId just like a regular ForEach would do?
thanks for your help.
It seems to me that this is what you're looking for:
Get-Content largefile.json | ForEach-Object {
$row = $_.TrimStart('[').TrimEnd(']') | ConvertFrom-Json
if ($null -ne $row) {
Set-Content -Path ($row.Id) -Value ($row.Body)
}
}
I am still not sure what you expect as an outcome.
But I think you want to do this:
#'
[{"Id":"1","ParentId":"parent1","Name":"1.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data1"}
{"Id":"2","ParentId":"parent2","Name":"2.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data2"}
{"Id":"3","ParentId":"parent3","Name":"3.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data3"}
{"Id":"4","ParentId":"parent4","Name":"4.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data4"}
{"Id":"5","ParentId":"parent5","Name":"5.txt","OwnerId":"owner","CreatedDate":"date","Body":"Data5"}
]
'# | Set-Content .\largefile.json
Get-Content .\largefile.json | ForEach-Object {
$_ = $_.TrimStart('[').TrimEnd(']')
If ($_) {
$Row = ConvertFrom-Json $_
Set-Content -Path ".\$($Row.Name)" -Value $Row.Body
}
}
The question has many errors. Assuming the json has the missing commas in, I would do it this way, if I understand the question. This should work with the new updates to the question. I also have a more unusual solution involving streaming json with jq here: Iterate though huge JSON in powershell Json streaming support may be added later: ConvertFrom-JSON high memory consumption #7698
[{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"},
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
]
get-content -Path largefile.json | ForEach-Object {
$_ = $_.TrimStart('[').TrimEnd(']').TrimEnd(',')
if ($_) {
$row = $_ | ConvertFrom-Json
Set-Content -Path ($Row.Id + '.txt') -Value $row.Body
}
}
get-content ID.txt
*******
As others already explained, your json example is invalid.
However, since this is a huge file to process, you can use switch for this.
switch -Regex -File D:\Test\largefile.json {
'"Id":"(\d+)".*"Body":"(\w+)"' {
Set-Content -Path ('D:\Test\{0}.txt' -f $matches[1]) -Value $matches[2]
}
}
Results using your example would be 5 files called 1.txt .. 5.txt, each having a single line data1 .. data5
I've the JSON file with the following structure:
{
"Url": "placeholder",
}
I'd like to write a Powershell script which replaces the value ("placeholder") by key ("Url").
I know how to replace content by I don't know how to save it back to the file.
$snapshot = (Get-Content settings.json | ConvertFrom-Json)
$snapshot.Url = "Test"
You can write it back using the Set-Content cmdlet. E. g.:
$snapshot | ConvertTo-Json | Set-Content
I have a PowerShell scripts which replaces
"version" : "xxx"
with
"version" : "myBuildNumber"
Now I encountered that I have multiple of these in my file.
I only want to replace the first occurrence.
I tried already Powershell - Replace first occurrences of String but it does not work with my regex.
Here's my script:
(Get-Content myFile.txt) -replace '(?<pre>"version"[\s]*:[\s]*)(?<V>"[^\"]*")', "`$1`"$Env:BUILD_VERSION`"" | Out-File myFile.txt
Since you are patching a JSON file, regex isn't the way to go. Instead you should parse the JSON, access and change the property you want and write it back:
$filePath = 'your_Path_To_project.json'
$json = (Get-Content $filePath -raw | ConvertFrom-Json)
$json.version = $Env:BUILD_VERSION
$json | ConvertTo-Json -Depth 10 | Set-Content $filePath
PowerShell newbie here,
I need to:
Get text files in recursive local directories that have a common string, students.txt in them.
Get another string, gc.student="name,name" in the resulting file set from #1 and get the name(s).
Put the filename from #1, and just the name,name from #2 (not gc.student="") into a hashtable where the filename is paired with its corresponding name,name.
Output the hashtable to an Excel spreadsheet with 2 columns: File and Name.
I've figured out, having searched and learned here and elsewhere, how to output #1 to the screen, but not how to put it into a hashtable with #2:
$documentsfolder = "C:\records\"
foreach ($file in Get-ChildItem $documentsfolder -recurse | Select String -pattern "students.txt" ) {$file}
I'm thinking to get name in #2 I'll need to use a RegEx since there might only be 1 name sometimes.
And for the output to Excel, this: | Export-Csv -NoType output.csv
Any help moving me on is appreciated.
I think this should get you started. The explanations are in the code comments.
# base directory
$documentsfolder = 'C:\records\'
# get files with names ending with students.txt
$files = Get-ChildItem $documentsfolder -recurse | Where-Object {$_.Name -like "*students.txt"}
# process each of the files
foreach ($file in $files)
{
$fileContents = Get-Content $file
$fileName = $file.Name
#series of matches to clean up different parts of the content
#first find the gc.... pattern
$fileContents = ($fileContents | Select-String -Pattern 'gc.student=".*"').Matches.Value
# then select the string with double quotes
$fileContents = ($fileContents | Select-String '".*"').Matches.Value
# then remove the leading and trailing double quotes
$fileContents = $fileContents -replace '^"','' -replace '"$',''
# drop the objects to the pipeline so that you can pipe it to export-csv
# I am creating custom objects so that your CSV headers will nave nice column names
Write-Output [pscustomobject]#{file=$fileName;name=$fileContents}
} | Export-Csv -NoType output.csv
So I recently have found the need to do a find and replace of mutliple items within a XML document. Currently I have found the code below which will allow me to do multiple find and replaces but these are hard coded within the powershell.
(get-content c:\temp\report2.xml) | foreach-object {$_ -replace "192.168.1.1", "Server1"} | foreach-object {$_ -replace "192.168.1.20", "RandomServername"} | set-content c:\temp\report3.xml
Ideally instead of hard coding the value I would like to find and replace from a list, ideally in a CSV or and XLSX. Maybe two txt file would be easier.
If it was from a CSV it could grab the value to find from A1 and the value to replace it with from B1 and keep looping down until the values are empty.
I understand I would have to use the get-content and the for each command I was just wondering if this was possible and how to go about it/ if anybody could help me.
Thanks in advance.
SG
#next line is to clear output file
$null > c:\temp\report3.xml
$replacers = Import-Csv c:\temp\replaceSource.csv
gc c:\temp\aip.xml | ForEach-Object {
$output = $_
foreach ($r in $replacers) {
$output = $output -replace $r.ReplaceWhat, $r.ReplaceTo
}
#the output has to be appended, not to rewrite everything
return $output | Out-File c:\temp\report3.xml -Append
}
Content of replaceSource.csv looks like:
ReplaceWhat,ReplaceTo
192.168.1.1,server1
192.168.1.20,SERVER2
Note the headers