I have spent a bit of time researching how to convert Json to a CSV file using powershell but am failing to have it complete properly. Below is the syntax I have created:
$pathToOutputFile = "C:\OrderLinQ\TESTING\IN\CatalogsRetrieved\test.txt"
$pathToJsonFile = "C:\OrderLinQ\TESTING\IN\CatalogsRetrieved\test.json"
Get-Content -Path $pathToJsonFile |
ConvertFrom-Json |
ConvertTo-Csv -NoTypeInformation |
Set-Content $pathToOutputFile
However, when I attempt to execute this, I get an argument exception error:
ConvertFrom-Json : Invalid array passed in, ']' expected. (1): [
At line:1 char:69
+ Get-Content "C:\OrderLinQ\TESTING\IN\CatalogsRetrieved\test.json" | ConvertFrom- ...
+ ~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand
Any thoughts as to what I'm doing wrong? I've verified my file using JSONLint and it shows verified. Example file can be found at https://api.myjson.com/bins/mo59w. Any help is appreciated!
Try this. I've also attached the output
$pathToOutputFile = "D:\output.csv"
$pathToJsonFile = "D:\test.json"
$json = Get-Content -Path $pathToJsonFile
function ConvertFrom-JsonToCsv {
param(
[Parameter(ValueFromPipeline)]
$json
)
Process {
($json | ConvertFrom-Json) | ConvertTo-Csv -NoTypeInformation
}
}
ConvertFrom-JsonToCsv $json | Set-Content $pathToOutputFile
Output
Related
I use a command to read a JSON file, this all works perfectly, until the file becomes large.
I currently have a JSON file of about 1.5GB. I read the file using Powershell using the following command:
get-content -Path C:\TEMP\largefile.json | out-string | ConvertFrom-Json
It returns the following error:
out-string : Exception of type 'System.OutOfMemoryException' was thrown.
+ ... oices = get-content -Path C:\TEMP\largefile.json | out-string | Conve ...
+ ~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Out-String], OutOfMemoryException
+ FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.OutStringCommand
I've increased the memory as shown here:
get-item wsman:localhost\Shell\MaxMemoryPerShellMB
WSManConfig: Microsoft.WSMan.Management\WSMan::localhost\Shell
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String MaxMemoryPerShellMB 8096
Any ideas on how to process this?
Edit (additions based on comments):
When I remove the out-string I get this error:
ConvertFrom-Json : Exception of type 'System.OutOfMemoryException' was thrown.
+ ... oices = get-content -Path C:\TEMP\largefile.json | ConvertFrom-Json ...
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [Out-String], OutOfMemoryException
+ FullyQualifiedErrorId : System.OutOfMemoryException,Microsoft.PowerShell.Commands.OutStringCommand
The Powershell version that I have is: 5.1.17763.1490
The file contains multiple columns regarding PDF files. These files are exported via an API into a JSON so it contains the file metadata such as owner and when it was created but also the actual PDF file in the column Body which later will be decoded to an actual PDF file. The structure is as followed:
[{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
{"Id":"ID","ParentId":"parent","Name":"filename","OwnerId":"owner","CreatedDate":"date","Body":"*******"}
]
Thank for the details.
For this issue I would try to convert each line separately and stream that through your process:
Get-Content C:\TEMP\largefile.json | ForEach-Object {
$_ = $_.Trim().TrimStart('[').TrimEnd(']')
if ($_) { $_ | ConvertFrom-Json }
}
As already suggested, I wouldn't be surprised if these memory issues wouldn't appear in PowerShell core. if possible, I recommend you to also give that a try.
Been trying all day to import my CSV file to my AD on Windows Server 2012.
But keep getting all kind of errors.
The error I get is this:
"New-ADUser : Cannot validate argument on parameter 'Name'. The argument is null
or empty. Provide an argument that is n ot null or empty, and then try the
command again.
At line:3 char:90
+ ... ipalname -Name $_.name -DisplayName $_.name -GivenName $_.cn -SurName $_.sn -Dep ...
+ ~~~~~~~
+ CategoryInfo : InvalidData: (:) [New-ADUser], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.ActiveDirectory.Management.Commands.NewADUsery
Can't seem to find any solution to this, so some help would be great. Here is my files.
This is my script:
Import-Csv .\nb1.csv | ForEach-Object {
$userprincipalname = $_.SamAccountName + "#hirethistire.local"
New-ADUser -SamAccountName $_.SamAccountName -UserPrincipalName $userprincipalname -Name $_.name -DisplayName $_.name -GivenName $_.cn -SurName $_.sn -Department $_.Department -Path "CN_Users,DC=HireThisTire,DC=local"-AccountPassword (ConvertTo-SecureString "Microsoft~1;" -AsPlainText -force) -Enabled $true -PasswordNeverExpires $true -PassThru
}
Sample Of the CSV File:
I found a PowerShell script that looks to be just what I need, but I am having trouble exporting the output to a CSV. I think it may be the foreach portion that is giving the trouble, was wondering if anyone can pinpoint the cause.
$Groups = Get-ADGroup -Properties * -Filter * -SearchBase "OU=Groups,DC=corp,DC=ourcompany,DC=Com"
foreach ($G in $Groups)
{
Write-Host $G.Name
Write-Host "-------------"
$G.Members
}
Specific error is:
An empty pipe element is not allowed.
At line:6 char:2
+ }|Export-csv -path c:\temp\adusers.csv
+ ~
+ CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : EmptyPipeElement
Give this one a try:
$Groups = Get-ADGroup -Properties * -Filter * -SearchBase "OU=Groups,DC=corp,DC=ourcompany,DC=Com"
$Groups | Select Name, Members | Export-Csv -Path "C:\temp\myfile.csv" -Delimiter ";"
if you want all attributes just use
$Groups | Export-Csv -Path "C:\temp\myfile.csv" -Delimiter ";"
or get your attributes as you like to:
$Groups | Select Name, Members, DistinguishedName, Email | Export-Csv -Path "C:\temp\myfile.csv" -Delimiter ";"
For some reason I have a script that started to give me problems all of a sudden. Maybe a co-worker modified it, I don't know but I need to get it running again.
This is my script that is causing the problem:
$result = ForEach-Object $filter {
Select-String -Pattern $pattern |
Select-Object Path, LineNumber, Line }
$result | Export-Csv "W:\test\search_results\$name.csv" -NoType
This is what $filter contains if I export it to CSV rather than $result:
FullName
\\omega.dce-eir.net\etm\ETMWinHeadPROD\CRA_WEB_UA\cra_htdocs\bt\fq-eng.html
\\omega.dce-eir.net\etm\ETMWinHeadPROD\CRA_WEB_UA\cra_htdocs\bt\ll_fq-eng.html
\\omega.dce-eir.net\etm\ETMWinHeadPROD\CRA_WEB_UA\cra_htdocs\bt\menu-eng.html
I am getting this error when running my code:
ForEach-Object : Cannot convert 'System.Object[]' to the type 'System.Management.Automation.ScriptBlock' required by parameter 'Process'. Specified method is not supported.
At C:\Tools\menu.ps1:41 char:37
+ $result = ForEach-Object <<<< $filter {
+ CategoryInfo : InvalidArgument: (:) [ForEach-Object], ParameterBindingException
+ FullyQualifiedErrorId : CannotConvertArgument,Microsoft.PowerShell.Commands.ForEachObjectCommand
What am I missing in this script? All it needs to do, is a search string on the files listed in the CSV.
$filter is a list of objects, not a scriptblock. You can't use it like that with ForEach-Object. Change your code to this:
$filter | Select-Object -Expand FullName |
ForEach-Object { Select-String -Path $_ -Pattern $pattern } |
Select-Object Path, LineNumber, Line |
Export-Csv "W:\test\search_results\$name.csv" -NoType
I have this file which I want to read with PowerShell:
var myMap =
[
{
"name": "JSON Example",
"attr": "Another attribute"
}
]
My PowerShell v3 Code:
$str = Get-Content $file | Select -Skip 1;
$str | ConvertFrom-Json;
But I'm always getting this error:
ConvertFrom-Json : Invalid array passed in, ']' expected. (1): [
At S:\ome\Path\script.ps1:60 char:8
+ $str | ConvertFrom-Json;
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand
If I copy and paste the JSON code manually into the code, everything is working fine:
'[
{
"name": "JSON Example",
"attr": "Another attribute"
}
]' | ConvertFrom-Json;
Try to pipe to Out-String before piping to ConvertFrom-Json:
Get-Content $file | Select -Skip 1 | Out-String | ConvertFrom-Json
In your working example the JSON code is a string while the non-working example returns a collection of lines. Piping to Out-String converts the collection to a single string, which is what the InputObject parameter accept.
Alternatively you can use Get-Content -Raw which will retrieve the JSON as a single string.
See this post for more info: http://blogs.technet.com/b/heyscriptingguy/archive/2014/04/23/json-is-the-new-xml.aspx
The var myMap = isn't json, it's javascript. Delete the first line and it will be fine.
EDIT:
Oh, you are skipping the first line. It may be that there's a carriage return missing in the last line of the file, and Powershell 3 is more sensitive to it. It works fine in Powershell 5.1 even without a carriage return at the end.
another answer that also works: use GC -raw <FILE> which will pass in as string