I have a JSON file which looks like this:
{
"body": {
"mode": "raw",
"raw": "{\n \"id\": \"middleware://so/998655555{{sguid}}\",\n \"reference\": \"998655555{{sguid}}\",\n \"branchCode\": \"1234\"\n }"
}
}
Now, I want to output the value of e.g. "branchCode".
I tried it with the following PowerShell commands:
$path = "file.json"
$raw = Get-Content $path -raw
$obj = ConvertFrom-Json $raw
$obj.body.raw.branchCode
Unfortunately, I don't get the value. It seems like PowerShell has a problem to retrieve the value from the raw json.
I would be very happy if someone could help me.
The value of "raw" is itself JSON.
You have to convert from JSON twice.
$data = Get-Content "file.json" -Raw -Encoding UTF8 | ConvertFrom-Json
$body = $data.body.raw | ConvertFrom-Json
$body.branchCode
prints
1234
Note that you should always explicitly specify an encoding when reading from and writing to text files. For the purposes of JSON (and, frankly, for pretty much every other purpose), UTF-8 is the appropriate choice.
Unfortunately Get-Content/Set-Content do not default to UTF-8. Save yourself from trouble by always specifying an encoding when working with text files.
Related
I have a JSON file, called data.json with contents:
{
"environment": "${environment}",
"url": "https://${environment}.example.com"
}
I have a PowerShell script, where I am attempting to read in the JSON file and use string interpolation to substitute the value of the PowerShell $environment variable into the JSON file.
script.ps1:
$environment = "qa"
$json = Get-Content -Path ".\data.json"
$formatted = Invoke-Expression "`"${json}`""
Write-host $formatted
When I run this, I can't seem to get around problems with the value being sent to Invoke-Expression as being null or invalid character ':' from the JSON.
Is there an easier/better way of trying to read in a JSON file and perform string interpolation on it?
I am trying to avoid using ExpandString(), string concatenation, string.Replace() and -f.
The end goal, I'm hoping to have a Json PowerShell object (using ConvertFrom-Json) to be:
{
"environment": "qa",
"url": "https://qa.example.com"
}
Thanks in advance!
Try like this:
$environment = 'qa'
$json = Get-Content -Path '.\data.json' -Raw # be sure to add -Raw switch
$formatted = Invoke-Expression "#""`n$json`n""#"
Write-Host $formatted
Thank you #Bill_Stewart and #Mathias R. Jessen - I explored a bit more and it seems like $ExecutionContext.InvokeCommand.ExpandString() works. To be honest, I'm not sure what I read yesterday that led me to believe that ExpandString() would not work. Thank you for your help, much appreciated.
How does one convert a text file's contents into a string and then insert this string into a JSON file?
For example, if a file contains:
this
is
a
sample
file
The script would generate:
"this\r\nis\r\na\r\nsample\r\nfile"
To insert into a JSON template:
"something":"<insertPoint>"
To produce:
"something":"this\r\nis\r\na\r\nsample\r\nfile"
I'm using Powershell 5 and have managed to load the file, generate some JSON and insert it by running:
# get contents and convert to JSON
$contentToInsert = Get-Content $sourceFilePath -raw | ConvertTo-Json
# write in output file
(Get-Content $outputFile -Raw).replace('<insertPoint>', $contentToInsert) | Set-Content $outputFile
However, a lot of other, unwanted fields are also added.
"something":"{
"value": "this\r\nis\r\na\r\nsample\r\nfile"
"PSPath": "C:\\src\\intro.md",
"PSParentPath": "C:\\src",
"PSChildName": "intro.md",
etc...
Ultimately, I'm trying to send small rich text segments to a web page via JSON but want to edit and store them locally using Markdown. If this doesn't make sense and there's a better way of sending these then please let me know also.
iRon's answer helpfully suggests not using string manipulation to create JSON in PowerShell, but to use hashtables (or custom objects) to construct the data and then convert it to JSON.
However, that alone does not solve your problem:
PS> #{ something = Get-Content -Raw $sourceFilePath } | ConvertTo-Json
{
"something": {
"value": "this\nis\na\nsample\nfile\n",
"PSPath": "/Users/mklement/Desktop/pg/lines.txt",
# ... !! unwanted properties are still there
}
The root cause of the problem is that Get-Content decorates the strings it outputs with metadata in the form of NoteProperty properties, and ConvertTo-Json currently invariably includes these.
A proposal to allow opting out of this decoration when calling Get-Content can be found in GitHub issue #7537.
Complementarily, GitHub issue #5797 suggests that ConvertTo-Json should ignore the extra properties for primitive .NET types such as strings.
The simplest workaround is to access the underlying .NET instance with .psobject.baseobject, which bypasses the invisible wrapper object PowerShell uses to supply the extra properties:
PS> #{ something = (Get-Content -Raw $sourceFilePath).psobject.baseobject } |
ConvertTo-Json
{
"something": "this\nis\na\nsample\nfile\n"
}
Just a general recommendation apart from the actually issue described by #mklement0 and metadata added to the Get-Content results:
Do not poke (replace, insert, etc.) in any Json content.
Instead, modify the object (if necessary, use ConvertFrom-Json to restore the object) prior converting it into (ConvertTo-Json) a Json file.
In this example, I would use a hash-table with a here-string for this:
#{'something' = #'
this
is
a
sample
file
'#
} | ConvertTo-Json
Result:
{
"something": "this\nis\na\nsample\nfile"
}
You can use the Out-String cmdlet to coerce the output of Get-Content into a flat string first:
#{ "something" = (Get-Content lines.txt | Out-String) } | ConvertTo-Json
This produces:
{
"something": "this\r\nis\r\na\r\nsample\r\nfile\r\n"
}
I have a PS script, which get JSON in variable ant then saves it in file.
Unfortunately, it get value in one string, like this:
{ "persistentdataapi": "https://somevalue.azurewebsites.net/", "collectioncountapi": "https://anothervalue.azurewebsites.net/", "eventserviceapi": "https://thirdvalue.azurewebsites.net/", "securityserviceapi": "https://fourthvalue.azurewebsites.net/" }
Is there any way, to process this value through some (preferably PS) JSON formatting, to get this one:
{
"persistentdataapi": "https://somevalue.azurewebsites.net/",
"collectioncountapi": "https://anothervalue.azurewebsites.net/",
"eventserviceapi": "https://thirdvalue.azurewebsites.net/",
"securityserviceapi": "https://fourthvalue.azurewebsites.net/",
}
Code to get value in Jenkins:
Import-Module "C:\Program Files\WindowsPowerShell\Modules\Octopus-Cmdlets\0.4.4\Octopus-Cmdlets.psd1"
connect-octoserver http://internal-Octopus.azure.com:8082 API-123456789012345678
$raw = (Get-OctoVariable var.Portal.Web DataAPIJson | Where-Object { $_.Environment -eq "QA" } )
$raw.Value | Out-File "$env:WORKSPACE\portal\var.Portal.Web\dataapi.json"
Powershell by default pretty-prints any JSON it produces.
So the correct way to do pretty-printing is to parse the JSON string into an object, and immediately convert it back to a JSON string.
$json = '{ "persistentdataapi": "https://somevalue.azurewebsites.net/", "collectioncountapi": "https://anothervalue.azurewebsites.net/", "eventserviceapi": "https://thirdvalue.azurewebsites.net/", "securityserviceapi": "https://fourthvalue.azurewebsites.net/" }'
$json | ConvertFrom-Json | ConvertTo-Json
produces
{
"persistentdataapi": "https://somevalue.azurewebsites.net/",
"collectioncountapi": "https://anothervalue.azurewebsites.net/",
"eventserviceapi": "https://thirdvalue.azurewebsites.net/",
"securityserviceapi": "https://fourthvalue.azurewebsites.net/"
}
or in your case
$file = "$env:WORKSPACE\portal\var.Portal.Web\dataapi.json"
$raw.Value | ConvertFrom-Json | ConvertTo-Json | Out-File $file -Encoding UTF8
As a side-effect this also makes sure that the JSON in the file is valid, because otherwise ConvertFrom-Json will throw an error.
Please always explicitly specify UTF8 encoding when reading and writing JSON files.
$data = Get-Content $file -Encoding UTF8 | ConvertFrom-Json
$data | ConvertTo-Json | Set-Content $file -Encoding UTF8
The reason for that is
By widely-accepted convention, JSON files ought to be UTF8.
Unless specified otherwise, Get-Content and Set-Content will use the system's default encoding to read/write text files.
The system default is very seldom UTF-8, most of the time it will be a legacy single-byte encoding like Windows-1252.
This creates the risk of
mangling Unicode characters, which are legal in JSON, when reading a JSON file.
creating JSON files that are not UTF-8, making them hard to consume by others.
In fact, always specify an encoding explicitly when working with text files, not only in the case of JSON.
Is there a way to minify (remove all whitespace in this case) a JSON file to turn this
[
0.000005,
0,
0
],
[
219.740502,
0.003449,
4.177065
],
[
45.210918,
0.003365,
-16.008996
],
[
344.552785,
0.030213,
277.614965
],
to this using PowerShell
[0.000005,0,0],[219.740502,0.003449,4.177065],[45.210918,0.003365,-16.008996],[344.552785,0.030213,277.614965],
I have tried several online "minifiers" however the file contains over 100,000 arrays and basically broke all the online minifiers. Any ideas?
Just for your information when you manipulate PowerShell object and convert them to JSON (ConvertTo-Json) you've got the -compress param :
New-Object -TypeName PSCustomObject -Property #{Name="Hugot";GivenName="Victor"} | ConvertTo-Json -Compress
gives :
{"GivenName":"Victor","Name":"Hugot"}
Adding to JP's answer,
(ConvertFrom-Json $json) | ConvertTo-Json -Compress
This is useful if you already have json. If you don't do the ConvertFrom-Json on the front, then it will encode the line breaks with \r\n and the quotes with \".
You can easily do so with a basic regex. If you have this in a file try the following. You must include the -Raw parameter or the file will be passed one line at a time which will prevent the regex from removing the newline character.
(Get-Content C:\Some\File.json -Raw) -replace '\s','' | out-file C:\some\outfile.json
When trying to convert a JSON file via PowerShell:
$json = Get-Content "C:\folder1\test.txt"
$json | ConvertFrom-Json
write-output $json
I'm getting the following error:
invalid json primitive : [.
(system.argunment.exception)
I'm going out on a limb here, since you didn't provide your input data or the complete error message, but I guess that your problem is caused by a format mismatch between the output Get-Content provides and the input ConvertFrom-Json expects.
Get-Content reads the input file into an array of strings, whereas ConvertFrom-Json expects the JSON data in a single string. Also, piping $json into ConvertFrom-Json does not change the value of $json.
Change your code to the following and the error should disapear (provided there is no syntactical error in your input data):
$json = Get-Content 'C:\folder1\test.txt' | Out-String | ConvertFrom-Json
Write-Output $json
You should check your JSON input file for characters that are not properly escaped with a "\"
I have also seen this issue with an input JSON file that was incorrectly formatted as follows:
{
Object1
}
{
Object2
}
Corrected format:
[{
Object1
},
{
Object2
}]
Once the format was corrected, I had no more issues.
I was also receiving this error, and upon investigating my json file noticed that some of the JSON was invalid. I was ending the last object in an array with a comma like so:
[{ ..},]
Removing the comma fixed the issue for myself.
So in short, invalid JSON caused this issue for me.
You will get this error if your input data starts like this:
data: [
{
...
},
{
...
}
]
You need to remove data: (and only have [and ] in this example):
[
{
...
},
{
...
}
]