How to check if file has valid JSON syntax in Powershell - json

I am trying to write a powershell script that reads a file and prints "true" if it is a valid JSON file. I am using Powershell v3.0 and this is what I have right now :
$text = Get-Content .\filename.txt -Raw
$powershellRepresentation = $text | ConvertFrom-Json
How do I check the return code? I mean I want something like this :
if(file not a JSON file){
Write-Host "not JSON"
}
else{
Write-Host "True"
}

UPDATE 2021: PowerShell 6 and newer versions
PowerShell 6 brings a brand new Test-Json cmdlet. Here is the reference.
You can simply pass the raw file content directly to the Test-Json cmdlet.
$text = Get-Content .\filename.txt -Raw
if ($text | Test-Json) {
$powershellRepresentation = ConvertFrom-Json $text -ErrorAction Stop;
Write-Host "Provided text has been correctly parsed to JSON";
} else {
Write-Host "Provided text is not a valid JSON string";
}
PowerShell 5 and earlier versions
There is no Test-Json cmdlet in these versions, so the best way is to put your ConvertFrom-Json cmdlet inside a try ... catch block
try {
$powershellRepresentation = ConvertFrom-Json $text -ErrorAction Stop;
$validJson = $true;
} catch {
$validJson = $false;
}
if ($validJson) {
Write-Host "Provided text has been correctly parsed to JSON";
} else {
Write-Host "Provided text is not a valid JSON string";
}

If you encounter this question and can use PowerShell 6 or later, there is now a Test-Json cmdlet. It can also not just validate that it's valid JSON, but that it conforms to a particular JSON schema using the -Schema param.
Example
$isValid = Get-Content .\filename.txt -Raw | Test-Json
if($isValid){
Write-Host "not JSON"
}
else{
Write-Host "True"
}
ARM Template Warning
A note for users looking to validate an ARM template via -Schema (I can't imagine a more perfect use case). At the time of writing, there are one or more bugs in the underlying library Test-Json uses for validation, NJsonSchema, and it is not possible to validate an ARM template.
GitHub Issues
PowerShell Issue #9560
NJsonSchema Issue #588

I don't think that it exists an other solution than catching the exception using ConvertFrom-Json.

ConvertFrom-JSON would work but only for a JSON object < 2MB in size.
For higher you can use JavaScriptSerializer class
try
{
$jsser = New-Object System.Web.Script.Serialization.JavaScriptSerializer
$jsser.MaxJsonLength = $jsser.MaxJsonLength * 10
$jsser.RecursionLimit = 99
$outObject = $jsser.DeserializeObject($json)
}
catch
{
Write-Host "Error converting $text to JSON"
}

ConvertFrom-Json shd be the appropriate way to go.
Test-Json unfortunately has alot of known unsupported JSON Types.
I.E. it cannot parse Json-Arrays or Primitives properly leading to falsely assuming it has wrong JSON-Syntax.

Related

Powershell 7.2: ConvertFrom-Json - Date Handling

With Powershell 7.2 there seems to be a change in how a JSON is deserialized into an object in terms of dates -> instead of string it is now datetime. But I want to have the "old" behavior, i.e. that it is handled as string and NOT datetime.
How can I achieve that when using ConvertFrom-Json in Powershell 7.2 all dates are deserialized as string and not datetime?
EDIT:
$val = '{ "date":"2022-09-30T07:04:23.571+00:00" }' | ConvertFrom-Json
$val.date.GetType().FullName
This is actually a known issue, see: #13598 Add a -DateKind parameter to ConvertFrom-Json to control how System.DateTime / System.DateTimeOffset values are constructed. Yet I think there is no easy solution for this. One thing you might do is just invoke (Windows) PowerShell. Which isn't currently straights forward as well therefore I have created a small wrapper to send and receive complex objects between PowerShell sessions (see also my #18460 Invoke-PowerShell purpose):
function Invoke-PowerShell ($Command) {
$SerializeOutput = #"
`$Output = $Command
[System.Management.Automation.PSSerializer]::Serialize(`$Output)
"#
$Bytes = [System.Text.Encoding]::Unicode.GetBytes($SerializeOutput)
$EncodedCommand = [Convert]::ToBase64String($Bytes)
$PSSerial = PowerShell -EncodedCommand $EncodedCommand
[System.Management.Automation.PSSerializer]::Deserialize($PSSerial)
}
Usage:
Invoke-PowerShell { '{ "date":"2022-09-30T07:04:23.571+00:00" }' | ConvertFrom-Json }
date
----
2022-09-30T07:04:23.571+00:00
Update
As commented by mklement0, I clearly complicated the answer.
Calling via powershell.exe is a pragmatic workaround (albeit slow and Windows-only), but note that you don't need a helper function: if you pass a script block to powershell.exe (or pwsh.exe) from PowerShell, Based64 CLIXML-based serialization happens automatically behind the scenes: try powershell.exe -noprofile { $args | ConvertFrom-Json } -args '{ "date":"2022-09-30T07:04:23.571+00:00" }' For that reason, I don't think there's a need for an Invoke-PowerShell cmdlet.
$Json = '{ "date":"2022-09-30T07:04:23.571+00:00" }'
powershell.exe -noprofile { $args | ConvertFrom-Json } -args $Json
date
----
2022-09-30T07:04:23.571+00:00
iRon's helpful answer provides a pragmatic solution via the Windows PowerShell CLI, powershell.exe, relying on the fact that ConvertFrom-Json there does not automatically transform ISO 8601-like timestamp strings to [datetime] instances.
Hopefully, the proposal in the GitHub issue he links to, #13598, will be implemented in the future, which would then simplify the solution to:
# NOT YET IMPLEMENTED as of PowerShell 7.2.x
'{ "date":"2022-09-30T07:04:23.571+00:00" }' |
ConvertFrom-Json -DateTimeKind None
However, a powershell.exe workaround has two disadvantages: (a) it is slow (a separate PowerShell instance in a child process must be launched), and (b) it is Windows-only. The solution below is a generalization of your own approach that avoids these problems.
Here's a generalization of your own in-process approach:
It injects a NUL character ("`0") at the start of each string that matches the pattern of a timestamp - the assumption is that the input itself never contains such characters, which is fair to assume.
This, as in your approach, prevents ConvertFrom-Json from recognizing timestamp strings as such, and leaves them untouched.
The [pscustomobject] graph that ConvertFrom-Json outputs must then be post-processed in order to remove the injected NUL characters again.
This is achieved with a ForEach-Object call that contains a helper script block that recursively walks the object graph, which has the advantage that it works with JSON input whose timestamp strings may be at any level of the hierarchy (i.e. they may also be in properties of nested objects).
Note: The assumption is that the timestamp strings are only ever contained as property values in the input; more work would be needed if you wanted to handle input JSON such as '[ "2022-09-30T07:04:23.571+00:00" ]' too, where the strings are input objects themselves.
# Sample JSON.
$val = '{ "date":"2022-09-30T07:04:23.571+00:00" }'
$val -replace '"(?=\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}[+-]\d{2}:\d{2}")', "`"`0" | #"
ConvertFrom-Json |
ForEach-Object {
# Helper script block that walks the object graph
$sb = {
foreach ($o in $args[0]) {
if ($o -is [Array]) { # nested array -> recurse
foreach ($el in $o) { & $sb $el } # recurse
}
elseif ($o -is [System.Management.Automation.PSCustomObject]) {
foreach ($prop in $o.psobject.Properties) {
if ($prop.Value -is [Array]) {
foreach ($o in $prop.Value) { & $sb $o } # nested array -> recurse
}
elseif ($prop.Value -is [System.Management.Automation.PSCustomObject]) {
& $sb $prop.Value # nested custom object -> recurse
}
elseif ($prop.Value -is [string] -and $prop.Value -match '^\0') {
$prop.Value = $prop.Value.Substring(1) # Remove the NUL again.
}
}
}
}
}
# Call the helper script block with the input object.
& $sb $_
# Output the modified object.
if ($_ -is [array]) {
# Input object was array as a whole (implies use of -NoEnumerate), output as such.
, $_
} else {
$_
}
}
Based on the input from #zett42 here my solution:
Assuming we know the regex pattern of the date used in the JSON I get the JSON as string, add a prefix so that ConvertFrom-Json does not convert dates to datetime but keeps it as string, convert it with ConvertFrom-Json to a PSCustomObject, do whatever I need to do on the object, serialize it back to a JSON string with ConvertTo-Json and then remove the prefix again.
[string]$json = '{ "date":"2022-09-30T07:04:23.571+00:00", "key1": "value1" }'
[string]$jsonWithDatePrefix = $json -replace '"(\d+-\d+.\d+T\d+:\d+:\d+\.\d+\+\d+:\d+)"', '"#$1"'
[pscustomobject]$jsonWithDatePrefixAsObject = $jsonWithDatePrefix | ConvertFrom-Json
$jsonWithDatePrefixAsObject.key1 = "value2"
[string]$updatedJsonString = $jsonWithDatePrefixAsObject | ConvertTo-Json
[string]$updatedJsonStringWithoutPrefix = $updatedJsonString -replace '"(#)(\d+-\d+.\d+T\d+:\d+:\d+\.\d+\+\d+:\d+)"', '"$2"'
Write-Host $updatedJsonStringWithoutPrefix
Two additional ways to change the date format:
Get-Node
Using this Get-Node which is quiet similar to mklement0 recursive function:
$Data = ConvertFrom-Json $Json
$Data |Get-Node -Where { $_.Value -is [DateTime] } | ForEach-Object {
$_.Value = GetDate($_.Value) -Format 'yyyy-MM-ddTHH\:mm\:ss.fffzzz' -AsUTC
}
$Data
DIY
Or do-it-yourself and build your own Json deserializer:
function ConvertFrom-Json {
[CmdletBinding()][OutputType([Object[]])] param(
[Parameter(ValueFromPipeLine = $True, Mandatory = $True)][String]$InputObject,
[String]$DateFormat = 'yyyy-MM-ddTHH\:mm\:ss.fffffffzzz', # Default: ISO 8601, https://www.newtonsoft.com/json/help/html/datesinjson.htm
[Switch]$AsLocalTime,
[Switch]$AsOrdered
)
function GetObject($JObject) {
switch ($JObject.GetType().Name) {
'JValue' {
switch ($JObject.Type) {
'Boolean' { $JObject.Value }
'Integer' { 0 + $JObject.Value } # https://github.com/PowerShell/PowerShell/issues/14264
'Date' { Get-Date $JObject.Value -Format $DateFormat -AsUTC:(!$AsLocalTime) } # https://github.com/PowerShell/PowerShell/issues/13598
Default { "$($JObject.Value)" }
}
}
'JArray' {
,#( $JObject.ForEach{ GetObject $_ } )
}
'JObject' {
$Properties = [Ordered]#{}
$JObject.ForEach{ $Properties[$_.Name] = GetObject $_.Value }
if ($AsOrdered) { $Properties } else { [PSCustomObject]$Properties } # https://github.com/PowerShell/PowerShell/pull/17405
}
}
}
GetObject ([Newtonsoft.Json.Linq.JObject]::Parse($InputObject))
}
Usage:
ConvertFrom-Json $Json -DateFormat 'yyyy-MM-ddTHH\:mm\:ss.fffzzz' |ConvertTo-Json -Depth 9

PowerShell - Read in JSOn file, use string interpolation

I have a JSON file, called data.json with contents:
{
"environment": "${environment}",
"url": "https://${environment}.example.com"
}
I have a PowerShell script, where I am attempting to read in the JSON file and use string interpolation to substitute the value of the PowerShell $environment variable into the JSON file.
script.ps1:
$environment = "qa"
$json = Get-Content -Path ".\data.json"
$formatted = Invoke-Expression "`"${json}`""
Write-host $formatted
When I run this, I can't seem to get around problems with the value being sent to Invoke-Expression as being null or invalid character ':' from the JSON.
Is there an easier/better way of trying to read in a JSON file and perform string interpolation on it?
I am trying to avoid using ExpandString(), string concatenation, string.Replace() and -f.
The end goal, I'm hoping to have a Json PowerShell object (using ConvertFrom-Json) to be:
{
"environment": "qa",
"url": "https://qa.example.com"
}
Thanks in advance!
Try like this:
$environment = 'qa'
$json = Get-Content -Path '.\data.json' -Raw # be sure to add -Raw switch
$formatted = Invoke-Expression "#""`n$json`n""#"
Write-Host $formatted
Thank you #Bill_Stewart and #Mathias R. Jessen - I explored a bit more and it seems like $ExecutionContext.InvokeCommand.ExpandString() works. To be honest, I'm not sure what I read yesterday that led me to believe that ExpandString() would not work. Thank you for your help, much appreciated.

JSON Prettifier Using Azure Function w/ PowerShell and HTTP Trigger

Thought this would be pretty simple, but alas, I can't figure it out. It appears that PowerShell will prettify JSON with a single cmdlet.
Goal: Prettify JSON using a PowerShell Azure Function app
Using Microsoft Flow, send an HTTP request (POST) to an Azure Function w/ "ugly", serialized JSON file in body
Azure Function reads file in (then uses ConvertToJson cmdlet to prettify?) and outputs the file back to Flow
Questions:
What do I put in the run.ps1 area of the Azure Function to make this happen?
What do I put in the functions.json area of the Azure Function to make this happen?
I have taken below serialize string
'{ "baz": "quuz", "cow": [ "moo", "cud" ], "foo": "bar" }'
which was mentioned in Prettify json in powershell 3
Here is my function which i used with HttpPost and send the request:
using namespace System.Net
# Input bindings are passed in via param block.
param($Request, $TriggerMetadata)
# Write to the Azure Functions log stream.
Write-Host "PowerShell HTTP trigger function processed a request."
# Interact with query parameters or the body of the request.
$name = $Request.Query.baz
if (-not $name) {
$name = $Request.Body.baz
}
if ($name) {
$status = [HttpStatusCode]::OK
$body = "Hello $name"
}
else {
$status = [HttpStatusCode]::BadRequest
$body = "Please pass a name on the query string or in the request body."
}
# Associate values to output bindings by calling 'Push-OutputBinding'.
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = $status
Body = $body
})
and below you can see , i am able to read it from the string which i posted.
You can use ConvertFrom-Json to convert it but i wondering if you even need it as you can access it by doing below:
$name = $Request.Query.baz
my binding is same as yours. Hope it helps.
Let me know if you still need any help.
Are you looking for something like this?
using namespace System.Net
param($Request, $TriggerMetadata)
Push-OutputBinding -Name Response -Value ([HttpResponseContext]#{
StatusCode = [HttpStatusCode]::OK
Body = $Request.RawBody | ConvertFrom-Json | ConvertTo-Json
})

ConvertFrom-Json : Invalid object passed in, ':' or '}' expected

Earlier this week, I asked this question to find the best way to go through a directory of text files with log information in JSON format and count how many of each unique messages there are.
I was able to do so with the answers provided. However, the problem I'm having now is that one of the files is formatted in a way that ConvertFrom-JSON doesn't like. It throws the error:
ConvertFrom-Json : Invalid object passed in, ':' or '}' expected.
Initially, I thought I could use 'erroraction -silentlycontinue' to skip that file and move on (there's just one line with nothing meaningful in it). However, it appears to be a known issue that this doesn't work with ConvertFrom-JSON and the alternative is to use a Try / Catch.
How would I use a try / catch to bypass the one bad file? Or is there another way to cleanly skip this file without having to remove it from the directory?
Here is what I have started out with. It's not much, but some guidance on this would be great. I have also seen some info online that I would use gc -Raw or Out-string before the ConvertFrom-JSON, but that gave me the same result.
try {
gci -Path "path" | gc | ConvertFrom-Json | Group-Object message -NoElement
}
catch {
write-host "can't convert file to JSON"
}
finally {
}
You'll have to insert your error handling into the pipeline. This is where ForEach-Object will come in handy:
Get-ChildItem -Path "path" |
ForEach-Object -Process {
try {
$_ | Get-Content | ConvertFrom-Json
} catch {
write-host "can't convert file '$_' to JSON"
}
} | Group-Object message -NoElement
The successful conversions will be passed through.
Additionally, consider using Write-Warning instead of Write-Host for your error condition. It seems to best fit this situation, and can be redirected or opted out by an invoker.
If you think this condition is even less serious than a warning, consider Write-Verbose so invokers can opt in instead.
The answer worked for me
ConvertFrom-Json cannot read my JSON
Use the -Raw parameter of the Get-Content cmdlet, otherwise Get-Content reads each line separately and it will be stored in the variable as an array.
$json = Get-Content c:\temp\net\cars.json -Raw
ConvertFrom-Json $json
Try this and you will end up with a nicely usable object:
$cars = Get-Content c:\temp\net\cars.json -Raw | ConvertFrom-Json | Select -ExpandProperty cars

Merging JSON Objects at runtime in Powershell

Following on from a question yesterday (JSON and references to other JSON objects).
Is it possible to merge JSON objects at runtime in a similar fashion?
In my test.json I wish to insert the $Defaults.wimos object into WIMS.wimos at runtime similar to what I did for the $Paths.drive value in $Defaults.
{
Paths: {drive: "W:"},
Defaults: {wimos: {dstdrive: "$($Paths.drive)"}
},
WIMS: {
winos: "$($Defaults.wimos)",
wimre: {dstdrive: "$($Paths.drive)"}
}
}
In the following code I cannot work out the syntax to have the object replaced at runtime.
$JSONConfig="test.json"
$rawJSON = (Get-Content $JSONConfig -Raw)
$pathJSON = $rawJSON | ConvertFrom-Json
#
# Load Paths from JSON into $Paths
#
$Paths=$pathJSON.Paths
#
# Merge JSON objects to have the defaults replaced
#
$DefaultsJSON=($ExecutionContext.InvokeCommand.ExpandString($rawJSON)) | ConvertFrom-Json
#
# Load Defaults into $Defaults
#
$Defaults=$DefaultsJSON.Defaults
#
# Merge JSON objects to have the System replaced
#
$JSON = ($ExecutionContext.InvokeCommand.ExpandString($rawJSON)) | ConvertFrom-Json
$JSON.WIMS
write-host ("JSON.WIMS.wimre.dstdrive =" + $JSON.WIMS.wimre.dstdrive)
write-host ("JSON.WIMS.wimos.dstdrive =" + $JSON.WIMS.wimos.dstdrive) #This fails to access and print "W:"
I would like to be able to replace $($Defaults.wimos) and be able to query the dstdrive member.
Is this possible to achieve this in powershell? Any suggestions on how?
Thanks
Stuart
There's a couple of issues in your code. First, the value that would be inserted into your template would be interpolated as a string and not a json-object.
To do this, you could use the ConvertTo-json within the template. I'll show you how in a minute.
If you're gonna be recreating and using different "templates" within the same file (like path, defaults etc), I would create a helper function for recreating / filling out the template on your session variables all the time (in order to avoid repeating code).
Create the following method:
function Get-ParsedJsonTemplate(){
return ($ExecutionContext.InvokeCommand.ExpandString((Get-Content $JSONConfig -Raw))) | ConvertFrom-Json
}
Change your template json to the following (showing you that you can use powershell code within your template file:
{
Paths: {drive: "W:"},
Defaults: {wimos: {dstdrive: "$($Paths.drive)"}},
WIMS: {
winos: $(
if($Defaults -and $Defaults.wimos){
$Defaults.wimos|ConvertTo-Json -Compress
} else {
#{"dstdrive"=""}|ConvertTo-Json -compress
}
),
wimre: {dstdrive: "$($Paths.drive)"}
}
}
You also had a typo in either your template or your printout (I see that you used wimos in your "printout" in powershell and "winos" in your template. I modified the code and the following should work for you (given the above test.json):
$JSONConfig="c:\tmp\test.json"
function Get-ParsedJsonTemplate(){
return ($ExecutionContext.InvokeCommand.ExpandString((Get-Content $JSONConfig -Raw))) | ConvertFrom-Json
}
$pathJSON = Get-ParsedJsonTemplate;
$Paths=$pathJSON.Paths
$DefaultsJSON=Get-ParsedJsonTemplate;
$Defaults=$DefaultsJSON.Defaults
$JSON = Get-ParsedJsonTemplate;
$JSON.WIMS
write-host ("JSON.WIMS.wimre.dstdrive =" + $JSON.WIMS.wimre.dstdrive)
write-host ("JSON.WIMS.wimos.dstdrive =" + $JSON.WIMS.winos.dstdrive)
The reason your script was failing was due to the fact that $Defaults.wimos was not the string { dstdrive: "W:" } it was instead a PSCustomObject which would be interpolated to a the value #{drive=W:}. To obtain the correct behavior, you need to convert the $Defaults.wimos value back to a json string.
$Defaults = $DefaultsJSON.Defaults
$Defaults.wimos = $Defaults.wimos | ConvertTo-Json -Compress