Earlier this week, I asked this question to find the best way to go through a directory of text files with log information in JSON format and count how many of each unique messages there are.
I was able to do so with the answers provided. However, the problem I'm having now is that one of the files is formatted in a way that ConvertFrom-JSON doesn't like. It throws the error:
ConvertFrom-Json : Invalid object passed in, ':' or '}' expected.
Initially, I thought I could use 'erroraction -silentlycontinue' to skip that file and move on (there's just one line with nothing meaningful in it). However, it appears to be a known issue that this doesn't work with ConvertFrom-JSON and the alternative is to use a Try / Catch.
How would I use a try / catch to bypass the one bad file? Or is there another way to cleanly skip this file without having to remove it from the directory?
Here is what I have started out with. It's not much, but some guidance on this would be great. I have also seen some info online that I would use gc -Raw or Out-string before the ConvertFrom-JSON, but that gave me the same result.
try {
gci -Path "path" | gc | ConvertFrom-Json | Group-Object message -NoElement
}
catch {
write-host "can't convert file to JSON"
}
finally {
}
You'll have to insert your error handling into the pipeline. This is where ForEach-Object will come in handy:
Get-ChildItem -Path "path" |
ForEach-Object -Process {
try {
$_ | Get-Content | ConvertFrom-Json
} catch {
write-host "can't convert file '$_' to JSON"
}
} | Group-Object message -NoElement
The successful conversions will be passed through.
Additionally, consider using Write-Warning instead of Write-Host for your error condition. It seems to best fit this situation, and can be redirected or opted out by an invoker.
If you think this condition is even less serious than a warning, consider Write-Verbose so invokers can opt in instead.
The answer worked for me
ConvertFrom-Json cannot read my JSON
Use the -Raw parameter of the Get-Content cmdlet, otherwise Get-Content reads each line separately and it will be stored in the variable as an array.
$json = Get-Content c:\temp\net\cars.json -Raw
ConvertFrom-Json $json
Try this and you will end up with a nicely usable object:
$cars = Get-Content c:\temp\net\cars.json -Raw | ConvertFrom-Json | Select -ExpandProperty cars
Related
I am stuck at one of the point in my PowerShell script where I need to update some part of the value for the property named restServiceURL and microServiceURL from http to https. (screenshot below)
I have the below script but somehow I am unable to figure out what needs to be added in order to replace the specific part of the value (http in this case) of the property from "http://VWMAIMPKG16SN/IMatchREST/" to "https://VWMAIMPKG16SN/IMatchREST/"
I know that set-content command should be able to do this but how to do I do it without changing the other part of the value is where I am stuck at.
Any suggestion on this would be helpful.
# Code to get Installation Directory path
$CommonNode=Get-ItemProperty -Path Registry::HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\AquagardePI\STeP\Platform\Common
$InstallationDir=$CommonNode.InstallationDir
#Path of Json File
$ConfigPath = $InstallationDir + "Web Client\www\\NextGen\assets\config.json"
#Get Content of the File
$file = Get-Content $ConfigPath -raw | ConvertFrom-Json
#Get the value of Property
$file = $file.restServiceURL
You can first get the JSON object, and then simply replace http with https for the two properties you are interested in:
$ConfigPath = $InstallationDir + "Web Client\www\\NextGen\assets\config.json"
$file = Get-Content $ConfigPath -raw | ConvertFrom-Json
$file.microServiceURL = $file.microServiceURL.Replace('http','https')
$file.restServiceURL = $file.restServiceURL.Replace('http','https')
Set-Content -Value ($file | ConvertTo-Json) -Path $ConfigPath
How does one convert a text file's contents into a string and then insert this string into a JSON file?
For example, if a file contains:
this
is
a
sample
file
The script would generate:
"this\r\nis\r\na\r\nsample\r\nfile"
To insert into a JSON template:
"something":"<insertPoint>"
To produce:
"something":"this\r\nis\r\na\r\nsample\r\nfile"
I'm using Powershell 5 and have managed to load the file, generate some JSON and insert it by running:
# get contents and convert to JSON
$contentToInsert = Get-Content $sourceFilePath -raw | ConvertTo-Json
# write in output file
(Get-Content $outputFile -Raw).replace('<insertPoint>', $contentToInsert) | Set-Content $outputFile
However, a lot of other, unwanted fields are also added.
"something":"{
"value": "this\r\nis\r\na\r\nsample\r\nfile"
"PSPath": "C:\\src\\intro.md",
"PSParentPath": "C:\\src",
"PSChildName": "intro.md",
etc...
Ultimately, I'm trying to send small rich text segments to a web page via JSON but want to edit and store them locally using Markdown. If this doesn't make sense and there's a better way of sending these then please let me know also.
iRon's answer helpfully suggests not using string manipulation to create JSON in PowerShell, but to use hashtables (or custom objects) to construct the data and then convert it to JSON.
However, that alone does not solve your problem:
PS> #{ something = Get-Content -Raw $sourceFilePath } | ConvertTo-Json
{
"something": {
"value": "this\nis\na\nsample\nfile\n",
"PSPath": "/Users/mklement/Desktop/pg/lines.txt",
# ... !! unwanted properties are still there
}
The root cause of the problem is that Get-Content decorates the strings it outputs with metadata in the form of NoteProperty properties, and ConvertTo-Json currently invariably includes these.
A proposal to allow opting out of this decoration when calling Get-Content can be found in GitHub issue #7537.
Complementarily, GitHub issue #5797 suggests that ConvertTo-Json should ignore the extra properties for primitive .NET types such as strings.
The simplest workaround is to access the underlying .NET instance with .psobject.baseobject, which bypasses the invisible wrapper object PowerShell uses to supply the extra properties:
PS> #{ something = (Get-Content -Raw $sourceFilePath).psobject.baseobject } |
ConvertTo-Json
{
"something": "this\nis\na\nsample\nfile\n"
}
Just a general recommendation apart from the actually issue described by #mklement0 and metadata added to the Get-Content results:
Do not poke (replace, insert, etc.) in any Json content.
Instead, modify the object (if necessary, use ConvertFrom-Json to restore the object) prior converting it into (ConvertTo-Json) a Json file.
In this example, I would use a hash-table with a here-string for this:
#{'something' = #'
this
is
a
sample
file
'#
} | ConvertTo-Json
Result:
{
"something": "this\nis\na\nsample\nfile"
}
You can use the Out-String cmdlet to coerce the output of Get-Content into a flat string first:
#{ "something" = (Get-Content lines.txt | Out-String) } | ConvertTo-Json
This produces:
{
"something": "this\r\nis\r\na\r\nsample\r\nfile\r\n"
}
I have a simple json file as follows:
[
{"ClientName": "Test Site 1", "ClientID": "000001"},
{"ClientName": "Test Site 2", "ClientID": "000002"},
{"ClientName": "Test Site 3", "ClientID": "000003"}
]
When I use the following PowerShell command:
ConvertFrom-Json (Get-Content TestSites.json -Raw)
I get back a System.Object[]. This doesn't allow me to pipe the output to another function I have which accepts "ClientName" and "ClientID" parameters.
However, when I assign that object to another variable, like this:
$myobj = ConvertFrom-Json (Get-Content TestSites.json -Raw)
$myobj is actually a System.Management.Automation.PSCustomObject which is capable of being passed to my function.
How can I just pipe the results of the original command without having to assign it to another variable first?
I hope that makes sense.
Your JSON is an array correct? PowerShell will unroll arrays in the pipeline unless you explicity change that behavior. Assuming your JSON is stored in the variable $json as a single string consider the following examples.
ConvertFrom-Json $json | ForEach-Object{$_.gettype().fullname}
System.Object[]
(convertFrom-Json $json) | ForEach-Object{$_.gettype().fullname}
System.Management.Automation.PSCustomObject
System.Management.Automation.PSCustomObject
System.Management.Automation.PSCustomObject
You should be able to wrap the expression in brackets to change the outcome. In the second example it should be sending the 3 objects individually down the pipe. In the first it is being sent as a single object array.
My explanation needs work but I am sure of the cause is how PowerShell deals with arrays and the pipeline. Unrolling being a common word used to describe it.
So depending on your use case you might just be able to wrap the expression in brackets so it gets processed before the pipe to ForEach in my example.
If you have an array such as System.Object[] you could try piping via foreach:
ConvertFrom-Json (Get-Content TestSites.json -Raw) | %{ $_ | your-function }
If you want to pass the whole array down the pipe as-is, you can try adding a comma (aka a unary comma before the variable:
,$myobj | whatever
You can probably see how the latter works by comparing the following:
$myobj | Get-Member # Shows the type of the elements of the array
Get-Member -InputObject $myobj # Shows the type of the array
,$myobj | Get-Member # Shows the type of the array
I am trying to write a powershell script that reads a file and prints "true" if it is a valid JSON file. I am using Powershell v3.0 and this is what I have right now :
$text = Get-Content .\filename.txt -Raw
$powershellRepresentation = $text | ConvertFrom-Json
How do I check the return code? I mean I want something like this :
if(file not a JSON file){
Write-Host "not JSON"
}
else{
Write-Host "True"
}
UPDATE 2021: PowerShell 6 and newer versions
PowerShell 6 brings a brand new Test-Json cmdlet. Here is the reference.
You can simply pass the raw file content directly to the Test-Json cmdlet.
$text = Get-Content .\filename.txt -Raw
if ($text | Test-Json) {
$powershellRepresentation = ConvertFrom-Json $text -ErrorAction Stop;
Write-Host "Provided text has been correctly parsed to JSON";
} else {
Write-Host "Provided text is not a valid JSON string";
}
PowerShell 5 and earlier versions
There is no Test-Json cmdlet in these versions, so the best way is to put your ConvertFrom-Json cmdlet inside a try ... catch block
try {
$powershellRepresentation = ConvertFrom-Json $text -ErrorAction Stop;
$validJson = $true;
} catch {
$validJson = $false;
}
if ($validJson) {
Write-Host "Provided text has been correctly parsed to JSON";
} else {
Write-Host "Provided text is not a valid JSON string";
}
If you encounter this question and can use PowerShell 6 or later, there is now a Test-Json cmdlet. It can also not just validate that it's valid JSON, but that it conforms to a particular JSON schema using the -Schema param.
Example
$isValid = Get-Content .\filename.txt -Raw | Test-Json
if($isValid){
Write-Host "not JSON"
}
else{
Write-Host "True"
}
ARM Template Warning
A note for users looking to validate an ARM template via -Schema (I can't imagine a more perfect use case). At the time of writing, there are one or more bugs in the underlying library Test-Json uses for validation, NJsonSchema, and it is not possible to validate an ARM template.
GitHub Issues
PowerShell Issue #9560
NJsonSchema Issue #588
I don't think that it exists an other solution than catching the exception using ConvertFrom-Json.
ConvertFrom-JSON would work but only for a JSON object < 2MB in size.
For higher you can use JavaScriptSerializer class
try
{
$jsser = New-Object System.Web.Script.Serialization.JavaScriptSerializer
$jsser.MaxJsonLength = $jsser.MaxJsonLength * 10
$jsser.RecursionLimit = 99
$outObject = $jsser.DeserializeObject($json)
}
catch
{
Write-Host "Error converting $text to JSON"
}
ConvertFrom-Json shd be the appropriate way to go.
Test-Json unfortunately has alot of known unsupported JSON Types.
I.E. it cannot parse Json-Arrays or Primitives properly leading to falsely assuming it has wrong JSON-Syntax.
I am pretty new with PowerShell. I was recently tasked with making a error message popup that would help a local user determine whether or not a MS SQL on-demand DB merge worked or not. I wrote a script that woudld do the following:
Run the batch file that conducted the merge
Read the results of a text log file into a variable
Check the variable for any instances of the word "ERROR" and pop a success or fail dialog box depending on whether or not it found the word error in the log file.
Quick and simple I thought but I appear to be struggling with the conditional statement. Here is the script:
cmd /c c:\users\PERSON\desktop\merge.bat
$c = get-content c:\replmerg.log
if ($c -contains ("ERROR"))
{
$a = new-object -comobject wscript.shell
$b = $a.popup(“ERROR - Database Merge“,0,”Please Contact Support”,1)
}
else
{
$a = new-object -comobject wscript.shell
$b = $a.popup(“SUCCESS - Database Merge“,0,”Good Job!”,1)
}
Right now what happens is that the script runs and just skips to the Success message. I can confirm that simply running the 'get-content' command in powershell will on its own produce a variable that I can then call and show the content of the log file. My script however does not appear as though it is actually checking the $c variable for the word and then popping the error message as intended. What am I missing here?
Christian's answer is correct. You could also use the -match operator. For example:
if ((Get-Content c:\replmerg.log) -match "ERROR")
{
'do error stuff'
}
else
{
'do success stuff'
}
You can use -cmatch if you want a case sensitive comparison.
You actually don't need to use Get-Content at all. Select-String can take a -path parameter. I created two very simple text files, one which has the word ERROR and one which does not
PS C:\> cat .\noerror.txt
not in here
PS C:\> cat .\haserror.txt
ERROR in here
this has ERROR in here
PS C:\> if ( Select-String -Path .\noerror.txt -Pattern ERROR) {"Has Error"}
PS C:\> if ( Select-String -Path .\haserror.txt -Pattern ERROR) {"Has Error"}
Has Error
PS C:\>
The one thing that might trip you up is that the -pattern actually takes a regular expression, so be careful of what you use for your pattern. THis will find ERROR anywhere in the log file, even if there are multiple instances, like in my "haserror.txt" file.
The -contains operator is used for looking for an exact match in a list (or array). As the other answers indicate, you should use -match, -like, or -eq to compare strings.
You can use the -quiet switch of select-string if you just wnat to test for the presence of a string in a file.
select-string -path c:\replmerg.log -pattern "ERROR" -casesensetive -quiet
Will return $true if the string is found in the file, and $false if it is not.
contain operator test only an identical value (not part of a value).
You can try this
$c = get-content c:\replmerg.log | select-string "ERROR" -casesensitive
if ($c.length -gt 0)