Powershell - Cannot parse JSON - Get NotSpecified error - json

Question

Code used to try parsing file
$myJson = Get-Content cleantest2.json -Raw | ConvertFrom-Json
Get below error when trying to parse JSON file.
$myJson = Get-Content cleantest2.json -Raw | ConvertFrom-Json
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand
EDIT: As noted, the above error was truncated because it was too long. Here is the full error.
ConvertFrom-Json : Invalid object passed in, ':' or '}' expected. (2): {"id":0,"resourcePackage
s":[{"resourcePackage":{"name":"SharedResources","type":2,"i
tems":[{"type":202,"path":"BaseThemes/CY18SU07.json","name"
:"CY18SU07"}],"disabled":false}}],"sections":[{"id":0,"name"
:"ReportSection","displayName":"Page 1","filters":"[]","ord
inal":0,"visualContainers":[{"x":10,"y":0,"z":1,"width":280,
"height":280,"config":"{\"name\":\"5a0b8ef6c97162087ff7\",\
"layouts\":[{\"id\":0,\"position\":{\"x\":10,\"y\":0,\"width
\":280,\"height\":280,\"z\":1}}],\"singleVisual\":{\"visual
Type\":\"tableEx\",\"projections\":{\"Values\":[{\"queryRef\
":\"servers_and_applications.App Name\"},{\"queryRef\":\"se
rvers_and_applications.Cust_Nme\"},{\"queryRef\":\"servers_a
nd_applications.MU_ID\"}]},\"prototypeQuery\":{\"Version\":
2,\"From\":[{\"Name\":\"s\",\"Entity\":\"servers_and_applica
tions\"}],\"Select\":[{\"Column\":{\"Expression\":{\"Source
Ref\":{\"Source\":\"s\"}},\"Property\":\"App
Name\"},\"Name\":\"servers_and_applications.App Name\"},{\"
Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},
\"Property\":\"Cust_Nme\"},\"Name\":\"servers_and_applicati
ons.Cust_Nme\"},{\"Column\":{\"Expression\":{\"SourceRef\":{
\"Source\":\"s\"}},\"Property\":\"MU_ID\"},\"Name\":\"serve
rs_and_applications.MU_ID\"}]},\"drillFilterOtherVisuals\":t
rue}}","filters":"[]","query":"{\"Commands\":[{\"SemanticQu
eryDataShapeCommand\":{\"Query\":{\"Version\":2,\"From\":[{\
"Name\":\"s\",\"Entity\":\"servers_and_applications\"}],\"S
elect\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Sourc
e\":\"s\"}},\"Property\":\"App
Name\"},\"Name\":\"servers_and_applications.App Name\"},{\"
Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},
\"Property\":\"Cust_Nme\"},\"Name\":\"servers_and_applicati
ons.Cust_Nme\"},{\"Column\":{\"Expression\":{\"SourceRef\":{
\"Source\":\"s\"}},\"Property\":\"MU_ID\"},\"Name\":\"serve
rs_and_applications.MU_ID\"}]},\"Binding\":{\"Primary\":{\"G
roupings\":[{\"Projections\":[0,1,2],\"Subtotal\":1}]},\"Da
taReduction\":{\"DataVolume\":3,\"Primary\":{\"Window\":{\"C
ount\":500}}},\"Version\":1}}}]}","dataTransforms":"{\"proj
ectionOrdering\":{\"Values\":[0,1,2]},\"queryMetadata\":{\"S
elect\":[{\"Restatement\":\"App
Name\",\"Name\":\"servers_and_applications.App Name\",\"Typ
e\":2048},{\"Restatement\":\"Cust_Nme\",\"Name\":\"servers_a
nd_applications.Cust_Nme\",\"Type\":2048},{\"Restatement\":
\"MU_ID\",\"Name\":\"servers_and_applications.MU_ID\",\"Type
\":2048}]},\"visualElements\":[{\"DataRoles\":[{\"Name\":\"
Values\",\"Projection\":0,\"isActive\":false},{\"Name\":\"Va
lues\",\"Projection\":1,\"isActive\":false},{\"Name\":\"Val
ues\",\"Projection\":2,\"isActive\":false}]}],\"selects\":[{
\"displayName\":\"App
Name\",\"queryName\":\"servers_and_applications.App Name\",
\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"un
derlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"S
ourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Prop
erty\":\"App Name\"}}},{\"displayName\":\"Cust_Nme\",\"quer
yName\":\"servers_and_applications.Cust_Nme\",\"roles\":{\"V
alues\":true},\"type\":{\"category\":null,\"underlyingType\
":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"
Entity\":\"servers_and_applications\"}},\"Property\":\"Cust
_Nme\"}}},{\"displayName\":\"MU_ID\",\"queryName\":\"servers
_and_applications.MU_ID\",\"roles\":{\"Values\":true},\"typ
e\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Col
umn\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_
and_applications\"}},\"Property\":\"MU_ID\"}}}]}"}],"config"
:"{}","displayOption":1,"width":1280,"height":720}],"config
":"{\"version\":\"3.19\",\"themeCollection\":{\"baseTheme\":
{\"name\":\"CY18SU07\",\"version\":\"3.22\",\"type\":2}},\"
activeSectionIndex\":0,\"defaultDrillFilterOtherVisuals\":tr
ue,\"settings\":{\"useStylableVisualContainerHeader\":true,
\"exportDataMode\":1}}","layoutOptimization":0}
At line:1 char:46
+ $myJson = Get-Content cleantest2.json -Raw | ConvertFrom-Json
+ ~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [ConvertFrom-Json], ArgumentException
+ FullyQualifiedErrorId : System.ArgumentException,Microsoft.PowerShell.Commands.ConvertFromJsonCommand
Below is file that was trying to get parsed.
{"id":0,"resourcePackages":[{"resourcePackage":{"name":"SharedResources","type":2,"items":[{"type":202,"path":"BaseThemes/CY18SU07.json","name":"CY18SU07"}],"disabled":false}}],"sections":[{"id":0,"name":"ReportSection","displayName":"Page 1","filters":"[]","ordinal":0,"visualContainers":[{"x":10,"y":0,"z":1,"width":280,"height":280,"config":"{\"name\":\"5a0b8ef6c97162087ff7\",\"layouts\":[{\"id\":0,\"position\":{\"x\":10,\"y\":0,\"width\":280,\"height\":280,\"z\":1}}],\"singleVisual\":{\"visualType\":\"tableEx\",\"projections\":{\"Values\":[{\"queryRef\":\"servers_and_applications.App Name\"},{\"queryRef\":\"servers_and_applications.Cust_Nme\"},{\"queryRef\":\"servers_and_applications.MU_ID\"}]},\"prototypeQuery\":{\"Version\":2,\"From\":[{\"Name\":\"s\",\"Entity\":\"servers_and_applications\"}],\"Select\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"App Name\"},\"Name\":\"servers_and_applications.App Name\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"Cust_Nme\"},\"Name\":\"servers_and_applications.Cust_Nme\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"MU_ID\"},\"Name\":\"servers_and_applications.MU_ID\"}]},\"drillFilterOtherVisuals\":true}}","filters":"[]","query":"{\"Commands\":[{\"SemanticQueryDataShapeCommand\":{\"Query\":{\"Version\":2,\"From\":[{\"Name\":\"s\",\"Entity\":\"servers_and_applications\"}],\"Select\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"App Name\"},\"Name\":\"servers_and_applications.App Name\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"Cust_Nme\"},\"Name\":\"servers_and_applications.Cust_Nme\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"MU_ID\"},\"Name\":\"servers_and_applications.MU_ID\"}]},\"Binding\":{\"Primary\":{\"Groupings\":[{\"Projections\":[0,1,2],\"Subtotal\":1}]},\"DataReduction\":{\"DataVolume\":3,\"Primary\":{\"Window\":{\"Count\":500}}},\"Version\":1}}}]}","dataTransforms":"{\"projectionOrdering\":{\"Values\":[0,1,2]},\"queryMetadata\":{\"Select\":[{\"Restatement\":\"App Name\",\"Name\":\"servers_and_applications.App Name\",\"Type\":2048},{\"Restatement\":\"Cust_Nme\",\"Name\":\"servers_and_applications.Cust_Nme\",\"Type\":2048},{\"Restatement\":\"MU_ID\",\"Name\":\"servers_and_applications.MU_ID\",\"Type\":2048}]},\"visualElements\":[{\"DataRoles\":[{\"Name\":\"Values\",\"Projection\":0,\"isActive\":false},{\"Name\":\"Values\",\"Projection\":1,\"isActive\":false},{\"Name\":\"Values\",\"Projection\":2,\"isActive\":false}]}],\"selects\":[{\"displayName\":\"App Name\",\"queryName\":\"servers_and_applications.App Name\",\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Property\":\"App Name\"}}},{\"displayName\":\"Cust_Nme\",\"queryName\":\"servers_and_applications.Cust_Nme\",\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Property\":\"Cust_Nme\"}}},{\"displayName\":\"MU_ID\",\"queryName\":\"servers_and_applications.MU_ID\",\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Property\":\"MU_ID\"}}}]}"}],"config":"{}","displayOption":1,"width":1280,"height":720}],"config":"{\"version\":\"3.19\",\"themeCollection\":{\"baseTheme\":{\"name\":\"CY18SU07\",\"version\":\"3.22\",\"type\":2}},\"activeSectionIndex\":0,\"defaultDrillFilterOtherVisuals\":true,\"settings\":{\"useStylableVisualContainerHeader\":true,\"exportDataMode\":1}}","layoutOptimization":0}
Reformatted:
{
"id": 0,
"resourcePackages": [
{
"resourcePackage": {
"name": "SharedResources",
"type": 2,
"items": [
{
"type": 202,
"path": "BaseThemes/CY18SU07.json",
"name": "CY18SU07"
}
],
"disabled": false
}
}
],
"sections": [
{
"id": 0,
"name": "ReportSection",
"displayName": "Page 1",
"filters": "[]",
"ordinal": 0,
"visualContainers": [
{
"x": 10,
"y": 0,
"z": 1,
"width": 280,
"height": 280,
"config": "{\"name\":\"5a0b8ef6c97162087ff7\",\"layouts\":[{\"id\":0,\"position\":{\"x\":10,\"y\":0,\"width\":280,\"height\":280,\"z\":1}}],\"singleVisual\":{\"visualType\":\"tableEx\",\"projections\":{\"Values\":[{\"queryRef\":\"servers_and_applications.App Name\"},{\"queryRef\":\"servers_and_applications.Cust_Nme\"},{\"queryRef\":\"servers_and_applications.MU_ID\"}]},\"prototypeQuery\":{\"Version\":2,\"From\":[{\"Name\":\"s\",\"Entity\":\"servers_and_applications\"}],\"Select\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"App Name\"},\"Name\":\"servers_and_applications.App Name\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"Cust_Nme\"},\"Name\":\"servers_and_applications.Cust_Nme\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"MU_ID\"},\"Name\":\"servers_and_applications.MU_ID\"}]},\"drillFilterOtherVisuals\":true}}",
"filters": "[]",
"query": "{\"Commands\":[{\"SemanticQueryDataShapeCommand\":{\"Query\":{\"Version\":2,\"From\":[{\"Name\":\"s\",\"Entity\":\"servers_and_applications\"}],\"Select\":[{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"App Name\"},\"Name\":\"servers_and_applications.App Name\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"Cust_Nme\"},\"Name\":\"servers_and_applications.Cust_Nme\"},{\"Column\":{\"Expression\":{\"SourceRef\":{\"Source\":\"s\"}},\"Property\":\"MU_ID\"},\"Name\":\"servers_and_applications.MU_ID\"}]},\"Binding\":{\"Primary\":{\"Groupings\":[{\"Projections\":[0,1,2],\"Subtotal\":1}]},\"DataReduction\":{\"DataVolume\":3,\"Primary\":{\"Window\":{\"Count\":500}}},\"Version\":1}}}]}",
"dataTransforms": "{\"projectionOrdering\":{\"Values\":[0,1,2]},\"queryMetadata\":{\"Select\":[{\"Restatement\":\"App Name\",\"Name\":\"servers_and_applications.App Name\",\"Type\":2048},{\"Restatement\":\"Cust_Nme\",\"Name\":\"servers_and_applications.Cust_Nme\",\"Type\":2048},{\"Restatement\":\"MU_ID\",\"Name\":\"servers_and_applications.MU_ID\",\"Type\":2048}]},\"visualElements\":[{\"DataRoles\":[{\"Name\":\"Values\",\"Projection\":0,\"isActive\":false},{\"Name\":\"Values\",\"Projection\":1,\"isActive\":false},{\"Name\":\"Values\",\"Projection\":2,\"isActive\":false}]}],\"selects\":[{\"displayName\":\"App Name\",\"queryName\":\"servers_and_applications.App Name\",\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Property\":\"App Name\"}}},{\"displayName\":\"Cust_Nme\",\"queryName\":\"servers_and_applications.Cust_Nme\",\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Property\":\"Cust_Nme\"}}},{\"displayName\":\"MU_ID\",\"queryName\":\"servers_and_applications.MU_ID\",\"roles\":{\"Values\":true},\"type\":{\"category\":null,\"underlyingType\":1},\"expr\":{\"Column\":{\"Expression\":{\"SourceRef\":{\"Entity\":\"servers_and_applications\"}},\"Property\":\"MU_ID\"}}}]}"
}
],
"config": "{}",
"displayOption": 1,
"width": 1280,
"height": 720
}
],
"config": "{\"version\":\"3.19\",\"themeCollection\":{\"baseTheme\":{\"name\":\"CY18SU07\",\"version\":\"3.22\",\"type\":2}},\"activeSectionIndex\":0,\"defaultDrillFilterOtherVisuals\":true,\"settings\":{\"useStylableVisualContainerHeader\":true,\"exportDataMode\":1}}",
"layoutOptimization": 0
}
Items attempted:
Used online formator on text and then re-ran command on resulting string. - Worked
Tried to import raw string and export as JSON using ConverTo-Json method but didn't work correctly.
Tried parsing using Python but had similar issue.

Answer 1

I was able to fix this by changing the encoding to unicode. See below sample execution.
$myJson = Get-Content cleantest2.json -Encoding unicode | ConvertFrom-Json

Answer 2

Your own answer implies that your input file, cleantest2.json, was malformed:
The fact that -Encoding Unicode solved your problem implies that the file is a UTF-16LE-encoded file that lacks a BOM, which means that it provides no indication to readers of the file what its encoding is, and causes Get-Content to (justifiably) misinterpret the file.
Consider correcting the problem at the source: make the creator of file cleantest2.json create it with the appropriate BOM, the way that Set-Content -Encoding Unicode does, for instance.
As for troubleshooting your symptom:
If a string's display output is seemingly correct, but processing of the string doesn't work as expected, the cause is usually the presence of hidden control characters.
Specifically, misreading a BOM-less UTF16-LE ("Unicode") file as either ANSI-encoded (which Windows PowerShell does by default) or UTF-8 (as PowerShell (Core) 7+ does by default) results in all ASCII-range characters being followed by a NUL character (code point 0x0)each, and these NUL characters are invisible when you print the string.
For testing, you can create a malformed UTF-16LE file as follows:
# Write 'foo' with UTF-16LE encoding to file 'cleantest2.json' *without a BOM*:
[IO.File]::WriteAllText("$PWD/cleantest2.json", "f`0o`0`o`0")
A simple test for the presence of at least one NUL character (expressed as "`0" in PowerShell) is:
(Get-Content cleantest2.json -Raw) -match "`0" # "`0" creates a NUL
Piping to Format-Hex can also help in this case:[1]
# Look for `00` values.
Get-Content cleantest2.json -Raw | Format-Hex
The Debug-String helper function, available from this MIT-licensed Gist offers a direct visualization:
# Download and define the Debug-String function.
# NOTE:
# I can personally assure you that doing this is safe, but you
# you should always check the source code first.
irm https://gist.github.com/mklement0/7f2f1e13ac9c2afaf0a0906d08b392d1/raw/Debug-String.ps1 | iex
Get-Content cleantest2.json -Raw | Debug-String
Output:
A note re spotting the problem with the help of text editors:
This may not help, because many editors correctly detect UTF-16LE files even without a BOM (e.g., Notepad, Visual Studio Code), and in their status bar report just "UTF-16 LE" whether or not a BOM is present.
However, if an editor offers a hex view (of the bytes that make up the file), you'll be able to tell: If an UTF-16LE file does have a BOM, the first two bytes are FF FE (for UTF-8 files that have a BOM, the first three bytes are EF BB BF).
Visual Studio Code has a Hex Editor extension
js2010 reports that Emacs has a hexl-mode.
[1] Use of Format-Hex to discover (non-ASCII-range) Unicode characters isn't an option in Windows PowerShell, because it represents any non-ASCII chars. as 3f, i.e. literal ?. In PowerShell (Core) it shows the ASCII rendering of the individual bytes that make up the UTF-8 encoding of non-ASCII characters, which can be confusing.