Background
I am using PowerShell 7.
Yesterday, I asked this question on help merging some JSON together and saving it, it worked great.
I have now ran into another problem that my new gained knowledge doesn't help with - escaped characters.
Problem
I have some JSON like so:
{
"toolcache": [
{
"name": "Python",
"url": "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json",
"platform": "linux",
"platform_version": "18.04",
"arch": "x64",
"versions": [
"2.7.*",
"3.6.*",
"3.7.*",
"3.8.*",
"3.9.*",
"3.10.*"
]
}
],
"powershellModules": [
{"name": "MarkdownPS"},
{"name": "Microsoft.Graph"},
{"name": "Pester"},
{"name": "PSScriptAnalyzer"}
]
}
And I try to apply the knowledge in the other post:
### Add to JSON Toolet
# Try to escape characters as plain text using heredoc - Is this is the issue?
$ToolsetsToAdd = #"
{"name": "SqlServer"}
"#
$ToolsetFile = "toolset-2004.json"
$ToolsetObject = $(Get-Content $ToolsetFile -Raw)
$ToolSetSystemObject =$($ToolsetObject | ConvertFrom-Json)
$ToolSetSystemObject.powershellModules += $ToolsetsToAddJson
$ToolSetSystemObject | ConvertTo-Json -Depth 100 | Set-Content $ToolsetFile
And when I run it, I get:
{
"toolcache": [
{
"name": "Python",
"url": "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json",
"platform": "linux",
"platform_version": "18.04",
"arch": "x64",
"versions": [
"2.7.*",
"3.6.*",
"3.7.*",
"3.8.*",
"3.9.*",
"3.10.*"
]
}
],
"powershellModules": [
{
"name": "MarkdownPS"
},
{
"name": "Microsoft.Graph"
},
{
"name": "Pester"
},
{
"name": "PSScriptAnalyzer"
},
"{\"name\": \"SqlServer\"}"
],
I have tried a variation of escapes using things like `"`" but couldn't figure this one out either.
This to my knowledge is expanded JSON, so I'm not sure its an error, but its not copying and escaping things like I expected. Whether this is avoidable is what I want to find out from someone who knows better!
What I want to do:
Ideally, I'd like a correctly formatted and parsed JSON file like so:
{
"toolcache": [
{
"name": "Python",
"url": "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json",
"platform": "linux",
"platform_version": "18.04",
"arch": "x64",
"versions": [
"2.7.*",
"3.6.*",
"3.7.*",
"3.8.*",
"3.9.*",
"3.10.*"
]
}
],
"powershellModules": [
{"name": "MarkdownPS"},
{"name": "Microsoft.Graph"},
{"name": "Pester"},
{"name": "PSScriptAnalyzer"}
{"name": "SqlServer"}
]
}
You are treating the array in $ToolSetSystemObject.powershellModules as if it were an array of strings, but in fact this is an array of objects
To add a new object to it, just do
$ToolSetSystemObject.powershellModules += #{name = "SqlServer"}
or
$ToolSetSystemObject.powershellModules += [PsCustomObject]#{name = "SqlServer"}
Related
Here is a simplified sample of the JSON data I'm working with:
[
{ "certname": "one.example.com",
"name": "fact1",
"value": "value1"
},
{ "certname": "one.example.com",
"name": "fact2",
"value": 42
},
{ "certname": "two.example.com",
"name": "fact1",
"value": "value3"
},
{ "certname": "two.example.com",
"name": "fact2",
"value": 10000
},
{ "certname": "two.example.com",
"name": "fact3",
"value": { "anotherkey": "anothervalue" }
}
]
The result I want to achieve, using jq preferably, is the following:
[
{
"certname": "one.example.com",
"fact1": "value1",
"fact2": 42
},
{
"certname": "two.example.com",
"fact1": "value3",
"fact2": 10000,
"fact3": { "anotherkey": "anothervalue" }
}
]
Its worth pointing out that not all elements have the same name/value pairs, by any means. Also, values are often complex objects in their own right.
If I was doing this in Python, it wouldn't be a big deal (and yes, I can hear the chorus of "do it in Python" ringing in my ears now). I would like to understand how to do this in jq, and it's escaping me at the moment.
... using jq preferably ...
That's the spirit! And in that spirit, here's a concise solution:
map( {certname, (.name): .value} )
| group_by(.certname)
| map(add)
Of course there are other reasonable solutions. If the above is at first puzzling, you might like to add a debug statement here or there, or you might like to explore the pipeline by executing the first line by itself, etc.
So I basically have JSON output from the JIRA Insights API, been digging around and found jq for parsing the JSON. Struggling to wrap my head around on how parse the following to only return values for the objectTypeAttributeId's that I am interested in.
For Example I'm only interested in the value of objectTypeAttributeId 887 provided that objectTypeAttributeId 911's name states as active, but then would like to return the name value of another objectTypeAttributeId
Can this be achieved using jq only? Or shoudl I be using something else?
I can filter down to this level which is the 'attributes' section of the JSON output and print each value, but struggling to find an example catering for my situation.
{
"id": 137127,
"objectTypeAttributeId": 887,
"objectAttributeValues": [
{
"value": "false"
}
],
"objectId": 9036,
"position": 16
},
{
"id": 137128,
"objectTypeAttributeId": 888,
"objectAttributeValues": [
{
"value": "false"
}
],
"objectId": 9036,
"position": 17
},
{
"id": 137296,
"objectTypeAttributeId": 911,
"objectAttributeValues": [
{
"status": {
"id": 1,
"name": "Active",
"category": 1
}
}
],
"objectId": 9036,
"position": 18
},
Can this be achieved using jq only?
Yes, jq was designed precisely for this kind of query. In your case, you could use any, select and if ... then ... else ... end, along the lines of:
if any(.[]; .objectTypeAttributeId == 911 and
any(.objectAttributeValues[]; .status.name == "Active"))
then map(select(.objectTypeAttributeId == 887))
else "whatever"
end
I am pulling data from an API using the New-WebServiceProxy in PowerShell 4.0 and then piping it out to a JSON file for review and import on another API service (same API version, etc, just a different host).
$tasklist.Taskconfig | ConvertTo-JSON-Depth 50 -As String | Out-File -FilePath $exportpath\$name.xml -Force
Gives me my XML containing the TaskConfig. In this case, TaskConfig is an object type automatically generated by the API I'm interfacing with. When I want to import the content I am using:
$taskconfig = (Get-Content "$taskjson") -join "`n" | ConvertFrom-Json
but when I run this it's unable to create the object. I assume this is because the JSON contains nested children, giving the error-
Cannot convert value "#{Name=plugindive; Value=;> Children=System.Object[]}" to type "Microsoft.PowerShell.Commands.NewWebserviceProxy.AutogeneratedTypes.WebServiceProxy1rcleWeb_WebClientAPI_asmx_wsdl.TaskConfig". Error: "Cannot convert the "#{Name=plugindive; Value=;Children=System.Object[]}" value of type "System.Management.Automation.PSCustomObject" to type "Microsoft.PowerShell.Commands.NewWebserviceProxy.AutogeneratedTypes.WebServiceProxy1rcleWeb_WebClientAPI_asmx_wsdl.TaskConfig"."
I've tried explictly stating the type of object:
$taskconfig = [Microsoft.PowerShell.Commands.NewWebserviceProxy.AutogeneratedTypes.WebServiceProxy1rcleWeb_WebClientAPI_asmx_wsdl.TaskConfig](Get-Content "$taskjson" | Out-string | ConvertFrom-Json)
as well as creating the object then trying to add the children from my JSON -
$taskconfig.children = $json.children
But these all fail in the same way.
I don't seem to get this same issue in PowerShell 5.0 interestingly enough, but I can't verify why - is there another way to approach this?
Added example JSON below
{"Name": "plugindive",
"Value": null,
"Children": [{
"Name": "auto",
"Value": "False",
"Children": [
]
},
{
"Name": "categories",
"Value": null,
"Children": [{
"Name": "Module Z",
"Value": "False",
"Children": [
]
},
{
"Name": "Module A",
"Value": "False",
"Children": [
]
},
{
"Name": "Module B",
"Value": "False",
"Children": [
]
},
{
"Name": "Module C",
"Value": "False",
"Children": [
]
}
]
}
]
}
It seems as if this doesn't work in PowerShell v3.0, so I simply ended up making posts with the explicit XML directly, rather than converting to JSON.
I have some JSON output I am trying to parse with jq. I read some examples on filtering but I don't really understand it and my output it more complicated than the examples. I have no idea where to even begin beyond jq '.[]' as I don't understand the syntax of jq beyond that and the hierarchy and terminology are challenging as well. My JSON output is below. I want to return the value for Valid where the ItemName equals Item_2. How can I do this?
"1"
[
{
"GroupId": "1569",
"Title": "My_title",
"Logo": "logo.jpg",
"Tags": [
"tag1",
"tag2",
"tag3"
],
"Owner": [
{
"Name": "John Doe",
"Id": "53335"
}
],
"ItemId": "209766",
"Item": [
{
"Id": 47744,
"ItemName": "Item_1",
"Valid": false
},
{
"Id": 47872,
"ItemName": "Item_2",
"Valid": true
},
{
"Id": 47872,
"ItemName": "Item_3",
"Valid": false
}
]
}
]
"Browse"
"8fj9438jgge9hdfv0jj0en34ijnd9nnf"
"v9er84n9ogjuwheofn9gerinneorheoj"
Except for the initial and trailing JSON scalars, you'd simply write:
.[] | .Item[] | select( .ItemName == "Item_2" ) | .Valid
In your particular case, to ensure the top-level JSON scalars are ignored, you could prefix the above with:
arrays |
Writting one shell script to automatically get list of name, current and latest available version from raw json data.
I am trying to format JSON data stored in file using shell script. I tried using JQ command line JSON parser.
I want to get formatted JSON data in script. Their is advanced option provided in JQ for same scenario. I am not able to use it properly.
Example: File containing Following JSON
{
"endpoint": {
"name": "test-plugin",
"version": "0.0.1"
},
"dependencies": {
"plugin1": {
"main": {
"name": "plugin1name",
"description": "Dummy text"
},
"pkgMeta": {
"name": "plugin1name",
"version": "0.0.1"
},
"dependencies": {},
"versions": [
"0.0.5",
"0.0.4",
"0.0.3",
"0.0.2",
"0.0.1"
],
"update": {
"latest": "0.0.5"
}
},
"plugin2": {
"main": {
"name": "plugin2name",
"description": "Dummy text"
},
"pkgMeta": {
"name": "plugin2name",
"version": "0.1.1"
},
"dependencies": {},
"versions": [
"0.1.5",
"0.1.4",
"0.1.3",
"0.1.2",
"0.1.1"
],
"update": {
"latest": "0.1.5"
}
}
}
}
Trying to get result in format
[{name: "plugin1name",
c_version: "0.0.1",
n_version: "0.0.5"
},
{name: "plugin2name",
c_version: "0.1.1",
n_version: "0.1.5"}]
Can someone suggest anything ?
Your json file is not valid at: .dependencies.pkgMeta.version.
After fixing your json file, try this command:
jq '
.dependencies |
to_entries |
map(.value |
{
name: .main.name,
c_version: .pkgMeta.version,
n_version: .update.latest
}
)' input.json
The result is:
[
{
"name": "plugin1name",
"c_version": "0.0.1",
"n_version": "0.0.5"
},
{
"name": "plugin2name",
"c_version": "0.1.1",
"n_version": "0.1.5"
}
]