I have a .json file that needs to be edited in User Data, so I will have to use powershell to accomplish this.
The json looks something like this:
{
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
{
"Id": "CustomLogs",
"FullName": "AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogDirectoryPath": "C:\\CustomLogs\\",
"TimestampFormat": "MM/dd/yyyy HH:mm:ss",
"Encoding": "UTF-8",
"Filter": "",
"CultureName": "en-US",
"TimeZoneKind": "Local"
}
}
],
"Flows": {
"Flows":
[
"(ApplicationEventLog,SystemEventLog),CloudWatchLogs"
]
}
}
}
I would like it to look like this --
{
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
{
"Id": "CustomLogs",
"FullName": "AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch",
"Parameters": {
"LogDirectoryPath": "C:\\ProgramData\\Amazon\\CodeDeploy\\deployment-logs",
"TimestampFormat": "[yyyy-MM-dd HH:mm:ss.fff]",
"Encoding": "UTF-8",
"Filter": "",
"CultureName": "en-US",
"TimeZoneKind": "Local"
}
}
],
"Flows": {
"Flows":
[
"(ApplicationEventLog,SystemEventLog, CustomLogs),CloudWatchLogs"
]
}
}
}
In the Custom Logs Parameters, the LogDirectoryPath and TimestampFormat have both changed. Also, in the Flows section, I have added the 'CustomLogs' to the CloudWatch Group.
I tried making it work with code like this:
$a = Get-Content 'C:\PATH\TO\file.json' -raw | ConvertFrom-Json
$a.EngineConfiguration.Components[0].Parameters = '{"LogDirectoryPath": "","TimestampFormat": "[yyyy-MM-dd HH:mm:ss.fff]","Encoding": "UTF-8","Filter": "","CultureName": "en-US","TimeZoneKind": "Local"}'
$a | ConvertTo-Json | set-content 'C:\PATH\TO\output.json'
But that produces a very ugly output
{
"EngineConfiguration": {
"PollInterval": "00:00:15",
"Components": [
"#{Id=CustomLogs; FullName=AWS.EC2.Windows.CloudWatch.CustomLog.CustomLogInputComponent,AWS.EC2.Windows.CloudWatch; Parameters={\"LogDirectoryPath\": \"\",\"TimestampFormat\": \"[yyyy-MM-dd HH:mm:ss.fff]\",\"Encoding\": \"UTF-8\",\"Filter\": \"\",\"CultureName\": \"en-US\",\"TimeZoneKind\": \"Local\"}}",
"#{Id=CloudWatchLogs; FullName=AWS.EC2.Windows.CloudWatch.CloudWatchLogsOutput,AWS.EC2.Windows.CloudWatch; Parameters=}"
],
"Flows": {
"Flows": "(ApplicationEventLog,SystemEventLog),CloudWatchLogs"
}
}
}
Is there a more elegant way to do this? Any advice would be greatly appreciated. Thanks!
Try using the -Depth switch for ConvertTo-Json. By default this compresses any child elements beyond a depth of 2 to the string representations of the object you have seen:
"#{Id=CustomLogs; etc."
By specifying a deeper depth you get a format more like the one you want. Combine this with something that compresses the excessive whitespace as so:
((ConvertFrom-Json $a) | ConvertTo-Json -Depth 4) -replace ((" "*4)," ")
It would be possible to reduce the leading whitespace with a regex. However, that does not really produce the reformatting, pretty-print that you say you want.
$a | ConvertTo-Json | % {$_ -replace " "," "} | set-content 'output.json'
Related
I am trying to parse the following JSON into a PSCustomObject
[
{
"tags": [
{
"tagName": "Microsoft Teams"
},
{
"tagName": "Worldwide (Standard Multi-Tenant)"
},
{
"tagName": "General Availability"
},
{
"tagName": "Web"
},
{
"tagName": "Desktop"
}
],
"tagsContainer": {
"products": [
{
"tagName": "Microsoft Teams"
}
],
"cloudInstances": [
{
"tagName": "Worldwide (Standard Multi-Tenant)"
}
],
"releasePhase": [
{
"tagName": "General Availability"
}
],
"platforms": [
{
"tagName": "Web"
},
{
"tagName": "Desktop"
}
]
},
"id": 51230,
"title": "Microsoft Teams: New file sharing experience",
"description": "Streamline sharing with Microsoft Teams. You can now create a shareable link for any file stored in Teams and directly set the appropriate permissions. Additionally, you can also set permissions for files stored in SharePoint or OneDrive while composing a private chat or starting a channel conversation.",
"status": "Launched",
"moreInfoLink": "https://techcommunity.microsoft.com/t5/microsoft-sharepoint-blog/rich-new-file-and-sharing-experiences-throughout-microsoft-365/ba-p/960129",
"publicRoadmapStatus": "Include this month",
"created": "2019-05-08T07:00:00",
"modified": "2022-01-13T00:05:19.663",
"publicDisclosureAvailabilityDate": "March CY2021",
"publicPreviewDate": ""
}
]
I have that JSON stored in a variable like this:
$RoadmapContent = Get-Content -Raw -Path ".\M365Roadmap_single.json" | ConvertFrom-Json
And this would be my PSCustomObject:
$RoadmapItems =[PSCustomObject]#{
Title = $($RoadmapContent.title)
Tags = $($RoadmapContent.tags)
}
I would like to have this tags in a single column, like this: Microsoft Teams, Worldwide (Standard Multi-Tenant), General Availability, Web, Desktop
I have tried to accomplish that (unsuccessfully) like this:
$RoadmapContent.Tags -join ";"
How can I parse this?
Many thanks!
Edit:
I can access the tags like this:
foreach ($tag in $RoadmapContent.tags) {Write-Host $Tag.tagName}
But cannot figure out how to use this in a PSCustomObject
Figured it out thanks to a comment in the original question, here's the full code:
$RoadmapContent = Get-Content -Raw -Path ".\M365Roadmap_single.json" | ConvertFrom-Json
$RoadmapItems =[PSCustomObject]#{
Title = $($RoadmapContent.title)
Tags = $($RoadmapContent.tags.tagname -join ', ')
}
$RoadmapItems
Background
I am using PowerShell 7.
Yesterday, I asked this question on help merging some JSON together and saving it, it worked great.
I have now ran into another problem that my new gained knowledge doesn't help with - escaped characters.
Problem
I have some JSON like so:
{
"toolcache": [
{
"name": "Python",
"url": "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json",
"platform": "linux",
"platform_version": "18.04",
"arch": "x64",
"versions": [
"2.7.*",
"3.6.*",
"3.7.*",
"3.8.*",
"3.9.*",
"3.10.*"
]
}
],
"powershellModules": [
{"name": "MarkdownPS"},
{"name": "Microsoft.Graph"},
{"name": "Pester"},
{"name": "PSScriptAnalyzer"}
]
}
And I try to apply the knowledge in the other post:
### Add to JSON Toolet
# Try to escape characters as plain text using heredoc - Is this is the issue?
$ToolsetsToAdd = #"
{"name": "SqlServer"}
"#
$ToolsetFile = "toolset-2004.json"
$ToolsetObject = $(Get-Content $ToolsetFile -Raw)
$ToolSetSystemObject =$($ToolsetObject | ConvertFrom-Json)
$ToolSetSystemObject.powershellModules += $ToolsetsToAddJson
$ToolSetSystemObject | ConvertTo-Json -Depth 100 | Set-Content $ToolsetFile
And when I run it, I get:
{
"toolcache": [
{
"name": "Python",
"url": "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json",
"platform": "linux",
"platform_version": "18.04",
"arch": "x64",
"versions": [
"2.7.*",
"3.6.*",
"3.7.*",
"3.8.*",
"3.9.*",
"3.10.*"
]
}
],
"powershellModules": [
{
"name": "MarkdownPS"
},
{
"name": "Microsoft.Graph"
},
{
"name": "Pester"
},
{
"name": "PSScriptAnalyzer"
},
"{\"name\": \"SqlServer\"}"
],
I have tried a variation of escapes using things like `"`" but couldn't figure this one out either.
This to my knowledge is expanded JSON, so I'm not sure its an error, but its not copying and escaping things like I expected. Whether this is avoidable is what I want to find out from someone who knows better!
What I want to do:
Ideally, I'd like a correctly formatted and parsed JSON file like so:
{
"toolcache": [
{
"name": "Python",
"url": "https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json",
"platform": "linux",
"platform_version": "18.04",
"arch": "x64",
"versions": [
"2.7.*",
"3.6.*",
"3.7.*",
"3.8.*",
"3.9.*",
"3.10.*"
]
}
],
"powershellModules": [
{"name": "MarkdownPS"},
{"name": "Microsoft.Graph"},
{"name": "Pester"},
{"name": "PSScriptAnalyzer"}
{"name": "SqlServer"}
]
}
You are treating the array in $ToolSetSystemObject.powershellModules as if it were an array of strings, but in fact this is an array of objects
To add a new object to it, just do
$ToolSetSystemObject.powershellModules += #{name = "SqlServer"}
or
$ToolSetSystemObject.powershellModules += [PsCustomObject]#{name = "SqlServer"}
I have a json file that looks like so:
[
{
"code": "1234",
"files": [
{
"fileType": "pdf",
"url": "http://.../a.pdf"
},
{
"fileType": "video",
"url": "http://.../b.mp4"
}
]
},
{
"code": "4321",
"files": [
{
"fileType": "pdf",
"url": "http://.../c.pdf"
},
{
"fileType": "video",
"url": "http://.../d.mp4"
}
]
},
{
"code": "9999",
"files": [
{
"fileType": "pdf",
"url": "http://.../e.pdf"
}
]
}
]
I would like to print out only the files that are of fileType == video in the files array such that I end up with output that looks like so:
1234, "http://.../b.mp4"
4321, "http://.../d.mp4"
So far I am only able to output something that looks like this:
1234, "http://.../a.pdf", "http://.../b.mp4",
4321, "http://.../c.pdf", "http://.../d.mp4"
Using the following:
jq -r '.[] | select(.files[]?.fileType == "video") | [.code, .files[].url] | #csv'
I was wondering how I can filter the .files[] based on the fileType as I am outputting them?
The following pipeline makes the solution fairly self-explanatory, assuming one understands the basic syntax and the -r command-line option:
< input.json jq -r '
.[]
| .code as $code
| .files[]
| select(.fileType == "video")
| "\($code), \"\(.url)\""
'
I have the following JSON held in a file "test.json":
{
"metadata": [
{
"src": [
{
"files": [
"src/**.csproj"
]
}
],
"dest": "api",
"disableGitFeatures": false,
"disableDefaultFilter": false
}
]
}
I'd like to modify the "src" element. Instead of:
"src": [
{
"files": [
"src/**.csproj"
]
}
],
It needs to be:
"src": [
{
"files": [
"*.csproj"
],
"cwd":".."
}
],
Where I modify the first element of "files" and add "cwd". This should be straight forward but I'm struggling to achieve this in powershell. Can anyone point me in the right direction of any examples of this?
Thanks for any pointers in advance.
You can do the following:
$JSONObject = Get-Content test.json -Raw | ConvertFrom-Json
$JSONObject.metadata.src.files = ,'*.csproj'
$JSONObject.metadata.src | Add-Member -Name 'cwd' -Value '..' -MemberType NoteProperty
$JSONObject | ConvertTo-Json -Depth 5 | Set-Content test.json
The tricky part is to make sure the .files value is an array of a single element. You can do this with the array subexpression operator #() or the unary operator ,.
I'm not sure how to name these elements properly, it'll be easier just to show it. I have following JSON:
{
"DEV": [
{
"GitEmail": "asd#asd.com"
}
],
"TEST": [
{
"GitEmail": "asd1#asd.com"
}
],
"PROD": [
{
"GitEmail": "asd2#asd.com"
}
]
}
I would like to get the "DEV" by providing it's email. How to implement that in powershell?
Something like below can help -
PS> $json = '{
"DEV": [
{
"GitEmail": "asd#asd.com"
}
],
"TEST": [
{
"GitEmail": "asd1#asd.com"
}
],
"PROD": [
{
"GitEmail": "asd2#asd.com"
}
]
}' | ConvertFrom-Json
PS> ($json.psobject.Properties | ? {$_.Value -match "asd#asd.com"}).Name
Depending on the email matches you can retrieve the environment names.
I can't promise there is an easier method, but this here is one way:
Given that you json is stored in a variable $json:
You can get every head object with $json.psobject.properties.name:
Input:
$json.psobject.properties.name
Output:
DEV
TEST
PROD
With this we can create a foreach loop and search for the Email:
foreach ($dev in $json.psobject.properties.name)
{
if($json.$dev.GitEmail -eq "asd#asd.com") {
echo $dev
}
}
I do not know any elegant way of doing it. ConvertFrom-Json does not create neat objects with easy ways to traverse them like convertfrom-xml, instead result is just a PsObject with bunch of noteproperties.
What I do in such cases is
$a= #"
{
"DEV": [
{
"GitEmail": "asd#asd.com"
}
],
"TEST": [
{
"GitEmail": "asd1#asd.com"
}
],
"PROD": [
{
"GitEmail": "asd2#asd.com"
}
]
}
"#
$JsonObject= ConvertFrom-Json -InputObject $a
$NAMES= $JsonObject|Get-Member |WHERE MemberType -EQ NOTEPROPERTY
$NAMES|Foreach-Object {IF($JsonObject.$($_.NAME).GITEMAIL -EQ 'asd#asd.com'){$_.NAME}}
Result of above is
DEV
Not pretty, not really re-usable but works.
If anyone knows a better way of going about it - I'll be happy to learn it:)