I am trying to use ConvertTo-Json in PowerShell. By default, ConvertTo-Json will escape special characters. However, it will not escape chinese.
For example:
"<>中文ABC" | ConvertTo-Json
The output is
"\u003c\u003e中文ABC"
But, what I really want is "\u003c\u003e\u4e2d\u6587ABC". Can anyone share the experience?
Found a way:
function ConvertTo-Rtf
{
[CmdletBinding()]
param (
[Parameter(ValueFromPipeline = $true)]
[string[]] $String
)
process
{
if ($null -eq $String) { $String = #() }
$stringBuilder = New-Object System.Text.StringBuilder
foreach ($s in $String)
{
$stringBuilder.Length = 0
foreach ($character in $s.GetEnumerator())
{
if ([int]$character -lt 0x7F)
{
$null = $stringBuilder.Append($character)
}
else
{
$null = $stringBuilder.AppendFormat('\u{0:x}', [int]$character)
}
}
$stringBuilder.ToString()
}
}
}
Then #{name="ABC<中文>" } | ConvertTo-Json -Compress| ConvertTo-Rtf
Output:
{"name":"ABC\u003c\u4e2d\u6587\u003e"}
Related
I have a JSON-File where I want to replace several values with a placeholder.
So i made a .csv with the parameters to replace. line[0] (if existing) is the path in the file, line[1] the element, line[2] the placeholder.
journeys.legs.origin.properties.downloads;url;placeholder;
;url;placeholder;
download;url;placeholder;
;psFileName;placeholder;
;serverTime;placeholder;
;calcTime;placeholder;
now I defined the following functions, to get the file, read the csv and replace the stuff.
$storage = "D:\Service\test.json"
$parameters2replace = "D:\Service\parameters2replace.csv"
function Get-JSONProperty([object] $InputObject, [string] $Property) {
$path = $Property -split '\.'
$obj = $InputObject
$path | %{ $obj = $obj.$_ }
$obj
}
function setParameter(){
foreach ($parameter in Get-Content $parameters2replace){
$line=$parameter.split(";")
$path = $line[0]
$elementName = $line[1]
$newValue = $line[2]
replaceElement $path $elementName $newValue
}
}
function replaceElement($path, $elementName, $newValue){
ForEach($JSONPath in Get-JSONProperty $JSON $path){
if (!$line[0]){
if($JSON.$elementName){
$JSON.$elementName = $newValue
}
}
else{
if($JSON.$path.$elementName){
echo $JSON.$path.$elementName
$JSON.$path.$elementName = $newValue
}
}
}
$JSON | ConvertTo-Json -depth 32| set-content $storage
}
$JSON = Get-Content $storage -raw | ConvertFrom-Json
setParameter
My problem now is, that the following if-argument won't work with the $path variable. If i put it in hardcoded it works just fine.
if($JSON.$path.$elementName)
I hope i could make everything clear, this was my first post.
$JSON.$path makes PowerShell look for a property with the literal name 'journeys.legs.origin.properties.downloads'. You need to split the string into it's individual parts and iterate (or recurse) through the path:
function Get-JSONPropertyValue {
param(
$JSON,
[string]$Path,
[string]$Name
)
$object = $JSON
foreach($propName in $Path.Split('.')){
$object = $object.$propName
}
return $object.$Name
}
function Set-JSONPropertyValue
{
param(
$JSON,
[string]$Path,
[string]$Name,
$Value
)
$object = $JSON
foreach($propName in $Path.Split('.')){
$object = $object.$propName
}
$object.$Name = $Value
}
Now you can do:
if($value = Get-JSONPropertyValue $JSON -Path $path -Name $elementName){
echo "Old value: $value"
Set-JSONPropertyValue $JSON -Path $path -Name $elementName -Value $newValue
}
I found a powershell deep compare object (https://github.com/kjelderg/Compare-DeepObject) and derived a recursive Json Compare Function for powershell.
The function generally seems to work fine, but its too slow for my purpose. For a batch compare it needs 8 minutes.
How can I speed up the processs?
Thanks
Alex
# walk through the two objects and generate a diff
# This extends compare-object which itself only does shallow comparisons.
# To use, impor t-module <thisFileName>
# Compare-DeepObject $foo $istar
function Compare-Json2 {
[CmdletBinding()]
param (
[Parameter(Mandatory=$true)][AllowNull()]$soll,
[Parameter(Mandatory=$true)][AllowNull()]$ist,
$isFile
# TODO: max-depth
)
PROCESS {
$diff = [PSCustomObject]#{
istJson = '{}'
sollJson = '{}'
difference = #()
}
if($isFile) {
if ($ist -eq $soll) {
$diff.difference += 'istFile == Soll File! Big Error in Testskript!'
return $diff
}
if(Test-Path $ist) {
$ist = Get-Content $ist
}
else {
if(Test-Path $soll) {
$diff.difference += 'istFileDoesNotExists'
}
return $diff
}
if(Test-Path $soll) {
$soll = Get-Content $soll
}
else {
$diff.difference += 'sollFileDoesNotExists'
return $diff
}
}
$diff.istJson = ($ist | ConvertFrom-Json) | ConvertTo-Json -Depth 100 -Compress
$diff.sollJson = ($soll | ConvertFrom-Json) | ConvertTo-Json -Depth 100 -Compress
$ist = $ist | ConvertFrom-Json -Depth 100
$soll = $soll | ConvertFrom-Json -Depth 100
$diff.difference = Compare-JsonObjectRecursive -soll $soll -ist $ist -node $null
# }
return $diff
} # End PROCESS
} # End function$
function Compare-JsonObjectRecursive {
[CmdletBinding()]
param (
[Parameter(Mandatory=$true)][AllowNull()]$soll,
[Parameter(Mandatory=$true)][AllowNull()]$ist,
$node
# TODO: max-depth
)
PROCESS {
# if one side is null, return the other side.
if($soll -eq $null -and $ist -ne $null) { return #{ soll=$null; ist=$ist} }
if($ist -eq $null -and $soll -ne $null) { return #{ ist=$null; soll=$soll} }
if($soll -eq $null -and $ist -eq $null) { return }
# compare data types
$differences = #{}
#$differences = New-Object PSCustomObject
if (($ist -is [PsCustomObject]) -and ($soll -is [array])) {
[array]$ist = $ist
}
if (($ist -is [array]) -and ($soll -is [PsCustomObject])) {
[array]$soll = $soll
}
if($soll -is [array]) { # Recurse for each element of an array
for($i = 0; $i -lt [math]::max($soll.length, $ist.length); $i++) {
if ($node) {$node = "$node[$i]"}
else {$node = "[$i]"}
Compare-JsonObjectRecursive -soll $soll[$i] -ist $ist[$i] -node $node
}
}
elseif ($soll -is [PSCustomObject]) { # Recurse for each property of a PSCustomObject
# walk both sets of keys^Wproperty names with this cool get-unique magic.
$keys = (#($soll.PSObject.properties.name) + #($ist.PSObject.properties.name)) | Sort-Object | get-unique
if ($node) {$node = "$node."}
else {$node = ""}
foreach($key in $keys) {
Compare-JsonObjectRecursive -soll $soll.$key -ist $ist.$key -node "$node$key"
}
}
else {
if($soll -ne $ist) {
$differences[$node] = #{soll = $soll; ist = $ist}
return $differences
}
}
} # End PROCESS
} # End function$
I built a recursive solution to read in a JSON doc and output a flattened table. Now I want to reverse the operation but am having a hard time figuring out a soliton. I need it to be in PowerShell as I just want to add it to my existing module.
I posted about my flatten code here: https://stackoverflow.com/a/63499156/13224569
I have a partial solution, but it falling well short. Here's that code:
$a = '.person.pets[0].color.black.toy.foo' -split '\.'
$b = '.john.harold.cravener' -split '\.'
$z = '.john.is.content' -split '\.'
$m = #{}
$c = #{}
$p = #{}
$pk = $null
$firstTime = $true
foreach($i in $a[1..($a.Count-1)]) {
$c = #{$i = $null}
if($firstTime){
$m = $c
$firstTime = $false
}
if($p -and $pk) {
$p[$pk] = $c
}
$p = $c
$pk = $i
}
$m
exit
And here's its output:
PS D:\> .\tester.ps1 | ConvertTo-Json -Depth 20
{
"person": {
"pets[0]": {
"color": {
"black": {
"toy": {
"foo": null
}
}
}
}
}
}
My first challenge, I can't figure out how to move on to the next row and store it to the exiting $m variable. I'm getting tripped up with the way this var is being referenced in PowerShell / .Net Core.
The second challenge is how to deal with arrays, but I haven't even started on this part yet.
Any and all help / suggestions would be greatly appreciated!
After some research, and a lot of work and debugging, I came to a solution. From my testing so far, it works quite well. It's based on simonw's python code here. And while my solution needed to be in PowerShell, it's not a straight port but pretty close. Most of the modifications are based on differences in how PowerShell and Python handles certain things. The main function is ConvertFrom-JhcUtilJsonTable which in turn depends on three helper functions. Here's all of them.
function ConvertFrom-JhcUtilJsonTable {
param (
[Parameter(Mandatory = $true, ValueFromPipeline = $true)]
[System.Collections.Hashtable]
$jsonHashTable
)
begin {}
process {
foreach ($h in $jsonHashTable) {
$m = #{}
foreach ($k in $h.Keys) {
$current = $m
$val = $h[$k]
$k = ($k -replace '\]', '') -replace '\[', '.'
$bits = $k.split('.')
$path = $bits[1..($bits.Count - 1)]
$count = 0
foreach ($bit in $path) {
$count++
if ($v = $current.item($bit)) {
$current[$bit] = $v
}
else {
if ($count -eq $path.Count) {
$current[$bit] = $val
}
else {
$current[$bit] = #{}
}
}
$current = $current[$bit]
}
}
intKeyHashToLists -obj $m
#--python code had a buit about handling root units e.g. {'$empty': '{}'} - need to add that
}
}
end {}
}
#---helper function for ConvertFrom-JhcUtilJsonTable
#
function isType {
param (
[Parameter(Mandatory)]
[System.Object]
$obj,
[Parameter(Mandatory)]
[ValidateSet('Hashtable', 'Object[]')]
[System.String]
$typeName
)
$t = $obj.GetType()
if ($t.Name -eq $typeName) {
return $true
}
return $false
}
#---helper function for ConvertFrom-JhcUtilJsonTable
#
function allKeysDigits {
param (
[Parameter(Mandatory)]
[System.Collections.Hashtable]
$h
)
foreach ($k in $h.Keys) {
if ($k -match '^0\d') {
return $false
}
if ($k -notmatch '^\d+$') {
return $false
}
}
return $true
}
#---helper function for ConvertFrom-JhcUtilJsonTable
#
function intKeyHashToLists {
param (
[Parameter(Mandatory)]
[System.Object]
$obj
)
if (isType -obj $obj -typeName 'Hashtable') {
if ($obj -and (allKeysDigits -h $obj)) {
$a = #()
foreach ($k in ($obj.Keys | Sort-Object) ) {
$a += intKeyHashToLists -obj $obj.item($k)
}
return ,$a #--- adding the comma forces this to retun an array even when it's a single element
}
else {
$h = #{}
foreach ($k in $obj.Keys) {
$h[$k] = intKeyHashToLists -obj $obj.item($k)
}
return $h
}
}
elseif (isType -obj $obj -typeName 'Object[]') {
return ( $obj | ForEach-Object { intKeyHashToLists -obj $_ } )
}
else {
return $obj
}
}
What is the easiest way to convert a PSCustomObject to a Hashtable? It displays just like one with the splat operator, curly braces and what appear to be key value pairs. When I try to cast it to [Hashtable] it doesn't work. I also tried .toString() and the assigned variable says its a string but displays nothing - any ideas?
Shouldn't be too hard. Something like this should do the trick:
# Create a PSCustomObject (ironically using a hashtable)
$ht1 = #{ A = 'a'; B = 'b'; DateTime = Get-Date }
$theObject = new-object psobject -Property $ht1
# Convert the PSCustomObject back to a hashtable
$ht2 = #{}
$theObject.psobject.properties | Foreach { $ht2[$_.Name] = $_.Value }
Keith already gave you the answer, this is just another way of doing the same with a one-liner:
$psobject.psobject.properties | foreach -begin {$h=#{}} -process {$h."$($_.Name)" = $_.Value} -end {$h}
Here's a version that works with nested hashtables / arrays as well (which is useful if you're trying to do this with DSC ConfigurationData):
function ConvertPSObjectToHashtable
{
param (
[Parameter(ValueFromPipeline)]
$InputObject
)
process
{
if ($null -eq $InputObject) { return $null }
if ($InputObject -is [System.Collections.IEnumerable] -and $InputObject -isnot [string])
{
$collection = #(
foreach ($object in $InputObject) { ConvertPSObjectToHashtable $object }
)
Write-Output -NoEnumerate $collection
}
elseif ($InputObject -is [psobject])
{
$hash = #{}
foreach ($property in $InputObject.PSObject.Properties)
{
$hash[$property.Name] = ConvertPSObjectToHashtable $property.Value
}
$hash
}
else
{
$InputObject
}
}
}
My extremely lazy approach, enabled by a new feature in PowerShell 6:
$myhashtable = $mypscustomobject | ConvertTo-Json | ConvertFrom-Json -AsHashTable
This works for PSCustomObjects created by ConvertFrom_Json.
Function ConvertConvertFrom-JsonPSCustomObjectToHash($obj)
{
$hash = #{}
$obj | Get-Member -MemberType Properties | SELECT -exp "Name" | % {
$hash[$_] = ($obj | SELECT -exp $_)
}
$hash
}
Disclaimer: I barely understand PowerShell so this is probably not as clean as it could be. But it works (for one level only).
My code:
function PSCustomObjectConvertToHashtable() {
param(
[Parameter(ValueFromPipeline)]
$object
)
if ( $object -eq $null ) { return $null }
if ( $object -is [psobject] ) {
$result = #{}
$items = $object | Get-Member -MemberType NoteProperty
foreach( $item in $items ) {
$key = $item.Name
$value = PSCustomObjectConvertToHashtable -object $object.$key
$result.Add($key, $value)
}
return $result
} elseif ($object -is [array]) {
$result = [object[]]::new($object.Count)
for ($i = 0; $i -lt $object.Count; $i++) {
$result[$i] = (PSCustomObjectConvertToHashtable -object $object[$i])
}
return ,$result
} else {
return $object
}
}
For simple [PSCustomObject] to [Hashtable] conversion Keith's Answer works best.
However if you need more options you can use
function ConvertTo-Hashtable {
<#
.Synopsis
Converts an object to a hashtable
.DESCRIPTION
PowerShell v4 seems to have trouble casting some objects to Hashtable.
This function is a workaround to convert PS Objects to [Hashtable]
.LINK
https://github.com/alainQtec/.files/blob/main/src/scripts/Converters/ConvertTo-Hashtable.ps1
.NOTES
Base ref: https://community.idera.com/database-tools/powershell/powertips/b/tips/posts/turning-objects-into-hash-tables-2
#>
PARAM(
# The object to convert to a hashtable
[Parameter(ValueFromPipeline = $true, Mandatory = $true)]
$InputObject,
# Forces the values to be strings and converts them by running them through Out-String
[switch]$AsString,
# If set, empty properties are Included
[switch]$AllowNulls,
# Make each hashtable to have it's own set of properties, otherwise,
# (default) each InputObject is normalized to the properties on the first object in the pipeline
[switch]$DontNormalize
)
BEGIN {
$headers = #()
}
PROCESS {
if (!$headers -or $DontNormalize) {
$headers = $InputObject | Get-Member -type Properties | Select-Object -expand name
}
$OutputHash = #{}
if ($AsString) {
foreach ($col in $headers) {
if ($AllowNulls -or ($InputObject.$col -is [bool] -or ($InputObject.$col))) {
$OutputHash.$col = $InputObject.$col | Out-String -Width 9999 | ForEach-Object { $_.Trim() }
}
}
} else {
foreach ($col in $headers) {
if ($AllowNulls -or ($InputObject.$col -is [bool] -or ($InputObject.$col))) {
$OutputHash.$col = $InputObject.$col
}
}
}
}
END {
return $OutputHash
}
}
Maybe this is overkill but I hope it Helps
Today, the "easiest way" to convert PSCustomObject to Hashtable would be so:
$custom_obj | ConvertTo-HashtableFromPsCustomObject
OR
[hashtable]$custom_obj
Conversely, you can convert a Hashtable to PSCustomObject using:
[PSCustomObject]$hash_table
Only snag is, these nifty options may not be available in older versions of PS
I have a dictionary like this:
function HashHandlerSHA256
{
param($Path, $Checksum)
$csp = new-object -TypeName System.Security.Cryptography.SHA256CryptoServiceProvider
$ComputedHash = $csp.ComputeHash([System.IO.File]::ReadAllBytes($Path))
$ComputedHash = [System.BitConverter]::ToString($ComputedHash).Replace("-", "").ToLower()
$result = $ComputedHash.CompareTo($Checksum)
return $result -eq 0
}
$HashHandler = #{"SHA256" = HashHandlerSHA256}
containing validation algorithms and the functions to be called for validation. The functions should all have the same parameter and return type.
Now when I have:
$Checksums = #{"SHA256" = "..."}
I'd like to call the correct function depending on which algorithms and values I have available. In this case I'd have a valid sha256 hash.
Now I want to do:
function Validate
{
param($Path, $Checksums)
foreach($Hash in $Checksums) {
$Type = $Hash.Name
$Value = $Hash.Value
if ($HashHandler.ContainsKey($Type)) {
$Handler = $HashHandler.Get_Item($Type)
if (-Not ($Handler -Path $Path -Checksum $Value)) {
return $FALSE
}
}
}
return $TRUE
}
The errormessage I get is:
At C:\Users\username\Desktop\hashtest.ps1:27 char:23
+ if (-Not ($Handler -Path $Path -Checksum $Value)) {
+ ~~~~~
Unexpected token '-Path' in expression or statement.
I am relatively new to PowerShell. I know how to call functions with parameters, but when stored in a variable, I didn't manage to solve this and when searching online for this, I didn't get the answers I needed.
Thanks for help.
if i understand you want Something like this
function HashHandlerSHA256
{
param($Path, $Checksum)
$csp = new-object -TypeName System.Security.Cryptography.SHA256CryptoServiceProvider
$ComputedHash = $csp.ComputeHash([System.IO.File]::ReadAllBytes($Path))
$ComputedHash = [System.BitConverter]::ToString($ComputedHash).Replace("-", "").ToLower()
$result = $ComputedHash.CompareTo($Checksum)
return $result -eq 0
}
function Validate
{
param($Path, $Checksums)
foreach($Hashkey in $Checksums.Keys)
{
$Value = $Checksums[$Hashkey]
if ($script:HashHandler.ContainsKey($Hashkey))
{
if (-Not (&$script:HashHandler[$Hashkey] -Path $Path -Checksum $Value)) { return $false}
}
}
return $TRUE
}
#add here your couples of algo/function
$script:HashHandler = #{"SHA256" = 'HashHandlerSHA256'}
#checksum to test
$Checksums=#{}
$Checksums["SHA256"]= 'd6a0a09fb1a7971b497674675d5b5621d865d6020e384137548de9c4ac6d4994'
$Checksums["MD5"]= 'xxxx'
#test list checksum and algo
Validate -Path "c:\temp\hello.csv" -Checksums $Checksums
an other solution
$file="C:\temp\exludevalue.txt"
$Checksums=#{}
$Checksums["SHA256"]= 'd6a0a09fb1a7971b497674675d5b5621d865d6020e384137548de9c4ac6d4994'
$Checksums["MD5k"]= '11A8D99F80F9B29FCF6A995D2F17B2E3'
$Checksums.Keys |
%{
if ($(gcm Get-FileHash).Parameters.Algorithm.Attributes.ValidValues -contains $_)
{
$algocalc=(Get-FileHash -path $file -Algorithm $_).Hash;
}
else
{
$algocalc='ALGO NOT FOUNDED'
}
new-object psobject -Property #{
Algo=$_
OldValue=$Checksums[$_]
CalculedValue=$algocalc
ResultComparison= $algocalc -eq $Checksums[$_]
}
}