Parsing JSON file in powershell with specific characters - json

I am trying to get the values in powershell within specific characters. Basically I have a json with thousands of objects like this
"Name": "AllZones_APOPreface_GeographyMatch_FromBRE_ToSTR",
"Sequence": 0,
"Condition": "this.TripOriginLocationCode==\"BRE\"&&this.TripDestinationLocationCode==\"STR\"",
"Action": "this.FeesRate=0.19m;this.ZoneCode=\"Zone1\";halt",
"ElseAction": ""
I want everything within \" \"
IE here I would see that BRE and STR is Zone1
All I need is those 3 things outputted.
I have been searching how to do it with ConvertFrom-Json but no success, maybe I just havent found a good article on this.
Thanks

Start by representing your JSON as a string:
$myjson = #'
{
"Name": "AllZones_APOPreface_GeographyMatch_FromBRE_ToSTR",
"Sequence": 0,
"Condition": "this.TripOriginLocationCode==\"BRE\"&&this.TripDestinationLocationCode==\"STR\"",
"Action": "this.FeesRate=0.19m;this.ZoneCode=\"Zone1\";halt",
"ElseAction": ""
}
'#
Next, create a regular expression that matches everything in between \" and \", that's under 10 characters long (else it'll match unwanted results).
$regex = [regex]::new('\\"(?<content>.{1,10})\\"')
Next, perform the regular expression comparison, by calling the Matches() method on the regular expression. Pass your JSON string into the method parameters, as the text that you want to perform the comparison against.
$matchlist = $regex.Matches($myjson)
Finally, grab the content match group that was defined in the regular expression, and extract the values from it.
$matchlist.Groups.Where({ $PSItem.Name -eq 'content' }).Value
Result
BRE
STR
Zone1
Approach #2: Use Regex Look-behinds for more accurate matching
Here's a more specific regular expression that uses look-behinds to validate each field appropriately. Then we assign each match to a developer-friendly variable name.
$regex = [regex]::new('(?<=TripOriginLocationCode==\\")(?<OriginCode>\w+)|(?<=TripDestinationLocationCode==\\")(?<DestinationCode>\w+)|(?<=ZoneCode=\\")(?<ZoneCode>\w+)')
$matchlist = $regex.Matches($myjson)
### Assign each component to its own friendly variable name
$OriginCode, $DestinationCode, $ZoneCode = $matchlist[0].Value, $matchlist[1].Value, $matchlist[2].Value
### Construct a string from the individual components
'Your origin code is {0}, your destination code is {1}, and your zone code is {2}' -f $OriginCode, $DestinationCode, $ZoneCode
Result
Your origin code is BRE, your destination code is STR, and your zone code is Zone1

Related

JMESPath join output and strip unwanted chars

I have the following json;
[
{
"compartment-id": "CompartmentID-123",
"defined-tags": {},
"display-name": "Test-123",
"freeform-tags": {},
"id": "ID-ABC",
"kms-key-id": "",
"lifecycle-state": "ACTIVE",
}
]
I am looking to join the id and display-name params into a comma seperate string with " and [] stripped, like so:
ID-ABC,Test-123
Closest I've managed to get so far is:
oci fs file-system list -c $compart --availability-domain $ad --query 'data[].[id,"display-name"][]' | tr -d '" '
[
ID-ABC,
Test-123
]
Wondering if there's a cleaner way of doing this all within JMESPath without piping output to tr, jq, sed or awk etc etc
UPDATE based on input from β.εηοιτ.βε
So close...
oci fs file-system list -c $compart --availability-domain $ad3 --query 'data[0].join(',', [id, "display-name"])'
Returns
ParseError: invalid token: Parse error at column 13, token "," (COMMA), for expression:
"data[0].join(,, [id, "display-name"])"
However playing around with quotes, best I can get is by using;
oci fs file-system list -c $compart --availability-domain $ad3 --query "data[0].join(',', [id, 'display-name'])"
Private key passphrase:
"ID-ABC,display-name"
I'm beginning to wonder if there's something wrong with my local settings or shell, whereby it's getting confused by the quotes marks?
Since you do have an array, you will first need to target the first element of the array, to get rid of it. Since an array is 0-based, this can easily be achieved via:
data[0]
which gives
{
"compartment-id": "CompartmentID-123",
"defined-tags": {},
"display-name": "Test-123",
"freeform-tags": {},
"id": "ID-ABC",
"kms-key-id": "",
"lifecycle-state": "ACTIVE"
}
Then, in order to create the string you are looking for, you can use the join function, that accepts, as parameter, a glue character and array.
You can get the array you look for pretty easily, using filters and multiselect lists:
data[0].[id, "display-name"]
which gives
[
"ID-ABC",
"Test-123"
]
Now we just need to apply the join function on top of all this:
data[0].join(',', [id, "display-name"])
which finally gives:
"ID-ABC,Test-123"
Additional notes on quoting in JMESPath:
'some-string'
is a raw literal string and won't be interpreted
`some-string`
is another form of raw literal string and won't be interpreted
"some-string"
is a literal expression and will be interpreted
So:
data[0].join(',', [id, "display-name"])
and
data[0].join(`,`, [id, "display-name"])
are two strictly equals queries.
While
data[0].join(',', [id, 'display-name'])
has a totally different meaning and would end you with the string display-name being the second element of your array, so it will result in
"ID-ABC,display-name"
All of this based on the JSON structure:
{
"data":[
{
"compartment-id":"CompartmentID-123",
"defined-tags":{},
"display-name":"Test-123",
"freeform-tags":{},
"id":"ID-ABC",
"kms-key-id":"",
"lifecycle-state":"ACTIVE"
}
]
}

I need to extract Data from a single line of json-data which is inbetween two variables (Powershell)

I need to extract Data from a single line of json-data which is inbetween two variables (Powershell)
my Variables:
in front of Data:
DeviceAddresses":[{"Id":
after Data:
,"
I tried this, but there needs to be some error because of all the special characters I'm using:
$devicepattern = {DeviceAddresses":[{"Id":{.*?},"}
#$deviceid = [regex]::match($changeduserdata, $devicepattern).Groups[1].Value
#$deviceid
As you've found, some character literals can't be used as-is in a regex pattern because they carry special meaning - we call these meta-characters.
In order to match the corresponding character literal in an input string, we need to escape it with \ -
to match a literal (, we use the escape sequence \(,
for a literal }, we use \}, and so on...
Fortunately, you don't need to know or remember which ones are meta-characters or escapable sequences - we can use Regex.Escape() to escape all the special character literals in a given pattern string:
$prefix = [regex]::Escape('DeviceAddresses":[{"Id":')
$capture = '(.*?)'
$suffix = [regex]::Escape(',"')
$devicePattern = "${prefix}${capture}${suffix}"
You also don't need to call [regex]::Match directly, PowerShell will populate the automatic $Matches variable with match groups whenever a scalar -match succeeds:
if($changeduserdata -match $devicePattern){
$deviceid = $Matches[1]
} else {
Write-Error 'DeviceID not found'
}
For reference, the following ASCII literals needs to be escaped in .NET's regex grammar:
$ ( ) * + . ? [ \ ^ { |
Additionally, # and (regular space character) needs to be escaped and a number of other whitespace characters have to be translated to their respective escape sequences to make patterns safe for use with the IgnorePatternWhitespace option (this is not applicable to your current scenario):
\u0009 => '\t' # Tab
\u000A => '\n' # Line Feed
\u000C => '\f' # Form Feed
\u000D => '\r' # Carriage Return
... all of which Regex.Escape() takes into account for you :)
To complement Mathias R. Jessen's helpful answer:
Generally, note that JSON data is much easier to work with - and processed more robustly - if you parse it into objects whose properties you can access - see the bottom section.
As for your regex attempt:
Note: The following also applies to all PowerShell-native regex features, such as the -match, -replace, and -split operators, the switch statement, and the Select-String cmdlet.
Mathias' answer uses [regex]::Escape() to escape the parts of the regex pattern to be used verbatim by the regex engine.
This is unequivocally the best approach if those verbatim parts aren't known in advance - e.g., when provided via a variable or expression, or passed as an argument.
However, in a regex pattern that is specified as a string literal it is often easier to individually \-escape the regex metacharacters, i.e. those characters that would otherwise have special meaning to the regex engine.
The list of characters that need escaping is (it can be inferred from the .NET Regular-Expression Quick Reference):
\ ( ) | . * + ? ^ $ [ {
If you enable the IgnorePatternWhiteSpace option (which you can do inline with
(?x), at the start of a pattern), you'll additionally have to \-escape:
#
significant whitespace characters (those you actually want matched) specified verbatim (e.g., ' ', or via string interpolation,"`t"); this does not apply to those specified via escape sequences (e.g., \t or \n).
Therefore, the solution could be simplified to:
# Sample JSON
$changeduserdata = '{"DeviceAddresses":[{"Id": 42,"More": "stuff"}]}'
# Note how [ and { are \-escaped
$deviceId = if ($changeduserdata -match 'DeviceAddresses":\[\{"Id":(.*?),"') {
$Matches[1]
}
Using ConvertFrom-Json to properly parse JSON into objects is both more robust and more convenient, as it allows property access (dot notation) to extract the value of interest:
# Sample JSON
$changeduserdata = '{"DeviceAddresses":[{"Id": 42,"More": "stuff"}]}'
# Convert to an object ([pscustomobject]) and drill down to the property
# of interest; note that the value of .DeviceAddresses is an *array* ([...]).
$deviceId = (ConvertFrom-Json $changeduserdata).DeviceAddresses[0].Id # -> 42

Yet another loop JSON object using Powershell

I am not able to twist my head into understanding how to get Powershell to loop the entire JSON Structure, it wont' loop the System.Object[]
$x = ConvertFrom-Json '{
"Car companies": {
"Name of Company": "Ford",
"Cars": [{
"Name of car": "Ranger",
"Config": "Pickup"
},
{
"Name of car": "Puma",
"Config": "Hatchback"
}]
}
}'
foreach( $rootProperty in #($x.psobject.properties | where-object {$_.MemberType -eq "NoteProperty"}) ) {
write-host " - '$($rootProperty.Name)' = '$($rootProperty.Value)'"
foreach( $childProperty in #($rootProperty.Value.psobject.properties ) ) {
write-host "'$($childProperty.Name)' = '$($childProperty.Value)'"
}
}
Outut I get now is just
- 'Brand' = '#{Name of Brand=Ford; Cars=System.Object[]}'
Name of Brand' = 'Ford'
Cars' = ' '
...as a follop How to iterate through a unknown JSON data/object?
tl;dr
You're seeing a bug that unexpectedly string-expands the Cars property value's array elements to the empty string.
A simple workaround - for display purposes only - is to pipe the property value to Out-String to get the usual display representation:
"'$($childProperty.Name)' = '$($childProperty.Value | Out-String)'"
You're seeing a bug in how arrays of [pscustomobject] instances are stringified (as of PowerShell Core 7.0.0-preview.6):
Generally, PowerShell arrays are stringified by joining the stringified element representations with the separator specified in the $OFS preference variable, which defaults to a space char.
Normally, [pscustomobject] instances have a string representation that resembles a hashtable literal (but isn't one); e.g.:
PS> $custObj = [pscustomobject] #{ foo = 'bar' }; "$custObj"
#{foo=bar} # string representation that *resembles* a hashtable literal
Unexpectedly - and this is the bug - when custom objects are the elements of an array, they stringify to the empty string, which is what you saw:
PS> $custObj = [pscustomobject] #{ foo = 'bar' }; $arr = $custObj, $custObj; "[$arr]"
[ ] # !! Bug: custom objects stringified to empty strings, joined with a space
This is an indirect manifestation of a long-standing bug reported in this GitHub issue: that is, elements of an array being stringified are stringified by calls to their .ToString() method, and calling .ToString() on custom objects unexpectedly yields the empty string (unlike the string representation you get when you directly reference a single custom object in an expandable string, as shown above).

jq split string by a pattern

I have a json object with one of field having values for example "countries-sapi-1.0", "inventory-list-api-1.0-snapshot"
Note that the first one has sapi and the other one has api.
Using jq, how can i get countries-sapi or inventory-list-api I mean whatever is there before the version. the version can be as simple as 1.0 or 1.0.1-snapshot etc..
I got here searching for how to split by regex instead of substring in jq, but I found out that you have to give two arguments to the split function (where the second argument contains flags for the regex, but it can be just an empty string).
$ jq -n '"axxbxxxc"|split("x+";"")'
[
"a",
"b",
"c"
]
From the manual:
split
Splits an input string on the separator argument.
jq 'split(", ")'
"a, b,c,d, e, "
=> ["a","b,c,d","e",""]
[...]
split(regex; flags)
For backwards compatibility, split splits on a string, not a regex.
It looks like you need to study up on regular expressions (regex); see for example https://regexone.com/ or https://github.com/zeeshanu/learn-regex or dozens of others.
Using jq, in your particular case, you could start with:
sub(" *- *[0-9]+\\.[0-9]+.*$"; "")
Note that two backslashes are required here because the "from" expression must be (or evaluate to) a valid JSON string.
For input "countries-sapi-1.0", use: .[] | match( "\\d"; "ig") which will give you the following output:
{
"offset": 15,
"length": 1,
"string": "1",
"captures": []
}
{
"offset": 17,
"length": 1,
"string": "0",
"captures": []
}
This uses the first object's offset value and tries to slice it from the starting position to the received offset.
Slice from beginning: $ jq -c '.[:15]'
In our case we got 15 as the offset for the first object so we used :15 for the slice.

Why does the JSON module quote some numbers but not others?

We recently switched to the new JSON2 perl module.
I thought all and everything gets returned quoted now.
But i encountered some cases in which a number (250) got returned as unquoted number in the json string created by perl.
Out of curiosity:
Does anyone know why such cases exist and how the json module decides if to quote a value?
It will be unquoted if it's a number. Without getting too deeply into Perl internals, something is a number if it's a literal number or the result of an arithmetic operation, and it hasn't been stringified since its numeric value was produced.
use JSON::XS;
my $json = JSON::XS->new->allow_nonref;
say $json->encode(42); # 42
say $json->encode("42"); # "42"
my $x = 4;
say $json->encode($x); # 4
my $y = "There are $x lights!";
say $json->encode($x); # "4"
$x++; # modifies the numeric value of $x
say $json->encode($x); # 5
Note that printing a number isn't "stringifying it" even though it produces a string representation of the number to output; print $x doesn't cause a number to be a string, but print "$x" does.
Anyway, all of this is a bit weird, but if you want a value to be reliably unquoted in JSON then put 0 + $value into your structure immediately before encoding it, and if you want it to be reliably quoted then use "" . $value or "$value".
You can force it into a string by doing something like this:
$number_str = '' . $number;
For example:
perl -MJSON -le 'print encode_json({foo=>123, bar=>"".123})'
{"bar":"123","foo":123}
It looks like older versions of JSON has autoconvert functionality that can be set. Did you not have $JSON::AUTOCONVERT set to a true value?