ruby json data lookup with regex - json

I have a json file ou.json:
{
"ProdInf": {
"regex": "^(?i)nyinf-p(\\d+)$",
"ou": "OU=Inf,OU=Prod,OU=Servers,DC=example,DC=local"
},
"ProdHyperV": {
"regex": "^(?i)nyhyp-p(\\d+)$",
"ou": "OU=HyperV,OU=Prod,OU=Servers,DC=example,DC=local"
},
"ProdRDS": {
"regex": "^(?i)nyrds-p(\\d+)$",
"ou": "OU=RDS,OU=Prod,OU=Servers,DC=example,DC=local"
}
"Default": {
"regex": "zzz",
"ou": "OU=Servers,DC=example,DC=local"
}
}
I have:
require 'json'
computername = "nyhyp-p24"
json_file = File.read("./ou.json")
json_hash = JSON.parse(json_file)
then be able to retrieve all the regex in the hash, and evaluate computername against each regex, and if it is a match return the appropriate ou.first
in the example above,
ou should equal "OU=HyperV,OU=Prod,OU=Servers,DC=example,DC=local"
if there is no match it should return default.ou

I add extra slashes to escape them in the JSON
Like this: "^(?i)nyrds-p(\\d+)$" --> "^(?i)nyrds-p(\\\\d+)$"
require "json"
json =
<<~JSON
{
"ProdInf": {
"regex": "^(?i)nyinf-p(\\\\d+)$",
"ou": "OU=Inf,OU=Prod,OU=Servers,DC=example,DC=local"
},
"ProdHyperV": {
"regex": "^(?i)nyhyp-p(\\\\d+)$",
"ou": "OU=HyperV,OU=Prod,OU=Servers,DC=example,DC=local"
},
"ProdRDS": {
"regex": "^(?i)nyrds-p(\\\\d+)$",
"ou": "OU=RDS,OU=Prod,OU=Servers,DC=example,DC=local"
},
"Default": {
"regex": "zzz",
"ou": "OU=Servers,DC=example,DC=local"
}
}
JSON
configs = JSON.parse(json, symbolize_names: true)
Then you can define some method like this
def find_ou(computer_name, configs)
configs
.find { |_, config| computer_name.match?(%r{#{config[:regex]}}) }
&.last
&.dig(:ou) || configs[:Default][:ou]
end
And then
find_ou("nyhyp-p24", configs)
# => "OU=HyperV,OU=Prod,OU=Servers,DC=example,DC=local"
find_ou("nyinf-p26", configs)
# => "OU=Inf,OU=Prod,OU=Servers,DC=example,DC=local"
find_ou("something else", configs)
# => "OU=Servers,DC=example,DC=local"

Related

pyjq - how to use "select" with both query and value as variables

I am writing a code in python3 where i am struggling with usage of variables with "pyjq", the code works without variables but variables are not getting parsed inside pyjq.
The documentation referred is https://github.com/doloopwhile/pyjq/blob/master/README.md#api
Please check the code given below and suggest -
My code
import json, os
import pyjq
from flask import Flask, request, jsonify
def query_records():
args = {"meta.antivirus.enabled": "true"}
for key, value in args.items():
with open('/tmp/data.txt', 'r') as f:
print (key)
print (value)
data = f.read()
records = json.loads(data)
query = ("." + key)
print (query)
#jq '.[]|select(.meta.antivirus.enabled=="true")' filename.json works,issue with variable substitution in python
match = pyjq.all('.[]|select(["$query"]==$value)', records, vars={"value": value,"query": query})
print (match)
query_records()
Content of file "/tmp/data.txt"
[
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "false"
},
"calories": 230,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "1g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "sandwich-nutrition"
},
{
"meta": {
"allergens": {
"eggs": "true",
"nuts": "false",
"seafood": "true"
},
"calories": 440,
"carbohydrates": {
"dietary-fiber": "4g",
"sugars": "2g"
},
"fats": {
"saturated-fat": "0g",
"trans-fat": "1g"
}
},
"name": "random-nutrition"
}
]
Expected output(which works without variables)
{
"name": "alpharetta",
"meta": {
"antivirus": {
"enabled": "true"
},
"limits": {
"cpu": {
"enabled": "true",
"value": "250m"
}
}
}
}
Current output []
seems like some issue with variables not being passed in case of "query" , help would be appreciated.
Edit 1
It works if I hardcode "query" -
match = pyjq.all('.[]|select(.meta.antivirus.enabled==$value)', records, vars={"value": value,"query": query})
but not vice-versa
which probably narrows it down to issue with the variable "query"
JQ is not a necessity and I can use other libraries too,given that json is returned
Variables are intended to be used for values, not for jq expressions (at least not directly).
I think the easiest option here is to go for an fstring:
match = pyjq.all(f'.[]|select({query}==$value)', records, vars={"value": value})
and it probably makes sense to prepend the period inside the fstring:
match = pyjq.all(f'.[]|select(.{key}==$value)', records, vars={"value": value})

Convert a JSON value to another JSON shape?

I have the following JSON file that I need to transform.
Original JSON file:
{
"configs": [
{
"ignore": "this",
"key": "abc123",
"default": false
},
{
"ignore": "that",
"key": "def123",
"default": "#F5F5F5"
},
{
"type": "color",
"key": "ghi123",
"default": "#3d4246"
}
]
}
I want to convert the JSON to look like this:
{
"configs": [
{
"abc123": false
},
{
"def123": "#F5F5F5"
},
{
"ghi123": "#3d4246"
}
]
}
How can I do this?
require 'json'
original_json = JSON.parse("...") #original JSON shape
There's likely a cleaner way, but here's a shot:
require 'json'
original_json = JSON.parse('{
"configs": [
{
"ignore": "this",
"key": "abc123",
"default": false
},
{
"ignore": "that",
"key": "def123",
"default": "#F5F5F5"
},
{
"type": "color",
"key": "ghi123",
"default": "#3d4246"
}
]
}')
original_json['configs'].each do |h|
k = h["key"]
v = h["default"]
h.clear
h[k] = v
end
original_json.to_json
#=> {"configs":[{"abc123":false},{"def123":"#F5F5F5"},{"ghi123":"#3d4246"}]}
With Enumerable#each_with_object you can create new hash as you want. Then you can convert it to json.
original_hash = JSON.parse(original_json)
new_hash = original_hash[:configs].each_with_object({ configs: [] }) do |item, new_hash|
new_hash[:configs] << {
item[:key] => item[:default]
# add another key value pair if you want
}
end
new_hash.to_json
#=> {
# "configs": [
# {
# "abc123": false
# },
# {
# "def123": "#F5F5F5"
# },
# {
# "ghi123": "#3d4246"
# }
# ]
# }

Filter nodes by a boolean value

I have to filter JSON by IsNew parameter (true or false)
Part of the JSON is below:
{
"data": [
{
"type": "users",
"attributes": {
"first-name": "student21",
"last-name": "student21",
"username": "student21",
"role": "student",
"IsNew": true
},
"relationships": {
"groups": {
"data": [
{
"type": "groups",
"id": "123f"
}
]
}
}
},
{
"type": "users",
"attributes": {
"first-name": "student23",
"last-name": "student23",
"email": "",
"avatar-url": null,
"username": "student23",
"role": "student",
"IsNew": false
},
"relationships": {
"groups": {
"data": [
{
"type": "groups",
"id": "456"
}
]
}
}
}
]
}
I've tried the following expressions:
$..data..[?(#.IsNew == true)].username,
$..data..[?(#.IsNew == 'true')].username,
$..data..[?(#.IsNew == "true")].username
All those expressions don't return any result.
I need to extract usernames for students with "IsNew" == true and "IsNew" == false separetely.
To extract usernames for students with "IsNew" == true, use a JSON Extractor with following settings:
JSON Path Expression :$.data..attributes[?(#.IsNew =~ /.*true/i)].username
Match No. -1 [to get all the matches for multiple students]
To extract usernames for students with "IsNew" == false, use a JSON Extractor with following settings:
JSON Path Expression :$.data..attributes[?(#.IsNew =~ /.*false/i)].username
Match No. -1 [to get all the matches for multiple students]
Use following variables to process further:
${FalseStudent_matchNr} or ${FalseStudent} if you have used Match No other than -1 in JSON Extractor
${TrueStudent_matchNr} or ${TrueStudent} if you have used Match No other than -1 in JSON Extractor

How to add keys to values and convert to json in PowerShell?

I have a string like this:
$string = "PackageName1,1,C:\Path
PackageName2,12,C:\Path2
PackageName3,3,C:\Path3"
(is a file with multilines, when I get the content I have the string above)
I want to convert this string to Json:
[
{
Pacakge: "PackageName1",
Branch: = "1",
LocalPath = "C:\Path"
}
{
Pacakge: "PackageName2",
Branch: = "2",
LocalPath = "C:\Path2"
}
]
I can get the values with this code:
$spiltted = $string.Split(' ')
ForEach ($s in $splitted)
{
$values = $s.Split(',')
}
How can I add the Json keys to each value and convert it to Json object?
Thank you.
As your String looks like a csv without headers:
$string | ConvertFrom-Csv -Header Package,Branch,LocalPath|ConvertTo-Json
Sample output
[
{
"Package": "PackageName1",
"Branch": "1",
"LocalPath": "C:\\Path"
},
{
"Package": "PackageName2",
"Branch": "12",
"LocalPath": "C:\\Path2"
},
{
"Package": "PackageName3",
"Branch": "3",
"LocalPath": "C:\\Path3"
}
]

Groovy JSONBuilder issues

I'm trying to use JsonBuilder with Groovy to dynamically generate JSON. I want to create a JSON block like:
{
"type": {
"__type": "urn",
"value": "myCustomValue1"
},
"urn": {
"__type": "urn",
"value": "myCustomValue2"
},
"date": {
"epoch": 1265662800000,
"str": "2010-02-08T21:00:00Z"
},
"metadata": [{
"ratings": [{
"rating": "NR",
"scheme": "eirin",
"_type": {
"__type": "urn",
"value": "myCustomValue3"
}
}],
"creators": [Jim, Bob, Joe]
}]
}
I've written:
def addUrn(parent, type, urnVal) {
parent."$type" {
__type "urn"
"value" urnVal
}
}
String getEpisode(String myCustomVal1, String myCustomVal2, String myCustomVal3) {
def builder = new groovy.json.JsonBuilder()
def root = builder {
addUrn(builder, "type", myCustomVal1)
addUrn(builder, "urn", "some:urn:$myCustomVal2")
"date" {
epoch 1265662800000
str "2010-02-08T21:00:00Z"
}
"metadata" ({
ratings ({
rating "G"
scheme "eirin"
addUrn(builder, "_type", "$myCustomVal3")
})
creators "Jim", "Bob", "Joe"
})
}
return root.toString();
}
But I've run into the following issues:
Whenever I call addUrn, nothing is returned in the string. Am I misunderstanding how to use methods in Groovy?
None of the values are encapsulated in double (or single) quotes in the returned string.
Anytime I use a {, I get a '_getEpisode_closure2_closure2#(insert hex)' in the returned value.
Is there something wrong with my syntax? Or can someone point me to some example/tutorial that uses methods and/or examples beyond simple values (e.g. nested values within arrays).
NOTE: This is a watered down example, but I tried to maintain the complexity around the areas that were giving me issues.
You have to use delegate in addUrn method instead of
passing the builder on which you are working.
It is because you are doing a toSting() or toPrettyString() on root instead of builder.
Solved if #2 is followed.
Sample:
def builder = new groovy.json.JsonBuilder()
def root = builder {
name "Devin"
data {
type "Test"
note "Dummy"
}
addUrn(delegate, "gender", "male")
addUrn(delegate, "zip", "43230")
}
def addUrn(parent, type, urnVal) {
parent."$type" {
__type "urn"
"value" urnVal
}
}
println builder.toPrettyString()
Output:-
{
"name": "Devin",
"data": {
"type": "Test",
"note": "Dummy"
},
"gender": {
"__type": "urn",
"value": "male"
},
"zip": {
"__type": "urn",
"value": "43230"
}
}