I'm trying to write a JQ filter allowing me to selectively filter object properties based on other of it's values.
For example, given following input
{
"object1": {
"Type": "Type1",
"Properties": {
"Property1": "blablabla",
"Property2": [
{
"Key": "Name",
"Value": "xxx"
},
{
"Key": "Surname",
"Value": "yyy"
}
],
"Property3": "xxx"
}
},
"object2": {
"Type": "Type2",
"Properties": {
"Property1": "blablabla",
"Property2": [
{
"Key": "Name",
"Value": "xxx"
},
{
"Key": "Surname",
"Value": "yyy"
}
],
"Property3": "xxx"
}
}
}
I would like to construct a filter, that based upon the object type, say "Type2", deletes or clears a property of that object, say Property2.
The resulting output would then be:
{
"object1": {
"Type": "Type1",
"Properties": {
"Property1": "blablabla",
"Property2": [
{
"Key": "Name",
"Value": "xxx"
},
{
"Key": "Surname",
"Value": "yyy"
}
],
"Property3": "xxx"
}
},
"object2": {
"Type": "Type2",
"Properties": {
"Property1": "blablabla",
"Property3": "xxx"
}
}
}
Any help greatly appreciated. Thanks in advance.
Pretty simple. Find the objects that you want to update, then update them.
Look through the values of your root object, filtering them based on your condition, the update the Properties property deleting the property you want.
(.[] | select(.Type == "Type2")).Properties |= del(.Property2)
Using .[] on an object yields all property values of an object. Also worth mentioning, when you update a value using assignments, the result of the expression just returns the input (in other words, it doesn't change the context).
A direct approach:
.[] |= (if .Type == "Type2" then delpaths([["Properties", "Property2"]]) else . end)
Related
I would like to change the values acceptable to the Value B field according to the values of the Value A field in the following Json.
{"maximumList": [
{
"ValueA": 232,
"ValueB": ["aa"]
}]}
To do so, I obtained Schema using the Json Schema Generator and modified it to make it as follows.
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"maximumList": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"ValueA": {
"type": "integer"
},
"ValueB": {
"type": "array",
"items": [
{
"type": "string"
}
]
}
},
"required": [
"ValueA",
"ValueB"
],
"if":{
"properties":{
"ValueA" :{"const":232}
}
},
"then":{
"ValueB":{
"enum":["bbb"]
}
}
}
]
}
},
"required": [
"maximumList"
]
}
definitely set the value of the ValueB field to allow only "bbb", but even if "aaa" is entered in the value of the ValueB field, it passes the validation.
Is there anything I'm missing?
I think your problem is that you're using draft 4 of JSON Schema (identified by the $schema keyword value of http://json-schema.org/draft-04/schema#.
The if/then/else keywords are only supported in draft 7 and later. If you change the $schema value to http://json-schema.org/draft-07/schema#, it should work for you.
How can I manipulate this chunk of json:
{
"id": "whatever",
"attributes": [
{
"key": "this",
"value": "A"
},
{
"key": "that",
"value": "B"
},
{
"key": "other",
"value": "C"
}
]
}
So that it matches on "that" and removes the key and value both in that grouping, leaving json like this:
{
"id": "whatever",
"attributes": [
{
"key": "this",
"value": "A"
},
{
"key": "other",
"value": "C"
}
]
}
I am attempting to use jq on linux.
Try this
.attributes |= map(select(.key != "that"))
Demo
Figured it out.
jq 'del(.attributes[] | select(.key == "that"))' test.json | sponge test.json
I have json schema structure that look like below.
{
"$schema": "https://json-schema.org/draft/2019-09/schema",
"description": "My sample Json",
"type": "object",
"properties": {
"eventId": {
"description": "The event Indetifier",
"type": [ "number", "string" ]
},
"serviceType": {
"description": "The service type. It can be either ABC or EFG",
"enum": [ "ABC", "EFG" ]
},
"parameters": { "$ref": "/schemas/parameters" }
},
"required": [ "eventId", "serviceType" ],
"$defs": {
"parameters": {
"$id": "/schemas/parameters",
"$schema": "http://json-schema.org/draft-07/schema#",
"description": "Other Parameters",
"type": "object",
"properties": {
"activityType": {
"description": "The activity type",
"type": [ "null", "string" ]
},
"activitySubType": {
"description": "The activity sub type",
"type": [ "null", "string" ]
}
}
}
}
}
Now I have requirement to perform some validation logic.
If eventId == "100" and serviceType == "ABC" then parameters.activityType should be not null and must have a minimum length of 10.
If eventId == "200" and serviceType == "EFG" then parameters.activitySubType should be not null and must have a minimum length of 20.
I am trying to perform the validation using "if then else" condition. I am not sure how to add that inside the Json Schema validator.
Can anyone help me with the syntax? Is it possible to do that?
This is definitely possible. For the first requirement:
{
...
"if": {
"required": [ "eventId", "serviceType" ],
"properties": {
"eventId": {
"const": "100"
},
"serviceType": {
"const": "ABC"
}
}
},
"then": {
"required": [ "parameters" ],
"properties": {
"parameters": {
"properties": {
"activityType": {
"type": "string",
"minLength": 10
}
}
}
},
...
}
The "required" keywords are there because if the property doesn't exist at all, the "if" subschema will validate as true, which you don't want -- you need to say "if this property exists, and its value is ..., then ...".
The second requirement is very similar to the first. You can use multiple "if"/"else" keywords in the schema at the same time by wrapping them in an allOf: "allOf": [ { ..schema 1..}, { ..schema 2.. } ]
I'm in a dire need of help for a script to basically convert JSON text to CSV text in an attempt to copy users from one AWS Cognito userpool to another.
The export JSON looks like this:
{
"Users": [
{
"Username": "user.name",
"Attributes": [
{
"Name": "sub",
"Value": "some-value"
},
{
"Name": "email_verified",
"Value": "true"
},
{
"Name": "custom:jobtitle",
"Value": Director"
},
{
"Name": "custom:user_id",
"Value": "38"
},
{
"Name": "email",
"Value": "foo.bar#email.com"
}
],
"UserCreateDate": some-value,
"UserLastModifiedDate": some-value,
"Enabled": some-value,
"UserStatus": "some-value"
}
[more lines down here]...
] }
Then the CSV file would contain these lines:
,,,,,,,,,foo.bar#email.com,TRUE,,,,,,FALSE,,,Director,,38,FALSE,foo.bar
[more lines down here]...
So, the variables would be like this for JSON:
{
"Users": [
{
"Username": "%USERNAME%",
"Attributes": [
{
"Name": "sub",
"Value": "some-value"
},
{
"Name": "email_verified",
"Value": "true"
},
{
"Name": "custom:jobtitle",
"Value": %JOB_TITLE%"
},
{
"Name": "custom:user_id",
"Value": "%USER_ID%"
},
{
"Name": "email",
"Value": %EMAIL%"
}
],
"UserCreateDate": some-value,
"UserLastModifiedDate": some-value,
"Enabled": some-value,
"UserStatus": "some-value"
}
...
]
}
And like this for CSV:
,,,,,,,,,%EMAIL%,TRUE,,,,,,FALSE,,,%JOB_TITLE%,,%USER_ID%,FALSE,%USERNAME%
where %EMAIL%, %JOB_TITLE%, %USER_ID%, and %USERNAME% are variables, everything else should be just string.
Appreciate your help in advanced guys.
Consider first this filter:
.Users[].Attributes
| map(select(.Name | . == "custom:jobtitle" or . == "custom:user_id" or . == "email") )
| from_entries
| [ .email, .["custom:jobtitle"], .["custom:user_id"] ]
| #csv
The trick used here is the use of from_entries to convert the array of Name/Value pairs to an object with the Names as keys.
Assuming valid JSON input along the lines shown in the Q, invoking jq with the -r option would yield:
"foo.bar#email.com","Director","38"
Unfortunately the precise requirements are not so clear to me, but you should be able to adapt the above in accordance with your needs.
I have a schema to validate a json.
For certain properties i need them to have values of certain types.
If "attr" property is "a" then "val" property should be "integer"
If "attr" property is "x" then "val" property should be "boolean"
If "attr" property is "b" then "val" property should be "string" with
format "ipv4"
and so on...
This, I can define with oneOff. For all other "attr" properties i need to them to be of a certain format, sort of like a catch all, with "val" property to be "string".
If "attr" matches pattern then "val" property should be "string".
can this be done.
This is the schema that i have at the moment.
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"name": {
"title": "name",
"type": "string"
},
"attribute": {
"title": "attributes",
"type": "object",
"$ref": "#/definitions/expr",
}
},
"definitions": {
"expr": {
"properties": {
"attr": {
"title": "attribute"
},
"val": {
"title": "val"
}
},
"required": ["val", "attr"],
"oneOf": [
{
"properties": {
"attr": {"enum": ["a","b"]},
"val": {"type": "integer"}
}
},
{
"properties": {
"attr": {"enum": ["x"]},
"val": {"type": "boolean"}
}
},
{
"properties": {
"attr": {"pattern": "^[-A-Za-z0-9_]*$", "maxLength": 255},
"val": {"type": "string"}
}
}
]
}
},
"additionalProperties": false,
"required": [
"name",
"attribute"
]
}
The problem is the properties for which i am trying to restrict the value type, also match the catchall format. so when i am expecting an integer value, it is passing with string value.
For Example:
the below json will pass the schema, based on the first item of oneOff
{
"name": "shouldpass",
"attribute": {
"attr": "a",
"val": 1
}
}
the below json will pass, based on the last item of oneOff.
{
"name": "shouldpass2",
"attribute": {
"attr": "h",
"val": "asd"
}
}
the below json should fail, based on the first item of oneOff, but it is also passing, because it is matching the last item of oneOff.
{
"name": "shouldfail",
"attribute": {
"attr": "a",
"val": "string"
}
}
how to acheive this?
You schema for attr in the last subschema could be:
{
"pattern": "^[-A-Za-z0-9_]*$",
"not": { "enum": ["a", "b", "x"] },
"maxLength": 255
}
Alternatively, instead of "oneOf" you can use "switch" keyword from the next JSON-schema version proposals: http://epoberezkin.github.io/ajv/keywords.html#switch-v5-proposal
It's implemented in Ajv (I am the author).