I'm trying to concatenate two properties for a list of objects.
Input
{
"image": "golang:1.3",
"unapproved": [
"CVE-2016-5421",
"CVE-2019-5010"
],
"vulnerabilities": [
{
"featurename": "curl",
"featureversion": "7.38.0-4+deb8u2",
"vulnerability": "CVE-2016-5421",
"namespace": "debian:8",
"description": "Use-after-free vulnerability in libcurl before 7.50.1 allows attackers to control which connection is used or possibly have unspecified other impact via unknown vectors.",
"link": "https://security-tracker.debian.org/tracker/CVE-2016-5421",
"severity": "High",
"fixedby": "7.38.0-4+deb8u4"
},
{
"featurename": "python2.7",
"featureversion": "2.7.9-2",
"vulnerability": "CVE-2019-5010",
"namespace": "debian:8",
"description": "Test description",
"link": "https://security-tracker.debian.org/tracker/CVE-2019-5010",
"severity": "Unknown",
"fixedby": ""
}
]
}
Desired output
The top-level image property should be used as a prefix for the
description of every object in the vulnerabilities list.
{
"image": "golang:1.3",
"unapproved": [
"CVE-2016-5421",
"CVE-2019-5010"
],
"vulnerabilities": [
{
"featurename": "curl",
"featureversion": "7.38.0-4+deb8u2",
"vulnerability": "CVE-2016-5421",
"namespace": "debian:8",
"description": "golang:1.3 - Use-after-free vulnerability in libcurl before 7.50.1 allows attackers to control which connection is used or possibly have unspecified other impact via unknown vectors.",
"link": "https://security-tracker.debian.org/tracker/CVE-2016-5421",
"severity": "High",
"fixedby": "7.38.0-4+deb8u4"
},
{
"featurename": "python2.7",
"featureversion": "2.7.9-2",
"vulnerability": "CVE-2019-5010",
"namespace": "debian:8",
"description": "golang:1.3 - Test description",
"link": "https://security-tracker.debian.org/tracker/CVE-2019-5010",
"severity": "Unknown",
"fixedby": ""
}
]
}
Current attempt
My current filter:
{image, unapproved, vulnerabilities: [{description: (.image + " - " + .vulnerabilities[].description)}] }
Output
{
"image": "golang:1.3",
"unapproved": [
"CVE-2016-5421",
"CVE-2019-5010"
],
"vulnerabilities": [
{
"description": "golang:1.3 - Use-after-free vulnerability in libcurl before 7.50.1 allows attackers to control which connection is used or possibly have unspecified other impact via unknown vectors."
},
{
"description": "golang:1.3 - Test description"
}
]
}
Unfortunately, I only get the description field back with my current filter. I want the full vulnerability object with the modified description field.
Question
How can I concatenate the nested field and keep the other properties of the object too?
jqPlay
The simplest solution would probably be:
.image as $prefix
| .vulnerabilities[].description |= $prefix + " - " + .
In words: update all the .description values using .image as shown.
Equivalently, and perhaps less esoterically:
.image as $prefix
| .vulnerabilities |= map(.description |= $prefix + " - " + .)
Related
I have a teams message card that I post using a web activity.
Currently, I do a lookup to determine whether or not I post the results, those results are stored in a variable, like so;
I want to include the results held in "Set variable" within my teams message, this will output the following as an array;
Using the web body, in expression builder, how can I include the values stored in the variable within my message, here is what I have generated so far;
{
"#type": "MessageCard",
"#context": "https://schema.org/extensions",
"summary": "This is the summary property",
"themeColor": "800000",
"sections": [
{
"heroImage": {
"image": "https://messagecardplayground.azurewebsites.net/assets/FlowLogo.png"
}
},
{
"startGroup": true,
"title": "**Checks - Requires Investigation**",
"facts": [
{
"name": "Date submitted:",
"value": "06/27/2017, 2:44 PM"
},
{
"name": "Details:",
"value": "The validation checks are hilighted below; "
},
{
"name": "Results:",
"value": "Include the output here"
},
{
"name": "Support Document:",
"value": "[Click here](https://dev.azure.com/supportDocument)"
}
]
}
]
}
You can use String interpolation to include the value of your variable (array). Please look at the following demonstration:
I have used a fake API in my web activity where I use PUT method. The following is the request body that is needed to be passed to the API.
{
"name": "morpheus",
"job": "leader"
}
Instead of using static data, if I want to use dynamic content (lets say, for 'name') i.e., use the result of my variable in the web body, we can do it in the following way. In the following, the variable req_name has value morpheus.
{
"name": "#{variables('req_name')}",
"job": "leader"
}
Now since you have the required result, which should be used in web body, stored in your variable, you can change your body as follows in the expression builder:
{
"#type": "MessageCard",
"#context": "https://schema.org/extensions",
"summary": "This is the summary property",
"themeColor": "800000",
"sections": [
{
"heroImage": {
"image": "https://messagecardplayground.azurewebsites.net/assets/FlowLogo.png"
}
},
{
"startGroup": true,
"title": "**Checks - Requires Investigation**",
"facts": [
{
"name": "Date submitted:",
"value": "06/27/2017, 2:44 PM"
},
{
"name": "Details:",
"value": "The validation checks are hilighted below; "
},
{
"name": "Results:",
"value": "#{variables('<your_variable_name>')}"
},
{
"name": "Support Document:",
"value": "[Click here](https://dev.azure.com/supportDocument)"
}
]
}
]
}
NOTE:
You can always check the debug input to check whether the body you are passing is the correct format.
If it is not, then you can manipulate the variable as required to convert it to required format and then include its value as demonstrated above using String Interpolation (#{......}).
I'm trying to extract values from nested arrays in JSON below and output as CSV.
Fields to extract:
templates.name
items.name
triggers.name
Output as:
templates.name; items.name; triggers.name
Anticipated output something like:
"Template App Agent"; "Host name of zabbix_agentd running"; "Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent"; "Agent ping"; "Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent"; "Version of zabbix_agent(d) running"; ""
Note:
Not every item has a trigger.
Several triggers may exist for an item.
I'm new to JQ. So far only success is extracting the template name.
jq '.[] | {templates: [.templates[].name]}'
Data:
{
"zabbix_export": {
"version": "5.4",
"date": "2022-05-17T06:25:59Z",
"groups": [
{
"uuid": "7df96b18c230490a9a0a9e2307226338",
"name": "Templates"
}
],
"templates": [
{
"uuid": "e60e6598cf19448089a5f5a6c5d796a2",
"template": "Template App Agent",
"name": "Template App Agent",
"groups": [
{
"name": "Templates"
}
],
"items": [
{
"uuid": "24c03ed734d54dc8868a282a83a02200",
"name": "Host name of zabbix_agentd running",
"key": "agent.hostname",
"delay": "1h",
"history": "1w",
"trends": "0",
"value_type": "CHAR",
"request_method": "POST",
"tags": [
{
"tag": "Application",
"value": "Zabbix agent"
}
],
"triggers": [
{
"uuid": "d2d12d9e7dfe4fedb252f19b85e5e6aa",
"expression": "(last(/Template App Agent/agent.hostname,#1)<>last(/Template App Agent/agent.hostname,#2))>0",
"name": "Host name of zabbix_agentd was changed on {HOST.NAME}",
"priority": "INFO"
}
]
},
{
"uuid": "abacad4ca5eb46d29864d8a4998f1cbb",
"name": "Agent ping",
"key": "agent.ping",
"history": "1w",
"description": "The agent always returns 1 for this item. It could be used in combination with nodata() for availability check.",
"valuemap": {
"name": "Zabbix agent ping status"
},
"request_method": "POST",
"tags": [
{
"tag": "Application",
"value": "Zabbix agent"
}
],
"triggers": [
{
"uuid": "6d2a73199f3b4288bf36331a142c1725",
"expression": "nodata(/Template App Agent/agent.ping,5m)=1",
"name": "Zabbix agent on {HOST.NAME} is unreachable for 5 minutes",
"priority": "AVERAGE"
}
]
},
{
"uuid": "2cc337555efd43d181c28c792f8cbbdb",
"name": "Version of zabbix_agent(d) running",
"key": "agent.version",
"delay": "1h",
"history": "1w",
"trends": "0",
"value_type": "CHAR",
"request_method": "POST",
"tags": [
{
"tag": "Application",
"value": "Zabbix agent"
}
]
}
],
"valuemaps": [
{
"uuid": "3d66c59a28c04b0ca8227c87902ddb4d",
"name": "Zabbix agent ping status",
"mappings": [
{
"value": "1",
"newvalue": "Up"
}
]
}
]
}
]
}
}
.zabbix_export.templates[] | .name as $tn | .items[] | [ $tn, .name, .triggers[]?.name? ] | join("; ")
Loop over the templates
.zabbix_export.templates[]
Save the template name in a var
.name as $tn
Loop over the items
.items[]
Create an array with fields you like (including the name from step 1
[ $tn, .name, .triggers[]?.name? ]
Join the array to a string
join("; ")
Will output:
"Template App Agent; Host name of zabbix_agentd running; Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent; Agent ping; Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent; Version of zabbix_agent(d) running"
Online demo
This is a nested structure, you need to iterate level by level and add up the items you want to be in one output line. Store values from previous levels in variables.
To account for an inexistent .triggers array, you may use the Error Suppression Operator ? in combination with Alternative Operator //.
Finally, wrap the items in quotes (here using map), join them using join, and output them as raw text using the -r option
jq -r '
.[].templates[] | .name as $t
| .items[] | .name as $i
| [$t, $i, (.triggers[].name)? // ""]
| map("\"\(.)\"") | join("; ")
'
"Template App Agent"; "Host name of zabbix_agentd running"; "Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent"; "Agent ping"; "Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent"; "Version of zabbix_agent(d) running"; ""
Demo
Also consider using the #csv builtin, which gives you valid CSV right away (properly encoded (not just quoted) items, but separated with commas, not semicolons):
jq -r '
.[].templates[] | .name as $t
| .items[] | .name as $i
| [$t, $i, (.triggers[].name)? // ""]
| #csv
'
"Template App Agent","Host name of zabbix_agentd running","Host name of zabbix_agentd was changed on {HOST.NAME}"
"Template App Agent","Agent ping","Zabbix agent on {HOST.NAME} is unreachable for 5 minutes"
"Template App Agent","Version of zabbix_agent(d) running",""
Demo
Trying to import a DLP rule via a JSON file - current contents below:
{
"DlpRules":
[
{
"Name": "Example",
"Comment": "THis is an Example",
"Policy": "[TEST] EXAMPLE POLICY",
"Disabled": "false",
"Priority": "0",
"BlockAccess": "true",
"BlockAccessScope": "All",
"AlertProperties": {
"AggregationType": "None"
},
"GenerateAlert": "true",
"NotifyUser": [
"example#example.com"
],
"NotifyEmailCustomText": "Test",
"ReportSeverityLevel": "Medium",
"ContentContainsSensitiveInformation": [{
"Groups": [{
"operator": "Or",
"labels": [
{
"name": "EXAMPLE - LABEL",
"id": "[PRETEND GUID IS HERE]",
"type": "Sensitivity"
}
],
"name": "Default"
}]
}]
}
]
}
When running I get the below the error:
The value specified in sensitive information is invalid.
+ CategoryInfo : NotSpecified: (:) [Set-DlpComplianceRule], CompliancePolicyValidationException
+ FullyQualifiedErrorId : Microsoft.Office.CompliancePolicy.PolicyEvaluation.CompliancePolicyValidationException,Microsoft.Office.Compli
ancePolicy.Tasks.SetDlpComplianceRule
+ PSComputerName : aus01b.ps.compliance.protection.outlook.com
Replacing the Labels block with 'sensitivetypes' targeting a Sensitive information type is successful.
Current file is based on manually creating it in the compliance portal exporting it and expanding the 'System.Collections.Hashtable' values with the below - is possible I'm doing something daft when it comes to combining these imputs:
(get-dlpcompliancerule "Example").ContentContainsSensitiveInformation | ConvertTo-Json
(get-dlpcompliancerule "Example").ContentContainsSensitiveInformation.groups | ConvertTo-Json
(get-dlpcompliancerule "Example").ContentContainsSensitiveInformation.groups.labels | ConvertTo-Json
is it possible to check, in an ARM Template, if the name for my Virtual Machine already exists?
I am developing a Solution Template for the Azure Marketplace. Maybe it is possible to set a paramter in the UiDefinition uniqe?
The goal is to reproduce this green Hook
A couple notes...
VM Names only need to be unique within a resourceGroup, not within the subscription
Solution Templates must be deployed to empty resourceGroups, so collisions with existing resources aren't possible
For solution templates the preference is that you simply name the VMs for the user, rather than asking - use something that is appropriate for the workload (e.g. jumpbox) - not all solutions do this but we're trying to improve that experience
Given that it's not likely we'll ever build a control that checks for naming collisions on resources without globally unique constraints.
That help?
This looks impossible, according to the documentation.
There are no validation scenarious.
I assume that you should be using the Microsoft.Common.TextBox UI element in your createUiDefinition.json.
I have tried to reproduce a green check by creating a simple createUiDefinition.json as below with a Microsoft.Common.TextBox UI element as shown below.
{
"$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json",
"handler": "Microsoft.Compute.MultiVm",
"version": "0.1.2-preview",
"parameters": {
"basics": [
{
"name": "textBoxA",
"type": "Microsoft.Common.TextBox",
"label": "VM Name",
"defaultValue": "",
"toolTip": "Please enter a VM name",
"constraints": {
"required": true
},
"visible": true
}
],
"steps": [],
"outputs": {}
}
}
I am able to reproduce the green check beside the VM Name textbox as shown below:
However, this green check DOES NOT imply the VM Name is Available.
This is because based on my testing, even if I use an existing VM Name in the same subscription, it is still showing the green check.
Based on the official documented constraints that are supported by the Microsoft.Common.TextBox UI element, it DOES NOT VALIDATE Name Availability.
Hope this helps!
While bmoore's point is correct that it's unlikely you would ever need this for a VM (nor is there an API for it), there are other compute resources that do have global naming requirements.
As of 2022 this concept is possible now with the use of the ArmApiControl UI element. It allows you to call ARM apis as part of validation in the createUiDefinition.json. Here is an example using the check name API for an Azure App service.
{
"$schema": "https://schema.management.azure.com/schemas/0.1.2-preview/CreateUIDefinition.MultiVm.json#",
"handler": "Microsoft.Azure.CreateUIDef",
"version": "0.1.2-preview",
"parameters": {
"basics": [
{}
],
"steps": [
{
"name": "domain",
"label": "Domain Names",
"elements": [
{
"name": "domainInfo",
"type": "Microsoft.Common.InfoBox",
"visible": true,
"options": {
"icon": "Info",
"text": "Pick the domain name that you want to use for your app."
}
},
{
"name": "appServiceAvailabilityApi",
"type": "Microsoft.Solutions.ArmApiControl",
"request": {
"method": "POST",
"path": "[concat(subscription().id, '/providers/Microsoft.Web/checknameavailability?api-version=2021-02-01')]",
"body": "[parse(concat('{\"name\":\"', concat('', steps('domain').domainName), '\", \"type\": \"Microsoft.Web/sites\"}'))]"
}
},
{
"name": "domainName",
"type": "Microsoft.Common.TextBox",
"label": "Domain Name Word",
"toolTip": "The name of your app service",
"placeholder": "yourcompanyname",
"constraints": {
"validations": [
{
"regex": "^[a-zA-Z0-9]{4,30}$",
"message": "Alphanumeric, between 4 and 30 characters."
},
{
"isValid": "[not(equals(steps('domain').appServiceAvailabilityApi.nameAvailable, false))]",
"message": "[concat('Error with the url: ', steps('domain').domainName, '. Reason: ', steps('domain').appServiceAvailabilityApi.reason)]"
},
{
"isValid": "[greater(length(steps('domain').domainName), 4)]",
"message": "The unique domain suffix should be longer than 4 characters."
},
{
"isValid": "[less(length(steps('domain').domainName), 30)]",
"message": "The unique domain suffix should be shorter than 30 characters."
}
]
}
},
{
"name": "section1",
"type": "Microsoft.Common.Section",
"label": "URLs to be created:",
"elements": [
{
"name": "domainExamplePortal",
"type": "Microsoft.Common.TextBlock",
"visible": true,
"options": {
"text": "[concat('https://', steps('domain').domainName, '.azurewebsites.net - The main app service URL')]"
}
}
],
"visible": true
}
]
}
],
"outputs": {
"desiredDomainName": "[steps('domain').domainName]"
}
}
}
You can copy the above code and test it in the createUiDefinition.json sandbox azure provides.
Given an example document in JSON similar to this:
{
"id": "post-1",
"type": "blog-post",
"tags": [
{
"id": "tag-1",
"name": "Tag 1"
},
{
"id": "tag-2",
"name": "Tag 2"
}
],
"heading": "Post 1",
"body": "this is my first blog post",
"links": [
{
"id": "post-2",
"heading": "Post 2",
"tags": [
{
"id": "tag-1",
"name": "Tag 1"
},
{
"id": "tag-3",
"name": "Tag 3"
}
]
}
],
"metadata": {
"user": {
"social": [
{
"id": "twitter",
"handle": "#user"
},
{
"id": "facebook",
"handle": "123456"
},
{
"id": "youtube",
"handle": "ABC123xyz"
}
]
},
"categories": [
{
"name": "Category 1"
},
{
"name": "Category 2"
}
]
}
}
I would like to select any object (regardless of depth) that has an attribute "id", as well as the attribute name of the parent object. The above example should be taken as just that, an example. The actual data, that I'm not at liberty to share, can have any depth and just about any structure. Attributes can be introduced and removed at any time. Using the Blog Post style is just because it is quite popular for examples and I have very limited imagination.
The attribute signifies a particular type within the domain, that might also be (but is not necessarily) coded into the value of the attribute.
If an object does not have the "id" attribute it is not interesting and should not be selected.
A very important special case is when the value of an attribute is an array of objects, in that case I need to keep the attribute name and associate it with each element in the array.
An example of the desired output would be:
[
{
"type": "tags",
"node": {
"id": "tag-1",
"name": "Tag 1"
}
},
{
"type": "tags",
"node": {
"id": "tag-2",
"name": "Tag 2"
}
},
{
"type": "links",
"node": {
"id": "post-2",
"heading": "Post 2",
"tags": [
{
"id": "tag-1",
"name": "Tag 1"
},
{
"id": "tag-3",
"name": "Tag 3"
}
]
}
},
{
"type": "tags",
"node": {
"id": "tag-1",
"name": "Tag 1"
}
},
{
"type": "tags",
"node": {
"id": "tag-3",
"name": "Tag 3"
}
},
{
"type": "social",
"node": {
"id": "twitter",
"handle": "#user"
}
},
{
"type": "social",
"node": {
"id": "facebook",
"handle": "123456"
}
},
{
"type": "social",
"node": {
"id": "youtube",
"handle": "ABC123xyz"
}
}
]
It isn't strictly necessary that the output is identical, order for instance is irrelevant for my use-case it could be grouped as well. Since the top level object has an attribute "id" it could be included with a special name, but I'd prefer if it was not included at all.
I've tried to use walk, reduce and recurse to no avail, I'm afraid my jq skills are too limited. But I imagine that a good solution would make use of at least one of them.
I would like an expression something like
to_entries[] | .value | .. | select(has("id")?)
which would select the correct objects, but with .. I'm no longer able to keep the associated attribute name.
The best I've come up with is
. as $document
| [paths | if length > 1 and .[-1] == "id" then .[0:-1] else empty end]
| map(. as $path
| $document
| { "type": [$path[] | if type == "string" then . else empty end][-1],
"node": getpath($path) })
Which works, but feels quite complicated and involves first extracting all paths, ignoring any path that does not have "id" as the last element, then remove the "id" segment to get the path to the actual object and storing the (by now last) segment that is a string, which corresponds to the parent objects attribute containing the interesting object. Finally the actual object is selected through getpath.
Is there a more elegant, or at the least shorter way to express this?
I should note that I'd like to use jq for the convenience of having bindings to other languages as well as being able to run the program on the command line.
For the scope of this question, I'm not really interested in alternatives to jq as I can imagine how to solve this differently using other tooling, but I would really like to "just" use jq.
Since the actual requirements aren’t clear to me, I’ll assume that the given implementation defines the functional requirements, and propose a shorter and hopefully sleeker version:
. as $document
| paths
| select(length > 1 and .[-1] == "id")
| .[0:-1] as $path
| { "type": last($path[] | strings),
"node": $document | getpath($path) }
This produces a stream, so if you want an array, you could simply enclose the above in square brackets.
last(stream) emits null if the stream is empty, which accords with the behavior of .[-1].
This works:
[
foreach (paths | select(.[-1] == "id" and length > 1)[:-1]) as $path ({i:.};
.o = {
type: last($path[] | strings),
node: (.i | getpath($path))
};
.o
)
]
The trick is to know that any numbers in the path indicates the value is part of an array. You'll have to adjust the path to get the parent name. But using last/1 with a string filter makes it simpler.