I am using ansible and community.mysql.mysql_query to perform some sanity on my database.
I already figured out that I need to register the output and the output holds a parameter named query_result that contains the returned data.
My problem is that all examples are for a standard select in which you use param.query_result['column'] but mine has a COUNT(*).
My output for this debug :
- name: debug in db role
debug:
msg: |
result : {{ first_query.query_result }}
is :
ok: [localhost] => {
"msg": "result : [[{u'COUNT(*)': 16}]]\n"
}
Since count has * in it I cannot access it in the playbook.
Any thoughts on to how I can accomplish it and actually use this '16' count number?
Thanks
That was fast on my part ...
- name: debug in db role
debug:
msg: |
result : {{ first_query.query_result[0][0]['COUNT(*)'] }}
Related
I have some ansible code involving a json_query in a loop that I'm trying to make conditional. But I think there is something I'm not understanding about when clauses and loops.
The following code works fine, when the condition is met (IaC.status == 400), the problem is that when the condition is not met, the 2nd task still runs, the loop tries to process IaC2, which doesn't exist, and the task fails with: "Invalid data passed to 'loop', it requires a list, got this instead:
I think this is actually expected behaviour for conditional statements and loops: (https://docs.ansible.com/ansible/latest/user_guide/playbooks_conditionals.html#using-conditionals-in-loops), but the solution described to skip the entire task is to use an empty iterator, but I have no idea how to merge that in with the json_query() statement.
Does anyone know how I can truly make the 2nd task below conditional?
# The preceding code has made a RESTful API call to create a gitlab group, if the group
# exists the return status is 400, so I need to look it up instead. In which case the
# following code works fine. The problem is when IaC.status = 201 for some reason the loop
# in the 2nd task below tries to run, despite the when clause, and fails
# because the variable IaC2 doesn't exist
- name: or if IaC already exists
when: IaC.status == 400
uri:
url: https://{{new_hostname}}/api/v4/groups
method: GET
headers:
Authorization: "Bearer {{token.json.access_token}}"
body_format: json
body:
name: "IaC"
top_level_only: true
status_code: 200
validate_certs: yes
register: IaC2
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC.status == 400
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
You should condition your second task on the first task:
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC2 is defined
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
I cannot reproduce your case on ansible 2.9.10 where the task is skipped as expected. But you can also help with a default. I guess this would work:
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
when: IaC2 is defined
debug:
var: item
loop: "{{ IaC2 | community.general.json_query(jmesquery) | default([]) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
I have an answer of sorts. Firstly there were two things I didn't fully understand;
the register var (IaC2) is always created even if the uri task is skipped. And IaC2 has a totally different structure depending upon whether this task is executed or skipped.
The loop in the following task is always executed, so the when clause on the second task is pretty meaningless.
The actual problem was the jmesquery looking for the IaC2 variable to have a particular json structure, which it doesn't have in the case where the uri task is skipped.
So the solution that seems to work is checking to see if the uri task was skipped (when: IaC2.skipped is defined), and if it was skipped re-defining IaC2 in a empty list form that doesn't break the loop.
Code examples shown below.
So, it works, but I can't help thinking there must be an easier way of doing this?
- name: If IaC already exists (IaC.status == 400) then get the list of existing groups.
when: IaC.status == 400
uri:
url: https://{{new_hostname}}/api/v4/groups
method: GET
headers:
Authorization: "Bearer {{token.json.access_token}}"
body_format: json
body:
name: "IaC"
top_level_only: true
status_code: 200
validate_certs: no
register: IaC2
# To avoid breaking the following loop that searches the groups in the case that
# the group was created, re-define IaC2 into a form that won't break it.
- name: Not created group "IaC", so redefine IaC2 var so as not to break the following loop
when: IaC2.skipped is defined
set_fact:
IaC2: '{{ {"json": []} }}'
- name: json_query to find the IaC group id when status = "{{IaC.status}}"
#when: IaC.status == 400 << Leave in or comment out, makes no difference
debug:
var: item
loop: "{{ (IaC2 | community.general.json_query(jmesquery)) }}"
vars:
jmesquery: "json[?name=='IaC'].id"
register: group_id
- name: record the IaC group's id (400)
when: IaC.status == 400
set_fact:
IaC_group_id: "{{group_id.results[0].item}}"
I try to set up chroot for sftp users, so that they can see user/group names on ls -l as per this article. To this end I need to get output of getent command and place it into /chroots/{{ user.username }}/etc/passwd file.
I try to use Ansible to replace this command getent passwd sftpuser > /chroots/sftpuser/etc/passwd as follows:
- name: get {{ user.username }} user info
getent:
database: passwd
key: "{{ user.username }}"
- debug:
var: getent_passwd
- name: create /chroots/{{ user.username }}/etc/passwd file
lineinfile:
path: /chroots/{{ user.username }}/etc/passwd
line: "{{ getent_passwd | from_json }}"
state: present
create: yes
owner: root
group: root
mode: '0644'
The 'getent_passwd' looks as follows:
ok: [cf1] => {
"getent_passwd": {
"testuser1": [
"x",
"1001",
"1002",
"",
"/home/testuser1",
"/usr/sbin/nologin"
]
}
}
But I get this error: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ getent_passwd | from_json }}): expected string or buffer"}
What is the proper way to get those values supplied by getent_passwd into one flat string joined by ":"?
Is it safe to use genent module with key: "root" this way instead of echo "root:x:0:0:not really root:::" >> /chroots/sftpuser/etc/passwd?
one can run getent passwd user1 user2 - is it possible to supply two keys to the ansible's getent module somehow?
What is the proper way to get those values supplied by getent_passwd into one flat string joined by ":"?
For example using a Jinja2 template with join filter:
- debug:
msg: "{{ user.username }}:{{getent_passwd[user.username]|join(':')}}"
One can run getent passwd user1 user2 - is it possible to supply two keys to the ansible's getent module somehow?
No. Either a single one or all.
Use an outer loop to request values in the first case, or filter the resulting list in the second.
I have a JSON reply from a GitHub repository with a list of possible downloads for a certain release (the assets array in the document).
I want to get the browser download URL when the name of an asset ends with x64.AppImage.
In Ansible, the filters are built apon jmespath and using its terminal tool, I can query the url with the following expression:
assets[?ends_with(name, 'x64.AppImage')].browser_download_url
With the following playbook, the JSON document is queried and stored in the json_reply variable.
---
- hosts: local
tasks:
- name: Get list of Rambox releases
uri:
url: "https://api.github.com/repos/saenzramiro/rambox/releases/latest"
body_format: json
register: json_reply
- name: Filter reply
debug: URL -> "{{ item }}"
with_items:
- "{{ json_reply.json | json_query(json_filter) }}"
vars:
- json_filter: assets[?ends_with(name, 'x64.AppImage')].browser_download_url
However, executing this gives the following error:
fatal: [localhost]: FAILED! => {
"msg": "JMESPathError in json_query filter plugin:\nIn function ends_with(), invalid type for value: latest-mac.json, expected one of: ['string'], received: \"unknown\""
}
Where latest-mac.json is the first object in the assets array.
How can I make Ansible to iterate over all the assets array and apply my filter?
PS:
If instead of querying if the name ends with a word I specify it directly, the filter works:
assets[?name == 'Rambox-0.5.13-x64.AppImage')].browser_download_url
JSON example:
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/8001922",
"prerelease": false,
"created_at": "2017-10-04T21:14:15Z",
"published_at": "2017-10-05T01:10:55Z",
"assets": [
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/assets/4985942",
"id": 4985942,
"name": "latest-mac.json",
"uploader": {
"login": "saenzramiro",
"id": 2694669
},
"browser_download_url": "https://github.com/saenzramiro/rambox/releases/download/0.5.13/latest-mac.json"
},
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/assets/4985640",
"id": 4985640,
"name": "Rambox-0.5.13-x64.AppImage",
"uploader": {
"login": "saenzramiro",
"id": 2694669
},
"browser_download_url": "https://github.com/saenzramiro/rambox/releases/download/0.5.13/Rambox-0.5.13-x64.AppImage"
}
],
"tarball_url": "https://api.github.com/repos/saenzramiro/rambox/tarball/0.5.13"
}
The problem of type errors in JMESPath filters is discussed in issue 27299.
You can use this patched json_query.py filter plugin.
Or apply double conversion to your object as a workaround: | to_json | from_json |.
This will convert object to JSON (thus plain strings) and back, so json_query will treat strings as supported type.
Loop through each asset
Print the browser URL of the item if it ends with x64.AppImage
Solution not using JMESPath:
- name: Filter reply
debug: var=item.browser_download_url
with_items: "{{ json_reply.json.assets }}"
when: item.browser_download_url | regex_search('x64.AppImage$')
As #helloV said, you can accomplish this using Ansible loops, although there's no reason to involve a regular expression match. You can use the same test you're already using:
- name: Filter reply
debug:
var: item.browser_download_url
with_items: "{{ json_reply.json.assets }}"
when: item.name.endswith('x64.AppImage')
The root problem would appear to be an Ansible bug. The error comes from the following check in the jmespath library:
if actual_typename not in allowed_types:
raise exceptions.JMESPathTypeError(
function_name, current,
self._convert_to_jmespath_type(actual_typename), types)
At the point this code is called, the data type of values in your json response is AnsibleUnsafeText, where as allowed_types is [str, unicode]. I think the transformation of values from native types to the AnsibleUnsafeText type probably is some sort of standard Ansible module behavior being imposed by the uri module. We can work around it by using curl instead, like this:
- name: Get list of Rambox releases
command: >
curl -s "https://api.github.com/repos/saenzramiro/rambox/releases/latest"
register: json_reply
And then:
- name: Filter reply
debug:
var: item.browser_download_url
with_items: >
{{ json_reply.stdout|from_json|json_query('assets[?ends_with(name, `x64.AppImage`)]') }}
I am trying to parse a json response from an API. The response in a browser looks like:
[{url: "abc.com/xyz"}]
I request it from ansible:
- name: Get url
uri:
url: my-url...
method: GET
force: yes
return_content: yes
#HEADER_Content-Type: "application/json"
register: json_response
I get a reply from ansible that looks like this (with debug):
- name: print reply
debug:
var: json_response
verbosity: 1
which gives:
ok: [server] => {
"json_response": {
... //removed for readability
"content": "({:url \"https://the-file-I-want\"})"
}
So it seems like some parsing happened already (note the colons :).
Accessing the content seem to work (with debug json_response['content']):
ok: [server] => {
"json_response['content']": "({:url \"https://the-file-I-want\"})"
}
But I cannot seems to access the json response url. If I try to take the first element of the array, I get "(" so it seems it is still a string.
- name: print reply2
debug:
var: json_response['content'][0]
verbosity: 1
from_json does not seem to work: fatal: [server]: FAILED! => {"failed": true, "msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined....
How do I parse a json reply like this one?
I created a json file response.json with the following contents:
{
content: ({:url \"https://the-file-I-want\"})
}
Then, in my playbook I loaded the file and to get the url you need, I created a custom jinja filter since Jinja2 does not have any filter for finding sub-string or regexp.
My custom filter named filter.py(you can name it anything) is in a dir called filter_plugins in the same directory as my playbook. My filter.py file is as follows:
import re
class FilterModule(object):
''' Custom filters are loaded by FilterModule objects '''
def filters(self):
return {'urlsubstr': self.urlsubstr}
def urlsubstr(self,content):
url = re.findall('http[s]?://(?:[a-zA-Z]|[0-9]|[$-_#.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', content)
return url[0]
After creating the custom filter, I got the url like this:
- hosts: localhost
vars:
json_response: "{{ lookup('file', 'response.json') | from_json }}"
tasks:
- debug: msg="{{ json_response.content | urlsubstr }}"
with_dict: "{{ json_response }}"
This is the output of running my playbook:
TASK [setup] *******************************************************************
ok: [localhost]
TASK [debug] *******************************************************************
ok: [localhost] => (item={'value': u'({:url "https://the-file-I-want"})', 'key': u'content'}) => {
"item": {
"key": "content",
"value": "({:url \"https://the-file-I-want\"})"
},
"msg": "https://the-file-I-want"
}
Hope this helps.
To make the response json use the to_json filter, then navigate to the url key. That should give the value you're looking for: https://.....
Here's the documentation:
http://docs.ansible.com/ansible/playbooks_filters.html#filters-for-formatting-data
The original response works on any case but it seems overkill unless there's a problem with the conversion to JSON, like in yours.
Nevertheless I think it might help me do something else I intend to do, which is why I was looking in here.
I'm trying to filter a list in ansible in Jinja2 when the elements contain a string, but the Jinja documentation doesn't seem clear enough for me to figure it out.
This is what I have so far:
- name: run script
command: /usr/tmp/run_script.py
register: script_results
- name: display run info
debug:
var: "{{script_results.stdout_lines | select(\"'running script' in script_results.stdout_lines\") }}"
But all I get is the error:
"<generator object _select_or_reject at 0x13851e0>": "VARIABLE IS NOT DEFINED!"
So for example, if stdout_lines contains ["apples","running script one","oranges","running script two"], I want to print
running script one
running script two
They have documentation for select and documentation for built-in-tests, but they don't display the "in" test, and I don't know how they work in the context of this ansible variable.
I tried solving it like this:
- name: display run info
debug:
var: item
with_items: "{{script_results.stdout_lines}}"
when: "'running script' in item"
But that displays "skipping" for every line that doesn't pass the test ... kinda defeating the purpose!
The select filter would take another filter. Like in the docs odd, which will return only the odd elements of the list. The filter you would like to combine select with is equalto.
Now here's the thing. Ansible bundles a very old version of Jinja2, which simply does not contain the equalto filter. Yes, that renders it useless unless you want to filter odd elements. (Which nobody ever in history wanted to...)
Furthermore I was yet unable to make custom filter plugins work in Ansible 2. So you're pretty much forced to hack something ugly together.
helloV already showed one option. Here is another idea:
- name: run script
shell: /usr/tmp/run_script.py | grep "running script"
register: script_results
Update:
I recently discovered you can use match (not a standard Jinja2 filter but added by Ansible) together with select. Thats a good replacement for the eualto filter plus you can use regular expressions. This should work:
{{ script_results.stdout_lines | select("match", ".*running script.*") }}
I understand there may be more than one way to do this. Will this work for you?
- debug: var={{item}}
when: item.find('running script') > -1
with_items: script_results.stdout_lines
I know this is an old thread but I too was looking for an answer to this and I was able to use the if in method. I think the other issue you were having was how to display the resulting list. You can do a jinja loop right in the ansible modules, debug for example. Of course since Ansible is horrible with presenting data using debug, you could also utilize the blockinfile module with the pipe (|). Hopefully this helps others and gives yet another option.
- debug:
msg: |
{% for item in (script_results.stdout_lines) %}
{% if 'running script' in item %}
{{ item }}
{% endif %}
{% endfor %}
tags: debug
or to add the filtered data to a file:
- name: Update the scriptStatus file.
delegate_to: localhost
run_once: TRUE
blockinfile:
path: '/tmp/scriptStatus.txt'
block: |
{% for item in (script_results.stdout_lines) %}
{% if 'running script' in item %}
{{ item }}
{% endif %}
{% endfor %}
tags: chkScripts
I ended up writing a python script to do it, because I couldn't get ansible or ancient-jinja2 to make the cut.
Ansible tasks:
- name: gather run info
command: "{{role_path}}/files/print_results.py {{script_results.stdout_lines}}"
register: script_print_results
delegate_to: 127.0.0.1
run_once: true
- name: display run info
debug:
var: script_print_results.stdout_lines
delegate_to: 127.0.0.1
run_once: true
Python script:
for result_line in sys.argv[1:]:
if "running script:" in result_line:
print result_line[1:-1]
You can build a new list with set_fact and print the elements of a new list.
- hosts: localhost
gather_facts: false
vars:
script_stdout_lines:
- apples
- running script one
- oranges
- running script two
tasks:
- set_fact:
new_list: "{{ new_list | default([]) + [item] }}"
with_items: "{{ script_stdout_lines }}"
when: '"running script" in item'
- debug: var=new_list
Result:
TASK [set_fact] *********************************************************************************************************************
skipping: [localhost] => (item=apples)
ok: [localhost] => (item=running script one)
skipping: [localhost] => (item=oranges)
ok: [localhost] => (item=running script two)
TASK [debug] ************************************************************************************************************************
ok: [localhost] => {
"new_list": [
"running script one",
"running script two"
]
}
It prints skipping during set_fact operation but at the end it provides a new list with the only matching items.