Ansible: Issues when importing boolean from json file - json

I want to use a json file for user management in ansible. Therefore I created a json file containing all the user and groups like this (user_group_management.json):
{
"linux_users": [
{
"name": "myuser",
"uid": 1003,
"group": "myuser",
"groups": "users,sudo",
"shell": "/bin/bash",
"password": <password as sha-512>,
"create_home": "yes",
"home": "/home/user",
"hosts": ["hostname1","hostname2","hostname3"]
},
{...},
...
],
"linux_groups": [...],
}
Now I wrote an ansible script for adding the users and groups (user_group_management.yml):
- name: User and group management
hosts: all
vars_files:
- user_group_management.json
tasks:
[part for adding groups (no problems here)]
- name: Add users
ignore_errors: yes
user:
name: item.name
uid: item.uid
group: item.group
groups: item.groups
shell: item.shell
password: item.password
create_home: item.create_home
home: item.home
state: present
when: ansible_hostname in item.hosts
loop: "{{ linux_users }}"
When running the ansible script using ansible-playbook --check user_management.yml I get these error messages concerning the "Add users" task:
"msg": "The value 'item.create_home' is not a valid boolean. Valid booleans include: 0, 'on', 'f', 'false', 1, 'no', 'n', '1', '0', 't', 'y', 'off', 'yes', 'true'"
I tried to fix this issue by replacing "create_home": "yes", with "create_home": 1, but it did not change anything. Now I added curly brackets around the variable (item.create_home -> "{{ item.create_home }}") which solved the issue. So now I do not get any error messages again.
Sadly I do not understand why this helped me. I thought that item.create_home is a string ("yes") in the first place and should be an integer (1) after editing the json file. But both give me the errors. Is there an explanation for this phenomenon?

If don't have a lot of experience with json manipulation in ansible however have you tried to change
"create_home": "yes",
to
"create_home": true,
Another solution it to convert the value you expect to be boolean with a jinja filter |bool when using it.
The official documentation provides
- debug:
msg: test
when: some_string_value | bool
as an example with when condition.
For your case it should be
create_home: item.create_home
should become
create_home: "{{ item.create_home | bool }}"
general advice: You should enclose your variables inside "{{ }}"

Related

Ansible: Invalid JSON when using --extra-vars

Hi community,
I have been struggling with an issue in ansible issue for days now.
Everything is executed wihtin a Jenkins pipeline.
The ansible command looks like:
sh """
ansible-playbook ${env.WORKSPACE}/cost-optimization/ansible/manage_dynamo_db.yml \
--extra-vars '{"projectNameDeployConfig":${projectNameDeployConfig},"numberOfReplicas":${numberOfReplicas},"dynamodbtask":${dynamodbtask}}'
"""
And the playbooks is:
playbook.yml
---
- hosts: localhost
vars:
numberOfReplicas: "{{numberOfReplicas}}"
dynamodbtask: "{{dynamodbtask}}"
namespace: "{{projectNameDeployConfig}}"
status: "{{status}}"
- tasks:
- name: "Get replica number for the pods"
command: aws dynamodb put-item --table-name pods_replicas
register: getResult
when: dynamodbtask == "get"
- name: "Update replica number for specified pods"
command: |
aws dynamodb put-item
--table-name pods_replicas
--item '{"ProjectNameDeployConfig":{"S":{{namespace}}},"NumberReplicas":{"N":{{numberOfReplicas}}}}'
register: updatePayload
when: dynamodbtask == "put" and getResult is skipped
However, there is always the following error:
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["aws", "dynamodb", "put-item", "--table-name",
"pods_replicas", "--item", "{\"ProjectNameDeployConfig\":{\"S\":LERN-PolicyCenterV10},\"NumberReplicas\":
{\"N\":0}}"], "delta": "0:00:01.702107", "end": "2020-02-09 16:58:26.055579",
"msg": "non-zero return code", "rc": 255, "start": "2020-02-09 16:58:24.353472", "stderr": "\nError parsing parameter '--item': Invalid JSON: No JSON object could be decoded\nJSON received: {\"ProjectNameDeployConfig\":{\"S\":LERN-PolicyCenterV10},\"NumberReplicas\":{\"N\":0}}", "stderr_lines": ["", "Error parsing parameter '--item': Invalid JSON: No JSON object could be decoded", "JSON received: {\"ProjectNameDeployConfig\":{\"S\":LERN-PolicyCenterV10},\"NumberReplicas\":{\"N\":0}}"], "stdout": "", "stdout_lines": []}
There are two answers to your question: the simple one and the correct one
The simple one is that had you actually fed the JSON into jq, or python -m json.tool, you would have observed that namespace is unquoted:
"{\"ProjectNameDeployConfig\":{\"S\": LERN-PolicyCenterV10 },\"NumberReplicas\": {\"N\":0}}"
where I added a huge amount of space, but didn't otherwise alter the quotes
The correct answer is that you should never use jinja2 to try and assemble structured text when there are filters that do so for you.
What you actually want is to use the to_json filter:
- name: "Update replica number for specified pods"
command: |
aws dynamodb put-item
--table-name pods_replicas
--item {{ dynamodb_item | to_json | quote }}
vars:
dynamodb_item:
"ProjectNameDeployConfig":
"S": '{{ projectNameDeployConfig }}'
"NumberReplicas":
"N": 0
register: updatePayload
when: dynamodbtask == "put" and getResult is skipped
although you'll notice that I changed your variable name because namespace is the name of a type in jinja2, so you can either call it ns or I just used the interpolation value from your vars: block at the top of the playbook, as it doesn't appear that it changed from then

Substitution of variables from JSON response (from uri) in ansible playbook to create local users account

I have JSON as output for query into CMDB, which contains informations about local users like UID, GID etc. And I would like to substitute them in ansible playbook to create those account. I know how to use it as one variable. But I would like to use it for more variables as parameters - one value from key for UID, GID, ssh public key etc. Example of JSON is attached.
Any suggestion or help?
JSON answer
$ curl -s -k "https://cmdb.test.loc/api/getusers.php" | jq -r '.'
{
"users": [
{
"login": "johnd",
"description": "John Doe",
"uid": "9004",
"sshpubkey": "ssh-rsa AAsQxHUC4mshf+mXy3+Evtd1l9/x/DF5tR1AMI9i0CQoggxX6GbxRb+lYVySUmCgnMqT/dVcOlL3ETzSnubeoC3o4Yjv9Q1NLrw== johnd#lin.x"
},
{
"login": "thomasm",
"description": "Thomas Mann",
"uid": "9003",
"sshpubkey": "ssh-rsa AAAdUG4PPLzrJo/M5StBXsgUcUQAPuU4y1vXIJIw+o5mx/IkxpZVvbWTqx6HbLJD8CL5jp4wpXb1UGjyYorvumrGyArYWAXWOQZNdzsIeTp1Z9tZb0qoVkGX1aR1HGst36FbcYDTB3yEmliYHSNQ== thomasm#lin.x"
},
{
"login": "trib",
"description": "Three Be",
"uid": "9002",
"sshpubkey": "ssh-rsa AAAAB3NzpyNyRsdvthGYzkm0rvEug1RRwURWZ9d++TzJtoWZrCKw7Ec8PJGgNJvHRAvFdSJ8mKXAK3U0WIc46P2Ij7BJzCR4o900Sad/qeYSlY7xL+oW+nXWPUuxCVT25pHAeTJDjHeHJmpqDpHGg8sLmQMXtwCb5r6YxFUOUSlM1L2Q2iVedC/IXKMTFP+7Wp9YvPX75MVjMWc6L1xCZMsBk+dQ== 3be#lin.x"
}
]
}
Begin of playbook.yml
---
- hosts: all
tasks:
- name: Take information about actual users from CMDB
uri:
url: "https://cmdb.test.loc/api/getusers.php"
validate_certs: false
method: GET
status_code: 200,400
body_format: json
return_content: yes
register: cmdb_users
- name: Filter and rename variables from source list
debug:
msg: "{{ cmdb_users.json | json_query('users[*].{login: login, description: description, uid: uid, sshpubkey: sshpubkey}') }}"
- set_fact:
users: "{{ cmdb_users.json | json_query('users[*].{login: login, description: description, uid: uid, sshpubkey: sshpubkey}') }}"
...
I would like have any suggestion for next steps to use it for account creating:
- name : Create local users
user:
name : {{ ??? }}
uid : {{ ??? }}
comment : {{ ??? }}
I know that authorized-keys file have to be created by other way, but I think, that it can be created as shell "echo" to specific file to specific path composed from variables.
The loop below should do the job
- name: Create local users
user:
name: "{{ item.login }}"
uid: "{{ item.uid|int }}"
comment: "{{ item.description }}"
loop: "{{ users }}"
Next loop will store the public keys to the files in pub_key_dir
- name: Store public keys to files
copy:
dest: "{{ pub_key_dir }}/{{ item.login }}.sshpubkey"
content: |
{{ item.sshpubkey }}
loop: "{{ users }}"
Notes
1) The correct name of the module is user
2) In the playbook, the registered variable is cmdb_users. The correct list is probably
loop: "{{ cmdb_users.json.users }}"

Ansible: parsing/concatenating output of getent module

I try to set up chroot for sftp users, so that they can see user/group names on ls -l as per this article. To this end I need to get output of getent command and place it into /chroots/{{ user.username }}/etc/passwd file.
I try to use Ansible to replace this command getent passwd sftpuser > /chroots/sftpuser/etc/passwd as follows:
- name: get {{ user.username }} user info
getent:
database: passwd
key: "{{ user.username }}"
- debug:
var: getent_passwd
- name: create /chroots/{{ user.username }}/etc/passwd file
lineinfile:
path: /chroots/{{ user.username }}/etc/passwd
line: "{{ getent_passwd | from_json }}"
state: present
create: yes
owner: root
group: root
mode: '0644'
The 'getent_passwd' looks as follows:
ok: [cf1] => {
"getent_passwd": {
"testuser1": [
"x",
"1001",
"1002",
"",
"/home/testuser1",
"/usr/sbin/nologin"
]
}
}
But I get this error: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ getent_passwd | from_json }}): expected string or buffer"}
What is the proper way to get those values supplied by getent_passwd into one flat string joined by ":"?
Is it safe to use genent module with key: "root" this way instead of echo "root:x:0:0:not really root:::" >> /chroots/sftpuser/etc/passwd?
one can run getent passwd user1 user2 - is it possible to supply two keys to the ansible's getent module somehow?
What is the proper way to get those values supplied by getent_passwd into one flat string joined by ":"?
For example using a Jinja2 template with join filter:
- debug:
msg: "{{ user.username }}:{{getent_passwd[user.username]|join(':')}}"
One can run getent passwd user1 user2 - is it possible to supply two keys to the ansible's getent module somehow?
No. Either a single one or all.
Use an outer loop to request values in the first case, or filter the resulting list in the second.

Filter a JSON document in Ansible

I have a JSON reply from a GitHub repository with a list of possible downloads for a certain release (the assets array in the document).
I want to get the browser download URL when the name of an asset ends with x64.AppImage.
In Ansible, the filters are built apon jmespath and using its terminal tool, I can query the url with the following expression:
assets[?ends_with(name, 'x64.AppImage')].browser_download_url
With the following playbook, the JSON document is queried and stored in the json_reply variable.
---
- hosts: local
tasks:
- name: Get list of Rambox releases
uri:
url: "https://api.github.com/repos/saenzramiro/rambox/releases/latest"
body_format: json
register: json_reply
- name: Filter reply
debug: URL -> "{{ item }}"
with_items:
- "{{ json_reply.json | json_query(json_filter) }}"
vars:
- json_filter: assets[?ends_with(name, 'x64.AppImage')].browser_download_url
However, executing this gives the following error:
fatal: [localhost]: FAILED! => {
"msg": "JMESPathError in json_query filter plugin:\nIn function ends_with(), invalid type for value: latest-mac.json, expected one of: ['string'], received: \"unknown\""
}
Where latest-mac.json is the first object in the assets array.
How can I make Ansible to iterate over all the assets array and apply my filter?
PS:
If instead of querying if the name ends with a word I specify it directly, the filter works:
assets[?name == 'Rambox-0.5.13-x64.AppImage')].browser_download_url
JSON example:
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/8001922",
"prerelease": false,
"created_at": "2017-10-04T21:14:15Z",
"published_at": "2017-10-05T01:10:55Z",
"assets": [
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/assets/4985942",
"id": 4985942,
"name": "latest-mac.json",
"uploader": {
"login": "saenzramiro",
"id": 2694669
},
"browser_download_url": "https://github.com/saenzramiro/rambox/releases/download/0.5.13/latest-mac.json"
},
{
"url": "https://api.github.com/repos/saenzramiro/rambox/releases/assets/4985640",
"id": 4985640,
"name": "Rambox-0.5.13-x64.AppImage",
"uploader": {
"login": "saenzramiro",
"id": 2694669
},
"browser_download_url": "https://github.com/saenzramiro/rambox/releases/download/0.5.13/Rambox-0.5.13-x64.AppImage"
}
],
"tarball_url": "https://api.github.com/repos/saenzramiro/rambox/tarball/0.5.13"
}
The problem of type errors in JMESPath filters is discussed in issue 27299.
You can use this patched json_query.py filter plugin.
Or apply double conversion to your object as a workaround: | to_json | from_json |.
This will convert object to JSON (thus plain strings) and back, so json_query will treat strings as supported type.
Loop through each asset
Print the browser URL of the item if it ends with x64.AppImage
Solution not using JMESPath:
- name: Filter reply
debug: var=item.browser_download_url
with_items: "{{ json_reply.json.assets }}"
when: item.browser_download_url | regex_search('x64.AppImage$')
As #helloV said, you can accomplish this using Ansible loops, although there's no reason to involve a regular expression match. You can use the same test you're already using:
- name: Filter reply
debug:
var: item.browser_download_url
with_items: "{{ json_reply.json.assets }}"
when: item.name.endswith('x64.AppImage')
The root problem would appear to be an Ansible bug. The error comes from the following check in the jmespath library:
if actual_typename not in allowed_types:
raise exceptions.JMESPathTypeError(
function_name, current,
self._convert_to_jmespath_type(actual_typename), types)
At the point this code is called, the data type of values in your json response is AnsibleUnsafeText, where as allowed_types is [str, unicode]. I think the transformation of values from native types to the AnsibleUnsafeText type probably is some sort of standard Ansible module behavior being imposed by the uri module. We can work around it by using curl instead, like this:
- name: Get list of Rambox releases
command: >
curl -s "https://api.github.com/repos/saenzramiro/rambox/releases/latest"
register: json_reply
And then:
- name: Filter reply
debug:
var: item.browser_download_url
with_items: >
{{ json_reply.stdout|from_json|json_query('assets[?ends_with(name, `x64.AppImage`)]') }}

extracting a variable from json output then debug and register the outout with ansible

Hi I have a problem of getting one of the variables extracted from a json output after doing a curl to be parsed and registered back to ansible
Playbook:
- name: debug stdout
debug:
msg: "{{ result.stdout | from_json }}"
register: dataresult
- name: debug fact
debug:
msg: "{{ dataresult.data.start_time_string }}"
output :
TASK [backup_api : debug stdout]
***********************************************
task path: /home/ansible/cm-dha/roles/backup_api/tasks/main.yml:36
ok: [127.0.0.1] => {
"msg": {
"data": [
{
"backup_id": 40362,
"certified": null,
"instance_id": 148,
"start_time": 1506985211,
"start_time_string": "10/03/2017 03:00:11 am"
}
],
"timestamp": 1507022232
}
}
error:
fatal: [127.0.0.1]: FAILED! => {
"failed": true,
"msg": "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'dict object' has no attribute 'data'\n\nThe error appears to have been in '/home/ansible/cm-dha/roles/backup_api/tasks/main.yml': line 48, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n - name: debug fact\n ^ here\n"
The error is happening when trying to extract the value start_time_string
so how to do it probably as I tried too many things like using with_items, with_dict , simulating the data[] output to debug and even doing a json query but without success
so any help here?
Don't use debug to assign facts, use set_fact instead:
- name: debug stdout
set_fact:
dataresult: "{{ result.stdout | from_json }}"
- name: debug fact
debug:
msg: "{{ dataresult.data[0].start_time_string }}"
Thanks to accepted answer, I've made it as below for my case. Leaving it here, maybe it helps someone.
- name: Connectors
shell: curl -X GET http://myserver:8083/connectors
register: out6
- set_fact:
dataresult: "{{ out6.stdout | from_json }}"
- debug: var=dataresult