Ansible: create a function - function

I found this page from answer number 4 by #cobbzilla useful to my use case.
Just want to ask if Ansible is capable to have a function that will handle this command:
2>&1 >> /tmp/debug.log
I have already applied this solution to my yml files and I was looking if the command can be wrapped on a function so that it will show much cleaner.
My Sample Ansible yml:
- name: Deploy
hosts: localhost
connection: local
gather_facts: false
tasks:
- name: perform 1st script
shell: bash -c "1st_script.sh 2>&1 >> /var/tmp/debug.log"
- name: perform 2nd script
shell: bash -c "2nd_script.sh 2>&1 >> /var/tmp/debug.log"
- name: perform 3rd script
shell: bash -c "3rd_script.sh 2>&1 >> /var/tmp/debug.log"

Preliminary note: I am answering your direct question below because this can be useful in other circumstances.
Meanwhile, in the ansible context, playing bash scripts and returning their output and error to a log file on the target is generally not a good idea. You will be left blind if something goes wrong, or if you want to analyze the output, as the module will return an empty stderr and stdout. You will have to rely on analyzing the log file later (and finding the correct output since you mix script output in there).
On an even wider level, you should use shell/command only when there is no other possibility to get the same job done using existing modules
If you don't mind writing some lines of python, a pretty easy and straight forward way to acheive your goal is to use a custom filter. The below example is quickNdirty. You will probably have to harden its code (escape command characters, check for specific errors...) but this should put you on track.
For the example, I am creating the filter in the filter_plugins folder adjacent to the demo playbook. If you need to distribute that filter across projects, search the ansible documentation to learn how to wrap that in a role or a collection.
In filter_plugins/shell_filters.py
def bash_n_log(command, log_file='/var/tmp/debug.log'):
"""Return a formatted string to play the script in bash and log its output"""
return f'bash -c "{command} 2>&1 >> {log_file}"'
class FilterModule(object):
"""collection of shell utility filters."""
def filters(self):
"""Return the filter list."""
return {
'bash_n_log': bash_n_log
}
Then the demo playbook.yml
- name: Custom shell filter demo
hosts: localhost
gather_facts: false
vars:
my_commands:
- echo Hello World
- ls -l /dev/null
tasks:
- name: Play my commands with my filter
shell: "{{ item | bash_n_log }}"
loop: "{{ my_commands }}"
- name: Same example with non default log file
shell: "{{ item | bash_n_log('/tmp/other.log') }}"
loop: "{{ my_commands }}"
- name: Get content of the log files
slurp:
path: "{{ item }}"
register: slurped_logs
loop:
- /var/tmp/debug.log
- /tmp/other.log
- name: Show log file content
debug:
msg: "{{ (item.content | b64decode).split('\n') }}"
loop: "{{ slurped_logs.results }}"
loop_control:
label: "{{ item.item }}"
gives:
$ ansible-playbook playbook.yml
PLAY [Custom shell filter demo] ********************************************************************************************************************************************************************************************************
TASK [Play my commands with my filter] *************************************************************************************************************************************************************************************************
changed: [localhost] => (item=echo Hello World)
changed: [localhost] => (item=ls -l /dev/null)
TASK [Same example with non default log file] ******************************************************************************************************************************************************************************************
changed: [localhost] => (item=echo Hello World)
changed: [localhost] => (item=ls -l /dev/null)
TASK [Get content of the log files] ****************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/var/tmp/debug.log)
ok: [localhost] => (item=/tmp/other.log)
TASK [Show log file content] ***********************************************************************************************************************************************************************************************************
ok: [localhost] => (item=/var/tmp/debug.log) => {
"msg": [
"Hello World",
"crw-rw-rw- 1 root root 1, 3 Mar 27 12:06 /dev/null",
""
]
}
ok: [localhost] => (item=/tmp/other.log) => {
"msg": [
"Hello World",
"crw-rw-rw- 1 root root 1, 3 Mar 27 12:06 /dev/null",
""
]
}
PLAY RECAP *****************************************************************************************************************************************************************************************************************************
localhost : ok=4 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

I just tried and tested this simple solution worked, I just added vars section witch points the variable to a command
- name: Deploy
hosts: localhost
connection: local
gather_facts: false
vars:
logger: "2>&1 >> /var/tmp/debug.log"
tasks:
- name: perform 1st script
shell: bash -c "1st_script.sh {{ logger }}"
- name: perform 2nd script
shell: bash -c "2nd_script.sh {{ logger }}"
- name: perform 3rd script
shell: bash -c "3rd_script.sh {{ logger }}"

Related

Checking the key value in a JSON file

I'm having trouble verifying the value of a json file on a remote server. I have to overwrite the file once on the remote machine from the template (j2). After that, I start a service that writes additional values to this file.
But when restarting ansible-playbook, this file is overwritten because it differs from the template. Before starting the task of writing a file from a template, I want to check the file for unique values.
For testing on a local machine, I do this and everything works:
- name: Check file
hosts: localhost
vars:
config: "{{ lookup('file','config.json') | from_json }}"
tasks:
- name: Check info
set_fact:
info: "{{ config.Settings.TimeStartUP }}"
- name: Print info
debug:
var: info
- name: Create directory
when: interfaces | length != 0
ansible.builtin.file:
...
But when I try to do the same in a task on a remote machine, for some reason ansible is looking for a file on the local machine
all.yml
---
config_file: "{{ lookup('file','/opt/my_project/config.json') | from_json }}"
site.yml
---
- name: Install My_project
hosts: server
tasks:
- name: Checking if a value exists
set_fact:
info: "{{ config_file.Settings.TimeStartUP }}"
- name: Print info
debug:
var: info
Error:
fatal: [server]: FAILED! => {"msg": "An unhandled exception occurred while templating '{{ lookup('file','/opt/my_project/config.json') | from_json }}'. Error was a <class 'ansible.errors.AnsibleError'>, original message: An unhandled exception occurred while running the lookup plugin 'file'. Error was a <class 'ansible.errors.AnsibleError'>, original message: could not locate file in lookup: /opt/my_project/config.json. could not locate file in lookup: /opt/my_project/config.json"}
Please tell me how to correctly check the key value in a JSON file on a remote server?
fetch the files from the remote hosts first. For example, given the files below for testing
shell> ssh admin#test_11 cat /tmp/config.json
{"Settings": {"TimeStartUP": "today"}}
shell> ssh admin#test_12 cat /tmp/config.json
{"Settings": {"TimeStartUP": "yesterday"}}
The playbook below
- hosts: test_11,test_12
gather_facts: false
tasks:
- file:
state: directory
path: "{{ playbook_dir }}/configs"
delegate_to: localhost
run_once: true
- fetch:
src: /tmp/config.json
dest: "{{ playbook_dir }}/configs"
- include_vars:
file: "{{ config_path }}"
name: config
vars:
config_path: "{{ playbook_dir }}/configs/{{ inventory_hostname }}/tmp/config.json"
- debug:
var: config.Settings.TimeStartUP
will create the directory configs in playbook_dir on the controller and will fetch the files from the remote hosts into this directory. See the parameter dest on how the path will be created
shell> cat configs/test_11/tmp/config.json
{"Settings": {"TimeStartUP": "today"}}
shell> cat configs/test_12/tmp/config.json
{"Settings": {"TimeStartUP": "yesterday"}}
Then include_vars and store the dictionary into the variable config
ok: [test_11] =>
config.Settings.TimeStartUP: today
ok: [test_12] =>
config.Settings.TimeStartUP: yesterday

Ansible: How to categorize files by permissions?

Im trying to categorize the files based on their permissions
and I have a problem with the JSON query.
The output I like to categorize
Example
user#test.example.com:~$ stat -c '%a %n' $(pwd)/*
644 /home/user/go
755 /home/user/sshified
644 /home/user/test.yaml
or
user#test.example.com:~$ find / -perm -4000 -type f -exec stat -c '%a %n' {} 2>/dev/null \;
4755 /usr/bin/mtr
4755 /bin/su
4777 /bin/app1
The query which doesn't give any output back.
Ansible Code
- name: Find binaries with suid bit set
shell:
cmd: stat -c '%a %n' folder/*
register: files-with-write
failed_when: files-with-write.rc != 1 and files-with-write.rc != 0
changed_when: false
- set_fact:
writeable_files: "{{files-with-write| to_json | from_json |json_query(\"[?ends_with(mode, '7') == `true`].{gr_name: gr_name, mode: mode, path: path }\") }}"
- debug:
msg:
- "files: {{writeable_files}}
Use find module and see what attributes are available in the registered results. For example, given the files
shell> stat -c '%a %n' test-476/*
644 test-476/go
755 test-476/sshified
664 test-476/test.yaml
the debug below lists the registered attributes of the files
- find:
paths: test-476
recurse: true
register: result
- debug:
var: result.files.0.keys()|list|to_yaml
gives
result.files.0.keys()|list|to_yaml: |-
[path, mode, isdir, ischr, isblk, isreg, isfifo, islnk, issock, uid, gid, size, inode,
dev, nlink, atime, mtime, ctime, gr_name, pw_name, wusr, rusr, xusr, wgrp, rgrp,
xgrp, woth, roth, xoth, isuid, isgid]
For example, use the attribute wgrp to select group-writable files
- set_fact:
group_writeable_files: "{{ result.files|selectattr('wgrp') }}"
- debug:
msg: "{{ group_writeable_files|map(attribute='path')|list }}"
gives
msg:
- test-476/test.yaml
If you just want to find files that are writeable, this can be much easier done on bash level:
- name: Find writable by others
command: find folder/ -perm /o+w
register: writable_others
- name: Find writable by others or group
command: find folder/ -perm /o+w,g+w
register: writable_others_group
Using Ansible modules like find or stat you may start your implementation with something like
---
- hosts: test
become: no
gather_facts: no
tasks:
- name: Return a list of files
find:
paths: "/home/{{ ansible_user }}/"
file_type: file
register: result
- name: Show result
debug:
msg: "{{ item.mode }} {{ item.path }}"
when: item.mode == "0755"
loop: "{{ result.files }}"

Ansible "set_fact" repository url from json file using filters like "from_json"

Using Ansible "set_fact" module, I need to get repository url from json file using filters like "from_json". I tried in couple ways, and still doesn't get it how is should work.
- name: initial validation
tags: bundle
hosts: localhost
connection: local
tasks:
- name: register bundle version_file
include_vars:
file: '/ansible/playbook/workbench-bundle/bundle.json'
register: bundle
- name: debug registered bundle file
debug:
msg: '{{ bundle }}'
I get json that I wanted:
TASK [debug registered bundle file] ************************************************
ok: [127.0.0.1] => {
"msg": {
"ansible_facts": {
"engine-config": "git#bitbucket.org/engine-config.git",
"engine-monitor": "git#bitbucket.org/engine-monitor.git",
"engine-server": "git#bitbucket.org/engine-server.git",
"engine-worker": "git#bitbucket.org/engine-worker.git"
},
"changed": false
}
}
And then I'm trying to select each value by key name to use this value as URL to "npm install" each package in separate instances.
- name: set_fact some paramater
set_fact:
engine_url: "{{ bundle.('engine-server') | from_json }}"
And then I get error:
fatal: [127.0.0.1]: FAILED! => {"failed": true, "msg": "template error
while templating string: expected name or number. String: {{
bundle.('engine-server') }}"}
I many others ways like this loopkup, and it still fails with others errors. Can someone help to understand, how I can find each parameter and store him as "set_fact"? Thanks
Here is a sample working code to set a variable like in the question (although I don't see much sense in it):
- name: initial validation
tags: bundle
hosts: localhost
connection: local
tasks:
- name: register bundle version_file
include_vars:
file: '/ansible/playbook/workbench-bundle/bundle.json'
name: bundle
- debug:
var: bundle
- debug:
var: bundle['engine-server']
- name: set_fact some paramater
set_fact:
engine_url: "{{ bundle['engine-server'] }}"
The above assumes your input data (which you did not include) is:
{
"engine-config": "git#bitbucket.org/engine-config.git",
"engine-monitor": "git#bitbucket.org/engine-monitor.git",
"engine-server": "git#bitbucket.org/engine-server.git",
"engine-worker": "git#bitbucket.org/engine-worker.git"
}

Ansible: parsing/concatenating output of getent module

I try to set up chroot for sftp users, so that they can see user/group names on ls -l as per this article. To this end I need to get output of getent command and place it into /chroots/{{ user.username }}/etc/passwd file.
I try to use Ansible to replace this command getent passwd sftpuser > /chroots/sftpuser/etc/passwd as follows:
- name: get {{ user.username }} user info
getent:
database: passwd
key: "{{ user.username }}"
- debug:
var: getent_passwd
- name: create /chroots/{{ user.username }}/etc/passwd file
lineinfile:
path: /chroots/{{ user.username }}/etc/passwd
line: "{{ getent_passwd | from_json }}"
state: present
create: yes
owner: root
group: root
mode: '0644'
The 'getent_passwd' looks as follows:
ok: [cf1] => {
"getent_passwd": {
"testuser1": [
"x",
"1001",
"1002",
"",
"/home/testuser1",
"/usr/sbin/nologin"
]
}
}
But I get this error: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ getent_passwd | from_json }}): expected string or buffer"}
What is the proper way to get those values supplied by getent_passwd into one flat string joined by ":"?
Is it safe to use genent module with key: "root" this way instead of echo "root:x:0:0:not really root:::" >> /chroots/sftpuser/etc/passwd?
one can run getent passwd user1 user2 - is it possible to supply two keys to the ansible's getent module somehow?
What is the proper way to get those values supplied by getent_passwd into one flat string joined by ":"?
For example using a Jinja2 template with join filter:
- debug:
msg: "{{ user.username }}:{{getent_passwd[user.username]|join(':')}}"
One can run getent passwd user1 user2 - is it possible to supply two keys to the ansible's getent module somehow?
No. Either a single one or all.
Use an outer loop to request values in the first case, or filter the resulting list in the second.

Is it possible to do a dynamic lookup

I have an environment where the sudo password of a user is not the same on several servers. This can be seen as a security feature :-D
I wanted to save the sudo password (vault/security aside at time point) in a file:
host1.sudopwd
host2.sudopwd
...
hostn.sudopwd
in my ansible playbook, I am using something like that:
---
- hosts: all
tasks:
- include: '{{ inventory_hostname }}'
vars:
ansible_become: true
ansible_become_user: root
ansible_become_pass: "{{ lookup('file','{{ inventory_hostname }}.sudopwd') }}"
And it works well if I have one file per hosts.
Now how can I make it more "dynamic"?
Let say that, that I have a script that is called searchsudopwd, which is a simple bash script (could be any language in fact).
#!/usr/bin/env bash
if [[ "$1" == "host1" ]]; then
echo "pwd1"
elif [[ "$1" == "host2" ]]; then
echo "pwd2"
else
echo "default pwd"
fi
How can I use it with ansible? What needs to be changed in my playbook to make it work?
I have tried something like:
---
- hosts: all
tasks:
- include: '{{ inventory_hostname }}'
vars:
ansible_become: true
ansible_become_user: root
ansible_become_pass: "{{ item }}"
with_lines: searchsudopwd '{{ inventory_hostname }}'
But it doesn't work...
If my idea doesn't work, is there a way to have like a "default" password, and only get the password for specific hosts?
Thank you for your help and time!
I think you are better off putting the passwords in the individual host_vars files, when needed, and put the default in group_vars/all. Put it all in Ansible Vault. (But then you have a shared password for Ansible Vault.)
I have a host called "ml01" so I changed your script:
$ cat /home/jscheible/searchsudopwd
#!/usr/bin/env bash
if [[ "$1" == "ml01" ]]; then
echo "pwd1"
elif [[ "$1" == "host2" ]]; then
echo "pwd2"
else
echo "default pwd"
fi
To call that local program with the passwords, just call that in a play with delegate_to: localhost:
$ cat show_pwd
---
- hosts: all
connection: ssh
gather_facts: false
tasks:
- name: Do passwd lookup
command: /home/jscheible/searchsudopwd {{ inventory_hostname }}
delegate_to: localhost
register: result
- name: Show results
debug:
var: result.stdout
delegate_to: localhost
Here are the results for my test:
$ ansible-playbook show_pwd
PLAY [all] *********************************************************************
TASK [Do passwd lookup] ********************************************************
changed: [al01 -> localhost]
changed: [ml01 -> localhost]
TASK [Show results] ************************************************************
ok: [al01 -> localhost] => {
"result.stdout": "default pwd"
}
ok: [ml01 -> localhost] => {
"result.stdout": "pwd1"
}
PLAY RECAP *********************************************************************
There are handy lookup plugins.
You can actually just replace your file lookup with pipe lookup:
ansible_become_pass: "{{ lookup('pipe','/path_on_ansible_machine/searchsudopwd '+inventory_hostname) }}"