Get Node attributes via knife - json

I have a requirement where I need to get hostname, memory, cores, storage, packages installed for multiple nodes(~1k).
I have approached the solution by using knife.
$ knife search node 'hostname:HostName1 OR hostname:HostName2 OR hostname:HostName3' -a hostname -a cpu.cores -a memory.total -a rpm -a filesystem.by_device -F j|jq '.'
And the typical output of this command is like;
{
"results": 3,
"rows": [
{
"MyHostName1": {
"hostname": "MyHostName1",
"cpu.cores": 4,
"memory.total": "15645184kB",
"rpm": {
"loger-multipath": [
{
"version": "0.4.9",
"release": "123.el7",
"arch": "x86_64"
}
],
"python": [
{
"version": "7.19.0",
"release": "19.el7",
"arch": "x86_64"
}
]
},
"filesystem.by_device": {
"/apps/logger/root_my-root": {
"kb_size": "8125880",
"kb_used": "2426760",
"kb_available": "5263308",
"percent_used": "32%",
"mount_options": [
"rw",
"discard",
"data=ordered"
],
"uuid": "87ujrf56-6yu6-654r-yu43-uy67yg43ws67",
"mounts": [
"/"
]
}
}
}
}
}
However, there are details which I do not need;
How can we set the display sequence same as that of the attribute list in the command, i.e. hostname then core, memory…
We get the file system names and their corresponding sizes, however, we are getting all the other tag values as well; how can we get just the file system name and the ( something similar to what we get from the df command; e.g. apps/logger/root_vg-apps: kb_size: 3997376 )
The output of the rpm attribute gives us the rpm package name, architecture, version and release information, how can we concatenate the output of multiple attributes in a single line ( something similar to the output when we run the yum list installed command; e.g loger-multipath.x86_64 0.4.9-123.el7 )
EDIT:
After much googling this is the progress:
knife search node 'HostName1 OR hostname:HostName2' -a cpu.cores -a memory.total -a filesystem.by_device -F j|jq '.rows[]|keys[] as $hostName|"\($hostName),\(.[$hostName]|."cpu.cores"),\(.[$hostName]|."memory.total"),\(.[$hostName]|."filesystem.by_device")"'
And the corresponding output
"HostName1,4,15645184kB,{\"/dev/mapper/root_vg-root\":{\"kb_size\":\"8125880\",\"kb_used\":\"2425220\",\"kb_available\":\"5264848\",\"percent_used\":\"32%\",\"total_inodes\":\"524288\",\"inodes_used\":\"88441\",\"inodes_available\":\"435847\",\"inodes_percent_used\":\"17%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\",\"discard\",\"data=ordered\"],\"uuid\":\"rthd-762c-41affff8-8927-065fsee20853c681\",\"mounts\":[\"/\"]},\"devtmpfs\":{\"kb_size\":\"7810756\",\"kb_used\":\"0\",\"kb_available\":\"7810756\",\"percent_used\":\"0%\",\"total_inodes\":\"1952689\",\"inodes_used\":\"403\",\"inodes_available\":\"1952286\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"devtmpfs\",\"mount_options\":[\"rw\",\"nosuid\",\"seclabel\",\"size=7810756k\",\"nr_inodes=1952689\",\"mode=755\"],\"mounts\":[\"/dev\"]},\"tmpfs\":{\"kb_size\":\"1564520\",\"kb_used\":\"0\",\"kb_available\":\"1564520\",\"percent_used\":\"0%\",\"total_inodes\":\"1955648\",\"inodes_used\":\"1\",\"inodes_available\":\"1955647\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"tmpfs\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"size=1564520k\",\"mode=700\",\"uid=627000\",\"gid=161\"],\"mounts\":[\"/dev/shm\",\"/run\",\"/sys/fs/cgroup\",\"/run/user/0\",\"/run/user/627000\"]},\"/dev/sda1\":{\"kb_size\":\"499656\",\"kb_used\":\"212068\",\"kb_available\":\"250892\",\"percent_used\":\"46%\",\"total_inodes\":\"32768\",\"inodes_used\":\"350\",\"inodes_available\":\"32418\",\"inodes_percent_used\":\"2%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"data=ordered\"],\"uuid\":\"857fgg-b2a2-42d8-9db2-dfrferf7544\",\"mounts\":[\"/boot\"]},\"/dev/mapper/root_vg-var\":{\"kb_size\":\"5029504\",\"kb_used\":\"4142128\",\"kb_available\":\"608848\",\"percent_used\":\"88%\",\"total_inodes\":\"327680\",\"inodes_used\":\"6191\",\"inodes_available\":\"321489\",\"inodes_percent_used\":\"2%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"noacl\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"dfef155456-ab4c-48f4-a7a5-5454sfdf\",\"mounts\":[\"/var\"]},\"/dev/mapper/root_vg-var--tmp\":{\"kb_size\":\"1998672\",\"kb_used\":\"6180\",\"kb_available\":\"1871252\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"21\",\"inodes_available\":\"131051\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"vfghhhht542-ea7c-4c8b-9afd-frfgvbbn\",\"mounts\":[\"/var/tmp\"]},\"/dev/mapper/root_vg-apps\":{\"kb_size\":\"3997376\",\"kb_used\":\"495044\",\"kb_available\":\"3276236\",\"percent_used\":\"14%\",\"total_inodes\":\"262144\",\"inodes_used\":\"3203\",\"inodes_available\":\"258941\",\"inodes_percent_used\":\"2%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nodev\",\"relatime\",\"seclabel\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"fvfbvfbv55444-a813-4d9c-a9ac-7d50cfbfe345\",\"mounts\":[\"/apps\"]},\"/dev/mapper/root_vg-kdump\":{\"kb_size\":\"1998672\",\"kb_used\":\"6144\",\"kb_available\":\"1871288\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"11\",\"inodes_available\":\"131061\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"frgrghg55-9673-47f5-aaac-4g4g4g1g1\",\"mounts\":[\"/kdump\"]},\"/dev/mapper/root_vg-home\":{\"kb_size\":\"1998672\",\"kb_used\":\"6544\",\"kb_available\":\"1870888\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"83\",\"inodes_available\":\"130989\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"frbj4874-fe4d-4f82-ad86-41554ffv\",\"mounts\":[\"/home\"]},\"/dev/mapper/root_vg-tmp\":{\"kb_size\":\"1998672\",\"kb_used\":\"7916\",\"kb_available\":\"1869516\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"51\",\"inodes_available\":\"131021\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"a995fd05-90a8-46a8-a192-0a02f68e476a\",\"mounts\":[\"/tmp\"]},\"/dev/mapper/root_vg-gcis2\":{\"kb_size\":\"20511312\",\"kb_used\":\"3191304\",\"kb_available\":\"16255048\",\"percent_used\":\"17%\",\"total_inodes\":\"1310720\",\"inodes_used\":\"33967\",\"inodes_available\":\"1276753\",\"inodes_percent_used\":\"3%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"fb4bb78a-7f33-47f1-87a6-dcbe50bc6349\",\"mounts\":[\"/apps/gcis2\"]},\"sysfs\":{\"fs_type\":\"sysfs\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\",\"seclabel\"],\"mounts\":[\"/sys\"]},\"proc\":{\"fs_type\":\"proc\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\"],\"mounts\":[\"/proc\"]},\"securityfs\":{\"fs_type\":\"securityfs\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\"],\"mounts\":[\"/sys/kernel/security\"]},\"devpts\":{\"fs_type\":\"devpts\",\"mount_options\":[\"rw\",\"nosuid\",\"noexec\",\"relatime\",\"seclabel\",\"gid=5\",\"mode=620\",\"ptmxmode=000\"],\"mounts\":[\"/dev/pts\"]},\"cgroup\":{\"fs_type\":\"cgroup\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\",\"seclabel\",\"net_prio\",\"net_cls\"],\"mounts\":[\"/sys/fs/cgroup/systemd\",\"/sys/fs/cgroup/perf_event\",\"/sys/fs/cgroup/blkio\",\"/sys/fs/cgroup/freezer\",\"/sys/fs/cgroup/cpu,cpuacct\",\"/sys/fs/cgroup/memory\",\"/sys/fs/cgroup/pids\",\"/sys/fs/cgroup/hugetlb\",\"/sys/fs/cgroup/cpuset\",\"/sys/fs/cgroup/devices\",\"/sys/fs/cgroup/net_cls,net_prio\"]},\"pstore\":{\"fs_type\":\"pstore\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\"],\"mounts\":[\"/sys/fs/pstore\"]},\"configfs\":{\"fs_type\":\"configfs\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/kernel/config\"]},\"selinuxfs\":{\"fs_type\":\"selinuxfs\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/fs/selinux\"]},\"systemd-1\":{\"fs_type\":\"autofs\",\"mount_options\":[\"rw\",\"relatime\",\"fd=30\",\"pgrp=1\",\"timeout=0\",\"minproto=5\",\"maxproto=5\",\"direct\",\"pipe_ino=16127\"],\"mounts\":[\"/proc/sys/fs/binfmt_misc\"]},\"debugfs\":{\"fs_type\":\"debugfs\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/kernel/debug\"]},\"hugetlbfs\":{\"fs_type\":\"hugetlbfs\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\"],\"mounts\":[\"/dev/hugepages\"]},\"mqueue\":{\"fs_type\":\"mqueue\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\"],\"mounts\":[\"/dev/mqueue\"]},\"binfmt_misc\":{\"fs_type\":\"binfmt_misc\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/proc/sys/fs/binfmt_misc\"]},\"fusectl\":{\"fs_type\":\"fusectl\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/fs/fuse/connections\"]},\"/dev/fd0\":{},\"/dev/sda\":{},\"/dev/sda2\":{\"fs_type\":\"LVM2_member\",\"uuid\":\"zrtcBv-4y6D-LG2a-Wt6M-h18d-K0BQ-zyl11h\"},\"/dev/mapper/root_vg-swap\":{\"fs_type\":\"swap\",\"uuid\":\"vfbvejgbg5456454-93a1-4a67-b33f-gtrbbbn\"},\"/dev/mapper/root_vg-pool0_tmeta\":{},\"/dev/mapper/root_vg-pool0-tpool\":{},\"/dev/mapper/root_vg-pool0\":{},\"/dev/mapper/root_vg-pool0_tdata\":{},\"rootfs\":{\"fs_type\":\"rootfs\",\"mount_options\":[\"rw\"],\"mounts\":[\"/\"]}}"
"HostName2,4,15645184kB,{\"/dev/mapper/root_vg-root\":{\"kb_size\":\"8125880\",\"kb_used\":\"2425220\",\"kb_available\":\"5264848\",\"percent_used\":\"32%\",\"total_inodes\":\"524288\",\"inodes_used\":\"88441\",\"inodes_available\":\"435847\",\"inodes_percent_used\":\"17%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\",\"discard\",\"data=ordered\"],\"uuid\":\"rthd-762c-41affff8-8927-065fsee20853c681\",\"mounts\":[\"/\"]},\"devtmpfs\":{\"kb_size\":\"7810756\",\"kb_used\":\"0\",\"kb_available\":\"7810756\",\"percent_used\":\"0%\",\"total_inodes\":\"1952689\",\"inodes_used\":\"403\",\"inodes_available\":\"1952286\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"devtmpfs\",\"mount_options\":[\"rw\",\"nosuid\",\"seclabel\",\"size=7810756k\",\"nr_inodes=1952689\",\"mode=755\"],\"mounts\":[\"/dev\"]},\"tmpfs\":{\"kb_size\":\"1564520\",\"kb_used\":\"0\",\"kb_available\":\"1564520\",\"percent_used\":\"0%\",\"total_inodes\":\"1955648\",\"inodes_used\":\"1\",\"inodes_available\":\"1955647\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"tmpfs\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"size=1564520k\",\"mode=700\",\"uid=627000\",\"gid=161\"],\"mounts\":[\"/dev/shm\",\"/run\",\"/sys/fs/cgroup\",\"/run/user/0\",\"/run/user/627000\"]},\"/dev/sda1\":{\"kb_size\":\"499656\",\"kb_used\":\"212068\",\"kb_available\":\"250892\",\"percent_used\":\"46%\",\"total_inodes\":\"32768\",\"inodes_used\":\"350\",\"inodes_available\":\"32418\",\"inodes_percent_used\":\"2%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"data=ordered\"],\"uuid\":\"857fgg-b2a2-42d8-9db2-dfrferf7544\",\"mounts\":[\"/boot\"]},\"/dev/mapper/root_vg-var\":{\"kb_size\":\"5029504\",\"kb_used\":\"4142128\",\"kb_available\":\"608848\",\"percent_used\":\"88%\",\"total_inodes\":\"327680\",\"inodes_used\":\"6191\",\"inodes_available\":\"321489\",\"inodes_percent_used\":\"2%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"noacl\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"dfef155456-ab4c-48f4-a7a5-5454sfdf\",\"mounts\":[\"/var\"]},\"/dev/mapper/root_vg-var--tmp\":{\"kb_size\":\"1998672\",\"kb_used\":\"6180\",\"kb_available\":\"1871252\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"21\",\"inodes_available\":\"131051\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"vfghhhht542-ea7c-4c8b-9afd-frfgvbbn\",\"mounts\":[\"/var/tmp\"]},\"/dev/mapper/root_vg-apps\":{\"kb_size\":\"3997376\",\"kb_used\":\"495044\",\"kb_available\":\"3276236\",\"percent_used\":\"14%\",\"total_inodes\":\"262144\",\"inodes_used\":\"3203\",\"inodes_available\":\"258941\",\"inodes_percent_used\":\"2%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nodev\",\"relatime\",\"seclabel\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"fvfbvfbv55444-a813-4d9c-a9ac-7d50cfbfe345\",\"mounts\":[\"/apps\"]},\"/dev/mapper/root_vg-kdump\":{\"kb_size\":\"1998672\",\"kb_used\":\"6144\",\"kb_available\":\"1871288\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"11\",\"inodes_available\":\"131061\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"frgrghg55-9673-47f5-aaac-4g4g4g1g1\",\"mounts\":[\"/kdump\"]},\"/dev/mapper/root_vg-home\":{\"kb_size\":\"1998672\",\"kb_used\":\"6544\",\"kb_available\":\"1870888\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"83\",\"inodes_available\":\"130989\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"frbj4874-fe4d-4f82-ad86-41554ffv\",\"mounts\":[\"/home\"]},\"/dev/mapper/root_vg-tmp\":{\"kb_size\":\"1998672\",\"kb_used\":\"7916\",\"kb_available\":\"1869516\",\"percent_used\":\"1%\",\"total_inodes\":\"131072\",\"inodes_used\":\"51\",\"inodes_available\":\"131021\",\"inodes_percent_used\":\"1%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"relatime\",\"seclabel\",\"discard\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"a995fd05-90a8-46a8-a192-0a02f68e476a\",\"mounts\":[\"/tmp\"]},\"/dev/mapper/root_vg-gcis2\":{\"kb_size\":\"20511312\",\"kb_used\":\"3191304\",\"kb_available\":\"16255048\",\"percent_used\":\"17%\",\"total_inodes\":\"1310720\",\"inodes_used\":\"33967\",\"inodes_available\":\"1276753\",\"inodes_percent_used\":\"3%\",\"fs_type\":\"ext4\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\",\"stripe=16\",\"data=ordered\"],\"uuid\":\"fb4bb78a-7f33-47f1-87a6-dcbe50bc6349\",\"mounts\":[\"/apps/gcis2\"]},\"sysfs\":{\"fs_type\":\"sysfs\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\",\"seclabel\"],\"mounts\":[\"/sys\"]},\"proc\":{\"fs_type\":\"proc\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\"],\"mounts\":[\"/proc\"]},\"securityfs\":{\"fs_type\":\"securityfs\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\"],\"mounts\":[\"/sys/kernel/security\"]},\"devpts\":{\"fs_type\":\"devpts\",\"mount_options\":[\"rw\",\"nosuid\",\"noexec\",\"relatime\",\"seclabel\",\"gid=5\",\"mode=620\",\"ptmxmode=000\"],\"mounts\":[\"/dev/pts\"]},\"cgroup\":{\"fs_type\":\"cgroup\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\",\"seclabel\",\"net_prio\",\"net_cls\"],\"mounts\":[\"/sys/fs/cgroup/systemd\",\"/sys/fs/cgroup/perf_event\",\"/sys/fs/cgroup/blkio\",\"/sys/fs/cgroup/freezer\",\"/sys/fs/cgroup/cpu,cpuacct\",\"/sys/fs/cgroup/memory\",\"/sys/fs/cgroup/pids\",\"/sys/fs/cgroup/hugetlb\",\"/sys/fs/cgroup/cpuset\",\"/sys/fs/cgroup/devices\",\"/sys/fs/cgroup/net_cls,net_prio\"]},\"pstore\":{\"fs_type\":\"pstore\",\"mount_options\":[\"rw\",\"nosuid\",\"nodev\",\"noexec\",\"relatime\"],\"mounts\":[\"/sys/fs/pstore\"]},\"configfs\":{\"fs_type\":\"configfs\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/kernel/config\"]},\"selinuxfs\":{\"fs_type\":\"selinuxfs\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/fs/selinux\"]},\"systemd-1\":{\"fs_type\":\"autofs\",\"mount_options\":[\"rw\",\"relatime\",\"fd=30\",\"pgrp=1\",\"timeout=0\",\"minproto=5\",\"maxproto=5\",\"direct\",\"pipe_ino=16127\"],\"mounts\":[\"/proc/sys/fs/binfmt_misc\"]},\"debugfs\":{\"fs_type\":\"debugfs\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/kernel/debug\"]},\"hugetlbfs\":{\"fs_type\":\"hugetlbfs\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\"],\"mounts\":[\"/dev/hugepages\"]},\"mqueue\":{\"fs_type\":\"mqueue\",\"mount_options\":[\"rw\",\"relatime\",\"seclabel\"],\"mounts\":[\"/dev/mqueue\"]},\"binfmt_misc\":{\"fs_type\":\"binfmt_misc\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/proc/sys/fs/binfmt_misc\"]},\"fusectl\":{\"fs_type\":\"fusectl\",\"mount_options\":[\"rw\",\"relatime\"],\"mounts\":[\"/sys/fs/fuse/connections\"]},\"/dev/fd0\":{},\"/dev/sda\":{},\"/dev/sda2\":{\"fs_type\":\"LVM2_member\",\"uuid\":\"zrtcBv-4y6D-LG2a-Wt6M-h18d-K0BQ-zyl11h\"},\"/dev/mapper/root_vg-swap\":{\"fs_type\":\"swap\",\"uuid\":\"vfbvejgbg5456454-93a1-4a67-b33f-gtrbbbn\"},\"/dev/mapper/root_vg-pool0_tmeta\":{},\"/dev/mapper/root_vg-pool0-tpool\":{},\"/dev/mapper/root_vg-pool0\":{},\"/dev/mapper/root_vg-pool0_tdata\":{},\"rootfs\":{\"fs_type\":\"rootfs\",\"mount_options\":[\"rw\"],\"mounts\":[\"/\"]}}"
I know this seems a bit messy; any help is welcome


looking at the json provided in the question, it seems that your query does not use the right values for the key that you specify -- namely, hostname -- you specified hostname:HostName1 where it feels it needs to be hostname:MyHostName1.
also, you can drop the node argument from the search command and specify it in the query, such as node:my.node.name.

Related

Is there a way to use AWS Step Functions Input to assembly command string on System Manager block?

I am creating a Step Function State machine to everytime an instance starts it copy a file from S3 to an specific folder inside this instance. The origin folder inside S3 bucket has a folder named with this instance ID. The instance ID I am passing as input for the System manager block, but I need to use it to create the command string that will be performed inside the EC2.
For example:
My input is: $.detail.instance-id (lets assume the following ID i-11223344556677889)
The Systems Manager API parameters are:
"CloudWatchOutputConfig": {
"CloudWatchLogGroupName": "myloggroup",
"CloudWatchOutputEnabled": true
},
"DocumentName": "AWS-RunShellScript",
"DocumentVersion": "$DEFAULT",
"InstanceIds.$": "States.Array($)",
"MaxConcurrency": "50",
"MaxErrors": "10",
"Parameters": {
"commands": [
{
"runuser -l ec2-user -c \"aws s3 cp s3://my-bucket/**MY_INSTANCEID**/myfile.xyz /home/ec2-user/myfolder/myfile.xyz\""
}
},
"TimeoutSeconds": 6000
}```
Summing up, I want to turn the line with the command replacing the MY_INSTANCEID by my input $.detail.instance-id, and perform the following command:
"runuser -l ec2-user -c "aws s3 cp s3://my-bucket/i-11223344556677889/myfile.xyz /home/ec2-user/myfolder/myfile.xyz""
Is there a way? I already tried to use the Fn::join withou success.
Thank you in advance,
kind regards,
Winner

It was necessary to use State.Format inside the State.Array so the it worked, and State.Format inside the State.Array cannot have quotes:
"CloudWatchOutputConfig": {
"CloudWatchLogGroupName": "myloggroup",
"CloudWatchOutputEnabled": true
},
"DocumentName": "AWS-RunShellScript",
"DocumentVersion": "$DEFAULT",
"InstanceIds.$": "States.Array($)",
"MaxConcurrency": "50",
"MaxErrors": "10",
"Parameters": {
"commands.$": "States.Array(States.Format('runuser -l ec2-user -c \"aws s3 cp s3://my-bucket/**MY_INSTANCEID**/myfile.xyz /home/ec2-user/myfolder/myfile.xyz\"', $))"
},
"TimeoutSeconds": 6000
}```
Was also necessary to use .$ after command.

AWS CLI events put-targets EcsParameters (structure)

I am trying to get the current Task Definition of an ECS Cluster then update the revision in the cloud bridge event target.
This is what I have so far:
james#LAPTOP:/mnt/c/Users/james$ target_id="xxx_hourly_cron"
james#LAPTOP:/mnt/c/Users/james$ aws events list-targets-by-rule --rule `echo $target_id` > rule-target.json
james#LAPTOP:/mnt/c/Users/james$ cat rule-target.json
{
"Targets": [
{
"Id": "xxx_hourly_cron",
"Arn": "arn:aws:ecs:eu-west-2:000000000000:cluster/xxx-cluster",
"RoleArn": "arn:aws:iam::000000000000:role/ecsEventsRole",
"EcsParameters": {
"TaskDefinitionArn": "arn:aws:ecs:eu-west-2:000000000000:task-definition/xxx-cron:55",
"TaskCount": 1,
"EnableECSManagedTags": false,
"EnableExecuteCommand": false,
"PropagateTags": "TASK_DEFINITION"
}
}
]
}
james#LAPTOP:/mnt/c/Users/james$ aws events put-targets --rule `echo $target_id` --targets --EcsParameters jsonfile?
The last command is where I am struggling:
within the AWS docs I am not to sure what it means by structure I have tried json and I have tried to escape it.
Here is the docs I am looking at:
https://docs.aws.amazon.com/cli/latest/reference/events/put-targets.html

With AWS CLI commands, you can often replace the majority of the arguments supplied to a command with a single JSON file using the --cli-input-json argument. This can make it far easier to work with complex structures as input arguments to CLI commands.
In the above example, you would modify the rule-target.json output to become an input (rule-target-input.json) for the next using something like the following:
{
"Rule": "xxx_hourly_cron",
"Targets": [
{
"Id": "xxx_hourly_cron",
"Arn": "arn:aws:ecs:eu-west-2:000000000000:cluster/xxx-cluster",
"RoleArn": "arn:aws:iam::000000000000:role/ecsEventsRole",
"EcsParameters": {
"TaskDefinitionArn": "arn:aws:ecs:eu-west-2:000000000000:task-definition/xxx-cron:55",
"TaskCount": 1,
"EnableECSManagedTags": false,
"EnableExecuteCommand": false,
"PropagateTags": "TASK_DEFINITION"
}
}
]
}
And then feeding that into the input using something like the following:
aws put-targets --cli-input-json file://rule-target-input.json

Retrieve one (last) value from influxdb

I'm trying to retrieve the last value inserted into a table in influxdb. What I need to do is then post it to another system via HTTP.
I'd like to do all this in a bash script, but I'm open to Python also.
$ curl -sG 'https://influx.server:8086/query' --data-urlencode "db=iotaWatt" --data-urlencode "q=SELECT LAST(\"value\") FROM \"grid\" ORDER BY time DESC" | jq -r
{
"results": [
{
"statement_id": 0,
"series": [
{
"name": "grid",
"columns": [
"time",
"last"
],
"values": [
[
"2018-01-17T04:15:30Z",
690.1
]
]
}
]
}
]
}
What I'm struggling with is getting this value into a clean format I can use. I don't really want to use sed, and I've tried jq but it complains the data is a string and not an index:
jq: error (at <stdin>:1): Cannot index array with string "series"
Anyone have a good suggestion?

Pipe that curl to the jq below
$ your_curl_stuff_here | jq '.results[].series[]|.name,.values[0][]'
"grid"
"2018-01-17T04:15:30Z"
690.1
The results could be stored into a bash array and used later.
$ results=( $(your_curl_stuff_here | jq '.results[].series[]|.name,.values[0][]') )
$ echo "${results[#]}"
"grid" "2018-01-17T04:15:30Z" 690.1
# Individual values could be accessed using "${results[0]}" and so, mind quotes
All good :-)

Given the JSON shown, the jq query:
.results[].series[].values[]
produces:
[
"2018-01-17T04:15:30Z",
690.1
]
This seems to be the output you want, but from the point of view of someone who is not familiar with influxdb, the requirements seem very opaque, so you might want to consider a variant, such as:
.results[-1].series[-1].values[-1]
which in this case produces the same result, as it happens.
If you just want the atomic values, you could simply append [] to either of the queries above.

Store JSON directly in bash script with variables?

I'm going to preface by saying that "no, find a different way to do it" is an acceptable answer here.
Is there a reliable way to store a short bit of JSON in a bash variable for use in a AWS CLI command running from the same script?
I'll be running a job from Jenkins that's updating an AWS Route53 record, which requires UPSERTing a JSON file with the change in records. Because it's running from Jenkins, there's no local storage where I can keep this file, and I'd really like to avoid needing to do a git checkout every time this project will run (which will be once an hour).
Ideally, storing the data in a variable ($foo) and calling it as part of the change-resource-record-sets command would be most convenient given the Jenkins setup, but I'm unfamiliar with exactly how to quote/store JSON inside bash - can it be done safely?
The specific JSON in this case is the following;
{"Comment":"Update DNSName.","Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"alex.","Type":"A","AliasTarget":{"HostedZoneId":"######","DNSName":"$bar","EvaluateTargetHealth":false}}}]}
As an added complication the DNSName value - $bar - needs to be expanded.

You could use a here-doc:
foo=$(cat <<EOF
{"Comment":"Update DNSName.","Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"alex.","Type":"A","AliasTarget":{"HostedZoneId":"######","DNSName":"$bar","EvaluateTargetHealth":false}}}]}
EOF
)
By leaving EOF in the first line unquoted, the contents of the here-doc will be subject to parameter expansion, so your $bar expands to whatever you put in there.
If you can have linebreaks in your JSON, you can make it a little more readable:
foo=$(cat <<EOF
{
"Comment": "Update DNSName.",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "alex.",
"Type": "A",
"AliasTarget": {
"HostedZoneId": "######",
"DNSName": "$bar",
"EvaluateTargetHealth": false
}
}
}
]
}
EOF
)
or even (first indent on each line must be a tab)
foo=$(cat <<-EOF
{
"Comment": "Update DNSName.",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "alex.",
"Type": "A",
"AliasTarget": {
"HostedZoneId": "######",
"DNSName": "$bar",
"EvaluateTargetHealth": false
}
}
}
]
}
EOF
)
and to show how that is stored, including quoting (assuming that bar=baz):
$ declare -p foo
declare -- foo="{
\"Comment\": \"Update DNSName.\",
\"Changes\": [
{
\"Action\": \"UPSERT\",
\"ResourceRecordSet\": {
\"Name\": \"alex.\",
\"Type\": \"A\",
\"AliasTarget\": {
\"HostedZoneId\": \"######\",
\"DNSName\": \"baz\",
\"EvaluateTargetHealth\": false
}
}
}
]
}"
Because this expands some shell metacharacters, you could run into trouble if your JSON contains something like `, so alternatively, you could assign directly, but be careful about quoting around $bar:
foo='{"Comment":"Update DNSName.","Changes":[{"Action":"UPSERT","ResourceRecordSet":{"Name":"alex.","Type":"A","AliasTarget":{"HostedZoneId":"######","DNSName":"'"$bar"'","EvaluateTargetHealth":false}}}]}'
Notice the quoting for $bar: it's
"'"$bar"'"
│││ │││
│││ ││└ literal double quote
│││ │└ opening syntactical single quote
│││ └ closing syntactical double quote
││└ opening syntactical double quote
│└ closing syntactical single quote
└ literal double quote

It can be stored safely; generating it is a different matter, since the contents of $bar may need to be encoded. Let a tool like jq handle creating the JSON.
var=$(jq -n --arg b "$bar" '{
Comment: "Update DNSName.",
Changes: [
{
Action: "UPSERT",
ResourceRecordSet: {
Name: "alex.",
Type: "A",
AliasTarget: {
HostedZoneId: "######",
DNSName: $b,
EvaluateTargetHealth: false
}
}
}
]
}')

How to extract the value of "name" from this puppet metadata.json without jq in shell?

For some extreme reason, I can't use jq or other cli tool. I need to extract the value of "name" from any json matching this puppet metadata.json. format.
the json might not be properly formatted and indented but will be valid. Meaning, white spaces, and line breaks, carriage backs might be inserted in eligible places.
Note that there could be "name" elements in dependencies array.
So, how to extract the value only using standard unix commands and/or shell script without installing any application like jq or other tools?
Thank you!!
{
"name": "examplecorp-mymodule",
"version": "0.0.1",
"author": "Pat",
"license": "Apache-2.0",
"summary": "A module for a thing",
"source": "https://github.com/examplecorp/examplecorp-mymodule",
"project_page": "https://forge.puppetlabs.com/examplecorp/mymodule",
"issues_url": "https://github.com/examplecorp/examplecorp-mymodule/issues",
"tags": ["things", "stuff"],
"operatingsystem_support": [
{
"operatingsystem":"RedHat",
"operatingsystemrelease":[ "5.0", "6.0" ]
},
{
"operatingsystem": "Ubuntu",
"operatingsystemrelease": [ "12.04", "10.04" ]
}
],
"dependencies": [
{ "name": "puppetlabs/stdlib", "version_requirement": ">=3.2.0 <5.0.0" },
{ "name": "puppetlabs/firewall", "version_requirement": ">= 0.0.4" }
]
}

It's ugly, awful, horrible, not structure-aware and will give you incorrect results if you have extra contents in your input file that look similar to what you're trying to find -- but...
#!/bin/bash
# ^- NOT /bin/sh; shell-native regexes are a bash extension
contents=$(<in.json)
if [[ $contents =~ '"name":'[[:space:]]*'"'([^\"]*)'"' ]]; then
echo "Found name: ${BASH_REMATCH[1]}"
fi
Now, let's talk about some of the ways this answer is broken (and using jq would be better):
It finds the first name, even if it's not one at an outer nesting layer. That is to say, if "dependencies": [ { "name": "puppetlabs/stdlib", "version_requirement": ">=3.2.0 <5.0.0" } ] comes before "name": "examplecorp-mymodule", guess which result is being found? (The easy workarounds to this would involve making assumptions about whitespace/formatting, and are thus not proof against all possible JSON expressions of the same data).
It won't unescape contents inside your name that require, well, unescaping (think about names containing symbols encoded as &foo;).
It isn't multibyte-character aware, and thus isn't guaranteed to emit output that aligns on codepoint boundaries.
If you have a name with an escaped \" subsequence... well, guess what happens there?
Etc. It's not quite as awful as trying to parse XML with regular expressions (JSON is easier!), but it's still quite a mess.

This should work for you:
jq '.name' metadata.json