Convert Multiline String (utmpdump results) into JSON [closed] - json

Question

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 months ago.
Improve this question
All,
This is my first time submitting a stack overflow question, so thanks in advance for taking the time to read/consider my question. I'm currently using the 'utmpdump' utility to dump linux authentication log results each hour from a bash script, which is done using the syntax shown below:
dateLastHour=$(date +"%a %b %d %H:" -d '1 hour ago')
dateNow=$(date +"%a %b %d %H:")
utmpdump /var/log/wtmp* | awk "/$dateLastHour/,/$dateNow/"
What I'm now trying to accomplish and the subject of this question is how can I take these results and delimited them by new line for each authentication log, before converting each authentication event into it's own JSON file to be exported to an external syslog collector for additional analysis and long term storage?
As an example, here's some of the test results I've been using:
[7] [08579] [ts/0] [egecko] [pts/0 ] [10.0.2.6 ] [1.1.1.1 ] [Fri Nov 04 23:40:29 2022 EDT]
[8] [08579] [ ] [ ] [pts/0 ] [ ] [0.0.0.0 ] [Fri Nov 04 23:55:16 2022 EDT]
[2] [00000] [~~ ] [reboot ] [~ ] [3.10.0-1160.80.1.el7.x86_64] [0.0.0.0 ] [Sat Dec 03 12:28:05 2022 EST]
[5] [00811] [tty1] [ ] [tty1 ] [ ] [0.0.0.0 ] [Sat Dec 03 12:28:12 2022 EST]
[6] [00811] [tty1] [LOGIN ] [tty1 ] [ ] [0.0.0.0 ] [Sat Dec 03 12:28:12 2022 EST]
[1] [00051] [~~ ] [runlevel] [~ ] [3.10.0-1160.80.1.el7.x86_64] [0.0.0.0 ] [Sat Dec 03 12:28:58 2022 EST]
[7] [02118] [ts/0] [egecko] [pts/0 ] [1.1.1.1 ] [1.1.1.1 ] [Sat Dec 03 12:51:22 2022 EST]
Any assistance or pointers here is greatly appreciated!
I've been using the following SED commands to trim out unnessecary whitespace, and I know that what I probably should do is using IDF to split the results string into new lines before using brackets as the delimeter:
utmpResults=$(echo "$utmpResults" | sed 's/ */ /g')
IFS="\n" read -a array <<< "$utmpResults"
echo $array
But when I echo $array it only returns the first line...?

Answer 1

With the help of jq (sed for json), it's an easy task:
#!/bin/bash
jq -R -c '
select(length > 0) | # remove empty lines
[match("\\[(.*?)\\]"; "g").captures[].string # find content within square brackets
| sub("^\\s+";"") | sub("\\s+$";"")] # trim content
| { # convert to json object
"type" : .[0],
"pid" : .[1],
"terminal_name_suffix" : .[2],
"user" : .[3],
"tty" : .[4],
"remote_hostname" : .[5],
"remote_host" : .[6],
"datetime" : .[7],
"timestamp" : (.[7] | strptime("%a %b %d %T %Y %Z") | mktime)
}' input.txt
Output
{"type":"7","pid":"08579","terminal_name_suffix":"ts/0","user":"egecko","tty":"pts/0","remote_hostname":"10.0.2.6","remote_host":"1.1.1.1","datetime":"Fri Nov 04 23:40:29 2022 EDT","timestamp":1667605229}
{"type":"8","pid":"08579","terminal_name_suffix":"","user":"","tty":"pts/0","remote_hostname":"","remote_host":"0.0.0.0","datetime":"Fri Nov 04 23:55:16 2022 EDT","timestamp":1667606116}
{"type":"2","pid":"00000","terminal_name_suffix":"~~","user":"reboot","tty":"~","remote_hostname":"3.10.0-1160.80.1.el7.x86_64","remote_host":"0.0.0.0","datetime":"Sat Dec 03 12:28:05 2022 EST","timestamp":1670070485}
{"type":"5","pid":"00811","terminal_name_suffix":"tty1","user":"","tty":"tty1","remote_hostname":"","remote_host":"0.0.0.0","datetime":"Sat Dec 03 12:28:12 2022 EST","timestamp":1670070492}
{"type":"6","pid":"00811","terminal_name_suffix":"tty1","user":"LOGIN","tty":"tty1","remote_hostname":"","remote_host":"0.0.0.0","datetime":"Sat Dec 03 12:28:12 2022 EST","timestamp":1670070492}
{"type":"1","pid":"00051","terminal_name_suffix":"~~","user":"runlevel","tty":"~","remote_hostname":"3.10.0-1160.80.1.el7.x86_64","remote_host":"0.0.0.0","datetime":"Sat Dec 03 12:28:58 2022 EST","timestamp":1670070538}
{"type":"7","pid":"02118","terminal_name_suffix":"ts/0","user":"egecko","tty":"pts/0","remote_hostname":"1.1.1.1","remote_host":"1.1.1.1","datetime":"Sat Dec 03 12:51:22 2022 EST","timestamp":1670071882}
Without the option -c you can create formatted output.
To save each line in a file, you can do it like this in bash.
I have chosen the timestamp as the file name.
INPUT_AS_JSON_LINES=$(
jq -R -c '
select(length > 0) | # remove empty lines
[match("\\[(.*?)\\]"; "g").captures[].string # find content within square brackets
| sub("^\\s+";"") | sub("\\s+$";"")] # trim content
| { # convert to json object
"type" : .[0],
"pid" : .[1],
"terminal_name_suffix" : .[2],
"user" : .[3],
"tty" : .[4],
"remote_hostname" : .[5],
"remote_host" : .[6],
"datetime" : .[7],
"timestamp" : (.[7] | strptime("%a %b %d %T %Y %Z") | mktime)
}' input.txt
)
while read line
do
FILENAME="$(jq '.timestamp' <<< "$line").json"
CONTENT=$(jq <<< "$line") # format json
echo "writing file '$FILENAME'"
echo "$CONTENT" > "$FILENAME"
done <<< "$INPUT_AS_JSON_LINES"
Output
writing file '1667605229.json'
writing file '1667606116.json'
writing file '1670070485.json'
writing file '1670070492.json'
writing file '1670070492.json'
writing file '1670070538.json'
writing file '1670071882.json'