Regexp match the output , while it shouldn't - tcl

my output looks like this :
+------+--------------+---------+------------+-----------------------+
| id | IP address | status | type | created at |
+------+--------------+---------+------------+-----------------------+
| sc-1 | | running | r4.2 | Aug 21, 2017 08:09:44 |
| sc-2 | 164.54.39.30 | running | r4.2 | Aug 21, 2017 08:09:44 |
+------+--------------+---------+------------+-----------------------+
I need to check if sc-1and sc2 are "runnig".
My solution for the first row is following:
proc check_if_exist_in_output{lines
String_to_Check} {
set err_msg ""
set Flag "False"
foreach line $lines { if { $line eq $String_to_Check} {
set Flag "True"
return $Flag }}
if {$Flag == "False"} {return "Error"}
Now this is working fine, but the thing is, that I may have this IP also in the first line , and then my script doesn't work .
So I tries a solution with REGEXP , I want to provide a line from the output, and to check if it contains the line I am looking for.
long story short :
if "| sc-2 | | running |" is part of line " | sc-2 | 164.54.39.30 | running | "
The answer should be true.
my new proc with REXEXP looks like this :
proc check_if_exist_in_output_with_reg_exspression {lines Str} {
set Flag "False"
foreach line $lines {if {[regexp "$line.*" $Str] == 1}{
set Flag "True"
return $Flag }}
if {$Flag == "False"} {
#FAil step
"Retuen Error"
}
}
#calling the proc:
set lines [split $OutPut "\n"]
set expected_output "| sc-2 | | running |"
set Result [check_if_exist_in_output_reg $lines $expected_output]
But the above proc always returns TRUE, doesn't meter what I send it.
While I expect to get False in case the line doesn't really exist.
I also though about sending the expected result as something like :
set expected_output "| sc-2 |[regexp {(?:\d+\.){3}\d+}]| running |"
But I am not sure how to write it in the correct way.

as was suggested by jasonmclose (I am not sure on how to tag here) .
I 've changed the regexp to amore simple one . and the proc is working just fine like this :
proc check_if_exist_in_output_with_reg_exspression {lines String_to_Check} {
set err_msg ""
set Flag "False"
foreach line $lines { if {[regexp ^.*sc-2.*running.* $String_to_Check ]==1} {
set Flag "True"
return $Flag }}
if {$Flag == "False"} {
#Fail step
return "Error"
}
}

This is not an answer to the OP's question on getting the regexp right, but it is worth mentioning that the whole fuzz can also be tackled without a regexp. You may want to consider sth. along the lines of:
set data {
+------+--------------+---------+------------+-----------------------+
| id | IP address | status | type | created at |
+------+--------------+---------+------------+-----------------------+
| sc-1 | | running | r4.2 | Aug 21, 2017 08:09:44 |
| sc-2 | 164.54.39.30 | running | r4.2 | Aug 21, 2017 08:09:44 |
+------+--------------+---------+------------+-----------------------+
}
set status [dict create]
foreach line [lrange [split [string trim $data] "\n"] 3 end-1] {
set line [lmap el [split [string trim $line |] |] {string trim $el}]
dict set status [lindex $line 0] [lindex $line 2]
}
puts [dict get $status "sc-2"]

set Output {+------+--------------+---------+------------+-----------------------+
| id | IP address | status | type | created at |
+------+--------------+---------+------------+-----------------------+
| sc-1 | | running | r4.2 | Aug 21, 2017 08:09:44 |
| sc-2 | 164.54.39.30 | running | r4.2 | Aug 21, 2017 08:09:44 |
+------+--------------+---------+------------+-----------------------+};
proc check_if_exist_in_output_reg {lines Str} {
set Flag "False"
if {[regexp -linestop $Str $lines a] ==1} { ; #-linestop for matching whithin a line, a for just storing match in a variable a
#puts $a
set Flag "True"
return $Flag
} else {
return "Error"
}
}
#calling the proc:
#set lines [split $Output "\n"] : not needed
set expected_output {sc-1.*running} ; #brackets for literal substitution
set Result [check_if_exist_in_output_reg $Output $expected_output]
puts $Result

Related

JQ - Groupby and concatenate text objects

Not quite getting it. I can produce multiple lines but cannot get multiple entries to combine. Looking to take Source JSON and output to CSV as shown:
Source JSON:
[{"State": "NewYork","Drivers": [
{"Car": "Jetta","Users": [{"Name": "Steve","Details": {"Location": "Home","Time": "9a-7p"}}]},
{"Car": "Jetta","Users": [{"Name": "Roger","Details": {"Location": "Office","Time": "3p-6p"}}]},
{"Car": "Ford","Users": [{"Name": "John","Details": {"Location": "Home","Time": "12p-5p"}}]}
]}]
Desired CSV:
"NewYork","Jetta","Steve;Roger","Home;Office","9a-7p;3p-6p"
"NewYork","Ford","John","Home","12p-5p"
JQ code that does not work:
.\[\] | .Drivers\[\] | .Car as $car |
.Users\[\] |
\[$car, .Name\] | #csv
You're looking for something like this:
.[] | [.State] + (
.Drivers | group_by(.Car)[] | [.[0].Car] + (
map(.Users) | add | [
map(.Name),
map(.Details.Location),
map(.Details.Time)
] | map(join(";"))
)
) | #csv
$ jq -r -f tst.jq file
"NewYork","Ford","John","Home","12p-5p"
"NewYork","Jetta","Steve;Roger","Home;Office","9a-7p;3p-6p"
$
Not quite optimised, but I though't I'd share the general idea:
jq -r 'map(.State as $s |
(.Drivers | group_by(.Car))[]
| [
$s,
(map(.Users[].Name) | join(";")),
(map(.Users[].Details.Location) | join(";")),
(map(.Users[].Details.Time) | join(";"))
])
[] | #csv' b
map() over each state, remember the name (map(.State as $s | )
group_by(.Car)
Create an array containing all your fields that is passed to #csv
Use map() and join() to create the fields for Name, Location and Time
This part could be improved so you don't need that duplicated part
Output (with --raw-output:
"NewYork","John","Home","12p-5p"
"NewYork","Steve;Roger","Home;Office","9a-7p;3p-6p"
JqPlay seems down, so I'm still searching for an other way of sharing a public demo
Far from perfect, but it builds the result incrementally so it should be easily debuggable and extensible:
map({State} + (.Drivers[] | {Car} + (.Users[] | {Name} + (.Details | {Location, Time}))))
| group_by(.Car)
| map(reduce .[] as $item (
{State:null,Car:null,Name:[],Location:[],Time:[]};
. + ($item | {State,Car}) | .Name += [$item.Name] | .Location += [$item.Location] | .Time += [$item.Time]))
| .[]
| [.State, .Car, (.Name,.Location,.Time|join(","))]
| #csv

Format number in thousands separators with jq json cli

Given {"a": 1234567890}, I want 1,234,567,890 in the result, how this can be done with jq
echo '{"a": 1234567890}' | jq '.a | FORMAT?'
Thanks for #peak's answer, the solution is
echo '{"a": 1234567890}' | jq -r 'def h: [while(length>0; .[:-3]) | .[-3:]] | reverse | join(","); .a | tostring | h'
//-> 1,234,567,890
Here's an idiomatic one-liner definition:
def h: tostring | [while(length>0; .[:-3]) | .[-3:]] | reverse | join(",");
Example
12, 123, 1234, 12345678 | h
Output (using -r option):
12
123
1,234
12,345,678
jq doesn't have (yet) a printf function to format according locale settings.
If that's an option for you can pass the number to the shell using printf:
echo '{"a": 12345}' | jq '.a' | xargs printf "%'.f\n"
12,345
Note that the printf conversion relies on the format %'.f that is explained in man 3 printf
Here's a generic solution for integers or integer-valued strings:
# "h" for "human-readable"
def h:
def hh: .[0] as $s | .[1] as $answer
| if ($s|length) == 0 then $answer
else ((if $answer == "" then "" else "," end) + $answer ) as $a
| [$s[0:-3], $s[-3:] + $a] | hh
end;
[ tostring, ""] | hh;
Example
12, 123, 1234, 12345678 | h
Result (using -r option):
12
123
1,234
12,345,678

table creation from CSV file with headers using awk

I have a comma separated CSV file with headers and want to include them in the table
Input:
header,word1,word2,word3
supercalifragi,black,white,red
adc,bad,cat,love
Output:
| header | word1 | word2 | word3 |
| -------------- | ----- | ----- | ----- |
| supercalifragi | black | white | red |
| adc | bad | cat | love |
I need to include the headers and I need to take into account the length of the words in the input file so that the finished table formats correctly
Here is the updated code:
function pr(){
for(i=1;i<=NF;i++)
printf "| %-"len[i]+1"s",$i;
printf "|\n"
}
NR==FNR{
for(i=1;i<=NF;i++)
if(len[i]<length($i)){
len[i]=length($i);
word[i]=$i
}next
}{pr()}
FNR==1{
for(i=1;i<=NF;i++){
gsub(/./,"-",word[i]);
$i=word[i]};
pr()
}
``
I took the freedom of rewriting the entire code from scratches. This should work:
BEGIN {
FS=","
OFS=" | "
for (i=1; i<=NF; i++) {
transientLength[i] = 0
}
}
{
if(NR==1) {
# read headers
for (i=0; i<NF; i++) {
headers[i] = $(i+1)
transientLength[i] = (length($(i+1))>=transientLength[i] ? length($(i+1)) : transientLength[i])
}
} else {
for (i=0; i<NF; i++) {
fields[NR][i] = $(i+1)
transientLength[i] = (length($(i+1))>=transientLength[i] ? length($(i+1)) : transientLength[i])
}
}
}
END {
# print header
for (j in headers) {
spaceLength = transientLength[j]-length(headers[j])
for (s=1;s<=spaceLength;s++) {
spaces = spaces" "
}
if (!printable) printable = headers[j] spaces
else printable = printable OFS headers[j] spaces
spaces = "" # garbage collection
}
printable = "| "printable" |"
print printable
printable = "" # garbage collection
# print alignments
for (j in transientLength) {
for (i=1;i<=transientLength[j];i++) {
sep = sep"-"
}
if (!printable) printable = sep
else printable = printable OFS sep
sep = "" # garbage collection
}
printable = "| "printable" |"
print printable
printable = "" # garbage collection
# print all rows
for (f in fields) {
for (j in fields[f]) {
spaceLength = transientLength[j]-length(fields[f][j])
for (s=1;s<=spaceLength;s++) {
spaces = spaces" "
}
if (!printable) printable = fields[f][j] spaces
else printable = printable OFS fields[f][j] spaces
spaces = "" # garbage collection
}
printable = "| "printable" |"
print printable
printable = "" # garbage collection
}
}
But please be aware: you need to clean your input file of unnecessary whitespaces. It should read:
header,word1,word2,word3
supercalifragi,black,white,red
adc,bad,cat,love
Alternatively, you might use FS=", ", but that would be actually limited to your example.
a shorter alternative with double scanning
$ awk -F' *, *' 'function pr()
{for(i=1;i<=NF;i++) printf "| %-"len[i]+1"s",$i; printf "|\n"}
NR==FNR{for(i=1;i<=NF;i++)
if(len[i]<length($i)) {len[i]=length($i); word[i]=$i} next}
{pr()}
FNR==1{for(i=1;i<=NF;i++) {gsub(/./,"-",word[i]); $i=word[i]}; pr()}' file{,}
| header | word1 | word2 | word3 |
| -------------- | ----- | ----- | ----- |
| supercalifragi | black | white | red |
| adc | bad | cat | love |
It's not exactly the output you asked for but maybe this is all you really need:
$ column -t -s, -o' | ' < file | awk '1; NR==1{gsub(/[^|]/,"-"); print}'
header | word1 | word2 | word3
---------------|-------|-------|------
supercalifragi | black | white | red
adc | bad | cat | love

Loop in database using perl

I need to extract each character from the input (input are numbers) and check it against my database, if the number is present the corresponding rows will be printed. But in my code the loop doesn't work and only the first character is printed.
use dbi;
my $seq=<stdin>;
my $r=my$seq;
my $db="hnf1a";
my $user="root";
my $password="";
my $host="localhost";
my $dbh = DBI->connect("DBI:mysql:database=$db:$host",$user,$password);
my #w= split(//, $r);
print #w;
foreach my $b(#w)
{
my $sth=$dbh -> prepare("select an,ano from mody having ano = '$b' ");
my $rv=$sth->execute();
while (my #row =$sth->fetchrow_array())
{
print #row;
}
}
my $rc=my $sth->finish;
}
print "database closed";`
Database:
mysql> select * from mody;
+----+---------+------+
| id | an | ano |
+----+---------+------+
| 1 | 123 | 456 |
| 2 | abc | 567 |
| 3 | hello | 5 |
| 4 | world | 5 |
| 5 | goodbye | 6 |
+----+---------+------+
5 rows in set (0.00 sec)
Code:
use strict;
use warnings;
use 5.020;
use autodie;
use Data::Dumper;
use DBI;
my $dsn = 'dbi:MariaDB:database=my_db;host=localhost';
my $dbh = DBI->connect($dsn, 'root', '');
my $sth = $dbh->prepare('SELECT an,ano FROM mody where ano = ?');
my $input = "56";
my #numbers = split //, $input;
for my $number(#numbers) {
say "rows matching input <$number>:";
$sth->execute($number);
while(my #data = $sth->fetchrow_array) {
say "\t#data";
}
};
Output:
rows matching input <5>:
hello 5
world 5
rows matching input <6>:
goodbye 6

Unable to fix the logic of the bash script

I have a table (say UserInputDetails) with the following entries:
+------------+-----------+----------+
| screenId | userInput | numInput |
+------------+-----------+----------+
| 13_1_2_1 | 2 | 9 |
| 13_1_2_2 | 2 | 9 |
| 13_1_2_2 | 3 | 2 |
| 13_1_2_2 | 9 | 2 |
| 13_1_2_2_2 | 3 | 3 |
| 13_1_2_2_2 | 5 | 2 |
| 13_2_2_2 | 4 | 4 |
| 13_2_2_2 | 5 | 4 |
| 13_2_2_2 | 7 | 2 |
+------------+-----------+----------+
I need to write a shell script which gives its expected output as:
13_1_2_1,0,0,9,0,0,0,0,0,0,0
13_1_2_2,0,0,9,2,0,0,0,0,0,2
13_1_2_2_2,0,0,0,3,0,2,0,0,0,0
13_2_2_2,0,0,0,0,4,4,0,2,0,0
Explanation for the output:
the first line of input denotes the numInputs for a particular userInput for screenId '13_1_2_1'. The line first prints the screenId and then corresponding NumInput for userInput 0-9. Since the numInput for userInput '2' is 9 and for the rest of 0-9 is 0, it gives the value 13_1_2_1,0,0,9,0,0,0,0,0,0,0
The bash script written for the following function is:
#!/bin/bash
MYSQL="mysql -uroot -proot -N Database1"
yesterday=""
if [ $# -ge 1 ]
then
yesterday="$1"
else
yesterday=`$MYSQL -sBe "select date_sub(date(now()), interval 1 day);"`
fi
echo "DATE: $yesterday"
PREVSCREENID=''
SCREENID=
ABC=tempSqlDataFile
$MYSQL -sBe "select screenId, userInput, numInput from userInputDetails group by screenID, userInput" > $ABC
for i in {0..9}
do
arr[$i]='0'
done
while read line
do
SCREENID=`echo $line | awk '{ print $1 }'`
i=`echo $line | awk '{print $2 }'`
arr[$i]=`echo $line | awk '{print $3}'`
if [[ $SCREENID != $PREVSCREENID ]]
then
echo "$SCREENID ${arr[*]}" | tr ' ' ','
for i in {0..9}
do
arr[$i]='0'
done
else
i=`echo $line | awk '{print $2 + 1}'`
arr[$i]=`echo $line | awk '{print $3}'`
fi
PREVSCREENID=$SCREENID
done < $ABC
The logic somewhere is going wrong and I am unable to get the logic right. the output from the above shell script is:
13_1_2_1,0,0,9,0,0,0,0,0,0,0,
13_1_2_2,0,0,9,0,0,0,0,0,0,0,
13_1_2_2_2,0,0,9,3,0,0,0,0,0,2,
13_2_2_2,0,0,0,3,4,2,0,0,0,0,
Please can you help me fix the logic in my script? Also, since I am new to scripting and programming, this may not be an efficient way to perform this task. Please suggest if there is an efficient way.
There are a number of errors in your script. Here's a rewrite of the latter part:
while read SCREENID i n; do
if [[ "$SCREENID" != "$PREVSCREENID" ]]; then
[ "$PREVSCREENID" ] && echo "$PREVSCREENID ${arr[*]}" | tr ' ' ,
for j in {0..9}; do arr[$j]=0; done
fi
arr[$i]="$n"
PREVSCREENID="$SCREENID"
done < "$ABC"
echo "$PREVSCREENID ${arr[*]}" | tr ' ' ,
You can avoid calling tr like this:
print_arr() { IFS=,; echo $PREVSCREENID,"${arr[*]}"; unset IFS; }
while read SCREENID i n; do
if [[ "$SCREENID" != "$PREVSCREENID" ]]; then
[ "$PREVSCREENID" ] && print_arr
for j in {0..9}; do arr[$j]=0; done
fi
arr[$i]="$n"
PREVSCREENID="$SCREENID"
done < "$ABC"
print_arr