I want to get the number of commits per day per hour. With the following command, I am able to get the output in json format. However, I would like to know if I can add the key to the values in json format using command line?
curl https://api.github.com/repos/test/myrepo/stats/punch_card
Current output:
[
0,
2,
32
]
Expected output:
[
day: 0,
hour: 2,
commits: 32
]
Since you haven't specified anything beyond "command line", I'm assuming you want a bash-based solution. This simple (though kind of ugly) script will do what you want, while maintaining indentation (apart from the closing square bracket of the overall response):
#!/bin/bash
resp=$(curl https://api.github.com/repos/test/myrepo/stats/punch_card)
nextPref=""
for val in $resp
do
echo "$nextPref $val"
if [[ $val == "[" && $nextPref == "" ]]
then
nextPref=" "
elif [[ $val == "[" && $nextPref == " " ]]
then
nextPref=" day:"
elif [[ $nextPref == " day:" ]]
then
nextPref=" hour:"
elif [[ $nextPref == " hour:" ]]
then
nextPref=" commits:"
elif [[ $nextPref == " commits:" ]]
then
nextPref=" "
fi
done
Related
#!/bin/bash
doc1=docum1
doc2=docum2
doc3=
pack1=
pack2=package2
pack3=
if [ -n $doc1 ] || [ -n $pack1 ]; then
echo "doc1 is $doc1"
echo "pack1 is $pack1"
else
echo "doc1 has empty value"
fi
enter image description here
How can I write a function if I have a bigger number of variables?
If I understand your question and you want to be able to coordinate many different document and package pairs, then one approach would be to use two indexed arrays, one holding the documents and the other holding packages and you coordinate the documents and packages through a common index.
For example the first index applies to the first document and first package, the second to the next document and next package and so on and so forth. (noting that arrays in bash are zero-indexed, so actually index 0 corresponds to document_1 and package_1)
Bash arrays can hold empty-values. So initializing the doc array as:
doc=( docum1 docum2 "" )
Results in a 3-element array with the 3rd element empty. And for your pac array:
pac=( "" package2 "" )
Where only the 2nd element has a value.
Putting it together with logic that checks if (1) both have values; (2) neither have a value; (3) only document has a value; and finally (4) only package has a value, you could do:
#!/bin/bash
doc=( docum1 docum2 "" ) ## index arrays for doc and pac
pac=( "" package2 "" )
docn=${#doc[#]} ## numer of elements in doc array
## validate it is equal to number of elements in pac
if [ "$docn" -ne "${#pac[#]}" ]; then
printf "error: unequal elements in 'doc' and 'pac' arrays\n" >&2
exit 1
fi
## loop over each index
for ((i=0; i<docn; i++)); do
## check if both have values
if [ -n "${doc[i]}" ] && [ -n "${pac[i]}" ]; then
printf "doc[%d] : %s\npac[%d] : %s\n" "$((i+1))" "${doc[i]}" "$((i+1))" "${pac[i]}"
## check if neither have values
elif [ -z "${doc[i]}" ] && [ -z "${pac[i]}" ]; then
printf "both doc[%d] and pack[%d] are empty\n" "$((i+1))" "$((i+1))"
## check if doc has value with empty package
elif [ -n "${doc[i]}" ]; then
printf "doc[%d] has empty value\n" "$((i+1))"
## otherwise doc is empty and package has value
else
print "pac[%d] is package without document\n" "$((i+1))"
fi
printf "\n" ## tidy up with newline between output
done
(remember arrays in bash are zero-indexed. So "$((i+1)) is used to print the results as 1, 2, 3, ...)
Bash arrays can easily handle tens or hundreds of thousands of elements. It won't be the fastest script on earth with that many elements, but it won't be bad either.
Example Use/Output
$ bash doc_and_pack.sh
doc[1] has empty value
doc[2] : docum2
pac[2] : package2
both doc[3] and pack[3] are empty
Look things over and let me know if I understood what you needed and if not leave a comment and I'm happy to help further. Let me know if you have further questions.
too simple .
Use this script :
#!/bin/bash
doc1=docum1
doc2=docum2
doc3=
# ...
doc102222=docum102222
pack1=
pack2=package2
pack3=
#...
pack30234324=package30234324
function checkExists() {
v1=doc$1
v2=pack$2
if [ -z ${!v1} ] ; then
echo "$v1 is empty"
elif [ -z ${!v2} ] ; then
echo "$v2 is empty"
else
echo "doc is ${!v1}"
echo "pack is ${!v2}"
fi
}
checkExists 1 1
echo "============================"
checkExists 2 2
echo "============================"
checkExists 102222 30234324
result is :
pack1 is empty
============================
doc is docum2
pack is package2
============================
doc is docum102222
pack is package30234324
I have the below variables in bash
source_FROM_batch_date='2020-06-06'
source_to_batch_date='2020-06-07'
min_batch_date_seq_num=2
max_batch_date_seq_num=3
My data loads run in a batches 1 to 4 and 4 being the max batch.
I want to generate a where clause dynamically based on the variables I have above
batch_date and batch_seq_num are the column I will filter the data on
conditions
1) read all the data where batch_date = '$source_FROM_batch_date' and batch_seq_num >= 'min_batch_date_seq_num'
2) read all the data where batch_date = '$source_to_batch_date' and batch_seq_num <= 'max_batch_date_seq_num'
3) read all the data that occurs between $source_FROM_batch_date and $source_to_batch_date
I have done like below.
#!/bin/bash
run_project_tmp_dir='/home/$USER'
source_FROM_batch_date='2020-06-06'
source_to_batch_date='2020-06-07'
min_batch_date_seq_num=2
max_batch_date_seq_num=3
export min_batch_date=${source_FROM_batch_date}
export max_batch_date=${source_to_batch_date}
export min_batch_date_seq_num=${min_batch_date_seq_num}
export max_batch_date_seq_num=${max_batch_date_seq_num}
####################Generate batch filter#######################
startdate=${min_batch_date}
enddate=${max_batch_date}
d=
n=0
loop_cnt=0
start_seq=${min_batch_date_seq_num}
end_seq=${max_batch_date_seq_num}
max_seq=4
max_seq_num=$((max_seq + 1))
batch_filter_file=${run_project_tmp_dir}/batch_filter_file.txt
if [ -f ${batch_filter_file} ]; then
rm -f ${batch_filter_file}
fi
until [ "$d" = "$enddate" ]
do
d=$(date -d "$startdate + $n days" +%Y-%m-%d)
## Case when only one batch to process
if [[ $d = ${enddate} && ${start_seq} = "${end_seq}" && ${loop_cnt} = 0 ]];then
echo "batch_date='$d' AND batch_seq_num='$start_seq'" >> ${batch_filter_file}
fi
## Case when multiple batches on same batch date
if [[ $d = ${enddate} && ${start_seq} -ne ${end_seq} && ${loop_cnt} = 0 ]];then
until [ "$start_seq" = $((end_seq + 1)) ]
do
echo "(batch_date='$d' AND batch_seq_num='$start_seq') OR " >> ${batch_filter_file}
((start_seq++))
done
fi
if [[ $d != "${enddate}" ]];then
until [ "$start_seq" = "$max_seq_num" ]
do
echo "(batch_date='$d' AND batch_seq_num='$start_seq') OR " >> ${batch_filter_file}
((start_seq++))
done
fi
if [[ $d = "${enddate}" && ${loop_cnt} != 0 ]];then
until [ "$start_seq" = $((end_seq + 1)) ]
do
echo "(batch_date='$d' AND batch_seq_num='$start_seq') OR " >> ${batch_filter_file}
((start_seq++))
done
fi
((n++))
((loop_cnt++))
start_seq=1
done
if [ -f ${batch_filter_file} ]; then
sed -i '$s/OR $//' ${batch_filter_file}
sed -i '1i (' ${batch_filter_file}
echo ")" >> ${batch_filter_file}
fi
output
(
(batch_date='2020-06-06' AND batch_seq_num='2') OR
(batch_date='2020-06-06' AND batch_seq_num='3') OR
(batch_date='2020-06-06' AND batch_seq_num='4') OR
(batch_date='2020-06-07' AND batch_seq_num='1') OR
(batch_date='2020-06-07' AND batch_seq_num='2') OR
(batch_date='2020-06-07' AND batch_seq_num='3')
)
required output
(
(batch_date='2020-06-06' AND batch_seq_num in ('2', '3', '4') OR
(batch_date='2020-06-07' AND batch_seq_num in ('1', '2', '3')
)
How can I achieve what I want
For multi-batch:
wclause="(batch_date=... and batch_seq_num in"
Then inside the loop for each seq:
wclause="${wclause}(${start_seq}" # for first seq
wclause="${wclause},${start_seq}" # rest of seq's
After you exit the loop:
wclause="${wclause}))"
echo "${wclause}"
How do I convert these lists of text strings into json
Text strings:
start filelist:
/download/2017/download_2017.sh
/download/2017/log_download_2017.json
/download/2017/log_download_2017.txt
start wget:
2017-05-15 20:42:00 URL:http://web.site.com/downloads/2017/file_1.zip [1024/1024] -> "file_1.zip" [1]
2017-05-15 20:43:21 URL:http://web.site.com/downloads/2017/file_2.zip [2048/2048] -> "file_2.zip" [1]
JSON output:
{
"start filelist": [
"download_2017.sh",
"log_download_2017.txt",
"log_download_2017.json",
],
}
{
"start wget": [
"2017-05-15 20:43:01 URL:http://web.site.com/downloads/2017/file_1.zip [1024/1024] -> "file_1.zip" [1]",
"2017-05-15 20:43:21 URL:http://web.site.com/downloads/2017/file_2.zip [2048/2048] -> "file_2.zip" [1]",
],
}
Appreciate any options and approaches
Here's a jq-only solution, which produces valid JSON along the lines of your example:
foreach (inputs,null) as $line ({};
if $line == null then .emit = {(.key): .value}
elif $line[-1:] == ":"
then (if .key then {emit: {(.key): .value}} else null end)
+ { key : $line[0:-1] }
else {key, value: (.value + [$line])}
end;
.emit // empty )
Invocation:
jq -n -R -f program.jq input.txt
Please note the -n option in particular.
Caveats
If the input does not begin with a "key" line, then the above jq program will report an error and terminate. If more fault-tolerance is required, then the following variant might be of interest:
foreach (inputs,null) as $line ({};
if $line == null then .emit = {(.key|tostring): .value}
elif $line[-1:] == ":"
then (if .key then {emit: {(.key): .value}} else null end)
+ { key : $line[0:-1] }
else {key, value: (.value + [$line])}
end;
.emit // empty )
Can't seem to understand the output of the following code snippet. trying to print the function return value in a loop
contains () {
local e
for e in "${#:2}"; do [[ "$e" == "$1" ]] && return 0; done
return 1
}
line="ayush"
line2="this is a line containing ayush"
contains $line $line2
echo $? #prints 0
for i in 1 2 3;do
contains "$line" "$line2"
echo $? #prints 1 everytime
done
#Ayush Goel
The Problem is here,
contains () {
local e
for e in "${#:2}"; do [[ "$e" == "$1" ]] && return 0; done
return 1
}
line="ayush"
line2="this is a line containing ayush"
contains $line $line2
echo $? #prints 0
for i in 1 2 3;do
contains $line $line2 # <------------------ ignore ""
echo $? # Now it will print 0
done
Difference between $var and "$var" :
1) $var case
var="this is the line"
for i in $var; do
printf $i
done
here it will print
this is the line
means $var is expanded using space
2)"$var" case
var="this is the line"
for i in "$var"; do
printf $i
done
this will print
this
here "$var" will be considered as a single argument and it will take only one value from the list.
I need to make this script create and then output to an html file data inside template.vars. The script is a template engine, but right now all it does is take in input from the keyboard, echoing out the name in template.vars when the input is #NAME#. Here is the code of the template engine at the moment:
#!/bin/bash
IFS=# #makes # a delimiter.
while read line
do
dataspace=$(awk '$0=$1' FS== <<< "$line")
value=$(awk '$0=$2' FS== <<< "$line")
printf -v $dataspace "$value" #make the value stored in value into the name of a dataspace.
done < 'template.vars' #read template.vars for standard input.
skipflag=false #initialize the skipflag to false
while read line #while it is reading standard input one line at a time
do
read -a dataspacearray <<< "$line" #make the line into an array.
if [ $skipflag == false ] && [ "${dataspacearray[1]}" != "ENDIF" ] ; then
if [[ ${dataspacearray[1]} == "IF "* ]] ; then #If second element of dataspacearray is "IF "(something)
dataspace=`echo ${dataspacearray[1]} | cut -d' ' -f2`
if [ -z "${!dataspace}" ] ; then #if dataspace not found, skip everything up to endif. -z to test string
skipflag=true
fi
else
for ((i=0; i<${#dataspacearray[#]}; i++))
do
dataspace=${dataspacearray[i]} #access to each dataspace in the array
even=`expr $i % 2`
if [ $even == '0' ] ; then #if it's even(f0,f2, f4.. etc. etc) then it's not a variable, so print it directly
if [ -n "${dataspace}" ] ; then
printf ${dataspace}
fi
else #(odd dataspaces(f1, f3... etc.) are variables, print their values if they exist
if [ -n "${!dataspace}" ] ; then
printf ${!dataspace}
else
printf "Error!!!"
fi
fi
done
printf "\n"
fi
else
skipflag=false
fi
done
The html file goes in as input like this:
<html>
<head>
<title>#NAME#</title>
</head>
<body>
<p>Name: #NAME#</p>
<p>Major: #MAJOR#</p>
<p>Classification: #CLASS#</p>
<p>Graduation date: #GDATE#</p>
#IF HONORS#
<p>Graduating with Honors</p>
#ENDIF#
</body>
</html>
With the script replacing all strings beginning and ending with # using the data in the template.vars file (or outputting error!!! if that isn't found). I have seen many instances where templates replace any text that has $ at the beginning, and think that I need to implement something similar, but with it reading the html file as input, and then saving it as output. How do I do this?
This isn't strictly what you asked for, but it's probably close enough to give you something to work with.
Leaving aside the #IF ...# ... #ENDIF# handling (which is a mostly untested afterthought), The logic goes something like this:
For each input line
1) initilise an empty holding space
2) if the line has fewer than two delimiters, goto #10
3) copy from the start of the line up to (but not including) the first delimiter to the hold space
4) remove the first segment of the line (including the delimiter). The leading segment of the line should now be a token.
5) copy the token (up to the delimiter) to a temporary variable
6) remove the token from the line (leave the trailing delimiter in place)
7) look up the token in a list of known tokens.
8) if it's a known token, append the token's expansion to the hold space and remove the delimiter from the line. If the token is unknown, append the delimiter (removed at #4) and the token text to the hold space but leave the non-token's trailing delimiter at the start of the line -- this is to allow (#name###domain#) and multiple runs with different variable definition files (./expstuff -v user.vars < template.in | ./expstuff -v global.vars > outputfile) but will need changing for your requirement.
9) goto #2
10) print hold space and whatever remains of the line.
typeset -A s
i=template.vars
d=#
while IFS='=' read t v ; do
[[ $t != \#* ]] && s[$t]=$v
done < "$i"
while IFS= read -r l ; do
# assuming the #IF blah# and corresponding #ENDIF# occupy entire lines.
if [[ $l == $d[Ee][Nn][Dd][Ii][Ff]$d ]]; then
c=''
continue
fi
[[ $c == '0' ]] && continue
if [[ $l == $d[Ii][Ff][[:space:]]*[![:space:]]*$d ]]; then
read _ t <<< "${l%$d}"
c=0
if [[ -n $t && ${s[$t]+t} == 't' ]]; then
case ${s[$t]} in [Yy]|[Yy][Ee][Ss]|[Tt][Rr][Uu][Ee]|1 ) c=1 ;; esac
fi
continue
fi
# Currently, given
# foo#domain#TLD# wibble
# with TLD=.co.uk
# the following loop outputs
# foo#domain.co.uk wibble
# You appear to require
# fooError!!!TLD#wibble
h=
while [[ $l == *$d*$d* ]]; do
h=$h${l%%$d*}
l=${l#*$d}
t=${l%%$d*}
l=${l#$t}
if [[ -n $t && ${s[$t]+t} == 't' ]]; then
h=$h${s[$t]}
l=${l#$d}
else
h=$h$d$t
# for apparent requirement change preceding line to (untested)
# h=$h"Error!!!" ; l=${l#$d}
fi
done
printf '%s\n' "$h$l"
done