Related
I want to be able to sort an input csv file that is comma separated by a values created in an extra column. Below is a sample of the input csv file
Timestamp,Email,Name,Year,Make,Model,Car_ID,Judge_ID,Judge_Name,Racer_Turbo,Racer_Supercharged,Racer_Performance,Racer_Horsepower,Car_Overall,Engine_Modifications,Engine_Performance,Engine_Chrome,Engine_Detailing,Engine_Cleanliness,Body_Frame_Undercarriage,Body_Frame_Suspension,Body_Frame_Chrome,Body_Frame_Detailing,Body_Frame_Cleanliness,Mods_Paint,Mods_Body,Mods_Wrap,Mods_Rims,Mods_Interior,Mods_Other,Mods_ICE,Mods_Aftermarket,Mods_WIP,Mods_Overall
8/5/2018 14:10,honoland13#japanpost.jp,Hernando,2015,Acura,TLX,48,J04,Bob,0,0,2,2,4,4,0,2,4,4,2,4,2,2,2,2,2,0,4,4,4,6,2,0,4
8/5/2018 15:11,nlighterness2q#umn.edu,Noel,2015,Jeep,Wrangler,124,J02,Carl,0,6,4,2,4,6,6,4,4,4,6,6,6,6,6,4,6,6,6,6,6,4,6,4,6
8/5/2018 17:10,eguest47#microsoft.com,Edan,2015,Lexus,Is250,222,J05,Adrian,0,0,0,0,0,0,0,0,6,6,6,0,0,6,6,6,0,0,0,0,0,0,0,0,4
8/5/2018 17:34,hchilley40#fema.gov,Hieronymus,1993,Honda,Civic eG,207,J06,Aaron,0,0,2,2,2,2,2,2,0,4,2,2,2,2,2,2,4,2,2,0,0,0,2,2,0
8/5/2018 14:30,nnowick3d#tuttocitta.it,Nickolas,2016,Ford,Mystang,167,J02,Carl,0,0,2,2,0,2,2,0,0,0,0,2,0,2,2,2,0,0,2,0,0,0,0,0,2
8/5/2018 16:12,mdearl39#amazon.co.uk,Martin,2013,Hyundai,Gen coupe,159,J04,Bob,0,0,2,0,0,0,2,0,0,0,0,2,0,2,2,0,2,0,2,0,0,0,0,0,0
8/5/2018 17:00,alynamg#blogtalkradio.com,Aldridge,2009,Infiniti,G37,20,J06,Aaron,2,0,2,2,0,0,2,0,0,2,2,2,2,2,2,2,2,2,4,2,2,0,2,0
What my code currently does is sift through the csv file, and pick out the car_id column, year, make, and model columns. Then it runs through every column from racer_turbo to the last, and for each row it adds up the values in those columns into a total value and prints that along side the other values (id, make, model, etc.). There is also a ranking column that precedes the other 5 when printed. Here is my code below.
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
{
rank;
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print ++rank,$7, $4, $5, $6, total;
}
rows[$5][total][$0]
}
END {
print "\n";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
ranking;
PROCINFO["sorted_in"] = "#ind_str_asc"
for (m in rows) {
n = asorti(rows[m], t, "#ind_num_desc");
n = (n>3) ? 3 : n
for(i = 1; i <= n; i++) for(s in rows[m][t[i]]) {
$0 = s;
$1 = ++r;
print ++ranking, $7, $4, $5, $6, total;
}
}
}
What I would like to do in the END block is print the output again, however, rank the cars by top three from each make using the total column which was created in the preceding block of the code. However, what I run my code now the output looks as follows
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
...
Ranking Car_ID Year Make Model Total
1 113 2012 Acura Tsx sportwagon 10
2 112 2008 Acura TL 10
3 50 2015 Acura TLX 10
4 15 2014 Audi S4 10
5 18 2015 Audi S3 10
6 116 2008 Audi A4 10
7 2 2016 Bmw M2 10
8 172 2014 Bmw 4 10
9 28 1995 Bmw 318xi 10
...
See how in the total column on the second printed section it shows total is 10 for each printed car, instead of being the same values as they were in the first printed section for each respective car, and the highest 3 totals for each make being displayed.
Below is the expected output
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
8 178 2009 Honda Oddesy 66
...
Ranking Car_ID Year Make Model Total
1 112 2008 Acura TL 110
2 50 2015 Acura TLX 102
3 127 2013 Acura Tsx 86
4 15 2014 Audi S4 120
5 18 2015 Audi S3 38
6 116 2008 Audi A4 28
7 2 2016 Bmw M2 24
8 172 2014 Bmw 4 22
9 111 2007 Bmw 328i 10
10 218 2010 Chevy Camaro 64
11 170 2014 Chevy Cruze 50
12 0 2015 Chevy Camaro 0
...
Is this salvagable with my current code? Or would a better approach be to create a separate awk file that will sort through the generated output and produce another file that is sorted by the top 3?
I'm running GNU AWK v4.0.2.
Assuming the Car_ID (hereinafter referred to as id) is unique across the rows, would you please try:
BEGIN {
FS = ","
OFS = "\t"
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total"
}
{
rank
total = 0
if (NR > 1) {
for (i = 8; i < NF; i++) {
total += $i
}
print ++rank, $7, $4, $5, $6, total
ttl[$5][$7] = total
row[$7] = $0
}
}
END {
print "\n"
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total"
ranking
id
PROCINFO["sorted_in"] = "#ind_str_asc"
for (m in ttl) {
n = asorti(ttl[m], t, "#val_num_desc")
n = (n>3) ? 3 : n
for (i = 1; i <= n; i++) {
id = t[i]
total = ttl[m][id]
$0 = row[id]
print ++ranking, $7, $4, $5, $6, total
}
}
}
I have slightly modified the data structure, assigning the id as the
main key. Then created a 2-D array ttl, which holds the value total
keyed by make and id. In the END loop, we can retrieve the
input data using the id.
As a side note, your original data structure uses total as an index.
If multiple rows with the same make happen to have the same value
of total, either of the indexes will be overwritten.
I want to be able to essentially print the line number along every printed line in the output after scanning and reworking an input csv file. The input csv file is comma separated, here's a sample
Timestamp,Email,Name,Year,Make,Model,Car_ID,Judge_ID,Judge_Name,Racer_Turbo,Racer_Supercharged,Racer_Performance,Racer_Horsepower,Car_Overall,Engine_Modifications,Engine_Performance,Engine_Chrome,Engine_Detailing,Engine_Cleanliness,Body_Frame_Undercarriage,Body_Frame_Suspension,Body_Frame_Chrome,Body_Frame_Detailing,Body_Frame_Cleanliness,Mods_Paint,Mods_Body,Mods_Wrap,Mods_Rims,Mods_Interior,Mods_Other,Mods_ICE,Mods_Aftermarket,Mods_WIP,Mods_Overall
8/5/2018 14:10,honoland13#japanpost.jp,Hernando,2015,Acura,TLX,48,J04,Bob,0,0,2,2,4,4,0,2,4,4,2,4,2,2,2,2,2,0,4,4,4,6,2,0,4
8/5/2018 15:11,nlighterness2q#umn.edu,Noel,2015,Jeep,Wrangler,124,J02,Carl,0,6,4,2,4,6,6,4,4,4,6,6,6,6,6,4,6,6,6,6,6,4,6,4,6
8/5/2018 17:10,eguest47#microsoft.com,Edan,2015,Lexus,Is250,222,J05,Adrian,0,0,0,0,0,0,0,0,6,6,6,0,0,6,6,6,0,0,0,0,0,0,0,0,4
8/5/2018 17:34,hchilley40#fema.gov,Hieronymus,1993,Honda,Civic eG,207,J06,Aaron,0,0,2,2,2,2,2,2,0,4,2,2,2,2,2,2,4,2,2,0,0,0,2,2,0
8/5/2018 14:30,nnowick3d#tuttocitta.it,Nickolas,2016,Ford,Mystang,167,J02,Carl,0,0,2,2,0,2,2,0,0,0,0,2,0,2,2,2,0,0,2,0,0,0,0,0,2
8/5/2018 16:12,mdearl39#amazon.co.uk,Martin,2013,Hyundai,Gen coupe,159,J04,Bob,0,0,2,0,0,0,2,0,0,0,0,2,0,2,2,0,2,0,2,0,0,0,0,0,0
8/5/2018 17:00,alynamg#blogtalkradio.com,Aldridge,2009,Infiniti,G37,20,J06,Aaron,2,0,2,2,0,0,2,0,0,2,2,2,2,2,2,2,2,2,4,2,2,0,2,0,2
What my code currently does is sift through the csv file, and pick out the car_id column, year, make, and model columns. Then it runs through every column from racer_turbo to the last, and for each row it adds up the values in those columns into a total value and prints that along side the other values (id, make, model, etc.). There is also a ranking column that precedes the other 5 when printed. Here is my code below.
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
FNR > 1 {
rank = 0;
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print ++rank,$7,$4,$5,$6,total;
}
}
END {
}
My current output is as follows
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
1 124 2015 Jeep Wrangler 118
1 222 2015 Lexus Is250 36
1 207 1993 Honda Civic eG 40
1 167 2016 Ford Mystang 18
1 159 2013 Hyundai Gen coupe 14
1 20 2009 Infiniti G37 36
1 178 2009 Honda Oddesy 66
My problem is that under the ranking column, it only shows 1 for each row, I need it to be able to increment starting at 1 and going down for as many lines as there are in the document. Right now as evident in my code, I have a rank variable that acts as a tracker that I want to increment up with each row printed, but it only prints 1 for each row. How can I fix that?
The expected output is this
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
8 178 2009 Honda Oddesy 66
Please be advised, my machine is running version 4.0.2 of AWK.
You do not need own variable in this case, just subtract 1 from NR to get desired output, that is
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
FNR > 1 {
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print NR-1,$7,$4,$5,$6,total;
}
}
As side note: END is optional, you do not have to have one in GNU AWK command.
My current code is:
count1 = 0
for i in range(30):
if i%26 == 0:
b = [i+1, i+2, i+3, i+4, i+5, i+6, i+7, i+8, i+9, i+10]
count1 += 1
print([count1])
print(*b, sep=' ')
elif (i-10)%26 == 0:
b = [i+1, i+2, i+3, i+4, i+5, i+6, i+7, i+8, i+9]
count1 += 1
print([count1])
print(*b, sep= ' ')
elif (i-16)%32 == 0:
b = [i+1, i+2, i+3, i+4, i+5, i+6, i+7, i+8, i+9, i+10]
count1 += 1
print([count1])
print(*b, sep= ' ')
which produces lines:
[1]
1 2 3 4 5 6 7 8 9 10
[2]
11 12 13 14 15 16 17 18 19
[3]
17 18 19 20 21 22 23 24 25 26
[4]
27 28 29 30 31 32 33 34 35 36
I'd like to output these lines in a simple text file. I'm familiar with the open and write functions, but do not know how to apply them to my specific example.
Thanks!
On GNU/Linux systems execute the program in the console, add > and the name of the file.
Example:
Assuming that you are in the directory wich contains the executable.
./[name of the program] > [name of the file]
./helloworld > helloworld.txt
This will save all the printed text in the console in a text file.
I have a script called "main.ksh" which returns "output.txt" file and I am sending that file via mail.(list contains 50+ records, I just give 4 records for example)
mail output I am getting is:
DATE | FEED NAMEs | FILE NAMEs | JOB NAMEs | SCHEDULED_TIME| TIMESTAMP| SIZE(MB)| COUNT| STATUS |
Dec 17 INVEST_AI_FUNDS_FEED amlfunds_iai_20161217.txt gdcpl3392_uxmow080_ori_inv_ai TUE-SAT 02:03 0.4248 4031 On_Time
Dec 17 INVEST_AI_SECURITIES_FEED amltxn_iai_20161217.txt gdcpl3392_uxmow080_ori_inv_ai TUE-SAT 02:03 0.0015 9 On_Time
Dec 17 INVEST_AI_CONNECTED_PARTIES_FEED amlbene_iai_20161217.txt gdcpl3392_uxmow080_ori_inv_ai TUE-SAT 02:03 0.0001 1 No_Records
I am implementing coloring for Delayed,On_Time and No_Records field and I wrote below script which gives me bottom output(output is correct but there is no space separated).
awk 'BEGIN {
print "<html>" \
"<body bgcolor=\"#333\" text=\"#f3f3f3\">" \
"<pre>"
}
NR == 1 { print $0 }
NR > 1 {
if ($NF == "Delayed") color="red"
else if ($NF == "On_time") color="green"
else if ($NF == "No_records") color="yellow"
else color="#003abc"
$NF="<span style=\"color:" color "\">" $NF "</span>"
print $0
}
END {
print "</pre>" \
"</body>" \
"</html>"
}
' output.txt > output.html
output with perfect coloring:
| DATE | FEED NAMEs | FILE NAMEs | JOB NAMEs | SCHEDULED_TIME| TIMESTAMP| SIZE(MB)| COUNT| STATUS |
Dec 17 INVEST_AI_FUNDS_FEED amlfunds_iai_20161217.txt gdcpl3392_uxmow080_ori_inv_ai On_Time
Dec 17 INVEST_AI_SECURITIES_FEED amltxn_iai_20161217.txt gdcpl3392_uxmow080_ori_inv_ai On_Time
Dec 17 INVEST_AI_CONNECTED_PARTIES_FEED amlbene_iai_20161217.txt gdcpl3392_uxmow080_ori_inv_ai No_Records
There are 4 columns are skipped automatically. Could you please help me on this please ? Thanks a lot !
When your code executes this
$NF="<span style=\"color:" color "\">" $NF "</span>"
print $0
the input line is rebuilt and therefore the multiple blanks between two consecutive fields are replaced by just ONE only blank space.
My solution copies the input line in a variable, deletes the last field (changing the value of the variable, not the input line), adds the modified last field and prints:
Dummy=$0
sub("[^ ]+$","",Dummy) # removes last field
Dummy=Dummy "<span style=\"color:" color "\">" $NF "</span>"
print Dummy
Best regards
update: the last two code lines can be reduced in this way:
print Dummy "<span style=\"color:" color "\">" $NF "</span>"
I'm using MySQL to store my data. I'm recorded dates with different offset according to each timezone. What is important on those dates are only "effective" time. Here is an example:
date1 = "Thu, 27 Oct 2016 07:00:00 CEST +02:00"
# I want to get 700
date2 = "Thu, 27 Oct 2016 22:00:00 CEST +02:00"
# I want to get 2200
I would like to get these values to compare them with my current time. Then if it's currently "2016-11-17 10:12:00 +0100", I would like to compare on a where clause:
"1012" > "2200"
# or
"1012" > "700"
Here is a fake record value:
# my_table
# | id | my_hour
# | XX | "Thu, 27 Oct 2016 07:00:00 CEST +02:00"
# | XY | "Thu, 27 Oct 2016 22:00:00 CEST +11:00"
Here is my code:
hour = Time.zone.now
# "2016-11-18 21:10:00 +0100"
where("CAST(DATE_FORMAT(my_hour,'%H%i') AS UNSIGNED) < ?", hour.strftime("%H%M").to_i)
# => Get XX record?
Use strftime to format date in Rails.
hour = Time.zone.now
MyTable.where("cast(strftime('%H%M', my_hour) as int) < ?", hour.strftime("%H%M").to_i)
here MyTable is your Model, my_hour is your date column and hour is current time.