Related
I want to be able to sort an input csv file that is comma separated by a values created in an extra column. Below is a sample of the input csv file
Timestamp,Email,Name,Year,Make,Model,Car_ID,Judge_ID,Judge_Name,Racer_Turbo,Racer_Supercharged,Racer_Performance,Racer_Horsepower,Car_Overall,Engine_Modifications,Engine_Performance,Engine_Chrome,Engine_Detailing,Engine_Cleanliness,Body_Frame_Undercarriage,Body_Frame_Suspension,Body_Frame_Chrome,Body_Frame_Detailing,Body_Frame_Cleanliness,Mods_Paint,Mods_Body,Mods_Wrap,Mods_Rims,Mods_Interior,Mods_Other,Mods_ICE,Mods_Aftermarket,Mods_WIP,Mods_Overall
8/5/2018 14:10,honoland13#japanpost.jp,Hernando,2015,Acura,TLX,48,J04,Bob,0,0,2,2,4,4,0,2,4,4,2,4,2,2,2,2,2,0,4,4,4,6,2,0,4
8/5/2018 15:11,nlighterness2q#umn.edu,Noel,2015,Jeep,Wrangler,124,J02,Carl,0,6,4,2,4,6,6,4,4,4,6,6,6,6,6,4,6,6,6,6,6,4,6,4,6
8/5/2018 17:10,eguest47#microsoft.com,Edan,2015,Lexus,Is250,222,J05,Adrian,0,0,0,0,0,0,0,0,6,6,6,0,0,6,6,6,0,0,0,0,0,0,0,0,4
8/5/2018 17:34,hchilley40#fema.gov,Hieronymus,1993,Honda,Civic eG,207,J06,Aaron,0,0,2,2,2,2,2,2,0,4,2,2,2,2,2,2,4,2,2,0,0,0,2,2,0
8/5/2018 14:30,nnowick3d#tuttocitta.it,Nickolas,2016,Ford,Mystang,167,J02,Carl,0,0,2,2,0,2,2,0,0,0,0,2,0,2,2,2,0,0,2,0,0,0,0,0,2
8/5/2018 16:12,mdearl39#amazon.co.uk,Martin,2013,Hyundai,Gen coupe,159,J04,Bob,0,0,2,0,0,0,2,0,0,0,0,2,0,2,2,0,2,0,2,0,0,0,0,0,0
8/5/2018 17:00,alynamg#blogtalkradio.com,Aldridge,2009,Infiniti,G37,20,J06,Aaron,2,0,2,2,0,0,2,0,0,2,2,2,2,2,2,2,2,2,4,2,2,0,2,0
What my code currently does is sift through the csv file, and pick out the car_id column, year, make, and model columns. Then it runs through every column from racer_turbo to the last, and for each row it adds up the values in those columns into a total value and prints that along side the other values (id, make, model, etc.). There is also a ranking column that precedes the other 5 when printed. Here is my code below.
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
{
rank;
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print ++rank,$7, $4, $5, $6, total;
}
rows[$5][total][$0]
}
END {
print "\n";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
ranking;
PROCINFO["sorted_in"] = "#ind_str_asc"
for (m in rows) {
n = asorti(rows[m], t, "#ind_num_desc");
n = (n>3) ? 3 : n
for(i = 1; i <= n; i++) for(s in rows[m][t[i]]) {
$0 = s;
$1 = ++r;
print ++ranking, $7, $4, $5, $6, total;
}
}
}
What I would like to do in the END block is print the output again, however, rank the cars by top three from each make using the total column which was created in the preceding block of the code. However, what I run my code now the output looks as follows
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
...
Ranking Car_ID Year Make Model Total
1 113 2012 Acura Tsx sportwagon 10
2 112 2008 Acura TL 10
3 50 2015 Acura TLX 10
4 15 2014 Audi S4 10
5 18 2015 Audi S3 10
6 116 2008 Audi A4 10
7 2 2016 Bmw M2 10
8 172 2014 Bmw 4 10
9 28 1995 Bmw 318xi 10
...
See how in the total column on the second printed section it shows total is 10 for each printed car, instead of being the same values as they were in the first printed section for each respective car, and the highest 3 totals for each make being displayed.
Below is the expected output
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
8 178 2009 Honda Oddesy 66
...
Ranking Car_ID Year Make Model Total
1 112 2008 Acura TL 110
2 50 2015 Acura TLX 102
3 127 2013 Acura Tsx 86
4 15 2014 Audi S4 120
5 18 2015 Audi S3 38
6 116 2008 Audi A4 28
7 2 2016 Bmw M2 24
8 172 2014 Bmw 4 22
9 111 2007 Bmw 328i 10
10 218 2010 Chevy Camaro 64
11 170 2014 Chevy Cruze 50
12 0 2015 Chevy Camaro 0
...
Is this salvagable with my current code? Or would a better approach be to create a separate awk file that will sort through the generated output and produce another file that is sorted by the top 3?
I'm running GNU AWK v4.0.2.
Assuming the Car_ID (hereinafter referred to as id) is unique across the rows, would you please try:
BEGIN {
FS = ","
OFS = "\t"
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total"
}
{
rank
total = 0
if (NR > 1) {
for (i = 8; i < NF; i++) {
total += $i
}
print ++rank, $7, $4, $5, $6, total
ttl[$5][$7] = total
row[$7] = $0
}
}
END {
print "\n"
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total"
ranking
id
PROCINFO["sorted_in"] = "#ind_str_asc"
for (m in ttl) {
n = asorti(ttl[m], t, "#val_num_desc")
n = (n>3) ? 3 : n
for (i = 1; i <= n; i++) {
id = t[i]
total = ttl[m][id]
$0 = row[id]
print ++ranking, $7, $4, $5, $6, total
}
}
}
I have slightly modified the data structure, assigning the id as the
main key. Then created a 2-D array ttl, which holds the value total
keyed by make and id. In the END loop, we can retrieve the
input data using the id.
As a side note, your original data structure uses total as an index.
If multiple rows with the same make happen to have the same value
of total, either of the indexes will be overwritten.
I want to be able to essentially print the line number along every printed line in the output after scanning and reworking an input csv file. The input csv file is comma separated, here's a sample
Timestamp,Email,Name,Year,Make,Model,Car_ID,Judge_ID,Judge_Name,Racer_Turbo,Racer_Supercharged,Racer_Performance,Racer_Horsepower,Car_Overall,Engine_Modifications,Engine_Performance,Engine_Chrome,Engine_Detailing,Engine_Cleanliness,Body_Frame_Undercarriage,Body_Frame_Suspension,Body_Frame_Chrome,Body_Frame_Detailing,Body_Frame_Cleanliness,Mods_Paint,Mods_Body,Mods_Wrap,Mods_Rims,Mods_Interior,Mods_Other,Mods_ICE,Mods_Aftermarket,Mods_WIP,Mods_Overall
8/5/2018 14:10,honoland13#japanpost.jp,Hernando,2015,Acura,TLX,48,J04,Bob,0,0,2,2,4,4,0,2,4,4,2,4,2,2,2,2,2,0,4,4,4,6,2,0,4
8/5/2018 15:11,nlighterness2q#umn.edu,Noel,2015,Jeep,Wrangler,124,J02,Carl,0,6,4,2,4,6,6,4,4,4,6,6,6,6,6,4,6,6,6,6,6,4,6,4,6
8/5/2018 17:10,eguest47#microsoft.com,Edan,2015,Lexus,Is250,222,J05,Adrian,0,0,0,0,0,0,0,0,6,6,6,0,0,6,6,6,0,0,0,0,0,0,0,0,4
8/5/2018 17:34,hchilley40#fema.gov,Hieronymus,1993,Honda,Civic eG,207,J06,Aaron,0,0,2,2,2,2,2,2,0,4,2,2,2,2,2,2,4,2,2,0,0,0,2,2,0
8/5/2018 14:30,nnowick3d#tuttocitta.it,Nickolas,2016,Ford,Mystang,167,J02,Carl,0,0,2,2,0,2,2,0,0,0,0,2,0,2,2,2,0,0,2,0,0,0,0,0,2
8/5/2018 16:12,mdearl39#amazon.co.uk,Martin,2013,Hyundai,Gen coupe,159,J04,Bob,0,0,2,0,0,0,2,0,0,0,0,2,0,2,2,0,2,0,2,0,0,0,0,0,0
8/5/2018 17:00,alynamg#blogtalkradio.com,Aldridge,2009,Infiniti,G37,20,J06,Aaron,2,0,2,2,0,0,2,0,0,2,2,2,2,2,2,2,2,2,4,2,2,0,2,0,2
What my code currently does is sift through the csv file, and pick out the car_id column, year, make, and model columns. Then it runs through every column from racer_turbo to the last, and for each row it adds up the values in those columns into a total value and prints that along side the other values (id, make, model, etc.). There is also a ranking column that precedes the other 5 when printed. Here is my code below.
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
FNR > 1 {
rank = 0;
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print ++rank,$7,$4,$5,$6,total;
}
}
END {
}
My current output is as follows
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
1 124 2015 Jeep Wrangler 118
1 222 2015 Lexus Is250 36
1 207 1993 Honda Civic eG 40
1 167 2016 Ford Mystang 18
1 159 2013 Hyundai Gen coupe 14
1 20 2009 Infiniti G37 36
1 178 2009 Honda Oddesy 66
My problem is that under the ranking column, it only shows 1 for each row, I need it to be able to increment starting at 1 and going down for as many lines as there are in the document. Right now as evident in my code, I have a rank variable that acts as a tracker that I want to increment up with each row printed, but it only prints 1 for each row. How can I fix that?
The expected output is this
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
8 178 2009 Honda Oddesy 66
Please be advised, my machine is running version 4.0.2 of AWK.
You do not need own variable in this case, just subtract 1 from NR to get desired output, that is
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
FNR > 1 {
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print NR-1,$7,$4,$5,$6,total;
}
}
As side note: END is optional, you do not have to have one in GNU AWK command.
i'm trying to import a txt-File to octave which contains a matrix of data.
The matrix looks like this:
49 ..1. ...1.......... ..... 49
47 ..12 ...1...... ... ..... 47
45 ..12....1...... 2....1... 45
43 ....2....1...... 2...1.... 43
41 .1..2.. .........2. .1..... 41
39 .1.12.2....1.....2. .1..... 39
37 .1..2.22...1.....2. .1..... 37
35 .1. 2222...2....2....1.1... 35
33 ....22.2...2....2....12.... 33
31 ....22.2...2..........21... 31
29 .....2.2...2.....2....21... 29
27 ........222222....2....21.... 27
25 .......22.2222....2.22.2..... 25
23 .......22.2222....2.2..2..... 23
21 .......222.222....2.2........ 21
19 ........22.222....2.......... 19
17 ..........2.2.2...22........... 17
15 ............................... 15
13 .......................2....... 13
11 .......................2......2 11
9 ........................2.....222 9
7 . ................. ..... ......222. 7
5 .................. ....1.. 5
3 ....... ......... .... 3
1 1
This is actually map/coordinate system (y-axis=azimuth, x-axis=latitude), which i have to plot. (blank=no data, .=no effects, 1=weak effects, 2=strong eff.)
The result should look like this
Because i failed to import this txt-File, i changed it into this.
49;1;1;1;1;1;1;1;2;2;3;2;1;2;2;2;3;2;2;2;2;2;2;2;2;2;2;1;2;2;2;2;2;1;1;1;1;1;1;1;49
47;1;1;1;1;1;1;1;2;2;3;4;1;2;2;2;3;2;2;2;2;2;2;1;2;2;2;1;2;2;2;2;2;1;1;1;1;1;1;1;47
45;1;1;1;1;1;1;1;2;2;3;4;2;2;2;2;3;2;2;2;2;2;2;1;4;2;2;2;2;3;2;2;2;1;1;1;1;1;1;1;45
43;1;1;1;1;1;1;2;2;2;2;4;2;2;2;2;3;2;2;2;2;2;2;1;4;2;2;2;3;2;2;2;2;1;1;1;1;1;1;1;43
39;1;1;1;1;1;1;2;3;2;3;4;2;4;2;2;2;2;3;2;2;2;2;2;4;2;1;2;3;2;2;2;2;2;1;1;1;1;1;1;39
37;1;1;1;1;1;1;2;3;2;2;4;2;4;4;2;2;2;3;2;2;2;2;2;4;2;1;2;3;2;2;2;2;2;1;1;1;1;1;1;37
and so on.
This is working with my code.
RawMap = dlmread('C:\Desktop\2576.map', ';', 0:80, 0:24)
Map = flipud(RawMap)
pcolor(Map(:,2:end-1))
To this automatically i don't want to the change the code. So i need to get the original file imported.
Any suggestions?
Thanks
Here is one approach to parse the file meaningfully:
S = fileread('testo.txt');
S = strsplit (S, "\n");
S = strvcat( S );
S = double(S);
S = S(:, 4:end-4);
S( S == double(" ") ) = 0;
S( S == double(".") ) = 1;
S( S == double("1") ) = 2;
S( S == double("2") ) = 3;
pcolor(S); axis ij;
I have a script called "main.ksh" which returns "output.txt" file and I am sending that file via mail (list contains 50+ records, I just give 3 records for example).
mail output I am getting is: (10 cols)
DATE FEED FILE_NAME JOB_NAME SCHEDULED TIME SIZE COUNT STATUS
Dec 17 INVEST iai guxmow080 TUE-SAT 02:03 0.4248 4031 On_Time
Dec 17 SECURITIES amltxn gdcpl3392 TUE-SAT 02:03 0.0015 9 Delayed
Dec 17 CONNECTED amlbene gdcpl3392 TUE-SAT 02:03 0.0001 1 No_Records
output with perfect coloring: (6 cols only)
DATE FEED FILE_NAME JOB_NAME SCHEDULED TIME SIZE COUNT STATUS
Dec 17 INVEST iai guxmow080 On_Time(green color)
Dec 17 SECURITIES amltxn gdcpl3392 Delayed(red color)
Dec 17 CONNECTED amlbene gdcpl3392 No_Records(yellow color)
I am implementing coloring for Delayed, On_Time and No_Records field and I wrote below script which gives me bottom output.
awk 'BEGIN {
print "<html>" \
"<body bgcolor=\"#333\" text=\"#f3f3f3\">" \
"<pre>"
}
NR == 1 { print $0 }
NR > 1 {
if ($NF == "Delayed") color="red"
else if ($NF == "On_time") color="green"
else if ($NF == "No_records") color="yellow"
else color="#003abc"
Dummy=$0
sub("[^ ]+$","",Dummy)
print Dummy "<span style=\"color:" color (bold ? ";font-weight:bold" : "")(size ? ";font-size:size" : "") (italic ? ";font-style:italic" : "") "\">" $NF "</span>"
}
END {
print "</pre>" \
"</body>" \
"</html>"
}
' output.txt > output.html
There are 4 columns are skipped automatically.
| date | feed_names | file_names | job_names | scheduled_time| timestamp| size| count| status |
Dec 19 ISS_BENEFICIAL_OWNERS_FEED amlcpbo_iss_20161219.txt gdcpl3392_uxmow080_ori_isz_dat WEEK_DAYS 00:03 9.3734 34758 On_Time
Dec 19 ISS_INVESTORS_FEED amlinvest_iss_20161219.txt gdcpl3392_uxmow080_ori_isz_dat WEEK_DAYS 00:01 0.0283 82 On_Time
Dec 19 ISS_TRANSACTIONS_FEED amltran_iss_1_20161219.txt gdcpl3392_uxmow080_ori_isz_dat WEEK_DAYS 00:12 14.022 36532 DELAYED
Dec 19 ISS_TRANSACTIONS_FEED amltran_iss_5_20161219.txt gdcpl3392_uxmow080_ori_isz_dat WEEK_DAYS 00:23 0.0010 3 DELAYED
Dec 19 IBS_CUSTOMER_FEED ibscust_aml_***_20161219.txt gdcpl3392_uxmow080_ori_sfp_ibc WEEK_DAYS (11 _out_of_11) -NA- ARRIVED
Dec 19 IBS_DDA_NOSTRO_ACCOUNT_FEED ibsacct_aml_***_20161219.txt gdcpl3392_uxmow080_ori_sfp_ibc WEEK_DAYS (44 _out_of_44) -NA- ARRIVED
Dec 19 GP__TRANSACTIONS_FEED amltrans__20161219.txt gdcpl3392_uxmow080_ori_sfp_glo WEEK_DAYS (3 _out_of_30) -NA- ARRIVED
But when I am trying to print in a sequential order by using below command
awk '{printf("%-5s%s\t%-33s%-35s%-39s%s\t%s%-3s\t%s\t%s\n", $1,$2,$3,$4,$5,$6,$7,$8,$9,$10)}' output.txt, I am getting the output in a sequential format
but 4 cols are skipped. Kindly suggest!!!
| date | feed_names | file_names | job_names | scheduled_time| timestamp| size| count| status |
Dec 19 ISS_BENEFICIAL_OWNERS_FEED amlcpbo_iss_20161219.txt gdcpl3392_uxmow080_ori_isz_dat On_Time
Dec 19 ISS_INVESTORS_FEED amlinvest_iss_20161219.txt gdcpl3392_uxmow080_ori_isz_dat On_Time
Dec 19 ISS_TRANSACTIONS_FEED amltran_iss_1_20161219.txt gdcpl3392_uxmow080_ori_isz_dat DELAYED
Dec 19 ISS_TRANSACTIONS_FEED amltran_iss_5_20161219.txt gdcpl3392_uxmow080_ori_isz_dat DELAYED
Dec 19 IBS_CUSTOMER_FEED ibscust_aml_***_20161219.txt gdcpl3392_uxmow080_ori_sfp_ibc ARRIVED
Dec 19 IBS_DDA_NOSTRO_ACCOUNT_FEED ibsacct_aml_***_20161219.txt gdcpl3392_uxmow080_ori_sfp_ibc ARRIVED
Dec 19 GP__TRANSACTIONS_FEED amltrans__20161219.txt gdcpl3392_uxmow080_ori_sfp_glo YET_TO_RECEIVE
When I run below octave code the command window displays :
>> first
x =
10
20
30
40
50
60
70
80
90
100
y =
14
17
18
14
15
14
13
12
11
4
m = 10
x =
1 10
1 20
1 30
1 40
1 50
1 60
1 70
1 80
1 90
1 100
-- less -- (f)orward, (b)ack, (q)uit
I'm required to continually press (f) to complete program and view plot : plot(x(:,2), x*theta, '-');
Octave code :
x = [10
20
30
40
50
60
70
80
90
100]
y = [14
17
18
14
15
14
13
12
11
4]
m = length(y)
x = [ones(m , 1) , x]
theta = zeros(2, 1);
iterations = 10;
alpha = 0.000007;
for iter = 1:iterations
theta = theta - ((1/m) * ((x * theta) - y)' * x)' * alpha;
#theta
end
#plot(x, y, 'o');
#ylabel('Response Time')
#xlabel('Time since 0')
plot(x(:,2), x*theta, '-');
How to prevent user interaction with command window so that program runs to completion and displays prompt and not requiring
user interaction ?
To prevent your variables from printing altogether, simply add a semicolon to the end of each variable assignment:
m = length(y) %// **will** print to the console
m = length(y); %// will *not* print to the console
To print your variables to the console, but avoid Octave pausing the output when it gets to the bottom of the screen, add more off to the beginning of your script to turn off paging.
https://www.gnu.org/software/octave/doc/interpreter/Paging-Screen-Output.html
Type more on to switch it back on.