Importing Textfile as Matrix - octave
i'm trying to import a txt-File to octave which contains a matrix of data.
The matrix looks like this:
49 ..1. ...1.......... ..... 49
47 ..12 ...1...... ... ..... 47
45 ..12....1...... 2....1... 45
43 ....2....1...... 2...1.... 43
41 .1..2.. .........2. .1..... 41
39 .1.12.2....1.....2. .1..... 39
37 .1..2.22...1.....2. .1..... 37
35 .1. 2222...2....2....1.1... 35
33 ....22.2...2....2....12.... 33
31 ....22.2...2..........21... 31
29 .....2.2...2.....2....21... 29
27 ........222222....2....21.... 27
25 .......22.2222....2.22.2..... 25
23 .......22.2222....2.2..2..... 23
21 .......222.222....2.2........ 21
19 ........22.222....2.......... 19
17 ..........2.2.2...22........... 17
15 ............................... 15
13 .......................2....... 13
11 .......................2......2 11
9 ........................2.....222 9
7 . ................. ..... ......222. 7
5 .................. ....1.. 5
3 ....... ......... .... 3
1 1
This is actually map/coordinate system (y-axis=azimuth, x-axis=latitude), which i have to plot. (blank=no data, .=no effects, 1=weak effects, 2=strong eff.)
The result should look like this
Because i failed to import this txt-File, i changed it into this.
49;1;1;1;1;1;1;1;2;2;3;2;1;2;2;2;3;2;2;2;2;2;2;2;2;2;2;1;2;2;2;2;2;1;1;1;1;1;1;1;49
47;1;1;1;1;1;1;1;2;2;3;4;1;2;2;2;3;2;2;2;2;2;2;1;2;2;2;1;2;2;2;2;2;1;1;1;1;1;1;1;47
45;1;1;1;1;1;1;1;2;2;3;4;2;2;2;2;3;2;2;2;2;2;2;1;4;2;2;2;2;3;2;2;2;1;1;1;1;1;1;1;45
43;1;1;1;1;1;1;2;2;2;2;4;2;2;2;2;3;2;2;2;2;2;2;1;4;2;2;2;3;2;2;2;2;1;1;1;1;1;1;1;43
39;1;1;1;1;1;1;2;3;2;3;4;2;4;2;2;2;2;3;2;2;2;2;2;4;2;1;2;3;2;2;2;2;2;1;1;1;1;1;1;39
37;1;1;1;1;1;1;2;3;2;2;4;2;4;4;2;2;2;3;2;2;2;2;2;4;2;1;2;3;2;2;2;2;2;1;1;1;1;1;1;37
and so on.
This is working with my code.
RawMap = dlmread('C:\Desktop\2576.map', ';', 0:80, 0:24)
Map = flipud(RawMap)
pcolor(Map(:,2:end-1))
To this automatically i don't want to the change the code. So i need to get the original file imported.
Any suggestions?
Thanks
Here is one approach to parse the file meaningfully:
S = fileread('testo.txt');
S = strsplit (S, "\n");
S = strvcat( S );
S = double(S);
S = S(:, 4:end-4);
S( S == double(" ") ) = 0;
S( S == double(".") ) = 1;
S( S == double("1") ) = 2;
S( S == double("2") ) = 3;
pcolor(S); axis ij;
Related
How can I sort output from variable?
I want to be able to sort an input csv file that is comma separated by a values created in an extra column. Below is a sample of the input csv file
Timestamp,Email,Name,Year,Make,Model,Car_ID,Judge_ID,Judge_Name,Racer_Turbo,Racer_Supercharged,Racer_Performance,Racer_Horsepower,Car_Overall,Engine_Modifications,Engine_Performance,Engine_Chrome,Engine_Detailing,Engine_Cleanliness,Body_Frame_Undercarriage,Body_Frame_Suspension,Body_Frame_Chrome,Body_Frame_Detailing,Body_Frame_Cleanliness,Mods_Paint,Mods_Body,Mods_Wrap,Mods_Rims,Mods_Interior,Mods_Other,Mods_ICE,Mods_Aftermarket,Mods_WIP,Mods_Overall
8/5/2018 14:10,honoland13#japanpost.jp,Hernando,2015,Acura,TLX,48,J04,Bob,0,0,2,2,4,4,0,2,4,4,2,4,2,2,2,2,2,0,4,4,4,6,2,0,4
8/5/2018 15:11,nlighterness2q#umn.edu,Noel,2015,Jeep,Wrangler,124,J02,Carl,0,6,4,2,4,6,6,4,4,4,6,6,6,6,6,4,6,6,6,6,6,4,6,4,6
8/5/2018 17:10,eguest47#microsoft.com,Edan,2015,Lexus,Is250,222,J05,Adrian,0,0,0,0,0,0,0,0,6,6,6,0,0,6,6,6,0,0,0,0,0,0,0,0,4
8/5/2018 17:34,hchilley40#fema.gov,Hieronymus,1993,Honda,Civic eG,207,J06,Aaron,0,0,2,2,2,2,2,2,0,4,2,2,2,2,2,2,4,2,2,0,0,0,2,2,0
8/5/2018 14:30,nnowick3d#tuttocitta.it,Nickolas,2016,Ford,Mystang,167,J02,Carl,0,0,2,2,0,2,2,0,0,0,0,2,0,2,2,2,0,0,2,0,0,0,0,0,2
8/5/2018 16:12,mdearl39#amazon.co.uk,Martin,2013,Hyundai,Gen coupe,159,J04,Bob,0,0,2,0,0,0,2,0,0,0,0,2,0,2,2,0,2,0,2,0,0,0,0,0,0
8/5/2018 17:00,alynamg#blogtalkradio.com,Aldridge,2009,Infiniti,G37,20,J06,Aaron,2,0,2,2,0,0,2,0,0,2,2,2,2,2,2,2,2,2,4,2,2,0,2,0
What my code currently does is sift through the csv file, and pick out the car_id column, year, make, and model columns. Then it runs through every column from racer_turbo to the last, and for each row it adds up the values in those columns into a total value and prints that along side the other values (id, make, model, etc.). There is also a ranking column that precedes the other 5 when printed. Here is my code below.
BEGIN {
FS = ",";
OFS = "\t";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
}
{
rank;
total = 0;
if(NR > 1) {
for(i = 8; i < NF; i++) {
total += $i;
}
print ++rank,$7, $4, $5, $6, total;
}
rows[$5][total][$0]
}
END {
print "\n";
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total";
ranking;
PROCINFO["sorted_in"] = "#ind_str_asc"
for (m in rows) {
n = asorti(rows[m], t, "#ind_num_desc");
n = (n>3) ? 3 : n
for(i = 1; i <= n; i++) for(s in rows[m][t[i]]) {
$0 = s;
$1 = ++r;
print ++ranking, $7, $4, $5, $6, total;
}
}
}
What I would like to do in the END block is print the output again, however, rank the cars by top three from each make using the total column which was created in the preceding block of the code. However, what I run my code now the output looks as follows
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
...
Ranking Car_ID Year Make Model Total
1 113 2012 Acura Tsx sportwagon 10
2 112 2008 Acura TL 10
3 50 2015 Acura TLX 10
4 15 2014 Audi S4 10
5 18 2015 Audi S3 10
6 116 2008 Audi A4 10
7 2 2016 Bmw M2 10
8 172 2014 Bmw 4 10
9 28 1995 Bmw 318xi 10
...
See how in the total column on the second printed section it shows total is 10 for each printed car, instead of being the same values as they were in the first printed section for each respective car, and the highest 3 totals for each make being displayed.
Below is the expected output
Ranking Car_ID Year Make Model Total
1 48 2015 Acura TLX 58
2 124 2015 Jeep Wrangler 118
3 222 2015 Lexus Is250 36
4 207 1993 Honda Civic eG 40
5 167 2016 Ford Mystang 18
6 159 2013 Hyundai Gen coupe 14
7 20 2009 Infiniti G37 36
8 178 2009 Honda Oddesy 66
...
Ranking Car_ID Year Make Model Total
1 112 2008 Acura TL 110
2 50 2015 Acura TLX 102
3 127 2013 Acura Tsx 86
4 15 2014 Audi S4 120
5 18 2015 Audi S3 38
6 116 2008 Audi A4 28
7 2 2016 Bmw M2 24
8 172 2014 Bmw 4 22
9 111 2007 Bmw 328i 10
10 218 2010 Chevy Camaro 64
11 170 2014 Chevy Cruze 50
12 0 2015 Chevy Camaro 0
...
Is this salvagable with my current code? Or would a better approach be to create a separate awk file that will sort through the generated output and produce another file that is sorted by the top 3?
I'm running GNU AWK v4.0.2.
Assuming the Car_ID (hereinafter referred to as id) is unique across the rows, would you please try:
BEGIN {
FS = ","
OFS = "\t"
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total"
}
{
rank
total = 0
if (NR > 1) {
for (i = 8; i < NF; i++) {
total += $i
}
print ++rank, $7, $4, $5, $6, total
ttl[$5][$7] = total
row[$7] = $0
}
}
END {
print "\n"
print "Ranking", "Car_ID", "Year", "Make", "Model", "Total"
ranking
id
PROCINFO["sorted_in"] = "#ind_str_asc"
for (m in ttl) {
n = asorti(ttl[m], t, "#val_num_desc")
n = (n>3) ? 3 : n
for (i = 1; i <= n; i++) {
id = t[i]
total = ttl[m][id]
$0 = row[id]
print ++ranking, $7, $4, $5, $6, total
}
}
}
I have slightly modified the data structure, assigning the id as the
main key. Then created a 2-D array ttl, which holds the value total
keyed by make and id. In the END loop, we can retrieve the
input data using the id.
As a side note, your original data structure uses total as an index.
If multiple rows with the same make happen to have the same value
of total, either of the indexes will be overwritten.
Encoding troubles with python, mysql and utf8mb4
I get the following warnings, when trying to save a simple dataframe to mysql.:
C:...\anaconda3\lib\site-packages\pymysql\cursors.py:170: Warning: (1366, "Incorrect string value: '\x92\xE9t\xE9)' for column 'VARIABLE_VALUE' at row 518")
result = self._query(query)
And
C:...anaconda3\lib\site-packages\pymysql\cursors.py:170: Warning:
(3719, "'utf8' is currently an alias for the character set UTF8MB3,
but will be an alias for UTF8MB4 in a future release. Please consider
using UTF8MB4 in order to be unambiguous.") result =
self._query(query)
Environment info : I use Mysql8, python3.6 (pymysql 0.9.2, sqlalchemy 1.2.1)
I visited posts like the one linked bellow, none of which seem to give a solution as to how to avoid this warning.
MySQL “incorrect string value” error when save unicode string in Django -> Indication is to use UTF8
N.B : The Collation in the table within mysql doesn't seem to be set to the one I specified in the create_db function within the Connection class.
The executable code:
import DataEngine.db.Connection as connection
import random
import pandas as pd
if __name__ == "__main__":
conn = connection.Connection(host="host_name", port="3306", user="username", password="password")
conn.create_db("raw_data")
conn.establish("raw_data")
l1 = []
for i in range(10):
l_nested = []
for j in range(10):
l_nested.append(random.randint(0, 100))
l1.append(l_nested)
df = pd.DataFrame(l1)
conn.save(df, "random_df")
df2 = conn.retrieve("random_df")
print(df2)
So the dataframe that is persisted in the database is :
index 0 1 2 3 4 5 6 7 8 9
0 0 11 57 75 45 81 70 91 66 93 96
1 1 51 43 3 64 2 6 93 5 49 40
2 2 35 80 76 11 23 87 19 32 13 98
3 3 82 10 69 40 34 66 42 24 82 59
4 4 49 74 39 61 14 63 94 92 82 85
5 5 50 47 90 75 48 77 17 43 5 29
6 6 70 40 78 60 29 48 52 48 39 36
7 7 21 87 41 53 95 3 31 67 50 30
8 8 72 79 73 82 20 15 51 14 38 42
9 9 68 71 11 17 48 68 17 42 83 95
My Connection class
import sqlalchemy
import pymysql
import pandas as pd
class Connection:
def __init__(self: object, host: str, port: str, user: str, password: str):
self.host = host
self.port = port
self.user = user
self.password = password
self.conn = None
def create_db(self: object, db_name: str, charset: str = "utf8mb4", collate:str ="utf8mb4_unicode_ci",drop_if_exists: bool = True):
c = pymysql.connect(host=self.host, user=self.user, password=self.password)
if drop_if_exists:
c.cursor().execute("DROP DATABASE IF EXISTS " + db_name)
c.cursor().execute("CREATE DATABASE " + db_name + " CHARACTER SET=" + charset + " COLLATE=" + collate)
c.close()
print("Database %s created with a %s charset" % (db_name, charset))
def establish(self: object, db_name: str, charset: str = "utf8mb4"):
self.conn = sqlalchemy.create_engine(
"mysql+pymysql://" + self.user + ":" + self.password + "#" + self.host + ":" + self.port + "/" + db_name +
"?charset=" + charset)
print("Connection with database : %s has been established as %s at %s." % (db_name, self.user, self.host))
print("Charset : %s" % charset)
def retrieve(self, table):
df = pd.read_sql_table(table, self.conn)
return df
def save(self: object, df: "Pandas.DataFrame", table: str, if_exists: str = "replace", chunksize: int = 10000):
df.to_sql(name=table, con=self.conn, if_exists=if_exists, chunksize=chunksize)
Some elements that might help:
Well, hex 92 and e9 is not valid utf8mb4 (UTF-8). Perhaps you were expecting ’été, assuming CHARACTER SETs cp1250, cp1256, cp1257, or latin1. Find out where that text is coming from, and let's decide whether it is valid latin1. Then we can fix the code to declare that the client is really using latin1, not utf8mb4? Or we can fix the client to use UTF-8, which would probably be better in the long run.
How do I convert my loop output to a .txt file?
My current code is: count1 = 0 for i in range(30): if i%26 == 0: b = [i+1, i+2, i+3, i+4, i+5, i+6, i+7, i+8, i+9, i+10] count1 += 1 print([count1]) print(*b, sep=' ') elif (i-10)%26 == 0: b = [i+1, i+2, i+3, i+4, i+5, i+6, i+7, i+8, i+9] count1 += 1 print([count1]) print(*b, sep= ' ') elif (i-16)%32 == 0: b = [i+1, i+2, i+3, i+4, i+5, i+6, i+7, i+8, i+9, i+10] count1 += 1 print([count1]) print(*b, sep= ' ') which produces lines: [1] 1 2 3 4 5 6 7 8 9 10 [2] 11 12 13 14 15 16 17 18 19 [3] 17 18 19 20 21 22 23 24 25 26 [4] 27 28 29 30 31 32 33 34 35 36 I'd like to output these lines in a simple text file. I'm familiar with the open and write functions, but do not know how to apply them to my specific example. Thanks!
On GNU/Linux systems execute the program in the console, add > and the name of the file. Example: Assuming that you are in the directory wich contains the executable. ./[name of the program] > [name of the file] ./helloworld > helloworld.txt This will save all the printed text in the console in a text file.
Trying to write to a Hex file(.bin) using PB12.5 using BlobEdit (Blob)
I'm trying to write to a hex file using PB12.5, I'm able to write to it without any issues but through testing noticed I will need to send a null value (00) to the file at certain points. I know if I assign null to a string, it will null out the entire string so I tried using a Blob where I can insert a null value when needed (BlobEdit(blb_data, ll_pos, CharA(0)) ) But BlobEdit() automatically inserts a null value in between each position, I don't want this as it's causing issues as I'm trying to update the hex file. I just need to add my CharA(lb_byte) to each consecutive position in the Blob. Is there any way around this or is PB just unable to do this? Below is the code: ll_test = 1 ll_pos = 1 ll_length = Len(ls_output) Do While ll_pos <= (ll_length) ls_data = Mid(ls_output, ll_pos, 2) lb_byte = Event ue_get_decimal_value_of_hex(ls_data) ll_test = BlobEdit(blb_data, ll_test, CharA(lb_byte), EncodingANSI!) ll_pos = ll_pos + 2 Loop Hex file appears as follows: 16 35 2D D8 08 45 29 18 35 27 76 25 30 55 66 85 44 66 57 A4 67 99 After Blob update: 16 00 48 00 5D 00 C3 92 00 08 00 48 00 51 00 E2
I hope to help you: ////////////////////////////////////////////////////////////////////////// // Function: f_longtohex // Description: LONG to HEXADECIMAL // Ambito: public // Argumentos: as_number //Variable long to convert to hexadecimal // as_digitos //Number of digits to return // Return: String // Example: // f_longtohex(198 , 2) --> 'C6' // f_longtohex(198 , 4) --> '00C6' ////////////////////////////////////////////////////////////////////////// long ll_temp0, ll_temp1 char lc_ret if isnull(as_digitos) then as_digitos = 2 IF as_digitos > 0 THEN ll_temp0 = abs(as_number / (16 ^ (as_digitos - 1))) ll_temp1 = ll_temp0 * (16 ^ (as_digitos - 1)) IF ll_temp0 > 9 THEN lc_ret = char(ll_temp0 + 55) ELSE lc_ret = char(ll_temp0 + 48) END IF RETURN lc_ret + f_longtohex(as_number - ll_temp1 , as_digitos - 1) END IF RETURN ''
Prevent user interaction on command window for octave
When I run below octave code the command window displays :
>> first
x =
10
20
30
40
50
60
70
80
90
100
y =
14
17
18
14
15
14
13
12
11
4
m = 10
x =
1 10
1 20
1 30
1 40
1 50
1 60
1 70
1 80
1 90
1 100
-- less -- (f)orward, (b)ack, (q)uit
I'm required to continually press (f) to complete program and view plot : plot(x(:,2), x*theta, '-');
Octave code :
x = [10
20
30
40
50
60
70
80
90
100]
y = [14
17
18
14
15
14
13
12
11
4]
m = length(y)
x = [ones(m , 1) , x]
theta = zeros(2, 1);
iterations = 10;
alpha = 0.000007;
for iter = 1:iterations
theta = theta - ((1/m) * ((x * theta) - y)' * x)' * alpha;
#theta
end
#plot(x, y, 'o');
#ylabel('Response Time')
#xlabel('Time since 0')
plot(x(:,2), x*theta, '-');
How to prevent user interaction with command window so that program runs to completion and displays prompt and not requiring
user interaction ?
To prevent your variables from printing altogether, simply add a semicolon to the end of each variable assignment: m = length(y) %// **will** print to the console m = length(y); %// will *not* print to the console To print your variables to the console, but avoid Octave pausing the output when it gets to the bottom of the screen, add more off to the beginning of your script to turn off paging. https://www.gnu.org/software/octave/doc/interpreter/Paging-Screen-Output.html Type more on to switch it back on.