How can I load a string array in Octave from a csv or txt file? The file has the format seen below. When using
dlmread("TimeData.csv", ",")
it results in useless data like 2019 - 11i.
`TIME`
2019-11-08-13.27.03 +0100
2019-11-08-13.27.08 +0100
2019-11-08-13.27.13 +0100
2019-11-08-13.27.18 +0100
2019-11-08-13.27.23 +0100
From the documentation on dlmread (emphasis by me):
Read numeric data from the text file file which uses the delimiter sep between data values.
Use textscan instead (depending on your OS, the delimiter might needs to be modified):
fid = fopen('TimeData.csv');
C = textscan(fid, '%s', 'Delimiter', '\n')
fclose(fid);
Output:
C =
{
[1,1] =
{
[1,1] = 2019-11-08-13.27.03 +0100
[2,1] = 2019-11-08-13.27.08 +0100
[3,1] = 2019-11-08-13.27.13 +0100
[4,1] = 2019-11-08-13.27.18 +0100
[5,1] = 2019-11-08-13.27.23 +0100
}
}
Hope that helps!
Similar to textscan, except it preserves empty lines.
function [textLines] = readFile (file_in_dir)
if nargin != 1 || isempty(file_in_dir)
return
end
if exist(file_in_dir, "file")
fid = fopen(file_in_dir, "r");
data = char(fread(fid));
fclose(fid);
if !strcmp(data(end), "\n")
data = [data ; "\n"];
end
data(data == "\r") = [];
newlines = [0, [1 : rows(data)](data' == "\n")];
textLines = cell(1, columns(newlines) - 1);
for i = 1 : columns(newlines) - 1
textLines{i} = data(newlines(i) + 1 : newlines(i + 1) - 1)';
end
else
textLines = {};
end
end
No need for ugly textscan shenanigans.
Just use the csv2cell function from the io package.
I've used it many times, works like a charm.
Related
I was able to take a text file, read each line, create a dictionary per line, update(append) each line and store the json file. The issue is when reading the json file it will not read correctly. the error point to a storing file issue?
The text file looks like:
84.txt; Frankenstein, or the Modern Prometheus; Mary Wollstonecraft (Godwin) Shelley
98.txt; A Tale of Two Cities; Charles Dickens
...
import json
import re
path = "C:\\...\\data\\"
books = {}
books_json = {}
final_book_json ={}
file = open(path + 'books\\set_of_books.txt', 'r')
json_list = file.readlines()
open(path + 'books\\books_json.json', 'w').close() # used to clean each test
json_create = []
i = 0
for line in json_list:
line = line.replace('#', '')
line = line.replace('.txt','')
line = line.replace('\n','')
line = line.split(';', 4)
BookNumber = line[0]
BookTitle = line[1]
AuthorName = line[-1]
file
if BookNumber == ' 2701':
BookNumber = line[0]
BookTitle1 = line[1]
BookTitle2 = line[2]
AuthorName = line[3]
BookTitle = BookTitle1 + ';' + BookTitle2 # needed to combine title into one to fit dict format
books = json.dumps( {'AuthorName': AuthorName, 'BookNumber': BookNumber, 'BookTitle': BookTitle})
books_json = json.loads(books)
final_book_json.update(books_json)
with open(path + 'books\\books_json.json', 'a'
) as out_put:
json.dump(books_json, out_put)
with open(path + 'books\\books_json.json', 'r'
) as out_put:
'books\\books_json.json', 'r')]
print(json.load(out_put))
The reported error is: JSONDecodeError: Extra data: line 1 column 133
(char 132) - adding this is right between the first "}{". Not sure
how json should look in a flat-file format? The output file as seen on
an editor looks like: {"AuthorName": " Mary Wollstonecraft (Godwin)
Shelley", "BookNumber": " 84", "BookTitle": " Frankenstein, or the
Modern Prometheus"}{"AuthorName": " Charles Dickens", "BookNumber": "
98", "BookTitle": " A Tale of Two Cities"}...
I ended up changing the approach and used pandas to read the text and then spliting the single-cell input.
books = pd.read_csv(path + 'books\\set_of_books.txt', sep='\t', names =('r','t', 'a') )
#print(books.head(10))
# Function to clean the 'raw(r)' inoput data
def clean_line(cell):
...
return cell
books['r'] = books['r'].apply(clean_line)
books = books['r'].str.split(';', expand=True)
I want to remove the directory part from a cell array of strings with filenames. Of course one way would be to loop over the cell arrray and use fileparts but I have over 1e5 files and speed really matters.
My current approach is:
fns = {"/usr/local/foo.lib", "~/baz.m", "home/rms/eula.txt", "bar.m"}
filenames = cellfun (#(fn, s) fn(s+1:end), fns,
num2cell (rindex (fns, filesep())),
"UniformOutput", false)
which gives the desired output:
fns =
{
[1,1] = /usr/local/foo.lib
[1,2] = ~/baz.m
[1,3] = home/rms/eula.txt
[1,4] = bar.m
}
filenames =
{
[1,1] = foo.lib
[1,2] = baz.m
[1,3] = eula.txt
[1,4] = bar.m
}
and takes approx 2e-5s per file. Is there a better (faster, more readable) way to do this?
EDIT I've added Sardars solution and my previous attempt with regex and some benchmark results:
fns = {"/usr/local/foo.lib", "~/baz.m", "home/rms/eula.txt", "bar.m"};
fns = repmat (fns, 1, 1e4);
tic
f1 = cellfun (#(fn, s) fn(s+1:end), fns,
num2cell (rindex (fns, "/")),
"UniformOutput", false);
toc
tic
[~, ~, ~, M] = regexp (fns, "[^\/]+$", "lineanchors");
f2 = cell2mat (M);
toc
tic
## Asnwer from Sardar Usama
f3 = regexprep(fns, '.*/', '');
toc
assert (f1, f2)
assert (f1, f3)
which gives
Elapsed time is 0.729995 seconds. (Original code with cellfun)
Elapsed time is 0.67545 seconds. (using regexp)
Elapsed time is 0.230487 seconds. (using regexprep)
Use regexprep to search the strings till the last / and replace the occurrences with an empty string.
filenames = regexprep(fns, '.*/', '');
I'm learning programming in computercraft (minecraft) and having some trouble with reading some storage cells.
The function that I'm working on shall go through all cells and add together the storage capacity to a variable in a for loop.
This is what I got so far
local cell1 = peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_2")
local cell2 = peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_3")
local cell3 = peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_4")
local cell4 = peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_5")
local cell5 = peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_6")
local cell6 = peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_7")
cells = {"cell1", "cell2", "cell3", "cell4", "cell5", "cell6"}
local totStorage
function getTotStorage(table)
for key = 1,6 do
x = table[key]
totStorage = totStorage + x.getMaxEnergyStored()
end
print(totStorage)
end
I get an error on this line
totStorage = totStorage + x.getMaxEnergyStored()
saying "Attempt to call nil".
Any suggestions?
cells = {"cell1", "cell2", "cell3", "cell4", "cell5", "cell6"}
Is shorthand for:
cells = {
-- key value
[1] = "cell1",
[2] = "cell2",
[3] = "cell3",
[4] = "cell4",
[5] = "cell5",
[6] = "cell6"
}
Assuming that your getTotStorage call is similar to below (you didn't post it)
getTotStorage(cells)
In the loop you try to call a method on x which is a string value:
for key = 1,6 do
x = table[key]
totStorage = totStorage + x.getMaxEnergyStored()
end
This is essentially trying to do:
totStorage = totStorage + ("cell1").getMaxEnergyStored()
What you could do is rearrange your code so that the values are the objects returned by peripheral.wrap:
local cells = {
peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_2"),
peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_3"),
peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_4"),
peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_5"),
peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_6"),
peripheral.wrap("tile_thermalexpansion_cell_reinforced_name_7"),
}
function getTotStorage(t)
local totStorage = 0
for i,v in ipairs(t) do
totStorage = totStorage + v.getMaxEnergyStored()
end
print(totStorage)
end
A few observations:
Use local when you can (i.e. totStorage) to limit the scope of a variable (no need to have a loop counter as a global)
Don't name variables that collide with the standard Lua library (i.e. string, table, math)
ipairs is a better way to loop through a sequence
I am trying to set a function to add min to current time object and return a new time object. I created a function for this but for some reason the vars in the functions are not been re-set / local each time I call the function.
each call to the function will use the past value of the local vars within the function, why ?
local function AddTime (MinAfter, BaseTime)
if (MinAfter == nil) then MinAfter = 0 end
if (BaseTime == nil) or (BaseTime.min == nil) or (BaseTime.hour == nil) then BaseTime = os.date("*t") end
BaseTime.hour = BaseTime.hour + math.floor((BaseTime.min + MinAfter)/60)
BaseTime.min = BaseTime.min + MinAfter - (60 * (math.floor((BaseTime.min + MinAfter)/60)))
if BaseTime.hour > 24 then BaseTime.hour = 24 end
return BaseTime
end
local sunriseHour = os.date("*t" ,os.time {year = 2014, month = 4, day = 19, yday = 259, wday = 4, hour = 6, min = 0, sec = 0, isdst = false});
-- this is the original time object in this case sunraiseHour
print ("sunriseHour time:" .. (string.format("%02d",sunriseHour.hour) .. ":" .. string.format("%02d", sunriseHour.min)));
-- first call
local newtime1= AddTime(10, sunriseHour);
print ("call 1 time:" .. string.format("%02d", newtime1.hour) .. ":" .. string.format("%02d", newtime1.min));
-- on the 1st call I get 07:10 which is right
-- 2nd call
local newtime2= AddTime(10, sunriseHour);
print ("call 1 time:" .. string.format("%02d", newtime2.hour) .. ":" .. string.format("%02d", newtime2.min));
-- on the 2nd call I get 07:20 and not 07:10 since this was the 2nd call to the function - the BaseTime var within the function was not local
When you pass sunriseHour into AddTime, it is passed by reference rather than by value, which means that any changes made to BaseTime inside of AddTime are changes to sunriseHour -- both varibles (sunriseHour and BaseTime) point to the same object.
So when you write the following in AddTime:
BaseTime.hour = BaseTime.hour + math.floor((BaseTime.min + MinAfter)/60)
BaseTime.min = BaseTime.min + MinAfter - (60 * (math.floor((BaseTime.min + MinAfter)/60)))
You're modifying sunriseHour.
It seems you don't quite understand this, because you also assign a new value to BaseTime inside of AddTime, which suggests you think you have a new object. If you want to create an altered copy of sunriseHour, then you'll need to either do that inside of AddTime, or create some kind of copy constructor for your time object.
thanks Mud,
I understood that I needed to copy my time object otherwise since I use a reference of it, it will modify the original object.
I found a copy function and use it to copy the object into a local one within the function
thanks
function table.copy(orig)
local orig_type = type(orig)
local copy
if orig_type == 'table' then
copy = {}
for orig_key, orig_value in pairs(orig) do
copy[orig_key] = orig_value
end
else -- number, string, boolean, etc
copy = orig
end
return copy
end
function AddTime (MinAdd, TimeObj)
local BaseTime = {};
local MinAfter = 0;
if (TimeObj == nil) or (TimeObj.min == nil) or (TimeObj.hour == nil) then BaseTime = table.copy(os.date("*t")) else BaseTime = table.copy(TimeObj) end;
if (MinAdd == nil) then MinAfter = 0 else MinAfter = MinAdd end;
BaseTime.hour = BaseTime.hour + math.floor((BaseTime.min + MinAfter)/60)
BaseTime.min = BaseTime.min + MinAfter - (60 * (math.floor((BaseTime.min + MinAfter)/60)))
if BaseTime.hour > 24 then BaseTime.hour = 24 end
return BaseTime
end
-- this is the original time object in this case sunraiseHour
local sunriseHour = os.date("*t" ,os.time {year = 2014, month = 4, day = 19, yday = 259, wday = 4, hour = 6, min = 0, sec = 0, isdst = false});
print ("sunriseHour time:" .. (string.format("%02d",sunriseHour.hour) .. ":" .. string.format("%02d", sunriseHour.min)));
-- first call
local newtime1= AddTime(10,sunriseHour);
print ("call 1 time:" .. string.format("%02d", newtime1.hour) .. ":" .. string.format("%02d", newtime1.min));
-- on the 1st call I get 07:10 which is right
-- 2nd call
local newtime2= AddTime(10,sunriseHour);
print ("call 2 time:" .. string.format("%02d", newtime2.hour) .. ":" .. string.format("%02d", newtime2.min));
-- on the 2nd call I get 07:20 and not 07:10 since this was the 2nd call to the function - the BaseTime var within the function become global
print ("Added time:" .. string.format("%02d", AddTime(20, sunriseHour).hour) .. ":" .. string.format("%02d", AddTime(20, sunriseHour).min));
Here is the Pseudocode for Lempel-Ziv-Welch Compression.
pattern = get input character
while ( not end-of-file ) {
K = get input character
if ( <<pattern, K>> is NOT in
the string table ){
output the code for pattern
add <<pattern, K>> to the string table
pattern = K
}
else { pattern = <<pattern, K>> }
}
output the code for pattern
output EOF_CODE
I am trying to code this in Lua, but it is not really working. Here is the code I modeled after an LZW function in Python, but I am getting an "attempt to call a string value" error on line 8.
function compress(uncompressed)
local dict_size = 256
local dictionary = {}
w = ""
result = {}
for c in uncompressed do
-- while c is in the function compress
local wc = w + c
if dictionary[wc] == true then
w = wc
else
dictionary[w] = ""
-- Add wc to the dictionary.
dictionary[wc] = dict_size
dict_size = dict_size + 1
w = c
end
-- Output the code for w.
if w then
dictionary[w] = ""
end
end
return dictionary
end
compressed = compress('TOBEORNOTTOBEORTOBEORNOT')
print (compressed)
I would really like some help either getting my code to run, or helping me code the LZW compression in Lua. Thank you so much!
Assuming uncompressed is a string, you'll need to use something like this to iterate over it:
for i = 1, #uncompressed do
local c = string.sub(uncompressed, i, i)
-- etc
end
There's another issue on line 10; .. is used for string concatenation in Lua, so this line should be local wc = w .. c.
You may also want to read this with regard to the performance of string concatenation. Long story short, it's often more efficient to keep each element in a table and return it with table.concat().
You should also take a look here to download the source for a high-performance LZW compression algorithm in Lua...