LZW Data Compression In Lua [duplicate] - function

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
LZW Compression In Lua
Here is my code for compressing data in Lua using the LZW compression method. My problem is that the function is returning the character 'T', instead of returning the full compressed string 'TOBEORNOTTOBEORNOT'. Thanks!
function compress(uncompressed)
local dict_size = 256
local dictionary = {}
w = ""
result = {}
for i = 1, #uncompressed do
local c = string.sub(uncompressed, i, i)
local wc = w .. c
if dictionary[wc] == true then
w = wc
else
dictionary[w] = ""
dictionary[wc] = dict_size
dict_size = dict_size + 1
w = c
end
if w then
dictionary[w] = ""
end
return w
end
end
compressed = compress('TOBEORNOTTOBEORTOBEORNOT')
print(compressed)

Just a hint: you return w in the for loop
EDIT some explanation
If you return your result in the loop, then the loop will do only one iteration. At the end of the first iteration, your function will finish. That makes no sense. So your return statement should come after the for loop.
Also, it is suspicious that you declare a variable result = {} and then you never use it.
So I suggest you put your return statement after the loop and you print the value of your variables at the end of in each iteration (you'd put the print statements where you have the return now), so you can see what is really happening.

Related

For loop that stores variable in progressive order

I am trying to work on a matlab script that calculates a 1x1854 matrix called N2. This routine has to be performed 1000 times because each iteration the input data files are different. I am trying to store the matrix N2 in progressive order for each iteration, like N2_1, N2_2 ecc. How should implement that?
for ii=1:1000
file1 = load(['/Users/gianmarcobroilo/Desktop/1000shifts/delays/GRV_JUGR_2021158_1648X35X35001KV03.NEWFES_delay_' num2str(ii) '.TXT']);
file2 = load(['/Users/gianmarcobroilo/Desktop/1000shifts/delays/GRV_JUGR_2021158_1648X35K35001KV03.NEWFES_delay_' num2str(ii) '.TXT']);
%%calculations...
[N,bind] = elecdensity(omega_new,closestapproach);
%
N2_num2str(ii) = N./1e6;
end
To generate those variables, change the code line
N2_num2str(ii) = N./1e6;
to
eval(['N2_' num2str(ii) '= ' 'N./1e6']);
This might be computational too expensive. Another approach I will use to avoid the usage of the "eval" command is to save the tables in a structure and each field of it will be the matrix (named N_NUMBER). So, the code will be
% Generate the struct object
myValues = struct;
% Start the for loops
for ii=1:1000
file1 = load(['/Users/gianmarcobroilo/Desktop/1000shifts/delays/GRV_JUGR_2021158_1648X35X35001KV03.NEWFES_delay_' num2str(ii) '.TXT']);
file2 = load(['/Users/gianmarcobroilo/Desktop/1000shifts/delays/GRV_JUGR_2021158_1648X35K35001KV03.NEWFES_delay_' num2str(ii) '.TXT']);
%%calculations...
[N,bind] = elecdensity(omega_new,closestapproach);
%
fieldName = ['N2_' num2str(ii)];
myValues.(fieldName) = N./1e6;
end
% Print the table 54
myValues.N2_54

Lua - Concatinate several arguments passed into a function, until condition is met, then pass string to next function

pvMessage is sent from another function, the message often comes in a few parts almost instantly. I am looking to store the pvMessages and concatinate the message to the last. Therefore a master string is created with all parts.
Example.
pvMessage #1 = thisispart1msg
pvMessage #2 = now part two is being received
pvMessage #3 = Part 3
MasterMessage = thisispart1msgnow part two is being receivedPart 3
I have tried several attempts at solving this issue. The storing of the message outside the function is proving harder then I though, I keep overwriting the previous message.
function ProcessClientMessage( pvMessage )
if StartMessage == "" then
StartMessage = pvMessage
pvMessage = ""
end
if pvMessage ~= "" then
if MiddleMessage == "" then
MiddleMessage = pvMessage
pvMessage = StartMessage .. MiddleMessage
pvMessage = ""
end
end
if pvMessage ~= "" then
if EndMessage == "" then
EndMessage = pvMessage
pvMessage = StartMessage .. MiddleMessage .. EndMessage
pvMessage = ""
end
end
if pvMessage ~= "" then
ProcessClientMessageReset()
end
end
If there are always three parts that you want to concatenate, something like this may work:
local buffer = {}
local function ProcessClientMessage(pvMessage)
if #buffer < 3 then
table.insert(buffer, pvMessage)
else
--ProcessClientMessageReset(table.concat(buffer, ""))
print(table.concat(buffer, ""))
buffer = {}
end
end
ProcessClientMessage("thisispart1msg")
ProcessClientMessage("now part two is being received")
ProcessClientMessage("Part 3")
ProcessClientMessage("more thisispart1msg")
ProcessClientMessage("some now part two is being received")
ProcessClientMessage("Part 4")
This should print:
thisispart1msgnow part two is being receivedPart 3
more thisispart1msgsome now part two is being receivedPart 4
This problem also be solved with coroutines:
local function consumer(consume)
print(consume()..consume()..consume())
return consumer(consume)
end
local process = coroutine.wrap(consumer)
process(coroutine.yield)
process('foo')
process('bar')
process('baz')
process('hello')
process('world')
process('test')
EDIT: As Egor pointed out, the order of evaluation isn't technically defined (although it's left to right in every Lua implementation I'm aware of) so here's an alternative that would work even if your interpreter was doing gcc -O3-like optimizations:
local function consumer(consume)
local a = consume()
local b = consume()
local c = consume()
print(a .. b .. c)
return consumer(consume)
end
You can make use of the "or" assignment to simplify initialization of the global variable and then concatenate the string to the result. Consider:
function ProcessClientMessage(msg)
msg_global_ = (msg_global_ or "")..msg
print(msg_global_) -- Just for debug purposes so we can print each step
end
do
ProcessClientMessage("thisispart1msg")
ProcessClientMessage("now part two is being received")
ProcessClientMessage("Part 3")
end
The variable msg_global_ contains the string being built. If it has not been added too yet, then it will be nil. In this case the or "" will be executed and default the string empty.
We then simply append the string msg.
The output looks like:
thisispart1msg
thisispart1msgnow part two is being received
thisispart1msgnow part two is being receivedPart 3
When you actually process the message, just set the global variable to nil and you are good to go again.

Spliting string of arithmetic operations in lua

The following is string of arithmetic operations in lua.
local str ='x+abc*def+y^z+10'
Can this string be splitted so that individual variables or numbers will appear? For example, say string str is splitted into table s. Then the output will be
s[1] = x
s[2] = abc
s[3] = def
s[4] = y
s[5] = z
s[6] = 10
The splitting is to be done with operators +,-,*,\,^,%
Try also this simpler pattern:
local str ='x+(abc*def)+y^z+10'
for w in str:gmatch("%w+") do
print(w)
end
You can use string.gmatch to iterate over your string.
Feel free to add other operators to the pattern.
Refer to https://www.lua.org/manual/5.3/manual.html#6.4.1
local str ='x+abc*def+y^z+10'
local s = {}
for operand in str:gmatch('[^%+%*%^]+') do
table.insert(s, operand)
end
You can use string.gmatch to do what your looking for. you would use the pattern %+%-%*%^/
local str ='x+abc*def+y^z+10'
local s = {}
for value in str:gmatch("[%+%-%*%^/]*(%w*)[%+%-%*%^/]*") do
s[#s + 1] = value
end
print(unpack(s))
Also not that if you need \ as operator as shown in your question it would need to be escaped using an additional \.
Resourse for leaning more about lua patterns: understanding_lua_patterns

Redis Lua Differetiating empty array and object

I encountered this bug in cjson lua when I was using a script in redis 3.2 to set a particular value in a json object.
Currently, the lua in redis does not differentiate between an empty json array or an empty json object. Which causes serious problems when serialising json objects that have arrays within them.
eval "local json_str = '{\"items\":[],\"properties\":{}}' return cjson.encode(cjson.decode(json_str))" 0
Result:
"{\"items\":{},\"properties\":{}}"
I found this solution https://github.com/mpx/lua-cjson/issues/11 but I wasn't able to implement in a redis script.
This is an unsuccessful attempt :
eval
"function cjson.mark_as_array(t)
local mt = getmetatable(t) or {}
mt.__is_cjson_array = true
return setmetatable(t, mt)
end
function cjson.is_marked_as_array(t)
local mt = getmetatable(t)
return mt and mt.__is_cjson_array end
local json_str = '{\"items\":[],\"properties\":{}}'
return cjson.encode(cjson.decode(json_str))"
0
Any help or pointer appreciated.
There are two plans.
Modify the lua-cjson source code and compile redis, click here for details.
Fix by code:
local now = redis.call("time")
-- local timestamp = tonumber(now[1]) * 1000 + math.floor(now[2]/1000)
math.randomseed(now[2])
local emptyFlag = "empty_" .. now[1] .. "_" .. now[2] .. "_" .. math.random(10000)
local emptyArrays = {}
local function emptyArray()
if cjson.as_array then
-- cjson fixed: https://github.com/xiyuan-fengyu/redis-lua-cjson-empty-table-fix
local arr = {}
setmetatable(arr, cjson.as_array)
return arr
else
-- plan 2
local arr = {}
table.insert(emptyArrays, arr)
return arr
end
end
local function toJsonStr(obj)
if #emptyArrays > 0 then
-- plan 2
for i, item in ipairs(emptyArrays) do
if #item == 0 then
-- empty array, insert a special mark
table.insert(item, 1, emptyFlag)
end
end
local jsonStr = cjson.encode(obj)
-- replace empty array
jsonStr = (string.gsub(jsonStr, '%["' .. emptyFlag .. '"]', "[]"))
for i, item in ipairs(emptyArrays) do
if item[1] == emptyFlag then
table.remove(item, 1)
end
end
return jsonStr
else
return cjson.encode(obj)
end
end
-- example
local arr = emptyArray()
local str = toJsonStr(arr)
print(str) -- "[]"

LZW Compression In Lua

Here is the Pseudocode for Lempel-Ziv-Welch Compression.
pattern = get input character
while ( not end-of-file ) {
K = get input character
if ( <<pattern, K>> is NOT in
the string table ){
output the code for pattern
add <<pattern, K>> to the string table
pattern = K
}
else { pattern = <<pattern, K>> }
}
output the code for pattern
output EOF_CODE
I am trying to code this in Lua, but it is not really working. Here is the code I modeled after an LZW function in Python, but I am getting an "attempt to call a string value" error on line 8.
function compress(uncompressed)
local dict_size = 256
local dictionary = {}
w = ""
result = {}
for c in uncompressed do
-- while c is in the function compress
local wc = w + c
if dictionary[wc] == true then
w = wc
else
dictionary[w] = ""
-- Add wc to the dictionary.
dictionary[wc] = dict_size
dict_size = dict_size + 1
w = c
end
-- Output the code for w.
if w then
dictionary[w] = ""
end
end
return dictionary
end
compressed = compress('TOBEORNOTTOBEORTOBEORNOT')
print (compressed)
I would really like some help either getting my code to run, or helping me code the LZW compression in Lua. Thank you so much!
Assuming uncompressed is a string, you'll need to use something like this to iterate over it:
for i = 1, #uncompressed do
local c = string.sub(uncompressed, i, i)
-- etc
end
There's another issue on line 10; .. is used for string concatenation in Lua, so this line should be local wc = w .. c.
You may also want to read this with regard to the performance of string concatenation. Long story short, it's often more efficient to keep each element in a table and return it with table.concat().
You should also take a look here to download the source for a high-performance LZW compression algorithm in Lua...