how to print all the entries in registry key - masm32

I've been working on a program using masm32, I`m trying to print out all the entries on the registry key, and append some string at the end of the output
sample entries
I expect the output to print out all the name on the image and append _KEY at the end
this is my expected output:
value 1_KEY
value 2_KEY
value 3_KEY
this is the code i've been working on so far
include \masm32\include\masm32rt.inc
include \masm32\include\advapi32.inc
includelib \masm32\lib\advapi32.lib
.data
concat db "_KEY",0
combine db 260 dup(?)
target_key db "SOFTWARE\Microsoft\Windows\CurrentVersion\Run\Run2",0
hkey dd ?
size_of_registry_value dd ?
size_of_registry_data dd ?
data_type_registry dd ?
registry_value db 260 dup (?)
registry_data db 260 dup (?)
line_break db 13,10,0
num1 DWORD ?
.code
start:
Process:
push offset hkey
push offset target_key
push offset HKEY_CURRENT_USER
call RegOpenKey
mov size_of_registry_value, 256
mov size_of_registry_data, 256
push offset size_of_registry_data
push offset registry_data
push offset data_type_registry
push 0
push offset size_of_registry_value
push offset registry_value
push num1; index which key
push hkey
call RegEnumValue
push offset registry_value
push offset combine
call szCatStr
push offset concat
push offset combine
call szCatStr
invoke StdOut, offset combine
push offset line_break
call StdOut
add num1, 1
jmp Process
exitx:
end start
this is the actual output from my code:
im trying to iterate through the index of the entries by adding 1 to the variable num1 and pushing the value to eax, it prints and append as expected, but it prints infinitely, I want to print only the names on the registry key and after that the program should stop after printing all the entries, hope somebody can help me

Related

training CNN model using word2vectorization,while invoking get_vector() .Showing error " KeyError: 'CALLDATASIZE' " while preparing train_x

set of word vectors are generated from github link:https://github.com/jianwei76/SoliAudit/blob/master/va/features/op.origin.csv.xz.
Converted this op.origin.csv.xz file to .txt file using gen_doc() function,
opfile=op.origin.csv.xz #downloaded and uploaded in google colab folder
binfile=model.bin # new binfile created to save the model generated from word2vec model
def op_name(op):
return op.rstrip('0123456789')
def filter_op(op_line):
filter_ops = [ op_name(op) for op in op_line.split() ]
return ' '.join(filter_ops)
def gen_doc(opfile, docfile):
op = pd.read_csv(opfile, compression='xz', index_col=0)
op.dropna(inplace=True)
op['Opcodes'] = op['Opcodes'].apply(filter_op)
def get_model(opfile, binfile, size=5):
docfile = 'op-doc.tmp.txt'
gen_doc(opfile, docfile)
logging.info('Training opcode word2vec...in=%s, out=%s, word-embed-size=%d' % (docfile, binfile, size))
word2vec.word2vec(docfile, binfile, size=size, verbose=True)
return word2vec.load(binfile)
```
For the Code snippet:
```
op_vecs = [ opline_to_vec(row['Opcodes'], w2v) for idx, row in data.iterrows() ]
```
invokes function
```
def opline_to_vec(line, w2v):
print('inside oplinetovec func')
ops = line.split()
print('ops and line.split done')
vec = np.zeros((len(ops), w2v.vectors.shape[1]))
print('vec computed')
for i, op in enumerate(ops):
print('each vec i values')
vec[i] = w2v.get_vector(op_name(op))***
print(vec[i])
print ('returning from opline_to_vec')
return vec
the output of op-doc-temp.txt-->
CALLDATASIZE SUB DUP ADD SWAP DUP DUP CALLDATALOAD PUSH AND SWAP PUSH ADD SWAP SWAP SWAP SWAP POP POP POP PUSH JUMP JUMPDEST PUSH MLOAD DUP DUP DUP MSTORE PUSH ADD SWAP POP POP PUSH MLOAD DUP SWAP SUB SWAP RETURN JUMPDEST PUSH PUSH DUP CALLDATASIZE SUB DUP ADD SWAP DUP DUP CALLDATALOAD PUSH AND SWAP PUSH ADD SWAP SWAP SWAP SWAP DUP CALLDATALOAD SWAP PUSH ADD SWAP DUP ADD DUP CALLDATALOAD SWAP PUSH ADD SWAP SWAP SWAP SWAP SWAP SWAP SWAP SWAP SWAP POP POP POP PUSH JUMP JUMPDEST STOP JUMPDEST CALLVALUE DUP ISZERO PUSH JUMPI PUSH DUP REVERT JUMPDEST POP PUSH PUSH DUP
I have highlighted the code snippet(vec[i] = w2v.get_vector(op_name(op))) which produces the error:
/usr/local/lib/python3.7/dist-packages/word2vec/wordvectors.py in ix(self, word)
36 Returns the index on `self.vocab` and `self.vectors` for `word`
37 """
---> 38 return self.vocab_hash[word]
39
40 def word(self, ix):
KeyError: 'CALLDATASIZE'
enter image description here
It would be really great if you could please help

It looks like you're asking a word-vectors model for the vector of a word, 'CALLDATASIZE', that it does not know.
Where did the set of word-vectors come from? (Did you train them yourself, or import them from elsewhere? How did you load them?)
Would you expect it to have a vector for that weird opcode-word? If so, skip the other wraparound steps and just check for that word, and go back to the prior steps that you thought should have created that word-vector.
If it's reasonable the set doesn't have that word, and you can't fix that gap, change your code to handle that case - perhaps by ignoring the word.

Dimension problem when converting a MATLAB .m script into an Octave compatible syntax

I want to run a MATLAB script M-file to reconstruct a point cloud in Octave. Therefore I had to rewrite some parts of the code to make it compatible with Octave. Actually the M-file works fine in Octave (I don't get any errors) and also the plotted point cloud looks good at first glance, but it seems that the variables are only half the size of the original MATLAB variables. In the attached screenshots you can see what I mean.
Octave:
MATLAB:
You can see that the dimension of e.g. M in Octave is 1311114x3 but in MATLAB it is 2622227x3. The actual number of rows in my raw file is 2622227 as well.
Here you can see an extract of the raw file (original data) that I use.
Rotation angle Measured distance
-0,090 26,295
-0,342 26,294
-0,594 26,294
-0,846 26,295
-1,098 26,294
-1,368 26,296
-1,620 26,296
-1,872 26,296
In MATLAB I created my output variable as follows.
data = table;
data.Rotationangle = cell2mat(raw(:, 1));
data.Measureddistance = cell2mat(raw(:, 2));
As there is no table function in Octave I wrote
data = cellfun(#(x)str2num(x), strrep(raw, ',', '.'))
instead.
Octave also has no struct2array function, so I had to replace it as well.
In MATLAB I wrote.
data = table2array(data);
In Octave this was a bit more difficult to do. I had to create a struct2array function, which I did by means of this bug report.
%% Create a struct2array function
function retval = struct2array (input_struct)
%input check
if (~isstruct (input_struct) || (nargin ~= 1))
print_usage;
endif
%convert to cell array and flatten/concatenate output.
retval = [ (struct2cell (input_struct)){:}];
endfunction
clear b;
b.a = data;
data = struct2array(b);
Did I make a mistake somewhere and could someone help me to solve this problem?
edit:
Here's the part of my script where I'm using raw.
delimiter = '\t';
startRow = 5;
formatSpec = '%s%s%[^\n\r]';
fileID = fopen(filename,'r');
dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'HeaderLines' ,startRow-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');
fclose(fileID);
%% Convert the contents of columns containing numeric text to numbers.
% Replace non-numeric text with NaN.
raw = repmat({''},length(dataArray{1}),length(dataArray)-1);
for col=1:length(dataArray)-1
raw(1:length(dataArray{col}),col) = mat2cell(dataArray{col}, ones(length(dataArray{col}), 1));
end
numericData = NaN(size(dataArray{1},1),size(dataArray,2));
for col=[1,2]
% Converts text in the input cell array to numbers. Replaced non-numeric
% text with NaN.
rawData = dataArray{col};
for row=1:size(rawData, 1)
% Create a regular expression to detect and remove non-numeric prefixes and
% suffixes.
regexstr = '(?<prefix>.*?)(?<numbers>([-]*(\d+[\.]*)+[\,]{0,1}\d*[eEdD]{0,1}[-+]*\d*[i]{0,1})|([-]*(\d+[\.]*)*[\,]{1,1}\d+[eEdD]{0,1}[-+]*\d*[i]{0,1}))(?<suffix>.*)';
try
result = regexp(rawData(row), regexstr, 'names');
numbers = result.numbers;
% Detected commas in non-thousand locations.
invalidThousandsSeparator = false;
if numbers.contains('.')
thousandsRegExp = '^\d+?(\.\d{3})*\,{0,1}\d*$';
if isempty(regexp(numbers, thousandsRegExp, 'once'))
numbers = NaN;
invalidThousandsSeparator = true;
end
end
% Convert numeric text to numbers.
if ~invalidThousandsSeparator
numbers = strrep(numbers, '.', '');
numbers = strrep(numbers, ',', '.');
numbers = textscan(char(numbers), '%f');
numericData(row, col) = numbers{1};
raw{row, col} = numbers{1};
end
catch
raw{row, col} = rawData{row};
end
end
end
You don't see any raw in my workspaces because I clear all temporary variables before I reconstruct my point cloud.
Also my original data in row 1311114 and 1311115 look normal.
edit 2:
As suggested here is a small example table to clarify what I want and what MATLAB does with the table2array function in my case.
data =
-0.0900 26.2950
-0.3420 26.2940
-0.5940 26.2940
-0.8460 26.2950
-1.0980 26.2940
-1.3680 26.2960
-1.6200 26.2960
-1.8720 26.2960
With the struct2array function I used in Octave I get the following array.
data =
-0.090000 26.295000
-0.594000 26.294000
-1.098000 26.294000
-1.620000 26.296000
-2.124000 26.295000
-2.646000 26.293000
-3.150000 26.294000
-3.654000 26.294000
If you compare the Octave array with my original data, you can see that every second row is skipped. This seems to be the reason for 1311114 instead of 2622227 rows.
edit 3:
I tried to solve my problem with the suggestions of #Tasos Papastylianou, which unfortunately was not successful.
First I did the variant with a struct.
data = struct();
data.Rotationangle = [raw(:,1)];
data.Measureddistance = [raw(:,2)];
data = cell2mat( struct2cell (data ).' )
But this leads to the following structure in my script. (Unfortunately the result is not what I would like to have as shown in edit 2. Don't be surprised, I only used a small part of my raw file to accelerate the run of my script, so here are only 769 lines.)
[766,1] = -357,966
[767,1] = -358,506
[768,1] = -359,010
[769,1] = -359,514
[1,2] = 26,295
[2,2] = 26,294
[3,2] = 26,294
[4,2] = 26,296
Furthermore I get the following error.
error: unary operator '-' not implemented for 'cell' operands
error: called from
Cloud_reconstruction at line 137 column 11
Also the approach with the dataframe octave package didn't work. When I run the following code it leads to the error you can see below.
dataframe2array = #(df) cell2mat( struct(df).x_data );
pkg load dataframe;
data = dataframe();
data.Rotationangle = [raw(:, 1)];
data.Measureddistance = [raw(:, 2)];
dataframe2array(data)
error:
warning: Trying to overwrite colum names
warning: called from
df_matassign at line 147 column 13
subsasgn at line 172 column 14
Cloud_reconstruction at line 106 column 20
warning: Trying to overwrite colum names
warning: called from
df_matassign at line 176 column 13
subsasgn at line 172 column 14
Cloud_reconstruction at line 106 column 20
warning: Trying to overwrite colum names
warning: called from
df_matassign at line 147 column 13
subsasgn at line 172 column 14
Cloud_reconstruction at line 107 column 23
warning: Trying to overwrite colum names
warning: called from
df_matassign at line 176 column 13
subsasgn at line 172 column 14
Cloud_reconstruction at line 107 column 23
error: RHS(_,2): but RHS has size 768x1
error: called from
df_matassign at line 179 column 11
subsasgn at line 172 column 14
Cloud_reconstruction at line 107 column 23
Both error messages refer to the following part of my script where I'm doing the reconstruction of the point cloud in cylindrical coordinates.
distLaserCenter = 47; % Distance between the pipe centerline and the blind zone in mm
m = size(data,1); % Find the length of the first dimension of data
zincr = 0.4/360; % z increment in mm per deg
data(:,1) = -data(:,1);
for i = 1:m
data(i,2) = data(i,2) + distLaserCenter;
if i == 1
data(i,3) = 0;
elseif abs(data(i,1)-data(i-1)) < 100
data(i,3) = data(i-1,3) + zincr*(data(i,1)-data(i-1));
else abs(data(i,1)-data(i-1)) > 100;
data(i,3) = data(i-1,3) + zincr*(data(i,1)-(data(i-1)-360));
end
end
To give some background information for a better understanding. The script is used to reconstruct a pipe as a point cloud. The surface of the pipe was scanned from inside with a laser and the laser measured several points (distance from laser to the inner wall of the pipe) at each deg of rotation. I hope this helps to understand what I want to do with my script.

Not sure exactly what you're trying to do, but here's a toy example of how a struct could be used in an equivalent manner to a table:
matlab:
data = table;
data.A = [1;2;3;4;5];
data.B = [10;20;30;40;50];
table2array(data)
octave:
data = struct();
data.A = [1;2;3;4;5];
data.B = [10;20;30;40;50];
cell2mat( struct2cell (data ).' )
Note the transposition operation (.') before passing the result to cell2mat, since in a table, the 'fieldnames' are arranged horizontally in columns, whereas the struct2cell ends up arranging what used to be the 'fieldnames' as rows.
You might also be interested in the dataframe octave package, which performs similar functions to matlab's table (or in fact, R's dataframe object): https://octave.sourceforge.io/dataframe/ (you can install this by typing pkg install -forge dataframe in your console)
Unfortunately, the way to display the data as an array is still not ideal (see: https://stackoverflow.com/a/55417141/4183191), but you can easily convert that into a tiny function, e.g.
dataframe2array = #(df) cell2mat( struct(df).x_data );
Your code can then become:
pkg load dataframe;
data = dataframe();
data.A = [1;2;3;4;5];
data.B = [10;20;30;40;50];
dataframe2array(data)

Fixing broken csv files using awk

I have some csv files which are broken since there are junk such as control characters, enters and delimiters in some of the fields. An example mockup data without control characters:
id;col 1;col 2;col 3
1;data 11;good 21;data 31
2;data 12;cut
in two;data 32
3;data 13;good 23;data 33
4;data 14;has;extra delimiter;data 34
5;data 15;good 25;data 35
6;data 16;cut
and;extra delimiter;data 36
7;data 17;data 27;data 37
8;data 18;cut
in
three;data 38
9;data 19;data 29;data 39
I am processing above crap with awk:
BEGIN { FS=OFS=";" } # delimiters
NR==1 { nf=NF; } # header record is fine, use the NF
NR>1 {
if(NF<nf) { # if NF less that header's NF
prev=$0 # store $0
if(getline==1) { # read the "next" line
succ=$0 # set the "next" line to succ
$0=prev succ # rebuild a current record
}
}
if(NF!=nf) # if NF is still not adequate
$0=succ # expect original line to be malformed
if(NF!=nf) # if the "next" line was malformed as well
next # well skip "next" line and move to next
} 1
Naturally above program will fail records 4 and 6 (as the actual data has several fields where the extra delimiter may lurk) and 8 (since I only read the next line if NF is too short. I can live with loosing 4 and 6 but 8 might be doable?
Also, three successive ifs scream for a for loop but it's Friday afternoon here and my day is nearing $ and I just can't spin my head around it anymore. Do you guys have any brain reserve left I could borrow? Any best practices I didn't think of?

The key her is to keep a buffer containing the lines that are still not "complete"; once they are, print them and clear the buffer:
awk -F';' 'NF>=4 && !nf {print; next} # normal lines are printed
{ # otherwise,
if (nf>0) { # continue with a "broken" line by...
buff=buff OFS $0 # appending to the buffer
nf+=NF-1 # and adding NF
} else { # new "broken" line, so...
buff=$0 # start buffer
nf=NF # set number of fields already seen
}
}
nf>=4{ # once line is complete
print buff # print it
buff=""; nf=0 # and remove variables
}' file
Here, buff is such buffer and nf an internal counter to keep track of how many fields have been seen so far for the current record (like you did in your attempt).
We are adding NF-1 when appending to the buffer (that is, from the 2nd line of a broken stream) because a line with NF==1 does not add any record but just concatenates with the last field of the previous line:
8;data 18;cut # NF==3 |
in # NF==1 but it just continues $3 | all together, NF==4
three;data 38 # NF==2 but $1 continues $3 |
With your sample input:
$ awk -F';' 'NF>=4 && !nf {print; next} {buff=(nf>0 ? buff OFS : "") $0; nf+=(nf>0 ? NF-1 : NF)} nf>=4{print buff; buff=""; nf=0}' a
id;col 1;col 2;col 3
1;data 11;good 21;data 31
2;data 12;cut in two;data 32
3;data 13;good 23;data 33
4;data 14;has;extra delimiter;data 34
5;data 15;good 25;data 35
6;data 16;cut and;extra delimiter;data 36
7;data 17;data 27;data 37
8;data 18;cut in three;data 38
9;data 19;data 29;data 39

Unknown CRC Calculation

I'm trying to reverse engineer the communication protocol from an old serial device. I've figured out most of it, but am stuck on the CRC algorithm used. I have host software that I can generate request messages, so I've included a dump of relatively short messages sent by the host software.
This appears to be entirely ASCII based, and is marginally similar to a modbus-type protocol in that it requests register data using an addressing scheme. Numbers are represented using ascii characters corresponding to a readable hex value.
| | 001| 08| 001| E948| |
|Header|Device|Func|Reg |Checksum|trailer|
| Char | Addr |code|Addr| ??? | Char |
| 0x0C | hex |hex |hex | | 0x0D |
Here are a bunch of request messages (host->device) for individual registers. Some registers are not valid, so this is not entirely contiguous, but for the most part, these requests only differ by one character/"digit". I'm 99.9% sure the last 4 characters are the checksum, but I cant figure out how they are calculated. I've tried the usual algorithms, and didn't have much luck with the CRC reveng program (although I was probably doing something wrong). Any thoughts would be greatly appreciated.
00108001E948
00108002DBD3
00108003CA5A
00108004BEE5
00108005AF6C
001080087489
0010800FEE70
00108010E119
00108011F090
00108012C20B
00108013D382
00108014A73D
00108015B6B4
0010801795A6
001080186D51
001080197CD8
0010801A8317
0010801BB18C
0010801CA005
0010801DD4BA
0010801EC533
00108022E863
00108023F9EA
00108026AE47
00108027BFCE
001080284739
0010802956B0
0010802AA97F
0010802B9BE4
0010802C8A6D
0010802DFED2
0010802EEF5B
00108032F1BB
00108033E032
00108034948D
0010803FC418
001080409FA1
001080418E28
00108042BCB3
00108043AD3A
00108044D985
00108045C80C
00108047EB1E
0010804813E9
001080490260
0010804AFDAF
0010804BCF34
0010804CDEBD
0010804DAA02
0010804EBB8B
0010804F8910
001080508679
00108053B4E2
00108054C05D
00108055D1D4
00108056E34F
00108057F2C6
001080580A31
001080591BB8
0010805AE477
0010805BD6EC
0010805CC765
0010805DB3DA
0010805F90C8
00108060AC11
00108061BD98
001080628F03
00108066C927
00108067D8AE
001080682059
0010806931D0
0010806ACE1F
0010806BFC84
0010806CED0D
0010806D99B2
0010806E883B
0010806FBAA0
00108070B5C9
001080783981
001080792808
0010807AD7C7
0010807BE55C
0010807CF4D5
0010807D806A
001080803601
001080812788
001080821513
001080847025
0010808561AC
001080865337
0010808742BE
00108088BA49
00108089ABC0
0010808A540F
0010808B6694
0010808C771D
0010808D03A2
0010808E122B
0010808F20B0
001080902FD9
001080913E50
001080920CCB
001080931D42
0010809469FD
001080964AEF
001080975B66
00108098A391
00108099B218
0010809A4DD7
0010809B7F4C
0010809C6EC5
0010809D1A7A
0010809F3968
001080A011DD
001080A10054
001080A232CF
001080A32346
001080A457F9
001080A54670
001080A674EB
001080AA73D3
001080AB4148
001080AC50C1
001080AD247E
001080B218A7
001080B3092E
001080B47D91
001080B56C18
001080B65E83
001080B74F0A
001080B8B7FD
001080B9A674
001080BA59BB
001080BB6B20
001080BC7AA9
001080BD0E16
001080BE1F9F
001080BF2D04
001080C0226D
001080C133E4
001080C2017F
001080C310F6
001080C46449
001080C575C0
001080C6475B
001080C756D2
001080C8AE25
001080CC6371
001080CD17CE
001080CE0647
001080CF34DC
001080D24C77
001080D35DFE
001080D42941
001080D538C8
001080D60A53
001080D71BDA
001080D8E32D
001080D9F2A4
001080DA0D6B
001080DB3FF0
001080DC2E79
001080DD5AC6
001080DE4B4F
001080DF79D4
001080E16734
001080E255AF
001080E34426
001080E6138B
001080E70202
001080E8FAF5
001080E9EB7C
001080EA14B3
001080EB2628
001080EC37A1
001080ED431E
001080EE5297
001080EF600C
001080F05CD5
001080F14D5C
001080F27FC7
001080F36E4E
001080F9C114
001080FA3EDB
001080FB0C40
001080FC1DC9
001080FD6976
001080FE78FF
00108104E439
00108105F5B0
00108106C72B
00108107D6A2
0010810E8637
0010810FB4AC
00108110BBC5
00108111AA4C
00108118378D
001081192604
0010811AD9CB
0010811BEB50
0010811CFAD9
0010811D8E66
0010811E9FEF
0010811FAD74
0010812091AD
001081218024
00108122B2BF
00108123A336
00108124D789
00108125C600
00108127E512
001081281DE5
001081290C6C
0010812AF3A3
0010812BC138
00108135DFD8
00108136ED43
00108137FCCA
00108138043D
0010813915B4
0010813AEA7B
0010813BD8E0
0010813CC969
0010813DBDD6
0010813EAC5F
0010813F9EC4
00108140C57D
00108141D4F4
00108142E66F
00108143F7E6
0010814592D0
00108146A04B
0010814AA773
0010814B95E8
0010814C8461
0010814DF0DE
0010814EE157
0010814FD3CC
00108150DCA5
00108151CD2C
00108152FFB7
00108153EE3E
001081549A81
001081558B08
00108156B993
00108157A81A
001081594164
0010815ABEAB
00108172CC07
00108173DD8E
00108174A931
00108175B8B8
001081779BAA
00108178635D
0010818609EB
001081871862
00108188E095
00108189F11C
0010818A0ED3
0010818B3C48
0010818C2DC1
0010818D597E
0010818E48F7
0010818F7A6C
001081907505
00108191648C
001081925617
00108193479E
001081943321
0010819522A8
001081961033
0010819701BA
00108198F94D
00108199E8C4
0010819A170B
0010819B2590
0010819C3419
0010819D40A6
0010819E512F
0010819F63B4
001081A04B01
001081A15A88
001081A26813
001081A3799A
001081A40D25
001081A51CAC
001081A62E37
001081A73FBE
001081A8C749
001081A9D6C0
001081AA290F
001081AB1B94
001081AD7EA2
001081AE6F2B
001081AF5DB0
001081B06169
001081B170E0
001081B715D6
001081B9FCA8
001081BA0367
001081BB31FC
001081BC2075
001081BD54CA
001081BE4543
001081BF77D8
001081C078B1
001081C16938
001081C25BA3
001081C34A2A
001081C43E95
001081C52F1C
001081C61D87
001081C70C0E
001081C8F4F9
001081C9E570
001081CA1ABF
001081CC39AD
001081CD4D12
001081D9A878
001081E02C61
001081E13DE8
001081E20F73
001081E31EFA
001081E46A45
001081E57BCC
001081E64957
001081E758DE
001081E8A029
001081EA4E6F
001081EB7CF4
001081EF3AD0
001081F00609
001081F11780
001081F4402D
001081F551A4
001081FC4715
001081FD33AA
001082037FE2
001082040B5D
001082051AD4
00108206284F
0010820739C6
00108208C131
00108209D0B8
0010820A2F77
0010820E6953
0010820F5BC8
0010821054A1
001082114528
0010821277B3
00108217201E
00108218D8E9
00108219C960
0010821C15BD
0010821D6102
0010821E708B
0010821F4210
001082207EC9
001082261BFF
0010822B2E5C
0010822D4B6A
0010822E5AE3
001082306711
001082317698
001082324403
00108233558A
001082342135
0010823530BC
001082360227
0010823713AE
00108238EB59
00108239FAD0
0010823A051F
0010823B3784
00108258BF89
00108259AE00
0010825A51CF
0010825B6354
0010825C72DD
0010825D0662
0010825E17EB
0010825F2570
0010826019A9
001082610820
001082623ABB
001082632B32
001082645F8D
0010826B493C
0010826C58B5
0010826D2C0A
0010826E3D83
0010826F0F18
001082700071
0010827111F8
001082722363
0010827332EA
001082744655
001082766547
0010827774CE
001082788C39
001082799DB0
0010827A627F
0010827B50E4
0010827C416D
0010827D35D2
0010827E245B
0010827F16C0
0010828083B9
001082819230
00108282A0AB
00108284C59D
00108285D414
00108286E68F
00108287F706
001082880FF1
001082891E78
00108294DC45
00108295CDCC
0010829907A0
0010829AF86F
0010829BCAF4
0010829CDB7D
0010829DAFC2
-----More data request strings-----
Here are some more data strings that I haven't fully decoded Again, the first 3 characters are the slave address (similar to a modbus device address scheme), the next 2 characters are the function code. "10" is a data buffer request and I have not decoded this. Interestingly, there are non-numeric characters in this particular request, which is probably a big clue as to the underlying checksum calculation.
00110PPF3000000500351
00210PPF300000050DB2F
00310PPF3000000509305
00410PPF30000005063C2
00510PPF3000000502BE8
00610PPF300000050F396
00710PPF300000050BBBC
00810PPF3000000501A09
00910PPF3000000505223
00A10PPF30000005077DE
00B10PPF300000050AFA0
00C10PPF300000050E78A
00D10PPF300000050174D
00E10PPF3000000505F67
00F10PPF3000000508719
01010PPF300000050C56B
01110PPF3000000508D41
01210PPF300000050553F
01310PPF3000000501D15
10010PPF3000000505B74
00110PF3F30000005017B2
00210PF3F3000000508D93
00310PF3F3000000500383
00410PF3F300000050B1C0
00510PF3F3000000503FD0
00610PF3F300000050A5F1
00710PF3F3000000502BE1
00810PF3F300000050C966
00910PF3F3000000504776
00A10PF3F3000000506B39
00B10PF3F300000050F118
00C10PF3F3000000507F08
00D10PF3F300000050CD4B
00E10PF3F300000050435B
00F10PF3F300000050D97A
01010PF3F30000005089AD
01110PF3F30000005007BD
01210PF3F3000000509D9C
01310PF3F300000050138C
10010PF3F3000000506145
00110SPF30000005084BF
00210SPF3000000505CC1
00310SPF30000005014EB
00410SPF300000050E42C
00510SPF300000050AC06
00610SPF3000000507478
00710SPF3000000503C52
00810SPF3000000509DE7
00910SPF300000050D5CD
00A10SPF300000050F030
00B10SPF300000050284E
00C10SPF3000000506064
00D10SPF30000005090A3
00E10SPF300000050D889
00F10SPF30000005000F7
01010SPF3000000504285
01110SPF3000000500AAF
01210SPF300000050D2D1
01310SPF3000000509AFB
10010SPF300000050DC9A
Function code "09" is a contiguous parameter group, which as best I can tell is followed by the register address (3 numeric characters, followed by the register count, also 3 numeric characters)
0010900013F5CB6 <==== Error, should be 0010900103F5CB6
0020900103F8AB1
0030900103FC74C
0040900103F2EAE
0050900103F6353
0060900103FB554
0070900103FF8A9
0080900103F6E81
0090900103F237C
00A0900103FB278
00B0900103F647F
00C0900103F2982
00D0900103FC060
00E0900103F8D9D
00F0900103F5B9A
0100900103F3D6C
0110900103F7091
0120900103FA696
0130900103FEB6B
1000900103F44DA
0010904003E5F86
0020904003E8981
0030904003EC47C
0040904003E2D9E
0050904003E6063
0060904003EB664
0070904003EFB99
0080904003E6DB1
0090904003E204C
00A0904003EB148
00B0904003E674F
00C0904003E2AB2
00D0904003EC350
00E0904003E8EAD
00F0904003E58AA
0100904003E3E5C
0110904003E73A1
0120904003EA5A6
0130904003EE85B
1000904003E47EA

Here you go, in C:
#include <stddef.h>
unsigned crc16old(unsigned crc, unsigned char *buf, size_t len)
{
int k;
if (buf == NULL)
return 0xffff;
while (len--) {
crc ^= *buf++;
for (k = 0; k < 8; k++)
crc = crc & 1 ? (crc >> 1) ^ 0x8408 : crc >> 1;
}
return crc;
}
You call crc16old() with buf equal to NULL to get the initial CRC. Then you update the CRC using the routine with a series of buffers and lengths.

I don't have a slam dunk for you, but here are some observations:
First off, the 4-byte messages all map to unique 2-byte suffixes (the possible CRCs) in your example, although none of the messages are repeated in the snippet provided, so that doesn't prove a great deal. It would be worth looking to see whether the same messages get the same 2-byte suffix on every run.
Initially, trying out searching with reveng with the 4 data-points the documentation said were needed looked promising:
>reveng -w 16 -s 00108001E948 00108002DBD3 00108003CA5A 00108004BEE5
width=16 poly=0x1189 init=0x18b1 refin=false refout=false xorout=0x0000 check=0xa5c2 name=(none)
Then testing this with the next messages in your list:
>reveng -w 16 -c -p 1189 -i 18b1 00108005
af6c
>reveng -w 16 -c -p 1189 -i 18b1 00108008
7489
which was correct, but unfortunately stopped being correct once we hit 0010800FEE70.
It occurs to me though that the init parameter here may be variable, possibly based on other data being transmitted, so I wrote the following Python script to see how messages got grouped based on the init, if we assume the poly remains the same:
import subprocess
with open("unknownprotocol.txt") as f:
inits = dict()
for line in f.readlines():
line = line.rstrip('\n')
if len(line) < 12:
continue
data = line[0:8]
cmd = "reveng -w 16 -p 1189 -s " + line[0:12]
process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
result = process.communicate()[0]
loc = result.find("init=0x")
init = result[loc+7:loc+11].upper()
if inits.has_key(init):
inits[init].append(line)
else:
inits[init] = [line]
morethanfour = 0
for key in sorted(inits):
print "%s: %s" % (key, inits[key])
if(len(inits[key]) > 4):
morethanfour += 1
print morethanfour, "inits with more than 4 data-points"
With that, we get the following output:
0026: ['001080EC37A1']
0567: ['0010811CFAD9']
06E1: ['001081BB31FC', '001081BD54CA', '001081BF77D8']
0AF6: ['001080AA73D3']
0F7A: ['0010819C3419']
0FB7: ['0010815ABEAB']
1030: ['0010826C58B5']
10E9: ['00108294DC45', '00108295CDCC', '0010829907A0']
11F3: ['001080DB3FF0', '001080DD5AC6', '001080DF79D4']
1275: ['0010807CF4D5']
12AC: ['001080803601', '001080812788', '001080821513', '001080847025', '0010808561AC', '001080865337', '0010808742BE', '00108088BA49', '00108089ABC0']
149B: ['001080FA3EDB', '001080FE78FF']
17D0: ['0010809B7F4C', '0010809D1A7A', '0010809F3968']
188C: ['001081EB7CF4', '001081EF3AD0']
18B1: ['00108001E948', '00108002DBD3', '00108003CA5A', '00108004BEE5', '00108005AF6C', '001080087489']
1AE0: ['0010822E5AE3']
1AF4: ['0010821054A1', '001082114528', '0010821277B3', '00108217201E', '00108218D8E9', '00108219C960']
1DCD: ['0010801BB18C', '0010801DD4BA']
1E4B: ['001080BC7AA9']
1F88: ['0010820F5BC8']
231B: ['0010814C8461']
23D6: ['0010818A0ED3', '0010818E48F7']
25E1: ['001081F00609', '001081F11780', '001081F4402D', '001081F551A4']
29CB: ['0010810E8637']
29DF: ['00108135DFD8', '00108136ED43', '00108137FCCA', '00108138043D', '0010813915B4']
2CA3: ['0010812BC138']
2E4B: ['001081CD4D12']
3409: ['0010802C8A6D']
36E1: ['001080CC6371']
39F4: ['0010825B6354', '0010825D0662', '0010825F2570']
3B8C: ['001081A04B01', '001081A15A88', '001081A26813', '001081A3799A', '001081A40D25', '001081A51CAC', '001081A62E37', '001081A73FBE', '001081A8C749', '001081A9D6C0']
3BB1: ['0010804BCF34', '0010804DAA02', '0010804F8910']
3C9C: ['0010827A627F', '0010827E245B']
3ECD: ['001080508679', '00108053B4E2', '00108054C05D', '00108055D1D4', '00108056E34F', '00108057F2C6', '001080580A31', '001080591BB8']
3ED9: ['0010806ACE1F', '0010806E883B']
4382: ['0010813CC969']
456C: ['001081BA0367', '001081BE4543']
4946: ['00108140C57D', '00108141D4F4', '00108142E66F', '00108143F7E6', '0010814592D0', '00108146A04B']
497B: ['001080AB4148', '001080AD247E']
4FBC: ['001081FC4715']
527E: ['001080DA0D6B', '001080DE4B4F']
545D: ['0010809A4DD7']
5490: ['0010805CC765']
5716: ['001080FB0C40', '001080FD6976']
596D: ['0010822B2E5C', '0010822D4B6A']
5B01: ['001081EA4E6F']
5B28: ['0010803FC418']
5C05: ['0010820A2F77', '0010820E6953']
5C11: ['001082306711', '001082317698', '001082324403', '00108233558A', '001082342135', '0010823530BC', '001082360227', '0010823713AE', '00108238EB59', '00108239FAD0']
5CBC: ['001080C0226D', '001080C133E4', '001080C2017F', '001080C310F6', '001080C46449', '001080C575C0', '001080C6475B', '001080C756D2', '001080C8AE25']
5E40: ['0010801A8317', '0010801EC533']
5E54: ['00108022E863', '00108023F9EA', '00108026AE47', '00108027BFCE', '001080284739', '0010802956B0']
605B: ['0010818B3C48', '0010818D597E', '0010818F7A6C']
6304: ['001081D9A878']
6527: ['001081907505', '00108191648C', '001081925617', '00108193479E', '001081943321', '0010819522A8', '001081961033', '0010819701BA', '00108198F94D', '00108199E8C4']
6A46: ['0010810FB4AC']
6A7B: ['001080E16734', '001080E255AF', '001080E34426', '001080E6138B', '001080E70202', '001080E8FAF5', '001080E9EB7C']
6DC6: ['001081CA1ABF']
6F2E: ['0010812AF3A3']
6F3A: ['00108110BBC5', '00108111AA4C', '00108118378D', '001081192604']
70A9: ['0010821C15BD']
7416: ['001080B218A7', '001080B3092E', '001080B47D91', '001080B56C18', '001080B65E83', '001080B74F0A', '001080B8B7FD', '001080B9A674']
7828: ['00108070B5C9', '001080783981', '001080792808']
783C: ['0010804AFDAF', '0010804EBB8B']
78F1: ['0010808C771D']
7A6D: ['0010826019A9', '001082610820', '001082623ABB', '001082632B32', '001082645F8D']
7A79: ['0010825A51CF', '0010825E17EB']
7AB4: ['0010829CDB7D']
7D54: ['0010806BFC84', '0010806D99B2', '0010806FBAA0']
7F11: ['0010827B50E4', '0010827D35D2', '0010827F16C0']
8081: ['001081C078B1', '001081C16938', '001081C25BA3', '001081C34A2A', '001081C43E95', '001081C52F1C', '001081C61D87', '001081C70C0E', '001081C8F4F9', '001081C9E570']
8269: ['0010812091AD', '001081218024', '00108122B2BF', '00108123A336', '00108124D789', '00108125C600', '00108127E512', '001081281DE5', '001081290C6C']
827D: ['0010811AD9CB', '0010811E9FEF']
8715: ['0010813BD8E0', '0010813DBDD6', '0010813F9EC4']
873C: ['001080EA14B3', '001080EE5297']
8860: ['0010819A170B', '0010819E512F']
8B2B: ['001081FD33AA']
8DEC: ['001080AC50C1']
9007: ['0010805BD6EC', '0010805DB3DA', '0010805F90C8']
9381: ['001080FC1DC9']
9546: ['001081AB1B94', '001081AD7EA2', '001081AF5DB0']
956F: ['0010807AD7C7']
957B: ['001080409FA1', '001080418E28', '00108042BCB3', '00108043AD3A', '00108044D985', '00108045C80C', '00108047EB1E', '0010804813E9', '001080490260']
972A: ['0010826E3D83']
973E: ['00108258BF89', '00108259AE00']
9951: ['001080BA59BB', '001080BE1F9F']
A401: ['0010814AA773', '0010814EE157']
A415: ['00108172CC07', '00108173DD8E', '00108174A931', '00108175B8B8', '001081779BAA', '00108178635D']
A4CC: ['0010818C2DC1']
A82B: ['001081B06169', '001081B170E0', '001081B715D6', '001081B9FCA8']
B142: ['001082037FE2', '001082040B5D', '001082051AD4', '00108206284F', '0010820739C6', '00108208C131', '00108209D0B8']
B156: ['0010823A051F']
B1FB: ['001080CE0647']
B307: ['00108010E119', '00108011F090', '00108012C20B', '00108013D382', '00108014A73D', '00108015B6B4', '0010801795A6', '001080186D51', '001080197CD8']
B313: ['0010802AA97F', '0010802EEF5B']
B43E: ['0010821D6102', '0010821F4210']
B646: ['001081E02C61', '001081E13DE8', '001081E20F73', '001081E31EFA', '001081E46A45', '001081E57BCC', '001081E64957', '001081E758DE', '001081E8A029']
B67B: ['0010800FEE70']
B91A: ['001080902FD9', '001080913E50', '001080920CCB', '001080931D42', '0010809469FD', '001080964AEF', '001080975B66', '00108098A391', '00108099B218']
B9C3: ['0010806CED0D']
BB5F: ['0010828083B9', '001082819230', '00108282A0AB', '00108284C59D', '00108285D414', '00108286E68F', '00108287F706', '001082880FF1', '001082891E78']
BB86: ['0010827C416D']
BC66: ['0010808B6694', '0010808D03A2', '0010808F20B0']
BE23: ['0010829BCAF4', '0010829DAFC2']
BF39: ['001080D24C77', '001080D35DFE', '001080D42941', '001080D538C8', '001080D60A53', '001080D71BDA', '001080D8E32D', '001080D9F2A4']
C1F0: ['0010811BEB50', '0010811D8E66', '0010811FAD74']
C276: ['001081BC2075']
C48C: ['00108104E439', '00108105F5B0', '00108106C72B', '00108107D6A2']
C498: ['0010813AEA7B', '0010813EAC5F']
C4B1: ['001080EB2628', '001080ED431E', '001080EF600C']
CBED: ['0010819B2590', '0010819D40A6', '0010819F63B4']
CE91: ['0010818609EB', '001081871862', '00108188E095', '00108189F11C']
D1DB: ['001082700071', '0010827111F8', '001082722363', '0010827332EA', '001082744655', '001082766547', '0010827774CE', '001082788C39', '001082799DB0']
D347: ['0010809C6EC5']
D38A: ['0010805AE477']
D39E: ['00108060AC11', '00108061BD98', '001080628F03', '00108066C927', '00108067D8AE', '001080682059', '0010806931D0']
D4A7: ['0010826B493C', '0010826D2C0A', '0010826F0F18']
D564: ['001080DC2E79']
D6CB: ['001081AA290F', '001081AE6F2B']
D6E2: ['0010807BE55C', '0010807D806A']
D95A: ['0010801CA005']
DADC: ['001080BB6B20', '001080BD0E16', '001080BF2D04']
E2F0: ['00108150DCA5', '00108151CD2C', '00108152FFB7', '00108153EE3E', '001081549A81', '001081558B08', '00108156B993', '00108157A81A', '001081594164']
E78C: ['0010814B95E8', '0010814DF0DE', '0010814FD3CC']
E7B1: ['001080A011DD', '001080A10054', '001080A232CF', '001080A32346', '001080A457F9', '001080A54670', '001080A674EB']
EADC: ['001081CC39AD']
F09E: ['0010802B9BE4', '0010802DFED2']
F276: ['001080CD17CE', '001080CF34DC']
F2DB: ['0010823B3784']
F5E2: ['00108032F1BB', '00108033E032', '00108034948D']
F7A7: ['001082207EC9', '001082261BFF']
F7B3: ['0010821E708B']
F9DC: ['001080F05CD5', '001080F14D5C', '001080F27FC7', '001080F36E4E', '001080F9C114']
FD63: ['0010825C72DD']
FDAE: ['0010829AF86F']
FF26: ['0010804CDEBD']
FFEB: ['0010808A540F', '0010808E122B']
29 inits with more than 4 data-points
29 cases where there are more than 4 data-points with the same init suggests that there might be something in this.
I'd suggest having a look at the data again and seeing whether it's possible that the CRC is XOR-ed with other data, e.g. data sent by the other side, or with some value that is sent at the start of a conversation, etc. as that would explain init being different in different cases.

Well, this one still has me perplexed. Fortunately I can generate lots of request messages, so I figured I'd do just that and see if I could get some checksum duplicates (the hope being some pattern in the message would pop out at me). I let it run for a few hours generating about 10000 unique messages. Of that, there were 484 duplicate checksums that were each generated by 2-3 different messages.
Recapping the general message structure as I understand it:
[005=device address, ascii encoded hex number][08=function code][2BD=register address, ascii encoded hex number][AB02=checksum]
function code 09 has the register address followed by the register count (also 3 characters)
Here is an interesting set of (4) messages. The 2 different checksums differ by one. I've been staring at these for a while and cant see the correlation. (These messages are ASCII strings, although these particular ones only contain 0-F characters)
005082BDAB02
00D09092002AB02
00B0827AAB03
00D08168AB03
Here are the hex equivalents of the above
303035303832424441423032
303044303930393230303241423032
303042303832374141423033
303044303831363841423033
reveng came up with "no models found" on these 4 messages, but I'm no expert on using that tool.
Heres a bunch more distinct messages that collide to the same checksum
0110930C0020014
005081FF0014
006081A5007D
011091CC002007D
007081870098
011090D60020098
00B0934A002009C
00D09058002009C
00A08284009C
01109004002010D
008080AD010D
009082170146
011090F50010146
005080A00171
00F092820010171
005082370302
00B092A40020302
00C0932E0020415
00A080F90415
00B093340020452
00A082A60452
011090CC0020456
007081A50456
00D0915800204B7
00B0924A00204B7
011091040020526
009080AD0526
004090FE001053C
00D0801D053C
0110907A00205C3
0080808F05C3
004092940020600
007081E30600
00D08124060F
00F0934C002060F
009083450630
00B090AC0020630
00B0919E0020652
00C0935E0010652
01009002002076A
00D0805B076A
00D0934800207A5
00B0905A00207A5
00B082A6082F
00B09034002082F
00D0811A084D
00B08208084D
00C0828408CA
00B0914A00208CA
00D0925800208CA
0050818708CE
011092D600208CE
007080A00927
00F090820010927
011090C40020932
009080F70932
010090BE0020975
009082A80975
00B091920020B36
007082B90B36
00B090A40020B54
007082370B54
007080E60B71
00C0918C0020B71
00D090480020BD8
00B0935A0020BD8
00B082840CE1
00B0904A0020CE1
00D093580020CE1
011091C40020D19
008080F70D19
008082A80D5E
010091BE0020D5E
0080833F0E0F
00B0902C0020E0F
00F0915A0010EF0
00A090880020EF0
006082B90F1D
00B090920020F1D
006080E60F5A
00C0908C0020F5A
006082370F7F
00B091A40020F7F
00D080F91092
010091AE0021092
0080819C116A
00C09060002116A
0110919E00212BC
009080C112BC
0100935E00112BC
0070835A13E9
0100913E00213E9
00C092C200213ED
00A0805B13ED
00B082941539
00F092BA0021539
00D092D8002167D
00B091EA002167D
0100916300116F9
009080EC16F9
0080832F17D7
00F092DC00217D7
00F091BA0021944
00A082941944
00D09168002197B
00B0927A002197B
0060825D1AB3
00F092D40021AB3
00F090740021ADA
005082271ADA
0040921E0021ADE
00C081241ADE
00F092250011AF8
007080E71AF8
00810E01F3F3000000501B88
009081EF1B88
00C090C20021BBB
00C0805B1BBB
0100933E0021BBF
0050835A1BBF
006081971D6B
004092D00021D6B
00C091C20021F90
00B0805B1F90
00F0927C0021FE8
009083551FE8
00A081272013
0040901E0012013
00F091BC0022032
007080D52032
010092F20022157
007081CC2157
00D091B80022198
00B092CA0022198
00B082382225
00D0812A2225
01009242002222C
01109082001222C
00B0811B222C
0110925C002226B
00D08074226B
008080C2240C
0100925E002240C
006080D52419
00F090BC0022419
005080CF2470
00F0921C0022470
00C092EE0022511
00C082D32511
00D08126251D
00B09225001251D
010093420022607
00C0811B2607
00608262266A
00C0927E002266A
00D092C800226A1
00B091BA00226A1
00F0911C002280D
006080CF280D
005081CC2901
010090F20022901
00F090B4002297D
008081C7297D
004090EE0022A6B
0080826F2A6B
010090420022A7A
011092820012A7A
00B092BA0022ADC
00D091C80022ADC
00C092720022B0E
008083102B0E
007080CF2C26
00F0901C0022C26
00D0804A2C29
00B083582C29
009080DA2C33
010092FE0022C33
00C090EE0022D47
00A082D32D47
00C0922D0012E1E
006080A22E1E
010091420022E51
00A0811B2E51
00D092080022E9E
00B0911A0022E9E
010090940012F0F
011092540022F0F
00B0933A0023040
00D090280023040
008081B730A5
00B0924400230A5
00C0804A30F8
00F0909200230F8
00B0829A3113
00D081883113
00E090880023115
00B0915A0013115
00C081B931FF
010092EE00231FF
00A082873207
00C0918A0023207
0040929A0023212
002091880023212
008081F132F3
00F0918E00232F3
00D092380023352
00B0912A0023352
00D09128002346B
00B0923A002346B
00B09344002348E
009081B7348E
00F0919200234D3
00B0804A34D3
00C0929A002353E
00E09188002353E
00B0834835F1
00D0805A35F1
00C0924200236C2
00B0908200136C2
00F0908E00236D8
009081F136D8
00B090FC00236EC
00A0807436EC
00D093380023779
00B0902A0023779
0090827F3798
011092EC0023798
00C0919A0023943
00E092880023943
00A081B939A9
010090EE00239A9
00A0813739CB
0110931C00139CB
00C082873A51
00E090980023A51
006080EB3AF9
00610F01PF3000000503AF9
011092E40023AFC
0070830D3AFC
003093280023B11
00910F01F3F3000000503B11
00B092540023BE1
00C090940013BE1
00D0813A3BFD
00B082283BFD
005081E03CCD
004090940013CCD
002092880023E6F
0040919A0023E6F
00B082873E7A
00E091980023E7A
00C090420023E94
00B092820013E94
011092CC0013E9B
005081A63E9B
00C080743EBA
00B092FC0023EBA
008082053FA7
0110926C0023FA7
00510F01F3F3000000503FF1
004092FE0023FF1
010090DA0024001
007083364001
00A080354005
00C093460024005
002090480024109
0040935A0024109
00C0904A002411C
00E09358002411C
004091FA0024160
00A0814C4160
0040914A002421B
00209258002421B
00B082FA421F
00F09320002421F
002090F80024272
007081A14272
00F091A00024276
00A082A24276
00C092AC002439B
00508086439B
0080808B43E7
0040933C00243E7
00C09017001440C
00B082D5440C
00E092580024537
00C0914A0024537
00D0816A4545
00B082784545
00C0935A0024625
00E090480024625
005081FB4630
002093580024630
0040904A0024630
00C082FA4634
00F092200024634
00A0828046B8
00F091FE00246B8
00E090A80014719
0060820F4719
0040923C00247CC
0090808B47CC
0100907A002483E
0060834E483E
005083364857
010092DA0024857
00E09158002494A
00C0924A002494A
00F09330002495B
00608233495B
00B091AE0024981
00910SF3F3000000504981
005081A14A24
002092F80024A24
006081FB4A4D
0040934A0024A4D
002090580024A4D
008081AE4A58
00E093480024A58
00C0905A0024A58
004090A60014A68
0060830C4A68
00B082804AC5
00F092FE0024AC5
00C0919E0024BAF
00B0935E0014BAF
00C090AC0024BCD
007080864BCD
009081EC4C25
00F090500014C25
0100909F0014C3C
006081584C3C
004092FA0024D1D
00B0814C4D1D
00C0934A0024D61
00E090580024D61
009082444D65
010090D60024D65
007082334D70
00F092300024D70
002093480024D74
0040905A0024D74
00B082A24E0B
00F092A00024E0B
007081FB4E66
002091580024E66
0040924A0024E66
00E092480024E73
009081AE4E73
00C090F60024F03
007080A44F03
00B083085291
00D0801A5291
0050821F52BC
00A0917800152BC
0040927A00253AA
0020916800253AA
00C0927A0025486
00E091680025486
00510NF3F3000000505586
006081E65586
00A0831956BD
002090C800156BD
0060823258D2
00C0909F00158D2
00B0903F0015987
008081EB5987
00C091EA0025B80
00E092D80025B80
002092D80025CAC
004091EA0025CAC
006081595DB5
004092E60025DB5
0090809B5E14
0110913E0025E14
0100919E0025F41
0110935E0015F41
00C082815F67
00E10PPF3000000505F67
004092BA002600D
002091C8002600D
00C10SPF3000000506064
0040901A0026064
00B081D96104
00910F006104
008082D9626F
00C10E00626F
00409106002626F
0110909400162F2
0100925400262F2
00C0911A0026363
00E092080026363
00F090600026372
00C080C86372
0040911A002644F
00D08070644F
00209208002644F
007082666465
00A092D80026465
00C081D9652F
00810F00652F
00B10E006644
009082D96644
00D080B8667B
01109230002667B
00B082D7671E
00A09268002671E
00E091C80026721
00C092BA0026721
011090420026787
010092820016787
00C09206002693E
00D08175693E
011092860026A0D
00D0804E6A0D
004093060026A39
00A10E006A39
00A080C86B24
00F092600026B24
004092CA0026B49
002091B80026B49
00C091BA0026B5C
00E092C80026B5C
00A091680026B63
00A082D76B63
009081C36B72
00B093000026B72
005082666C33
00A090D80026C33
00A092780026C5A
0060821E6C5A
00E091B80026C65
00C092CA0026C65
004091BA0026C70
002092C80026C70
005082206E65
00B092E00016E65
00B082486F2D
00D0815A6F2D
008081C36F59
00B092000026F59
011092420026FD1
010090820016FD1
0090830470D9
00C0933600270D9
00E091A800271A9
0070820E71A9
00A0810F71AD
0100910600271AD
0090827B71BC
004091AA00271BC
00C080277299
00E091E90017299
00A082E772BB
00C090AA00272BB
007081E472BF
0100931600272BF
00E0913800272D2
00B082BD72D2
00B090420027369
00C092820017369
009080EE73CF
010092BA00273CF
0030919800274AB
005081FA74AB
00D0928800274BE
00B0919A00274BE
00C0923600274F2
0080830474F2
004090AA0027597
0080827B7597
00C0913A00275EB
0050827675EB
00C0817575EF
010092C600275EF
00C09254002761C
00B09094001761C
00F0925000276E8
00F10E0076E8
00B0825876F5
00D0814A76F5
00C0932A00276F9
00C082BD76F9
00D0918800278C3
008081AF78C3
00B0929A00278C3
00D0802A78F9
00B0833878F9
0040912A0027983
008082017983
002092380027983
006082767996
00E091280027996
00C0923A0027996
00B0919600279DA
005080FD79DA
0100930600279FB
00C0810F79FB
00E093380027A84
00C0902A0027A84
00B090E10017A88
00A082587A88
002090280027A91
0040933A0027A91
00F091500027A95
00E10E007A95
00C092AA0027AED
00C082E77AED
011091DE0027B3B
00410NPF3000000507B3B
00C090820017B3F
00B092420027B3F
008082E97BB7
011092000027BB7
0040915A0017BC4
003090880027BC4
00D090880027CE8
009081AF7CE8
00C0915A0017CE8
009082017DA8
002093380027DA8
0040902A0027DA8
00A081757DB9
010090C60027DB9
00E090280027DBD
007082767DBD
00C0933A0027DBD
00B0810F7DD0
010092060027DD0
00E092380027EAF
00A082BD7EAF
00C0912A0027EAF
002091280027EBA
0040923A0027EBA
009082E97F9C
011093000027F9C
00D080C87FA3
004091360027FA3
0100931A0027FDB
009080B67FDB
006082D48028
00A092020028028
003092EC0028045
00C081CD8045
00D092250018102
00A082328102
00D090740028120
00C080D28120
00D08158816B
00B0824A816B
00D091BA00282BE
00B092C800282BE
002090E20028310
005083198310
0060811E833E
0030927C002833E
00C082658353
00A092B20028353
003090DC0028357
005081668357
00A093020028403
007082D48403
00D09174002850B
00B080D2850B
00B091B80028587
00D092CA0028587
003092D40028665
009080148665
003090EC0028813
00A081CD8813
00209272002883D
00D080BA883D
00B081D7887A
0030926C002887A
009081D98920
00B10F008920
00810E008A4B
00C082D98A4B
00D0911A0028A81
00B092080028A81
003091E60018AB1
0080829D8AB1
003092DC0028B01
007081668B01
00A090B20028B05
00A082658B05
00D091EC0028B14
009081118B14
00B0821F8B6C
00A092120028B6C
00C10F008D0B
008081D98D0B
00B082D98E60
00910E008E60
00D092BA0028EC3
00B091C80028EC3
00D090EC0028F3F
008081118F3F
00A093120028F47
00C0821F8F47
00B091280029074
00D0923A0029074
00A08233908B
00A0918C002908B
00E0929A0029121
00C091880029121
0020934200291DA
00D10F0091DA
004090880029226
0030915A0019226
00E0924200292DD
00D0908200192DD
00B093380029366
00D0902A0029366
00B09028002945F
00D0933A002945F
00C09088002950A
00D0915A001950A
0030908200195F1
00E10F0095F1
0020924200295F1
00409188002960D
0020929A002960D
00D081939618
00E0918A0029618
00D0802896BF
00B0833A96BF
00D0912A002974D
00B09238002974D
006082FA97B2
00A0929C00297B2
0020909400198D2
0030925400298D2
00209142002998C
00F10F00998C
0080804299C0
0020902E00199C0
00E090F200299F0
006082E499F0
004092880029A70
0020919A0029A70
00E090420029A8B
00D092820019A8B
005082FA9BCF
00A0919C0029BCF
00C0830C9BEE
002092FE0029BEE
00E0919A0029D5C
00C092880029D5C
003092820019DA7
002090420029DA7
00E091F20029DDB
007082E49DDB
00B081FB9FE0
00F10NPF3000000509FE0
00E090940019FFE
00D092540029FFE
00E0932E002A00A
00B080FDA00A
00B09158002A0A8
00D0924A002A0A8
00209332002A106
00A081FAA106
00C0831AA124
00209163001A124
00E09232002A201
00908144A201
00E0935E001A24D
00D0919E002A24D
00710NPF300000050A345
00210E00A345
00B09348002A3BA
00D0905A002A3BA
00B09058002A483
00D08190A483
00D0934A002A483
00A08085A4AD
00E091F0002A4AD
00B0831AA50F
00D08008A50F
0030919E002A561
0020935E001A561
00E09332002A62A
00808144A62A
00D092A4002A71D
00D10NF3F300000050A71D
00508249A774
00D09024002A774
00E092F0002A8D0
00B08085A8D0
00B09358002A8FE
00D0904A002A8FE
005082BDAB02
00D09092002AB02
00B0827AAB03
00D08168AB03
00608249AB09
00D09324002AB09
00B09258002ACD5
00D0914A002ACD5
00B081FAAD7B
00209032002AD7B
00B09048002AFC7
00D0935A002AFC7
00A08190B004
011092F8002B004
00D090E4001B0C0
00608060B0C0
00B092D8002B262
00D091EA002B262
003090F5001B69B
00C081E5B69B
003092C4002B6B9
00A08325B6B9
00A090EE002B6BD
00708026B6BD
00B08220B7B5
00D092BC002B7B5
00D0901C002B7DC
00A0825CB7DC
011090F8002B852
00C08190B852
00209053001B8C3
00808279B8C3
003091C4002BAC4
00B08325BAC4
00A08220BBC8
00D091BC002BBC8
00908154BBD9
00A090E2002BBD9
0080806DBCC1
00309274001BCC1
00B09168002BD64
00D0927A002BD64
00508026BEEB
00A092EE002BEEB
00C08325BEEF
003090C4002BEEF
00D0921C002BF8A
00C0825CBF8A
00808154BFF2
00A091E2002BFF2
0020911A002C050
00409208002C050
01009328002C131
0070811AC131
00E09106002C15C
00B08261C15C
0020916A001C217
00908056C217
002092C6002C232
009082C7C232
00B0835AC26F
00D08048C26F
00E092BA002C33E
00C091C8002C33E
002092BA002C412
004091C8002C412
002092CC001C4BB
00A08197C4BB
00E090B6002C60C
006082D0C60C
00E09047001C647
0070804CC647
00D0819AC68D
00B08288C68D
00C082C9C743
00A092DA002C743
00C09208002C77C
00E0911A002C77C
004092C8002C86F
002091BA002C86F
00E092CA002C87A
00C091B8002C87A
0050811AC967
01009128002C967
008082DDCA0D
00209206002CA0D
00D08128CC63
00B0823ACC63
00209135001CCEA
00A08186CCEA
00A08261CD21
00E09206002CD21
00209306002CE26
009082DDCE26
00A090DA002CF15
00A082C9CF15
00C092C8002CF43
00E091BA002CF43
002092CA002CF56
004091B8002CF56
00209282001D05A
00309042002D05A
00B09288002D0A1
00D0919A002D0A1
0080807CD1A6
00309230002D1A6
00D09094001D203
00E09254002D203
00A08271D4F9
00A090F6002D4F9
00309094001D52F
00209254002D52F
0090807CD58D
00309330002D58D
00A080E6D68A
00D09230002D68A
00908143D6BE
00A090A6001D6BE
00C09138002D6CD
00A082B9D6CD
00D09042002D776
00E09282001D776
0030919A002D78D
00F10SF3F300000050D78D
00209236002D7C1
00D080CED7C1
00309242002D80C
00209082001D80C
00B09088002D8F7
00E0915A001D8F7
00C09028002D9A2
00E0933A002D9A2
0020902A002D9B7
00409338002D9B7
00708208D9CB
00C092A8002D9CB
007080F9D9EE
00D09096002D9EE
0020923A002DAA5
00409128002DAA5
00E0912A002DAB0
00B082B9DAB0
00C09238002DAB0
00E09190002DBA2
01010F00DBA2
010092C8002DBAD
009080E8DBAD
00D08058DBB7
00209280002DBB7
00B0834ADBB7
007082E0DBFF
00A09346002DBFF
00D0929A002DCDC
00B09188002DCDC
00C09128002DD89
00E0923A002DD89
00908137DD92
01010NPF300000050DD92
00409238002DD9C
0020912A002DD9C
00D09196002DDC5
006080F9DDC5
0020933A002DE8E
00409028002DE8E
00B0825EDE97
00D090E1001DE97
00E0902A002DE9B
00C09338002DE9B
00C082B9DE9B
00E09082001DF20
00D09242002DF20
00E09090002DF89
01110F00DF89
00A09246002DFD4
006082E0DFD4
00A091A6002E00E
00808140E00E
01009068002E05C
00608348E05C
00A092AA002E117
00508032E117
00E0914A002E128
00C09258002E128
0060804AE17E
00A0902A002E17E
0020904A002E22F
00409358002E22F
00C09048002E23A
008081A8E23A
00E0935A002E23A
00A09344002E289
00A08081E289
00C09228001E36F
00910E01F3F300000050E36F
01009168002E477
00708348E477
00C09358002E503
00E0904A002E503
0020935A002E516
00409048002E516
00A0834BE551
00309340002E551
0070804AE555
00A0912A002E555
00409258002E604
0020914A002E604
00B0820AE60B
00D08118E60B
00E0925A002E611
009081A8E611
00A0933A002E647
00A080A3E647
00F09088002E712
00A0915A001E712
00508287E7AA
00D09032002E7AA
00A09126002E831
0090813CE831
0030935E001E89C
0020919E002E89C
00B0834BE92C
00309040002E92C
00708032E941
00A090AA002E941
00409348002E96B
0020905A002E96B
00C09058002E97E
00E0934A002E97E
00409158002EA79
0020924A002EA79
00A09144002EADF
00C08081EADF
00608287EBD7
00D09332002EBD7
00C10F01F3F300000050EC0E
00A081AAEC0E
00A09026002EC1A
0080813CEC1A
00A0932A002ED03
0050804AED03
00309140002ED07
00C0834BED07
00E0924A002ED55
00C09158002ED55
00608032ED6A
00A091AA002ED6A
00A0913A002EE11
00C080A3EE11
00C09348002EE47
00E0905A002EE47
00409058002EE52
0020934A002EE52
0050802DEE7E
00E10F01F3F300000050EE7E
00B08081EEF4
00A09044002EEF4
00E0919E002EFB0
00D0935E001EFB0
00708287EFFC
00D09232002EFFC
00409278002F08C
00908055F08C
00708022F099
00E0927A002F099
00C09168002F099
00B10PF3F300000050F118
00A0819CF118
00808055F4A7
0020906A002F4A7
00C09068002F4B2
00608022F4B2
0050805AF4DB
00C092E8002F4DB
00D08081F60E
00309012002F60E
00B08091F72C
00E092B4002F72C
0060811BF781
01009228001F781
0020927A002F7B5
00409168002F7B5
0060805AF8A6
00E092DA002F8A6
004092D8002F8B3
002091EA002F8B3
00508022F8CF
00E0907A002F8CF
00608127F9C3
00209066002F9C3
002091CC002FA5D
00B08194FA5D
0070805AFC8D
00C090E8002FC8D
00209166002FDE8
00708127FDE8
00908150FDFD
00E09276002FDFD
002090CC002FE76
00C08194FE76
00E091EA002FF9F
00C092D8002FF9F

A shorter non-repeating alphanumeric code than UUID in MySQL

Is it possible for MySQL database to generate a 5 or 6 digit code comprised of only numbers and letters when I insert a record? If so how?
Just like goo.gl, bit.ly and jsfiddle do it. For exaple:
http://bit.ly/3PKQcJ
http://jsfiddle.net/XzKvP
cZ6ahF, 3t5mM, xGNPN, xswUdS...
So UUID_SHORT() will not work because it returns a value like 23043966240817183
Requirements:
Must be unique (non-repeating)
Can be but not required to be based off of primary key integer value
Must scale (grow by one character when all possible combinations have been used)
Must look random. (item 1234 cannot be BCDE while item 1235 be BCDF)
Must be generated on insert.
Would greatly appreciate code examples.

Try this:
SELECT LEFT(UUID(), 6);

I recommend using Redis for this task, actually. It has all the features that make this task suitable for its use. Foremost, it is very good at searching a big list for a value.
We will create two lists, buffered_ids, and used_ids. A cronjob will run every 5 minutes (or whatever interval you like), which will check the length of buffered_ids and keep it above, say, 5000 in length. When you need to use an id, pop it from buffered_ids and add it to used_ids.
Redis has sets, which are unique items in a collection. Think of it as a hash where the keys are unique and all the values are "true".
Your cronjob, in bash:
log(){ local x=$1 n=2 l=-1;if [ "$2" != "" ];then n=$x;x=$2;fi;while((x));do let l+=1 x/=n;done;echo $l; }
scale=`redis-cli SCARD used_ids`
scale=`log 16 $scale`
scale=$[ scale + 6]
while [ `redis-cli SCARD buffered_ids` -lt 5000 ]; do
uuid=`cat /dev/urandom | tr -cd "[:alnum:]" | head -c ${1:-$scale}`
if [ `redis-cli SISMEMBER used_ids $uuid` == 1]; then
continue
fi
redis-cli SADD buffered_ids $uuid
done
To grab the next uid for use in your application (in pseudocode because you did not specify a language)
$uid = redis('SPOP buffered_ids');
redis('SADD used_ids ' . $uid);
edit actually there's a race condition there. To safely pop a value, add it to used_ids first, then remove it from buffered_ids.
$uid = redis('SRANDMEMBER buffered_ids');
redis('SADD used_ids ' . $uid);
redis('SREM buffered_ids ' . $uid);