Mysql offers a UUID() function,
which returns an rfc 4122 version 1 guid.
This is an easily guessed timestamp + node_id bit string.
How may we insert randomized version 4 guids?
(Defining a new function requires permissions and is out-of-scope.
Ben Johnson offers an expression that is very nice but a little verbose.)
This inserts a version-4 random string, without dashes.
In the interest of conciseness it uses a slightly reduced portion of the key space,
just 120 bits.
(So 8-bits are predictable, they're constant.)
-- Produces version 4 guids for mysql, as its UUID() only offers version 1.
--
-- See https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
-- We consider variant 1 only, ignoring the Microsoft proprietary variant 2.
-- Version 1 is predictable timestamp.
-- Version 4 is 122 random bits + 6 constant bits.
--
-- The nil guid comes out like this:
-- UUID('00000000-0000-4000-8000-000000000000') # 8-4-4-4-12
-- The nybble '4' is constant version.
-- The nybble '8' has hi bit set, next bit cleared, plus two wasted bits.
-- We deliberately choose to emit just 120 random bits, for simplicity.
-- The RAND() function returns about 53 bits of entropy in the mantissa,
-- so for 15 nybbles we call it twice to obtain 106 ( > 60 ) unguessable bits.
-- The standard spelling of a guid, with four '-' dashes, is 36 characters.
-- We emit 32 hex characters, sans dashes.
INSERT INTO guid_test (guid) VALUES (
concat(substr(sha2(rand(), 256), 1, 12),
'4', substr(sha2(rand(), 256), 1, 3),
'8', substr(sha2(concat(rand(), rand()), 256), 1, 15)
)
);
Related
I'm processing data from a MySQL table where each row has a UUID associated with it. EDIT: the "UUID" is in fact an MD5 hash (VARCHAR) of the job text.
My select query looks something like:
SELECT * FROM jobs ORDER BY priority DESC LIMIT 1
I am only running one worker node right now, but would like to scale it out to several nodes without altering my schema.
The issue is that the jobs take some time, and scaling out beyond one right now would introduce a race condition where several nodes are working on the same job before it completes and the row is updated.
Is there an elegant way to effectively "shard" the data on the client-side, by specifying some modifier config value per worker node? My first thought was to use the MOD function like this:
SELECT * FROM jobs WHERE UUID MOD 2 = 0 ORDER BY priority DESC LIMIT 1
and SELECT * FROM jobs WHERE UUID MOD 2 = 1 ORDER BY priority DESC LIMIT 1
In this case I would have two workers configured as "0" and "1". But this isn't giving me an even distribution (not sure why) and feels clunky. Is there a better way?
The problem is you're storing the ID as a hex string like acbd18db4cc2f85cedef654fccc4a4d8. MySQL will not convert the hex for you. Instead, if it starts with a letter you get 0. If it starts with a number, you get the starting numbers.
select '123abc' + 0 = 123
select 'abc123' + 0 = 0
6 out of 16 will start with a letter so they will all be 0 and 0 mod anything is 0. The remaining 10 of 16 will be some number so will be distributed properly, 5 of 16 will be 0, 5 of 16 will be 1. 6/16 + 5/16 = 69% will be 0 which is very close to your observed 72%.
To do this right we need to convert the 128 hex string into a 64 bit unsigned integer.
Slice off 64 bits with either left(uuid, 16) or right(uuid, 16).
Convert the hex (base 16) into decimal (base 10) using conv.
cast the result to an unsigned bigint. If we skip this step MySQL appears to use a float which loses accurracy.
select cast(conv(right(uuid, 16), 16, 10) as unsigned) mod 2
Beautiful.
That will only use 64 bits of the 128 bit checksum, but for this purpose that should be fine.
Note this technique works with an MD5 checksum because it is pseudorandom. It will not work with the default MySQL uuid() function which is a UUID version 1. UUIDv1 is a timestamp + a fixed ID and will always mod the same.
UUIDv4, which is a random number, will work.
Convert the hex string to decimal before modding:
where CONV(substring(uuid, 1, 8), 16, 10) mod 2 = 1
A reasonable hashing function should distribute evenly enough for this purpose.
Use substring to convert only a small part so the conv doesn't overflow decimal range and maybe behave badly. Any subset of bits should also be well distributed.
I wanted to create an table
create table Oceny_projekty
(
OProj_Id int not null comment '',
ID_Projektu int comment '',
ID_Studenta int comment '',
OProj_Ocena decimal(1,1) comment '',
OProj_Data date comment '',
primary key (OProj_Id)
);
And fill it with sample data generated by PowerDesigner
insert into Oceny_projekty (OProj_Id, ID_Projektu, ID_Studenta, OProj_Ocena, OProj_Data)
values (2, 18, 10, '2.5', '1857-12-25');
And I've got this:
insert into Oceny_projekty (OProj_Id, ID_Projektu, ID_Studenta,
OProj_Ocena, OProj_Data) values (2, 18, 10, '2.5', '1857-12-25') Error
Code: 1264. Out of range value for column 'OProj_Ocena' at row 1
How can I modify command that can accept decimal numbers? (with integer numbers there is no problem)
Using MySQL Workbench 6.3, PowerDesigner 16.6
Thanks in advance!
Declaring OProj_Ocena as decimal(1,1) means that it has 0 integer digits (precision - scale = 1 - 1 = 0) and 1 fractional digit. It can only store values 0, 0.1, 0.2 ... 0.9.
The datatype you need is probably decimal(2,1).
Putting Marek Grzenkowicz's answer in another way, DECIMAL(M, D):
M = D + (the number of whole number digits). See 12.25.2 DECIMAL Data Type Characteristics
So, determine the number of whole number digits you want, those to the left of the decimal point, and the number of decimal places (D) that you need and add them to get M. IMO, they should have used the phrase "whole number digits" or "integer digits" to make this a bit more obvious even though what they say means that as M is for ALL digits. I misinterpreted this at first until I reread their description a few times.
I'm using a MySQL binary column (tinyblob) to store a 128-bit perceptual image hash for about 200,000 images, and then doing a SELECT query to find images whose hash value is within a certain number of bits different (the hamming distance is less than a given delta).
To count the number of bits different, you can XOR the two values and then count the number of 1 bits in the result. MySQL has a handy function called BIT_COUNT that counts the number of 1 bits in an unsigned 64-bit integer.
So I'm currently using the following query to split the 128-bit hash into two 64-bit parts, doing the two XOR and BIT_COUNT operations, and adding the results to get the total bit delta:
SELECT asset_id, dhash8
FROM assets
WHERE
BIT_COUNT(CAST(CONV(HEX(SUBSTRING(dhash8, 1, 8)), 16, 10)
AS UNSIGNED) ^ :dhash8_0) + -- high part
BIT_COUNT(CAST(CONV(HEX(SUBSTRING(dhash8, 9, 8)), 16, 10)
AS UNSIGNED) ^ :dhash8_1) -- plus low part
<= :delta -- less than threshold?
But doing a substring, and especially converting it to a hex string and back is kind of annoying (and inefficient). Is there a better way to do this using MySQL?
I want to know what's the best datatype to store these:
null
0
/* the length of other numbers is always 7 digits */
0000000
0000001
0000010
0000011
/* and so on */
1111111
I have tested, INT works as well. But there is a better datatype. Because all my numbers are made of 0 or 1 digits. Is there any better datatype?
What you are showing are binary numbers
0000000 = 0
0000001 = 2^0 = 1
0000010 = 2^1 = 2
0000011 = 2^0 + 2^1 = 3
So simply store these numbers in an integer data type (which is internally stored with bits as shown of course). You could use BIGINT for this, as recommended in the docs for bitwise operations (http://dev.mysql.com/doc/refman/5.7/en/bit-functions.html).
Here is how to set flag n:
UPDATE mytable
SET bitmask = POW(2, n-1)
WHERE id = 12345;
Here is how to add a flag:
UPDATE mytable
SET bitmask = bitmask | POW(2, n-1)
WHERE id = 12345;
Here is how to check a flag:
SELECT *
FROM mytable
WHERE bitmask & POW(2, n-1)
But as mentioned in the comments: In a relational database you usually use columns and tables to show attributes and relations rather than an encoded flag list.
As you've said in a comment, the values 01 and 1 should not be treated as equivalent (which rules out binary where they would be), so you could just store as a string.
It actually might be more efficient than storing as a byte + offset since that would take up 9 characters, whereas you need a maximum of 7 characters
Simply store as a varchar(7) or whatever the equivalent is in MySql. No need to be clever about it, especially since you are interested in extracting positional values.
Don't forget to bear in mind that this takes up a lot more storage than storing as a bit(7), since you are essentially storing 7 bytes (or whatever the storage unit is for each level of precision in a varchar), not 7 bits.
If that's not an issue then no need to over-engineer it.
You could convert the binary number to a string, with an additional byte to specify the number of leading zeros.
Example - the representation of 010:
The numeric value in hex is 0x02.
There is one leading zero, so the first byte is 0x01.
The result string is 0x01,0x02.
With the same method, 1010010 should be represented as 0x00,0x52.
Seems to me pretty efficient.
Not sure if it is the best datatype, but you may want to try BIT:
MySQL, PostgreSQL
There are also some useful bit functions in MySQL.
SELECT COUNT(*) FROM area
WHERE ROUND(SQRT(POWER(('71' - coords_x), 2) +
POWER(('97' - coords_y), 2))) <= 17
==> 51
SELECT COUNT(*) FROM area
WHERE ROUND(SQRT(POWER((71 - coords_x), 2) +
POWER((97 - coords_y), 2))) <= 17
==> 22
coords_x and coords_y are both TINYINT fields containing values in the range [1, 150]. Usually MySQL doesn't care if numbers are quoted or not.. but apparently it does in this case.
The question is just: Why?
MySQL always cares about data types. What happens is that your code relies in automatic type casting and performs math on strings (which can hold a number or not). This can lead to all sort of unpredictable results:
SELECT POW('Hello', 'World') -- This returns 1
To sum up: you need to learn and use the different data types MySQL offers. Otherwise, your application will never do reliable calculations.
Update:
One more hint:
TINYINT[(M)] [UNSIGNED] [ZEROFILL]
A very small integer. The signed range
is -128 to 127. The unsigned range is
0 to 255.
URL:
http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html
I hope you are not trying to store 150 in a signed tinyint column.