MySQL Convert Bytes to Kilobytes, Megabytes, Gigabytes - mysql

I have a logs table that stores various file sizes in bytes. I want to be able to query the database and return the smallest possible float which has been converted to MB GB etc. At present I can return the value in MB but how do I continue to divide further to smallest value and append the unit?
SELECT CONCAT( ROUND( SUM( data_transferred ) /1048576, 2 ) , ' MB' )
FROM `logs`
Any help would be appreciated.
UPDATE:
Based on the link voodoo417 provided I updated my query to the following, which will output the most relevant file size to two decimal places and append the unit (1000 Bytes, 1 KB, 500 MB, 2 GB, etc):
SET #bytes := (SELECT SUM(data_transferred) FROM wp_ddownload_statistics);
SELECT
CASE
WHEN ABS(#bytes) < 1024 THEN CONCAT( ROUND( #bytes, 2 ), ' Bytes')
WHEN ABS(#bytes) < 1048576 THEN CONCAT( ROUND( (#bytes/1024), 2 ), ' KB')
WHEN ABS(#bytes) < 1073741824 THEN CONCAT( ROUND( (#bytes/1048576), 2 ), ' MB')
WHEN ABS(#bytes) < 1099511627776 THEN CONCAT( ROUND( (#bytes/1073741824), 2 ), ' GB' )
WHEN ABS(#bytes) < 1125899906842624 THEN CONCAT( ROUND( (#bytes/1099511627776), 2 ), ' TB')
WHEN ABS(#bytes) < 1152921504606846976 THEN CONCAT( ROUND( (#bytes/1125899906842624), 2 ), ' PB' )
WHEN ABS(#bytes) < 1180591620717411303424 THEN CONCAT( ROUND( (#bytes/1152921504606846976) ,2), ' EB' )
WHEN ABS(#bytes) < 1208925819614629174706176 THEN CONCAT( ROUND( (#bytes/1180591620717411303424), 2), ' ZB' )
WHEN ABS(#bytes) < 1237940039285380274899124224 THEN CONCAT( ROUND( (#bytes/1208925819614629174706176), 2), ' YB' )
WHEN ABS(#bytes) < 1267650600228229401496703205376 THEN CONCAT( ROUND( (#bytes/1237940039285380274899124224), 2), ' BB' )
END

I know this is an old question, but I was looking for the same thing recently, and found out that MySQL 5.7 added format_bytes function exactly for this purpose:
mysql> SELECT format_bytes(512), format_bytes(18446644073709551615);
+-------------------+------------------------------------+
| format_bytes(512) | format_bytes(18446644073709551615) |
+-------------------+------------------------------------+
| 512 bytes | 16383.91 PiB |
+-------------------+------------------------------------+

I have a more elegant solution (also using a user defined function):
CREATE FUNCTION `format_filesize`(filesize FLOAT) RETURNS varchar(20) CHARSET utf8
BEGIN
DECLARE n INT DEFAULT 1;
LOOP
IF filesize < 1024 THEN
RETURN concat(round(filesize, 2), ' ', elt(n, 'Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB', 'BB'));
END IF;
SET filesize = filesize / 1024;
SET n = n + 1;
END LOOP;
END
UPDATE:
Even better, and can be used outside procedures:
SET #filesize = 536870912;
SET #log = IFNULL(TRUNCATE(LOG(1024, #filesize), 0),0);
SELECT CONCAT(ROUND(#filesize / POW(1024, #log), 2), ' ',
ELT(#log + 1, 'Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB', 'BB'));

Renaat's code is failing when filesize is 0 (obviously you can't do LOG from zero). Therefore #log is filled with null and CONCAT produce null as well.
Correct fix is:
SET #filesize = 536870912;
SET #log = IFNULL(TRUNCATE(LOG(1024, #filesize), 0),0);
SELECT CONCAT(ROUND(#filesize / POW(1024, #log), 2), ' ',
ELT(#log + 1, 'Bytes', 'KB', 'MB', 'GB', 'TB', 'PB', 'EB', 'ZB', 'YB', 'BB'));

select concat(round(data*1048576/1073741824,2),' GB')
for example: 1024 = 1gb
select concat(round(1024*1048576/1073741824,2),' GB')
1gb

The top answer about using format_bytes, despite being answered in 2016, is sadly still not found in MariaDB.
I adapted the query from here to make a function I could use.
FUNCTION `format_bytes`(val float) RETURNS varchar(20) CHARSET latin1
BEGIN
DECLARE pw smallint;
IF val < 1024 THEN
return CONCAT(val, ' B');
END IF;
SET pw = LEAST(7, FLOOR(LOG(val) / LOG(1024)));
RETURN CONCAT(ROUND(val / POW(1024, pw), 2), ' ', SUBSTR('KMGTPEZY', pw, 1), 'B');
END
>>> SELECT format_bytes(512), format_bytes(18446644073709551615);
+-------------------+------------------------------------+
| format_bytes(512) | format_bytes(18446644073709551615) |
+-------------------+------------------------------------+
| 512 B | 16.00 EB |
+-------------------+------------------------------------+
A few things could be tweaked to more closely mimic the MySQL format_bytes, but it's not what I was aiming for.

Related

How to auto increment a string with sql query

I am stuck at a point where i have to increment a string, and my strings are of type C001,SC001,B001
in my data base they are defined like
what i am trying to do do is write a query which check the previous highest code present into my db and the incriment it to +1
for example C001 -> C002,C009->C010,C099`->C100 and so on
Similarly for SC001->SC002,SC009->SC010,SC099->SC100 and so on
Similarly fro B001 -> B002,B009->B010,B099`->B100 and so on
I have a query which my friend has suggested me to use but that query only incriminating AAAA->AAAA01 , AAAA09->AAAA10
query is
SELECT id AS PrevID, CONCAT(
SUBSTRING(id, 1, 4),
IF(CAST(SUBSTRING(id, 5) AS UNSIGNED) <= 9, '0', ''),
CAST(SUBSTRING(id, 5) AS UNSIGNED) + 1
) AS NextID
FROM (
-- since you allow strings such as AAAA20 and AAAA100 you can no longer use MAX
SELECT id
FROM t
ORDER BY SUBSTRING(id, 1, 4) DESC, CAST(SUBSTRING(id, 5) AS UNSIGNED) DESC
LIMIT 1
) x
when i am replacing ID with CategoryCode it is giving me PrevID-C004 NextID-C00401 which is not my requirement i want PrevID-C004 and NextID->C005
NOTE i am using my sqlServer 5.1
Just try this one ,
SELECT
CategoryCode,CAST(CONCAT(LPAD(CategoryCode,1,0),LPAD(MAX(RIGHT(CategoryCode,
3)) + 1, 3, 0) ) AS CHAR),
FROM test
SELECT
SubCategoryCode,CAST(CONCAT(LPAD(SubCategoryCode,2,0),
LPAD(MAX(RIGHT(CategoryCode, 3)) + 1, 3, 0) ) AS CHAR),
FROM test
SELECT
BrandCode,CAST(CONCAT(LPAD(BrandCode,1,0), LPAD(MAX(RIGHT(BrandCode, 3)) +
1, 3, 0)) AS CHAR) FROM test

How do I multiply data in mySQL database to simulate scale?

Input:
A database consisting of
static tables that do not scale with number of users or time
dynamic tables that grow when users interact with the application (so scale with number of users and time)
a database with real life data for x users
Task:
scale the database to simulate larger number of users
Example:
Tables:
t_user (scale target)
UserId , Name
1 , John
2, Terry
t_post (dynamic)
AuthorId, PostId, TagId
1, 1 , 1
1, 2 , 2
1, 3 , 2
2, 4 , 1
t_tag (static)
TagId, Name
1, C#
2, Java
Desired output with scale factor = 2
t_user
UserId , Name
1 , John
2, Terry
3 , John
4, Terry
t_post (dynamic)
AuthorId, PostId, TagId
1, 1 , 1
1, 2 , 2
1, 3 , 2
2, 4 , 1
1, 5 , 1
1, 6 , 2
1, 7 , 2
2, 8 , 1
t_tag (static)
TagId, Name
1, C#
2, Java
Ofcourse for such a small database this can be done in MySQL but I need a solution that will work for a database with 150+ tables (writing a scaling routine for each is not a solution) and scale factors that will bring a database form 100 to up to 10 000 users.
Does anyone know a dedicated tool or hack that can accomplish this?
Benchmark Factory for Databases looks like it might do what you need it to, or you could give the MySQL Benchmark Tool a try.
I ended up with writing my own script. Below you will find a simplified version (many columns in tables are ommited for clarity). This worked very well. I was able to scale the DB by a factor of 100 quite efficiently. Hope this helps
SET autocommit = 0;
START TRANSACTION;
SET #UMAX = (SELECT MAX(UserID) AS MX FROM t_user);
SET #QSMAX = (SELECT MAX(QuestionSetID) AS MX FROM t_question_set);
SET #QGMAX = (SELECT MAX(QuestionGroupID) AS MX FROM t_question_group);
SET #QMAX = (SELECT MAX(QuestionID) AS MX FROM t_question);
SET #TMAX = (SELECT MAX(TestID) AS MX FROM t_test);
DROP TABLE IF EXISTS t_seq;
CREATE table t_seq AS
(
SELECT
1 S
);
INSERT INTO t_seq (S) VALUES (2),(3),(4),(5),(6),(7),(8),(9),(10);
INSERT INTO `t_user`
(
`UserID`,
`Login`,
`Password`,
)
SELECT
`UserID` + 1000000 + #UMAX * t_seq.S,
concat(if(Login is null, '', Login), `UserID` + 1000000 + #UMAX * t_seq.S),
`Password`,
FROM t_user,
t_seq;
INSERT INTO `t_question_set`(`QuestionSetID`)
SELECT `QuestionSetID` + 1000000 + #QSMAX * t_seq.S
FROM t_question_set,t_seq;
INSERT INTO `t_question_group`(
`QuestionGroupID`,
`QuestionSetID`
)
SELECT
`QuestionGroupID` + 1000000 + #QGMAX * t_seq.S,
`QuestionSetID` + 1000000 + #QSMAX * t_seq.S,
FROM t_question_group,t_seq;
INSERT INTO `t_question`(`QuestionID`, `QuestionGroupID`)
SELECT
`QuestionID` + 1000000 + #QMAX * t_seq.S,
`QuestionGroupID` + 1000000 + #QGMAX * t_seq.S,
FROM t_question, t_seq;
INSERT INTO `t_test`
(
`TestID`,
`QuestionSetID`,
`UserID`,
)
SELECT
`TestID` + 1000000 + #TMAX * t_seq.S,
`QuestionSetID` + 1000000 + #QSMAX * t_seq.S,
`UserID` + 1000000 + #UMAX * t_seq.S,
FROM t_test,t_seq;
INSERT INTO `t_question_answer`(
`QuestionID`,
`TestID`
)
SELECT
`QuestionID` + 1000000 + #QMAX * t_seq.S,
`TestID` + 1000000 + #TMAX * t_seq.S,
FROM t_question_answer,t_seq;
COMMIT;

Retrieve only a portion of a string

I want to retrieve only the first part of a VARCHAR(15) string. Which string operation is fastest? My data is like this:
80:0:0:0
100:00:00:00
0:00:25:60
I'd like the results to be:
80
100
0
DECLARE #str TABLE(x VARCHAR(15));
INSERT #str VALUES ('80:0:0:0'), ('100:00:00:00'), ('0:00:25:60');
SELECT FirstPart = SUBSTRING(x, 1, CHARINDEX(':', x)-1) FROM #str;
Results:
FirstPart
---------
80
100
0
If you need to show the whole string even if it doesn't contain : then you can do this instead:
SELECT SUBSTRING(x, 1, COALESCE(NULLIF(CHARINDEX(':', x), 0), 15)) FROM #str;

How can I speed up my MySQL UUID v4 stored function?

I'm attempting to write a MySQL stored function to generate v4 UUIDs as described in RFC 4122's section 4.4 ( http://www.ietf.org/rfc/rfc4122.txt ). My initial naive effort after a few tweaks is the following:
CREATE FUNCTION UUID_V4()
RETURNS BINARY(16)
READS SQL DATA
BEGIN
SET #uuid = CONCAT(
LPAD( HEX( FLOOR( RAND() * 4294967296 ) ), 8, '0' ),
LPAD( HEX( FLOOR( RAND() * 4294967296 ) ), 8, '0' ),
LPAD( HEX( FLOOR( RAND() * 4294967296 ) ), 8, '0' ),
LPAD( HEX( FLOOR( RAND() * 4294967296 ) ), 8, '0' )
);
SET #uuid = CONCAT(
SUBSTR( #uuid FROM 1 FOR 12 ),
'4',
SUBSTR( #uuid FROM 14 FOR 3 ),
SUBSTR( 'ab89' FROM FLOOR( 1 + RAND() * 4 ) FOR 1 ),
SUBSTR( #uuid FROM 18 )
);
RETURN UNHEX(#uuid);
END
The above function is quite slow: almost 100 times slower than the built-in UUID(), according to MySQL's BENCHMARK() feature. Short of writing a UDF using MySQL's C API, are there any improvements I can make here to, say, shave off an order of magnitude from its runtime?
If there is an already existing, well-regarded UUID UDF or stored procedure, I'd be happy to hear about that, too.
I didn't test this for correctness or for performance. It is just the idea of doing one only concatenation in instead of two.
create function uuid_v4()
returns binary(16)
begin
set #h1 = lpad(hex(floor(rand() * 4294967296)), 8, '0');
set #h2 = lpad(hex(floor(rand() * 4294967296)), 8, '0');
set #h3 = lpad(hex(floor(rand() * 4294967296)), 8, '0');
set #h4 = lpad(hex(floor(rand() * 4294967296)), 8, '0');
set #uuid = concat(
#h1,
substr(#h2 from 1 for 4),
'4',
substr(#h2 from 6),
substr('ab89' from floor(1 + rand() * 4) for 1 ),
substr(#h3 from 2),
#h4
);
return unhex(#uuid);
end
;
Also why do you use READS SQL DATA in your function?

Implementing parts of rfc4226 (HOTP) in mysql

Like the title says, I'm trying to implement the programmatic parts of RFC4226 "HOTP: An HMAC-Based One-Time Password Algorithm" in SQL. I think I've got a version that works (in that for a small test sample, it produces the same result as the Java version in the code), but it contains a nested pair of hex(unhex()) calls, which I feel can be done better. I am constrained by a) needing to do this algorithm, and b) needing to do it in mysql, otherwise I'm happy to look at other ways of doing this.
What I've got so far:
-- From the inside out...
-- Concatinate the users secret, and the number of time its been used
-- find the SHA1 hash of that string
-- Turn a 40 byte hex encoding into a 20 byte binary string
-- keep the first 4 bytes
-- turn those back into a hex represnetation
-- convert that into an integer
-- Throw away the most-significant bit (solves signed/unsigned problems)
-- Truncate to 6 digits
-- store into otp
-- from the otpsecrets table
select (conv(hex(substr(unhex(sha1(concat(secret, uses))), 1, 4)), 16, 10) & 0x7fffffff) % 1000000
into otp
from otpsecrets;
Is there a better (more efficient) way of doing this?
I haven't read the spec, but I think you don't need to convert back and forth between hex and binary, so this might be a little more efficient:
SELECT (conv(substr(sha1(concat(secret, uses)), 1, 8), 16, 10) & 0x7fffffff) % 1000000
INTO otp
FROM otpsecrets;
This seems to give the same result as your query for a few examples I tested.
This is absolutely horrific, but it works with my 6-digit OTP tokens. Call as:
select HOTP( floor( unix_timestamp()/60), secret ) 'OTP' from SecretKeyTable;
drop function HOTP;
delimiter //
CREATE FUNCTION HOTP(C integer, K BINARY(64)) RETURNS char(6)
BEGIN
declare i INTEGER;
declare ipad BINARY(64);
declare opad BINARY(64);
declare hmac BINARY(20);
declare cbin BINARY(8);
set i = 1;
set ipad = repeat( 0x36, 64 );
set opad = repeat( 0x5c, 64 );
repeat
set ipad = insert( ipad, i, 1, char( ascii( substr( K, i, 1 ) ) ^ 0x36 ) );
set opad = insert( opad, i, 1, char( ascii( substr( K, i, 1 ) ) ^ 0x5C ) );
set i = i + 1;
until (i > 64) end repeat;
set cbin = unhex( lpad( hex( C ), 16, '0' ) );
set hmac = unhex( sha1( concat( opad, unhex( sha1( concat( ipad, cbin ) ) ) ) ) );
return lpad( (conv(hex(substr( hmac, (ascii( right( hmac, 1 ) ) & 0x0f) + 1, 4 )),16,10) & 0x7fffffff) % 1000000, 6, '0' );
END
//
delimiter ;