SUM on VARCHAR field converted to DECIMAL not adding numbers after decimal - mysql

I have a VARCHAR field that stores a value like 0.00000000.
I want to run a report query to SUM all those VARCHAR fields, which means I have to convert them to a number to add them.
Here's my query, which works as far as giving no errors, but it gives the wrong number back:
SELECT SUM(CAST(IFNULL(tx.received_amount, '0.00000000') AS DECIMAL(16, 16)))
FROM account
JOIN account_invoice
ON account_invoice.account_id = account.id
JOIN withdrawal
ON withdrawal.invoice_id = account_invoice.invoice_id
JOIN tx
ON tx.id = withdrawal.tx_id
AND tx.currency = 'BTC'
AND tx.created_at > DATE_SUB(NOW(), INTERVAL 7 DAY)
WHERE account.id = 1
This is what I get: 100 x 1.12345678 = 100.00000000
This is what I should get: 100 x 1.12345678 = 112.34567800
Why is the SUM not adding the numbers after the decimal?

You are not using the DECIMAL datatype accordingly to your use case. DECIMAL(16, 16) declares a decimal number with a total of 16 digits and with 16 decimal digits. This cannot hold a value greater than 1.
Consider:
SELECT CAST('1.12345678' AS DECIMAL(16, 16))
Returns: 0.9999999999999999.
You probably want something like DECIMAL(16, 8) instead, since your strings seem to have 8 decimals.
From the MySQL documentation:
The declaration syntax for a DECIMAL column is DECIMAL(M,D). The ranges of values for the arguments are as follows:
M is the maximum number of digits (the precision). It has a range of 1 to 65.
D is the number of digits to the right of the decimal point (the scale). It has a range of 0 to 30 and must be no larger than M.

GMB's answer is usually the best choice, but if you truly need to output a (a_really_precise_number)*100 you can do it application-side by actually passing it as a string into a language that supports arbitrarily large numbers, then cast it application side. If you have numbers more precise than 16 digits in your database, you are likely already using one that supports this in your application.
In some cases, you are looking at data from another source and you have more precise numbers than your language of choice is designed for. Many languages that don't support these larger numbers natively may have libraries available that do fancy parsing to perform math on strings as strings but they tend to be a bit slow if you need to work with really large numbers or data sets.
A third option if you are just multiplying it by a power of 10 such as N*100 and outputting the result is to pass it to the application as a string, then just parse it to move that decimal over 2 places like this:
function shiftDec(str, shift){
// split on decimal point
var decPoint = str.indexOf(".");
var decInt = str.substr(0, decPoint);
var decMod = str.substr((decPoint+1));
// move decimal 'shift' places to simulate N*100.
if(shift > 0){
var shiftCopy = decInt .substr(0,shift);
decInt = decInt + shiftCopy;
decMod = decMod .substr(shift);
} else {
var shiftCopy = decInt .substr((decInt.length + shift));
decInt = decInt .substr(0,(decInt.length + shift));
decMod = shiftCopy + decMod;
}
return decInt + '.' + decMod;
}
var result = shiftDec("1234567891234567.8912345678912345", 2);
document.write(result);

You should not use DECIMAL(16,16)
SELECT 100 * CAST('1.123' AS DECIMAL(16,16))
99.999...
SELECT 100 * CAST('1.123' AS DECIMAL(16, 10))
112.300...

Related

Best datatype to store a long number made of 0 and 1

I want to know what's the best datatype to store these:
null
0
/* the length of other numbers is always 7 digits */
0000000
0000001
0000010
0000011
/* and so on */
1111111
I have tested, INT works as well. But there is a better datatype. Because all my numbers are made of 0 or 1 digits. Is there any better datatype?
What you are showing are binary numbers
0000000 = 0
0000001 = 2^0 = 1
0000010 = 2^1 = 2
0000011 = 2^0 + 2^1 = 3
So simply store these numbers in an integer data type (which is internally stored with bits as shown of course). You could use BIGINT for this, as recommended in the docs for bitwise operations (http://dev.mysql.com/doc/refman/5.7/en/bit-functions.html).
Here is how to set flag n:
UPDATE mytable
SET bitmask = POW(2, n-1)
WHERE id = 12345;
Here is how to add a flag:
UPDATE mytable
SET bitmask = bitmask | POW(2, n-1)
WHERE id = 12345;
Here is how to check a flag:
SELECT *
FROM mytable
WHERE bitmask & POW(2, n-1)
But as mentioned in the comments: In a relational database you usually use columns and tables to show attributes and relations rather than an encoded flag list.
As you've said in a comment, the values 01 and 1 should not be treated as equivalent (which rules out binary where they would be), so you could just store as a string.
It actually might be more efficient than storing as a byte + offset since that would take up 9 characters, whereas you need a maximum of 7 characters
Simply store as a varchar(7) or whatever the equivalent is in MySql. No need to be clever about it, especially since you are interested in extracting positional values.
Don't forget to bear in mind that this takes up a lot more storage than storing as a bit(7), since you are essentially storing 7 bytes (or whatever the storage unit is for each level of precision in a varchar), not 7 bits.
If that's not an issue then no need to over-engineer it.
You could convert the binary number to a string, with an additional byte to specify the number of leading zeros.
Example - the representation of 010:
The numeric value in hex is 0x02.
There is one leading zero, so the first byte is 0x01.
The result string is 0x01,0x02.
With the same method, 1010010 should be represented as 0x00,0x52.
Seems to me pretty efficient.
Not sure if it is the best datatype, but you may want to try BIT:
MySQL, PostgreSQL
There are also some useful bit functions in MySQL.

MySQL: compare a mixed field containing letters and numbers

I have a field in the mysql database that contains data like the following:
Q16
Q32
L16
Q4
L32
L64
Q64
Q8
L1
L4
Q1
And so forth. What I'm trying to do is pull out, let's say, all the values that start with Q which is easy:
field_name LIKE 'Q%'
But then I want to filter let's say all the values that have a number higher than 32. As a result I'm supposed to get only 'Q64', however, I also get Q4, Q8 and so for as I'm comparing them as strings so only 3 and the respective digit are compared and the numbers are in general taken as single digits, not as integers.
As this makes perfect sense, I'm struggling to find a solution on how to perform this operation without pulling all the data out of the database, stripping out the Qs and parsing it all to integers.
I did play around with the CAST operator, however, it only works if the value is stored as string AND it contains only digits. The parsing fails if there's another character in there..
Extract the number from the string and cast it to a number with *1 or cast
select * from your_table
where substring(field_name, 1, 1) = 'Q'
and substring(field_name, 2) * 1 > 32

Anomalous mysql behaviour on replace query

im using a v simple database and i have 3 columns A(bigINT 20) , B(bigInt 20) and c(DECIMAL(5,4)) , when i fire the following query i get the below mentioned results :
REPLACE INTO `my_table` SET `A` = 8,`B` = 44,`C` = 14;
i get these values in mysql A =8 , b= 44 and c as 9.9999 ! ?
any ideas as to why is this happening and what can i do to resolved this ?
DECIMAL(5,4) means that the number has at most 5 digits, 4 of them after decimal point. So 14 is simply overflow as it would require DECIMAL(6,4).
It must be cleared that 14 is overflow, because as constant precision point decimal it is internally 14.0000 here (so six digits over five).
So if you try to put 14.0000 (six digits) in DECIMAL(5,4) (five digits max) -> MySQL chooses value closest to the one you request. Therefore 14.0000 gets "rounded" to 9.9999.
To fit 14 in your column you can either extend it do DECIMAL(6,4) (to allow more digits in general) or change to DECIMAL(5,3) (which will allow one more digit before decimal point, but loses some precision of course).

Why doesn't this sql query return any results comparing floating point numbers?

I have this in a mysql table:
id and bolag_id are int. lat and lngitude are double.
If I use the the lngitude column, no results are returned:
lngitude Query: SELECT * FROM location_forslag WHERElngitude= 13.8461208
However, if I use the lat column, it does return results:
lat Query: SELECT * FROM location_forslag WHERElat= 58.3902782
What is the problem with the lngitude column?
It is not generally a good idea to compare floating point numbers with = equals operator.
Is it correct to compare two rounded floating point numbers using the == operator?
Dealing with accuracy problems in floating-point numbers
For your application, you need to consider how close you want the answer to be.
1 degree is about 112km, and 0.00001 degrees is about 1.1 metres (at the equator). Do you really want your application to say "not equal" if two points are different by 0.00000001 degrees = 1mm?
set #EPSLION = 0.00001 /* 1.1 metres at equator */
SELECT * FROM location_forslag
WHERE `lngitude` >= 13.8461208 -#EPSILON
AND `lngitude` <= 13.8461208 + #EPSILON
This will return points where lngitude is within #epsilon degrees of the desired value.
You should choose a value for epsilon which is appropriate to your application.
Floating points are irritating....
WHERE ABS(lngitude - 13.8461208) < 0.00000005
Convert float to decimal for compare. I had the same problem and solved like this:
SELECT
[dbo].[Story].[Longitude],
[dbo].[Story].[Latitude],
[dbo].[Story].[Location],
FROM
[dbo].[Story],
[dbo].[Places]
WHERE
convert(decimal, [dbo].[Story].[Latitude]) = convert(decimal, [dbo].[Places].[Latitude])
and
convert(decimal, [dbo].[Story].[Longitude]) = convert(decimal, [dbo].[Places].[Longitude])
and
[dbo].[Places].[Id] = #PlacesID
and
[dbo].[Story].IsDraft = 0
ORDER BY
[dbo].[Story].[Time] desc
Look at the first 3 rows after the WHERE clausule.
Hope it helps.

Single quotes affecting the calculations in Select query

SELECT COUNT(*) FROM area
WHERE ROUND(SQRT(POWER(('71' - coords_x), 2) +
POWER(('97' - coords_y), 2))) <= 17
==> 51
SELECT COUNT(*) FROM area
WHERE ROUND(SQRT(POWER((71 - coords_x), 2) +
POWER((97 - coords_y), 2))) <= 17
==> 22
coords_x and coords_y are both TINYINT fields containing values in the range [1, 150]. Usually MySQL doesn't care if numbers are quoted or not.. but apparently it does in this case.
The question is just: Why?
MySQL always cares about data types. What happens is that your code relies in automatic type casting and performs math on strings (which can hold a number or not). This can lead to all sort of unpredictable results:
SELECT POW('Hello', 'World') -- This returns 1
To sum up: you need to learn and use the different data types MySQL offers. Otherwise, your application will never do reliable calculations.
Update:
One more hint:
TINYINT[(M)] [UNSIGNED] [ZEROFILL]
A very small integer. The signed range
is -128 to 127. The unsigned range is
0 to 255.
URL:
http://dev.mysql.com/doc/refman/5.1/en/numeric-type-overview.html
I hope you are not trying to store 150 in a signed tinyint column.