I save very large numbers in my database (usually with 50+ digits) and then need to run queries like (id is the label of column where the numbers are saved):
WHERE id % 2 = 0
I tried to use varchar data type for this column and although no error is generated while running the query, the returned result is mathematically wrong (the returned ids are not even).
does MySQL convert varchar to int while running my query and so is overflow the reason of the mistake in results?
what is the best choice for saving such large numbers on which I can do arithmetic operation latter? if decimal is the best candidate then what if I need to save the numbers with 100 digits?
Your only choice is to cast to a numeric/decimal value explicitly. In MySQL, that supports up to 65 digits of precision.
Here is a db<>fiddle with an example. Or an example using %.
Related
I have the following SQL query.
SELECT SUM(final_insurance_total) as total
FROM `leads`
GROUP BY leads.status
I have a single row of data in the lead table with a value for final_insurance_total of 458796. The data type for final_insurance_total is float.
For some reason, MySQL is summing a single row as "458796.375".
If I change the query to
SELECT (final_insurance_total) as total
FROM `leads`
GROUP BY leads.status
the correct value is returned. What in the world is going on?
The FLOAT and DOUBLE types in MySQL (as well as in other databases and programming language runtimes) are represented in a special way, which leads to the values stored being approximations, not exact values. See MySQL docs, as well as general information on floating-point arithmetics.
In order to store and operate with exact values, use the type DECIMAL (see https://dev.mysql.com/doc/refman/5.1/en/precision-math-decimal-characteristics.html).
EDIT: I have run some tests, and while floating-point precision errors are quite common, this particular one looks to be specific to the implementation of SUM() in MySQL. In other words, it is a bug that has been there for a long time. In any case, you should use DECIMAL as your field type.
FLOAT does not guarantee precision where any calculation is made. If you use a simple SELECT, no calculation is made, so you get the original value. But if you use SUM(), even with one row, at least one addition is executed (0 + current_value).
Do you really need FLOAT? For example, if you have 2 decimal digits, you could use INT and multiply all values by 100 before all INSERTs. When SELECTing results, you will divide by 100.
If the user is not a sysadmin and cannot change the datatype of the field such as FLOAT, the user can use CAST to produce the desired output.
I have an old database table with column, which type is BIGINT. There's a lot of stored procedures and views that use that table and that column.
For some reason I need to change the type of that column to NUMERIC(38,0).
Is it safe to do it? Should I cast in any stored procedure and view existing BIGINT to NUMERIC(38,0)?
According to me numeric data type is identical with decimal which represents a fixed precision number, which will scale numeric data from -10^38 +1 through 10^38 –1
I don't think that the number types you mention are using fixed precision number and therefore BIGINT is probably the most efficient way to store the number especially if you want to perform some computation in your application.
I don't see really any use for computation with those number and therefore you may even use a string of appropriate length which requires more space in the database but you may be able to allow grouping characters in the numbers.
using BIGINT datatype instead of string you can create efficient indexes.
As you write you're already using numeric datatype and therefore if you upgrade to SQL 2008R2 / 2012 you should consider switching to BIGINT as you don't need fraction in your number. The BIGINT data type is intended for use when integer values might exceed the range that is supported by the int data type.
EDIT:
You can change the data type from BIGINT to NUMERIC(38,0) but be ensure that a Arthimetic overflow error shouldn't occur while converting.
Yes, it is.
According to this table on MSDN an numeric(38,0) has an higher capacity than a bigint.
I calculated the maximum values based on the numbers in the matrix:
9223372036854775808 (bigint, 2^63-1, 8 bytes)
1000000000000000000000000000000000000000 (numeric(38,0), 10^38–1, 17 bytes)
I have a mysql table that gets populated from a flat file with 20 million rows a daily. I have one field 'app_value' that is a varchar(24) that is a mix of text and numbers. I want to run a batch task every day to normalize all the values that are equivalent of zeros. Checking the database I have seen at least the following zero equivalent values but I think there are others.
0
0.0
0.000000
My plan was to cast to decimal and check if that that cast was equal to 0. If so I will update the value to '0'. To test my theory I ran
SELECT d.id, d.pp_value, CAST('d.app_value' as DECIMAL(20,6))
It seemed to work okay on the zero equivalent numbers however I was not that surprised to see that when 'app_value' is a character it is also cast to 0.000000. Is there a better way to do this? I need to protect against null, blanks and characters. I also need to be concerned about efficiency as I have to do this against 20 million rows every day.
You could match against a regular expression:
WHERE d.app_value RLIKE '^0+(\\.0*)?$'
Of course, this would not be particularly efficient (as it will require a full table scan on every invocation: indexes are of no help). If at all possible, I'd suggest checking for zeroes when loading data into the table (either directly within LOAD DATA itself, or using triggers, or else through some external preprocessing).
Second way to do it is with an NOT IN subquery.
This may scale better vs the regex engine on the larger datasets.. Regex engine startup/matching is relatively costly for the CPU compared to normal string matching..
note this one is bit hacky because we "trust" on MySQL auto cast conversion
select
*
from
data
where
number not in (
select
number
from
data
where
number >= 'a'
) and number = 0
see demo http://sqlfiddle.com/#!2/7118e7/2
Good day, I am confused with the datatype for MySQL.
I am using decimal as apparently it is the safest bet for accuracy in a business application. However, I find that when fields are returned I have values of 999999999.99, where my datatype is DECIMAL(10,2). So the actual value has overflowed outside the (10, 2) parameter.
Would it be correct that even though I have specified 10 places before the comma and 2 places after the comma. MySQL still stores the complete number?
Also would it be possible to turn off the maximum amount of digits displayed before and after the comma?
Would it be correct that even though I have specified 10 places before the comma and 2 places after the comma. MySQL still stores the complete number?
No, it wouldn't.
First, you specified 10 digits altogether; two are to the right of the decimal point, and eight are to the left.
Standard SQL requires that DECIMAL(5,2) be able to store any value with five digits and two decimals, so values that can be stored in the salary column range from -999.99 to 999.99.
Second, MySQL will silently convert the least significant digits to scale if there are more than two. That will probably look like MySQL truncates, but the actual behavior is platform-dependent. It will raise an error if you supply too many of the most significant digits.
Finally, when you're working with databases, the number of digits displayed has little to do with what a data type is or with what range of values it stores.
I have a need to store a provider name and the country(ies) they are able to provide services in. There are 92 counties. Rather than store up to 92 rows, I'd like to store 2^91 so if they provide only in county 1 and 2, I'd store 3.
The problem I'm having is the largest number is 2475880078570760549798248448, which is way too big for the largest BigInt.
In the past when I've had smaller numbers of options I've been able to do something like....
SELECT * FROM tblWhatever WHERE my_col & 2;
If my_col had 2, 3, 6, etc. stored (anything with bit 2) it would be found.
I guess I'm not sure of 2 things... how to store AND how to query if stored in a way other than an INT.
You could use BINARY(13) as a datatype to store up to 94 bits. But MySQL bitwise operators only support BIGINT which is 64 bits.
So if you want to use bitwise operators, you'd have to store your county bitfield in two BIGINT columns (or perhaps one BIGINT and one INT), and in application code work out which of the two columns to search. That seems like it's awkward.
However, I'll point out a performance consideration: using bitwise operators for searching isn't very efficient. You can't make use of an index to do the search, so every query is be forced to perform a table-scan. As your data grows, this will become more and more costly.
It's the same problem as searching for a substring with LIKE '%word%', or searching for all dates with a given day of the month.
So I'd suggest storing each county in a separate row after all. You don't have to store 92 rows for each service provider -- you only have to store as many rows as the number of counties they service. The absence of a row indicates no service in that county.