I am trying to create a query to populate unique data from one table into part of another table. The two rows from the source table I'm selecting are of types mediumint(8) and varchar(20). In the destination table I have a Binary(2) and a varchar(20) that I'm trying to populate from the source table. I'm also only selecting the top 100 unique values.
The problem I have is with the mediumint(8) to Binary(2) conversion. I can see all my mediumint(8) values are unique, and they are all less than 200 numerically, but for some reason I can't get them to fit in a 2 byte unique binary column. My query keeps rejecting the insert because it says the values are larger than 2 bytes.
sample query:
select distinct id, part from partsTable
group by id
limit 100;
This comes back with a table that looks kind of like this
id| part
_________
1 | A
2 | B
3 | C
21 | AB
22 | AC
...
I have tried doing things like ( id & 0xFFFF) to force my id column as 2 bytes, and CAST(id as UNSIGNED) or CAST(id as BINARY(2)), but these always truncate the data so that i get duplicate values. Not really sure what I'm missing or how my unique id values that I can see are all unique get translated to non unique values when I try to cast them into binary.
What's even more confusing is as a test I just inserted two records into my destination table:
INSERT into table1 (id, part) VALUES (0xFFFF, 'A');
INSERT into table1 (id, part) VALUES (0xFFFE, 'B');
And then I just wrote a query to see what that data looked like:
SELECT bin(id) as id, hex(id) as hex, part FROM table1;
Which returns a result like:
id | hex | part
__________________
0 | 0xFFFF| A
0 | 0xFFFE| B
So I have no idea what's going on
2 bytes would be a numeric range of 65535. The problem here is the binary data type in MySQL still contains a string, not a number, but with collation of binary instead of something like utf8 as the char data type would. The difference is that the binary data type restricts the value based on the number of bytes rather than the number of characters in the string that it is storing. In other words, a 3 digit value such as 150 as a string would be 3 characters, totaling 3 bytes. So if this value was converted to BINARY(2), it would lose anything past its 2nd byte, making it 15.
Because it stores a string, the binary data type is not restricted to numeric characters (0,1) as you would expect either. So you could make the values fit in a BINARY(2) by using HEX(). HEX(150) would be '96', and would therefore fit in a BINARY(2). You would then have to use UNHEX() to retreive any of these values. If this table already has other values stored in it I doubt this would work for you though.
Related
Database has two tables with same schema.
date VARCHAR(20),
track VARCHAR(150),
race_number INT,
horse_number INT,
early_last5 FLOAT(10,1),
PRIMARY KEY (track, race_number, horse_number, date)
One is named sectional_table and another is window_sectional_table.
I want to copy all contents of sectional table to window_sectional_table.
I do the most logical thing possible.
INSERT INTO window_sectional_table SELECT * FROM sectional_table;
Unfortunately, I am terrorized by this error.
Data truncated for column 'early_last5' at row 1
I investigate row 1. Row 1 looks like this.
+------------+----------+-------------+--------------+-------------+
| date | track | race_number | horse_number | early_last5 |
+------------+----------+-------------+--------------+-------------+
| 2021-05-03 | GUNNEDAH | 1 | 1 | 0.0 |
+------------+----------+-------------+--------------+-------------+
How do I proceed? I believe the value 0.0 should have been auto filled for null value.
The Data truncated for column 'early_last5' at row 1 error is referring to your INSERT statement rather than which specific value in the database is being truncated.
I believe the value 0.0 should have been auto filled for null value.
Nope. That’s not defined in your INSERT and it’s not defined in the table creation statement.
How do I proceed?
The simplest method would be to convert the FLOAT value to a DECIMAL for the INSERT, complete with an IFNULL if you want NULL to be converted to 0.0. The FLOAT data type has long been a thorn in the side of many people who expect it to work like a typical decimal.
Here’s a quick SQL statement that should get you moving again:
INSERT INTO `window_sectional_table` (`date`, `track`, `race_number`, `horse_number`, `early_last5`)
SELECT `date`, `track`, `race_number`, `horse_number`,
CAST(IFNULL(`early_last5`, 0) AS DECIMAL(10,1))
FROM `sectional_table`
ORDER BY `track`, `race_number`, `horse_number`, `date`;
Suppose we have 2 numbers of 3 bits each attached together like '101100', which basically represents 5 and 4 combined. I want to be able to perform aggregation functions like SUM() or AVG() on this column separately for each individual 3-bit column.
For instance:
'101100'
'001001'
sum(first three column) = 6
sum(last three column) = 5
I have already tried the SUBSTRING() function, however, speed is the issue in that case as this query will run on millions of rows regularly. And string matching will slow the query.
I am also open for any new databases or technologies that may support this functionality.
You can use the function conv() to convert any part of the string to a decimal number:
select
sum(conv(left(number, 3), 2, 10)) firstpart,
sum(conv(right(number, 3), 2, 10)) secondpart
from tablename
See the demo.
Results:
| firstpart | secondpart |
| --------- | ---------- |
| 6 | 5 |
With the current understanding I have of your schema (which is next to none), the best solution would be to restructure your schema so that each data point is its own record instead of all the data points being in the same record. Doing this allows you to have a dynamic number of data points per entry. Your resulting table would look something like this:
id | data_type | value
ID is used to tie all of your data points together. If you look at your current table, this would be whatever you are using for the primary key. For this answer, I am assuming id INT NOT NULL but yours may have additional columns.
Data Type indicates what type of data is stored in that record. This would be the current tables column name. I will be using data_type_N as my values, but yours should be a more easily understood value (e.g. sensor_5).
Value is exactly what it says it is, the value of the data type for the given id. Your values appear to be all numbers under 8, so you could use a TINYINT type. If you have different storage types (VARCHAR, INT, FLOAT), I would create a separate column per type (val_varchar, val_int, val_float).
The primary key for this table now becomes a composite: PRIMARY KEY (id, data_type). Since your previously single record will become N records, the primary key will need to adjust to accommodate that.
You will also want to ensure that you have indexes that are usable by your queries.
Some sample values (using what you placed in your question) would look like:
1 | data_type_1 | 5
1 | data_type_2 | 4
2 | data_type_1 | 1
2 | data_type_2 | 1
Doing this, summing the values now becomes trivial. You would only need to ensure that data_type_N is summed with data_type_N. As an example, this would be used to sum your example values:
SELECT data_type,
SUM(value)
FROM my_table
WHERE id IN (1,2)
GROUP BY data_type
Here is an SQL Fiddle showing how it can be used.
I was doing some system testing and expecting empty results from MySQL(5.7.21) but got surprised to get results.
My transactions table looks like this:
Column Data type
----------------------------
id | INT
fullnames | VARCHAR(40)
---------------------------
And I have some records
--------------------------------
id | fullnames
--------------------------------
20 | Mutinda Boniface
21 | Boniface M
22 | Some-other Guy
-------------------------------
My sample queries:
select * from transactions where id = "20"; -- gives me 1 record which is fine
select * from transactions where id = 20; -- gives me 1 record - FINE as well
Now it gets interesting when I try with these:
select * from transactions where id = "20xxx"; -- gives me 1 record - what is happening here?
What does MySQL do here??
MySQL plays fast and loose with type conversions. When implicitly converting a char to a number, it will take characters from the beginning of the string as long as they are digits, and ignore the rest. In your example, xxx aren't digits, so MySQL only takes the initial "20".
One way around this (which is horrible for performance, since you lose the usage on the index you may have on your column), is to explicitly cast the numeric side to a character:
SELECT * FROM transactions WHARE (CAST id AS CHAR) = 20;
EDIT:
Referencing the discussion about performance from the comments - performing the cast to a number on the client-side is probably the best approach, as it will allow you to avoid sending queries to the database when you know no rows should be returned (i.e., when your input is not a valid number, such as "20x").
An alternative hack could be to cast the input to a number and back again to a string, and compare the lengths. If the lengths are the same it means the input string was fully converted into a number and no characters were omitted. This should be OK WRT performance, since this comparison is performed on an inputted string, not on a value from the column, and the column's index can still be used if the condition passes the short-circuit evaluation of the input:
SELECT *
FROM transactions
WHERE LENGTH(:input) = LENGTH(CAST(:input AS SIGNED)) AND id = :input;
How to select rows without using the double quotes("1").
I am design the status column in enum type.
status
enum('0', '1')
My Query:
SELECT * FROM `user` WHERE activation_key='123456' AND status="1";
Result :
Display 1 row
I am Try :
SELECT * FROM `user` WHERE activation_key='123456' AND status=1;
Result :
Display 0 row
Is it possible to get the data without double or single quotes in status column?
I am a beginner of MYSQL, Sorry for my bad question!
An enumeration value must be a quoted string literal. And to query a string you must enclose it in quotes.
If you make enumeration values that look like numbers, it is easy to mix up the literal values with their internal index numbers.
numbers ENUM('0','1','2')
If you store 2, it is interpreted as an index value, and becomes '1'
(the value with index 2). If you store '2', it matches an enumeration
value, so it is stored as '2'. If you store '3', it does not match any
enumeration value, so it is treated as an index and becomes '2' (the
value with index 3).
mysql> INSERT INTO t (numbers) VALUES(2),('2'),('3');
mysql> SELECT * FROM t;
+---------+
| numbers |
+---------+
| 1 |
| 2 |
| 2 |
+---------+
More details ENUM in MySQL
I've stumbled on a previously asked and answered question here:
How to use comparison operator for numeric string in MySQL?
I absolutely agree with the answer being the best mentioned. But it left me with a question myself while I was trying to create my own answer. I was trying to select the first number and convert it to an integer. Next I wanted to compare that integer with a number (3 in case of the question).
This is the query I've created:
SELECT experience,
CONVERT(SUBSTRING_INDEX(experience,'-',1), UNSIGNED INTEGER) AS num
FROM employee
WHERE #num >= 3;
For the sake of simplicity, asume the data inside experience is: 4-8
The query doesn't return any errors. But it doesn't return the data either. I know it's possible to compare the data inside a column with a user defined variable. But is it possible to compare data (the integer in this case) with the variable like I'm trying to do?
This is purely out of curiousity and to learn something.
Yes, a derived table will do. The inner select block below is a derived table. And every derived table needs a name. In my case, xDerived.
The strategy is to let the derived table cleanse the use of the column name. Coming out of the derived chunk is a clean column named num which the outer select is free to use.
Schema
create table employee
( id int auto_increment primary key,
experience varchar(20) not null
);
-- truncate table employee;
insert employee(experience) values
('4-5'),('7-1'),('4-1'),('6-5'),('8-6'),('5-9'),('10-4');
Query
select id,experience,num
from
( SELECT id,experience,
CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER) AS num
FROM employee
) xDerived
where num>=7;
Results
+----+------------+------+
| id | experience | num |
+----+------------+------+
| 2 | 7-1 | 7 |
| 5 | 8-6 | 8 |
| 7 | 10-4 | 10 |
+----+------------+------+
Note, your #num concept was faulty but hopefully I interpreted what you meant to do above.
Also, I went with 7 not 3 because all your sample data would have returned, and I wanted to show you it would work.
The AS num instruction names the result of convert as num, not a variable named #num.
You could repeat the convert
SELECT experience,CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER)
FROM employee
WHERE CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER) >= 3;
Or use a partial (derived) table (only one convert)
SELECT experience,num
FROM (select experience,
CONVERT(SUBSTRING_INDEX(experience,'-',1),UNSIGNED INTEGER) as num
FROM employee) as partialtable WHERE num>=3;
Much simpler. (Or at least much shorter.) This will work for the data as described, namely "number, -, other stuff".
SELECT experience,
0+experience AS 'FirstPart'
FROM employee
WHERE 0+experience >= 3
Why? 0+string is parsed as "convert the string to a number, then add it to 0". Converting a string will extract the digits up to the first non-digit, then convert that as numeric.