I have a table that does not have any index or primary key in my MySQL database. I cannot change the schema of the table (it is not "my" table). As the table stores data that arrives in intervals, there can be (are) duplicates.
For example:
+--------------+--------------+--------+----------+----------+---------+
| first_seen | last_seen | type | name | hitcnt | data |
+--------------+--------------+--------+----------+----------+---------+
| 15:12:02 | 16:02:32 | 5 | foo | 3 | difank |
+--------------+--------------+--------+----------+----------+---------+
| 19:52:23 | 22:06:20 | 5 | foo | 4 | difank |
+--------------+--------------+--------+----------+----------+---------+
Now I would like to "reduce" this to:
+--------------+--------------+--------+----------+----------+---------+
| first_seen | last_seen | type | name | hitcnt | data |
+--------------+--------------+--------+----------+----------+---------+
| 15:12:02 | 22:06:20 | 5 | foo | 7 | difank |
+--------------+--------------+--------+----------+----------+---------+
And I would like to do this "in situ" (i.e. in place) if possible.
Using GROUP BY, MIN(), MAX(), etc. I can write a query that returns exactly what I want to end up with:
SELECT
MIN(first_seen),
MAX(last_seen),
type,
name,
SUM(hitcnt) as hit,
data
FROM <table>
GROUP BY type, name, data
ORDER BY hit desc, type;
The question is: how can I replace the existing data (efficiently) with the result of that query?
Do I have to use a temporary table (i.e. move the data to a temporary table, truncate the existing table and SELECT INTO from the temporary table)?
Can I do this in a transaction (to prevent data loss if something goes wrong)?
Are there other (better?) options than a temporary table?
TRUNCATE TABLE table_name;
INSERT INTO table_name (column1.....)
SELECT
MIN(first_seen),
MAX(last_seen),
type,
name,
SUM(hitcnt) as hit,
data
FROM <table>
GROUP BY type, name, data
ORDER BY hit desc, type;
Make sure number of columns of insert and select staement matches.
I ended up using the following approach:
CREATE TABLE tmp_table IF NOT EXISTS AS (SELECT ...);
then move the "new" table into place using ALTER TABLE ... RENAME ...;
followed by a DROP TABLE ...; for the old/original table.
That seems to work.
Related
I need to sort a query's results by a varchar that may contain numbers, but also sort by other fields beforehand.
Say my table looks like:
+------------------------+-----------------+
| AnotherField | VarCharWithNums |
+------------------------+-----------------|
| Same Data in Every Row | Numbers 5-10 |
| Same Data in Every Row | Numbers 10-13 |
| Same Data in Every Row | Numbers 13-15 |
+------------------------+-----------------|
This query:
SELECT VarCharWithNums, AnotherField
FROM MyTable
ORDER BY CAST(VarCharWithNums AS UNSIGNED) ASC
Gives me this:
+------------------------+-----------------+
| AnotherField | VarCharWithNums |
+------------------------+-----------------|
| Same Data in Every Row | Numbers 5-10 |
| Same Data in Every Row | Numbers 10-13 |
| Same Data in Every Row | Numbers 13-15 |
+------------------------+-----------------|
This query:
SELECT VarCharWithNums, AnotherField
FROM MyTable
ORDER BY AnotherField ASC, CAST(VarCharWithNums AS UNSIGNED) ASC
Gives me this:
+------------------------+-----------------+
| AnotherField | VarCharWithNums |
+------------------------+-----------------|
| Same Data in Every Row | Numbers 10-13 |
| Same Data in Every Row | Numbers 5-10 |
| Same Data in Every Row | Numbers 13-15 |
+------------------------+-----------------|
It doesn't matter what priority I give the fields in the ORDER BY clause, it never sorts VarCharWithNums correctly when I order it alongside other fields.
You mentioned it in your last paragraph, but truly the only error in what you've shown is that, to get the sort you described, CAST(VarCharWithNums AS UNSIGNED) ASC needs to be the first thing in the ORDER BY clause. It should work, and it works when I create a contrived example on my machine.
I have a small doubt in MySQL. While loading data from one table to another table I faced one issue
first table: emp
id | name | sal | deptno | loc | referby
1 | abc | 100 | 10 | hyd | xyz
2 | mnc | 200 | 20 |chen | pqr
second table:emprefers
id | name | sal | deptno | loc | referby
Now I want to load the emp table data into the emprefers table. I wrote a query like
insert into emprefers select * from emp after
I ran the query, the data was loaded into the emprefers table like below:
id | name | sal |deptno | loc |referby
1 | abc | 100 | 10 | hyd | xyz
2 | mnc | 200 | 20 | chen | pqr
Now I ran the same query a second time. It has failed. The reason is the name column is deleted from the emp table.
I edited the query like:
insert into emprefers select id,'null'as name,sal,deptno,loc,referby from emp
After I ran the edited query again, now records are loading into the emprefers table and the data looks like:
id | name | sal |deptno | loc |referby
1 | null | 100 |10 | hyd | xyz
2 | null | 200 |20 |chen | pqr
Every time before loading the emprefers table I truncate the emprefers table data. And the emprefers table structure never changed.
Again, a third time I ran the same query again. The query has failed, the reason was that the sal and deptno columns were missing in the emp table.
I don't want to edit the query again, reason is we don't know which columns are/get deleted from the emp table.
This time we want solve the issue.
We want to load the data into the second table if the columns are available in the emp table, then load the data - otherwise we need to pass null or empty values for those columns.
Please tell me how to write a query to check if a column exist or not, and if it exists to retrieve the same column, otherwise assign null values for that column.
Rather than changing the existing query and truncating the table, it might be a better idea to make delete the whole table, make a copy of the original emp table and then insert the data into it. That way they'll always be the same.
DROP TABLE emprefers IF EXISTS
CREATE TABLE emprefers LIKE emp;
INSERT INTO emprefers SELECT * FROM emp
This statement will create the table over the fly.
CREATE TABLE databasename.emprefers SELECT * FROM databasename.emp;
I have a table which is many to many and my table looks like this
+----+--------+
| Customers |
+----+--------+
| id | name |
+----+--------+
| 1 | john |
| 1 | john |
| 1 | james |
| 2 | george |
| 2 | michael|
+----+--------+
What i want is to remove the duplicate rows with the same name.
Unfortunately, you have no way to distinguish one row from another. So, the easiest way to do this is the temporary table approach:
create table temp as
select distinct id, name
from customers;
truncate table customers;
insert into customers(id, name)
select id, name
from temp;
drop table temp;
Take a look at GROUP BY aggregate function
Well a small variation #Gordon Linoff answer, is by avoiding the "Insert into" and doing a "Rename table" and making the queries to work on any table.
Solution-1: By using a temp table
CREATE TABLE table_name_clean AS SELECT DISTINCT
*
FROM
table_name;
DROP TABLE table_name;
RENAME TABLE table_name_clean TO table_name;
Solution-2: Adding a UNIQUE INDEX (recommended as it will prevent the creation of duplicate entries in your table)
ALTER IGNORE TABLE table_name ADD UNIQUE INDEX u_id (id,name);
Is there a way to link two tables, so when i alter base table, view will altered too? Something like that:
CREATE TABLE t (qty INT, price INT);
INSERT INTO t VALUES(3, 50);
CREATE VIEW v AS SELECT * FROM t;
SELECT * FROM v;
+------+-------+
| qty | price |
+------+-------+
| 3 | 50 |
+------+-------+
ALTER TABLE t ADD COLUMN comm INT;
SELECT * FROM t;
+------+-------+------+
| qty | price | comm |
+------+-------+------+
| 3 | 50 | NULL |
+------+-------+------+
SELECT * FROM v;
+------+-------+
| qty | price |
+------+-------+
| 3 | 50 |
+------+-------+
Last two SELECT-s should be equal.
PS. I am aware that MySQL says:
The view definition is “frozen” at creation time, so changes to the underlying tables afterward do not affect the view definition.
And creating trigger is also not possible, because trigger events does not include ALTER TABLE
You need to recreate view when you alter table as stated in manual
The view definition is “frozen” at
creation time, so changes to the
underlying tables afterward do not
affect the view definition. For
example, if a view is defined as
SELECT * on a table, new columns added
to the table later do not become part
of the view.
Either drop & recreate view or ALTER view too.
I've got a table in MySQL that looks roughly like:
value | count
-------------
Fred | 7
FRED | 1
Roger | 3
roger | 1
That is, it was created with string ops outside of MySQL, so the values are case- and trailing-whitespace-sensitive.
I want it to look like:
value | count
-------------
Fred | 8
Roger | 4
That is, managed by MySQL, with value a primary key. It's not important which one (of "Fred" or "FRED") is kept.
I know how to do this in code. I also know how to generate a list of problem values (with a self-join). But I'd like to come up with a SQL update/delete to migrate my table, and I can't think of anything.
If I knew that no pair of records had variants of one value, with the same count (like ("Fred",4) and ("FRED",4)), then I think I can do it with a self-join to copy the counts, and then an update to remove the zeros. But I have no such guarantee.
Is there something simple I'm missing, or is this one of those cases where you just write a short function outside of the database?
Thanks!
As an example of how to obtain the results you are looking for with a SQL query alone:
SELECT UPPER(value) AS name, SUM(count) AS qty FROM table GROUP BY name;
If you make a new table to hold the correct values, you INSERT the above query to populate the new table as so:
INSERT INTO newtable (SELECT UPPER(value) AS name, SUM(count) AS qty FROM table GROUP BY name);
Strangely, MySQL seems to do this for you. I just tested this in MySQL 5.1.47:
create table c (value varchar(10), count int);
insert into c values ('Fred',7), ('FRED',1), ('Roger',3), ('roger',1);
select * from c;
+-------+-------+
| value | count |
+-------+-------+
| Fred | 7 |
| FRED | 1 |
| Roger | 3 |
| roger | 1 |
+-------+-------+
select value, sum(count) from c group by value;
+-------+------------+
| value | sum(count) |
+-------+------------+
| Fred | 8 |
| Roger | 4 |
+-------+------------+
I was surprised to see MySQL transform the strings like that, and I'm not sure I can explain why it did that. I was expecting to have to get four distinct rows, and to have to use some string functions to map the values to a canonical form.