In a webpage I want to display several tables with information.
To do that, at the moment I do
SELECT * FROM tableONE WHERE field LIKE 'criteria'
then I process the result in a foreach loop. Then I do another query to the next table and process it again
SELECT * FROM tableTWO WHERE field LIKE 'criteria'
....
SELECT * FROM tableTWENTY WHERE field LIKE 'criteria'
I've the feeling that making 20 connections with the database is suboptimal and I could make a single query and in the foreach loop put each result in the appropriate table. The issues I found to do it are:
There is no related column amongst them, so I can not do a JOIN ON.
If I do a cartesian join I get many redundant rows
I can not use UNION because the columns on each table are different.
What is the most efficient way to do this?
Thanks
I've tried JOIN, UNION and separating the tables with comas (cartesian join), but they don't give the expected result
Just use UNION Clause
SELECT field1, field2, filed3 FROM table1
UNION
SELECT field1, NULL as filed2, field4 as filed3 FROM table2
;
Beware that
Every SELECT statement within UNION must have the same number of columns
The columns must also have similar data types
I am playing around with ms-access (MS-Office Professional Plus 2013) trying to figure out if I have duplicate rows before I merge one table into another table. I want to collect the rows that are duplicates and give an error with the duplicates before the merge happens. I have two scenarios to cover. The first scenario is duplicates on a single column. The second scenario is duplicates on two columns. Any help on the first scenario would be appreciated.
Scenario 1:
The two tables have the exact same column structure so to keep it simple I will use the following table structure. ( I simply added two tables inside access and run the query to figure out the correct syntax.)
Duplicates based upon one column:
Table1 Table2
ID ID
1 1
2 3
Running the query:
Select ID from Table1
Union ALL
Select ID from Table2
group by ID having count(*) > 1
The result set is always the records from the first select statement. In other words it always returns Id=1 and Id=2. If you change Table1 to Table2 the result set is always from table2. If I change "Union all" to union same results. I tried changing the ID column names as well as change the type to be number instead of auto. Any idea what am I doing wrong?
Scenario 2: I know what the value should be in the second column so it is hard-coded. I added this here to show access appears to work as expected in this scenario but not in scenario 1.
Duplicates based upon two columns:
Table1 Table2
ID Field1 ID Field1
1 abc 1 abc
2 bcd 3 abc
Running the query below works as expected. The row with ID=1 is only returned.
select ID, Field1 from Table1 where Field1 = 'abc'
union all
select ID, Field1 from Table2 where Field1 = 'abc'
group by ID, Field1 having count(*) > 1
The GROUP BY is only being applied to the second table. You need to do the UNION ALL first, and then the GROUP BY and HAVING on a SELECT from the combined results.
Not Access specific, but something like this works:
SELECT id FROM
(
SELECT id FROM a
UNION ALL
SELECT id FROM b
) AS c
GROUP BY id HAVING COUNT(*) > 1
My preferred way to do things like that is to use the build in Query Wizard:
Query Wizard, Find Duplicates Query Wizard
Let Access create the SQL statement for you and then you can modify it and/or move it into code.
What I am trying to do is select each distinct column1 value from table1 and then select all the columns from those rows returned from the above. Is this possible at all?
What I have so far, however, nothing is returned:
SELECT * FROM (SELECT DISTINCT column1 FROM table1)
I've thought about putting a unique/distinct restriction in the where clause of the query:
SELECT * FROM table1 WHERE some_unique_determiner column1
Any ideas how I could go about achieving the desired output?
Ok so answering my own question. What I need to do was to group the data by column1, without use of a nested query. Many thanks to #VR46 for the help.
SELECT * FROM table1 GROUP BY column1
Returned all columns from each unique value from column1
In your next posts, it will be better if you post your table structures, input and desired out put so it will be easier for us to understand.
If I did understand, there is one of two options:
Either you have duplicates, and you want to eliminate them so your correct query should be
select distinct COLUMNa,COLUMNb,COLUMNc... ETC
which will drop duplicates(that the entire row is the same).
Or you want to eliminate rows that have the same column1 and it doesn't matter if all the rest is the same or not.
In that case, You need to tell us which one of the result you want to keep, The up to date one,the older, random ETC.. because right now its impossible to make you a query that selects all the columns after you distinct, since all the duplicates will return like this:
SELECT * FROM TABLE WHERE COLUMN1 IN(SELECT DISTINCT COLUMN1 FROM TABLE)
Which is a wrong query since it doesn't do anything.
I read all the relevant duplicated questions/answers and I found this to be the most relevant answer:
INSERT IGNORE INTO temp(MAILING_ID,REPORT_ID)
SELECT DISTINCT MAILING_ID,REPORT_IDFROM table_1
;
The problem is that I want to remove duplicates by col1 and col2, but also want to include to the insert all the other fields of table_1.
I tried to add all the relevant columns this way:
INSERT IGNORE INTO temp(M_ID,MAILING_ID,REPORT_ID,
MAILING_NAME,VISIBILITY,EXPORTED) SELECT DISTINCT
M_ID,MAILING_ID,REPORT_ID,MAILING_NAME,VISIBILITY,
EXPORTED FROM table_1
;
M_ID(int,primary),MAILING_ID(int),REPORT_ID(int),
MAILING_NAME(varchar),VISIBILITY(varchar),EXPORTED(int)
But it inserted all rows into temp (including duplicates)
The best way to delete duplicate rows by multiple columns is the simplest one:
Add an UNIQUE index:
ALTER IGNORE TABLE your_table ADD UNIQUE (field1,field2,field3);
The IGNORE above makes sure that only the first found row is kept, the rest discarded.
(You can then drop that index if you need future duplicates and/or know they won't happen again).
This works perfectly in any version of MySQL including 5.7+. It also handles the error You can't specify target table 'my_table' for update in FROM clause by using a double-nested subquery. It only deletes ONE duplicate row (the later one) so if you have 3 or more duplicates, you can run the query multiple times. It never deletes unique rows.
DELETE FROM my_table
WHERE id IN (
SELECT calc_id FROM (
SELECT MAX(id) AS calc_id
FROM my_table
GROUP BY identField1, identField2
HAVING COUNT(id) > 1
) temp
)
I needed this query because I wanted to add a UNIQUE index on two columns but there were some duplicate rows that I needed to discard first.
For Mysql:
DELETE t1 FROM yourtable t1
INNER JOIN yourtable t2 WHERE t1.id < t2.id
AND t1.identField1 = t2.identField1
AND t1.identField2 = t2.identField2;
You will first need to find your duplicates by grouping on the two fields with a having clause.
Select identField1, identField2, count(*) FROM yourTable
GROUP BY identField1, identField2
HAVING count(*) >1
If this returns what you want, you can then use it as a subquery and
DELETE FROM yourTable WHERE field in (Select identField1, identField2, count(*) FROM yourTable
GROUP BY identField1, identField2
HAVING count(*) >1 )
you can always get the primary ids by grouping that two unique fields
select count(*), id as count from table group by col a, col b having count(*)>1;
and then
delete from table where id in ( select count(*), id as count from table group by col a, col b having count(*)>1) limit maxlimit;
you can also use max() in place of limit
NOTE: This solution is an alternative & old school solution.
If you couldn't achieve what you wanted, then you can try my "oldschool" method:
First, run this query to get the duplicate records:
select column1,
column2,
count(*)
from table
group by column1,
column2
having count(*) > 1
order by count(*) desc
After that, select those results and paste them into the notepad++:
Now by using the find and replace specialty of the notepad++ replace them with; first "delete" then "insert" queries like this (from now on, for security reasons, my values will be AAAA).
Special Note: Please make another new line for the end of the last line of your data inside notepad++ because regex matched the '\r\n' at the end of the each line:
Find what regex: \D*(\d+)\D*(\d+)\D*\r\n
Replace with string: delete from table where column1 = $1 and column2 = $2; insert into table set column1 = $1, column2 = $2;\r\n
Now finally, paste those queries to your MySQL Workbench's query console and execute. You will see only one occurrences of each duplicate record.
This answer is for a relation table constructed of just two columns without ID. I think you can apply it to your situation.
In a large data set if you are selecting the multiple columns in the select clause ex:
select x,y,z from table1.
And the requirement is to remove duplicate based on two columns:from above example let y,z
then you may use below instead of using combo of "group by" and "sub query", which is bad in performance:
select x,y,z
from (
select x,y,z , row_number() over (partition by y,z) as index_num
from table1) main
where main.index_num=1
I need to perform a GROUP BY on 2 columns separately...
In common terms, I'd like the query to say: GROUP BY column 1, then once this grouping has been performed, and the rows returned have been refined, go back to the top and GROUP BY column 2 to refine the rows returned again.
For instance, instead of stating:
GROUP BY column_1, column_2
I want to state (I Understand this is incorrect syntax):
GROUP BY column_1
GROUP BY column_2
If this is unclear I can include a sample query with expected returned results.
Are you trying to do something like this?
select ...
from (
select ...
from some_table
where ...
group by column1
) as dt
group by column2
That's the closest thing I can think of that matches what your question appears to be asking.
Mostly you can group by multiple columns in mysql. The query is:select * from table group by col1, col2
But you can't get answer as you want as. So you've another chance to get correct answer in mysql. That is, you've to use subqueries.select * from (select * from table group by col2) tabl group by col1