I am building a website which populates from a database. I'm testing now, and I'd like to see what my site will look like with a lot of data (mainly so I can watch performance, build out pagination, and address any issues with presentation). I have about 10 pieces of data in my table, which is great, but I'd like to display about 2,000 on my page.
Is there a way I can read from the same SELECT * FROM table statement over and over again in the same query in order to read the table multiple times?
I can do this by feeding all my results into a variable and echoing that variable multiple times, but it won't allow me to set a LIMIT or give me the proper count of rows from the query.
I'm surprised I haven't found a way to do this by Googling. It seems like it would be an easy, built-in thing.
If there's not, can you suggest any other way I can do this without modifying my original table?
Please use Cross Join. Cross Join will give you a cartesian product of rows from tables joined. Cross Join can generate a lot of data in quick amount of time. Can be useful for extensive testing.
Example:
SELECT * FROM A
CROSS JOIN B;
You can cross join on the same table as well.
As of MySQL 8 you can use a recursive query to get your rows multifold:
with recursive cte (a, b, c) as
(
select a, b, 1 from mytable
union all
select a, b, c + 1 from cte where c < 10 -- ten times as many
)
select a, b from cte;
(You can of course alter the generated values in the part after union all, e.g.: select a + 5, b * 2, c + 1 from cte where c < 10.)
Demo: https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=3a2699c167e1f4a7ffbe4e9b17ac7241
Related
I am trying to display a combination of two tables in which all rows will be there from first table and only 1 row from second table on some condition.
I was using left join suggest me some solutions
I think what you are looking for is UNION or UNION ALL.
It basically appends the results of two queries.
So your query would look somewhat like
select a, b, c from oneTable
UNION ALL
select x, y, z from otherTable where id = 23
The individual select can have where clauses and all kinds of stuff you know from SQL.
I'm running the exact same query four times, twice as a subquery, gathering different information each time. What is the best way to pass the results of the first query to the other three so it doesn't have to run three more times?
On the average, it returns around 2,000 rows, but can be anywhere from 0 (in which case I skip the other three) to all. The primary table has nearly 300,000 rows, is growing by about 800 per day, rows are never deleted, and thousands of rows are updated throughout the day, many multiple times.
I looked into query cache, but it doesn't look like it has a bright future:
disabled-by-default since MySQL 5.6 / MariaDB 10.1.7
depreciated as of MySQL 5.7.20
removed in MySQL 8.0
I considered using GROUP_CONCAT with IN, but somehow I doubt that would work very well (if at all) with larger queries.
This is in a library I use to format the results for other scripts, so the original query can be nearly anything. Usually, it is on indexed columns, but can be horribly complicated using stored functions and take several minutes. It always involves the primary table, but may also join in other tables (but only to filter results from the primary table).
I am using Perl 5.16 and MariaDB 10.1.32 (will upgrade to 10.2 shortly) on CentOS 7. I am using prepare_cached and placeholders. The user this library runs as has SELECT-only access to tables plus EXECUTE on a couple stored functions, but I can change that if needed.
I've minimized the below as much as I can and used metasyntactic variables (inside angle brackets) as much as possible in an attempt to make the logic clear. id is 16 bytes and the primary key of the primary table (labeled a below).
I'm accepting three parameters as input. <tables> always includes a and may include a join like a join b on a.id=b.id. <where> can be simple like e=3 or horribly complex. I'm also getting an array of data for the placeholders, but I've left that out of the below because it doesn't affect the logic.
<search> = FROM <tables> WHERE (<where>)
<foo> = k < NOW() - INTERVAL 3 HOUR
<bar> = j IS NOT NULL OR <foo>
<baz> = j IS NULL AND k > NOW() - INTERVAL 3 HOUR
so <baz> is !<bar>. Every row should match one or the other
<where> often includes 1 or more of foo/bar/baz
SELECT a.id, b, c, d, <foo> x <search> ORDER BY e, id
SELECT COUNT(*) <search> AND <baz>
I really only need to know if any of the above rows match <baz>
SELECT c, COUNT(*) t, SUM(<bar>) o FROM a WHERE c IN (SELECT c <search> GROUP BY c) GROUP BY c
SELECT d, COUNT(*) t, SUM(<bar>) o FROM a WHERE d IN (SELECT d <search> GROUP BY d) GROUP BY d
The last two get a list of all unique c or d from the rows in the original query and then count how many total rows (not just the ones in the original query) have matching c or d and how many of those match <bar>. Those results get dumped into hashes so I can look up those counts while I iterate through the rows from the original query. I'm thinking running those two queries once is more efficient than running two smaller queries for each row.
Thank you.
Edited to add solution:
A temporary table was the answer, just not quite in the way Raymond suggested. Using EXPLAIN on my queries indicates that MariaDB was already using a temporary table for each, and deleting it when each was complete.
An inner join only returns rows that exist in both tables. So by making a temporary table of IDs that match my first SELECT, and then joining it to the primary table for the other SELECTs, I only get the data I want, without having to copy all that data to the temporary table.
"To create a temporary table, you must have the CREATE TEMPORARY TABLES privilege. After a session has created a temporary table, the server performs no further privilege checks on the table. The creating session can perform any operation on the table, such as DROP TABLE, INSERT, UPDATE, or SELECT." - https://dev.mysql.com/doc/refman/5.7/en/create-temporary-table.html
I also figured out that GROUP BY sorts by default, and you can get better performance if you don't need the data sorted by telling it not to.
DROP TEMPORARY TABLE IF EXISTS `temp`;
CREATE TEMPORARY TABLE temp AS ( SELECT a.id FROM <tables> WHERE <where> );
SELECT a.id, b, c, d, <foo> x FROM a JOIN temp ON a.id=temp.id ORDER BY e, id;
SELECT COUNT(*) FROM a JOIN temp WHERE <baz>;
SELECT c, COUNT(*) t, SUM(<bar>) o FROM a WHERE c IN (SELECT c FROM a JOIN temp GROUP BY c ORDER BY NULL) GROUP BY c ORDER BY NULL;
SELECT d, COUNT(*) t, SUM(<bar>) o FROM a WHERE d IN (SELECT d FROM a JOIN temp GROUP BY d ORDER BY NULL) GROUP BY d ORDER BY NULL;
DROP TEMPORARY TABLE IF EXISTS `temp`;
The best i could think of is by using a TEMPORARY table.
p.s iám using valid MySQL SQL code mixed with the same pseudo code as the topicstarter
CREATE TEMPORARY TABLE <name> AS ( SELECT FROM <tables> WHERE (<where>) )
<foo> = k < NOW() - INTERVAL 3 HOUR
<bar> = j IS NOT NULL OR <foo>
<baz> = j IS NULL AND k > NOW() - INTERVAL 3 HOUR
so <baz> is !<bar>. Every row should match one or the other
<where> often includes 1 or more of foo/bar/baz
SELECT a.id, b, c, d, <foo> x FROM <name> ORDER BY e, id
SELECT COUNT(*) FROM <name> WHERE <baz>
SELECT c, COUNT(*) t, SUM(<bar>) o FROM a WHERE c IN (SELECT c FROM <name> GROUP BY c) GROUP BY c
SELECT d, COUNT(*) t, SUM(<bar>) o FROM a WHERE d IN (SELECT d FROM <name> GROUP BY d) GROUP BY d
I have two tables, my_table1 and my_table2.
my_table1 contains numbers from 1 to 10 and my_table2 contains letters a, b, c and d.
I want to do a query which returns the following:
1 a
1 b
1 c
1 d
2 a
2 b
2 c
2 d
All the way until the end.
Is there any possible way to do this in SQL?
Thanks in advance.
That is a cross join. You can write that in the simple (old) form by just selecting select * from table1, table2, but this is outdated syntax, and your queries will become very hard to read if you mix this syntax with the more modern explicit joins that were introduced in 1992. So, I'd chose to write the explicit cross join.
Also, it looks like you want the results sorted. If you're lucky this happens automatically, but you cannot be sure that this will always happen, so best to specify it if you need it. If not, omit the order by clause, because it does make the query slower.
select
n.nr,
l.letter
from
my_table1 n
cross join my_table2 l
order by
n.nr,
l.letter
That is a CROSS JOIN, in MySQL equivalent to an INNER JOIN or a regular comma:
SELECT * FROM my_table1, my_table2;
cf. https://dev.mysql.com/doc/refman/5.7/en/join.html
I have a short question an I am stuck at this point. I have the following query:
SELECT A, B
FROM my_table AS r
LEFT JOIN my_table_2 AS p ON r.rs_id = p.rs_id
WHERE r.rs_id = $DB->quote($_SESSION['mysession']['id']) EXISTS (SELECT A, B, date FROM my_table WHERE date = CURDATE())';
What I am trying to do is to get results from the database. If there is a row however that has a record that matches the current date I only want to get this rows, so not all A + B anymore. If there are NO rows with a current date I want to load all A + B by default.
Does anyone knows the answer? Thanks!
To do this in a clean way, you would check on application level, if the query searching for records with date equal to current date returns any records. If not, execute a second query.
You can do the same in MySQL, but since this is in a way a bit "dirty" (the techniques used are not meant to be used this way), I would recommend solving doing this on application level. In case you're curious, here's how:
SELECT SQL_CALC_FOUND_ROWS whatever
FROM table
WHERE date_column = CURDATE()
UNION ALL
SELECT whatever
FROM table
WHERE FOUND_ROWS() = 0;
read more about it here
CREATE PROCEDURE Test
AS
BEGIN
SELECT * FROM (
SELECT 1 AS a,'test1' as b, 'query1' as c
UNION ALL
SELECT 2 AS a,'test22' as b, 'query22' as c
UNION ALL
SELECT 2 AS a,'test2' as b, 'query2' as c
UNION ALL
SELECT 3 AS a,'test3' as b, 'query3' as c
UNION ALL
SELECT 4 AS a,'test4' as b, 'query4' as c
) As sample
FOR XML RAW
END
Can we guarantee that the stored procedure returns results in given order?
Normally it says when we insert these select query to temporary table we can't guarantee its inserting order. So we have to use order by clause. But most of time it gives same order. Can we enforce to give it some different order? Is this related with clustered and non clustered indices.
In second case can we enforce inserting order by adding Identity column?
When you insert data, SQL refers to it as a set. When even writing data to disc it tries to take minimum space and starts inserting rows in free pages it finds in non-uniform extents at first. So when you query data the result depends on the order of the information which is in the cash and the order of the information which is read from hard disc. I think it is almost impossible to predict that orders as it depends on the work of OS , other programs and so on.