Is there a manner to add something like a where clause as a 'global' parameter for a mysql session.
For example an company has multiple user and you want to query for the user in this company, normally you would use a statement like:
SELECT * FROM users WHERE users.companyId = 2;
The issue is that adding the WHERE clauses would mean a huge impact on the code. Though, we defined the relations and thus I image (though, I don't think it exists), that you could create a session with the "global" constrained that all queries in that session should comply to.
You can create a view
CREATE VIEW view2 AS SELECT * FROM table1 WHERE companyid = 2;
If slowness is your curse, there are a few things you can do:
put an index on the where field(s) in this case companyid.
if you need more speed you can partition the table by companyid.
make the table a memory table, and use hash indexes for = and IN fields.
use InnoDB, instead of MyISAM. InnoDB has faster indexes.
Do not use select *, explicitly select only the fields you need.
See: http://dev.mysql.com/doc/refman/5.0/en/create-view.html
http://dev.mysql.com/doc/refman/5.5/en/index-btree-hash.html
http://dev.mysql.com/doc/refman/5.0/en/memory-storage-engine.html
http://dev.mysql.com/doc/refman/5.1/en/partitioning.html
The answer is NO.
As said before, you should put an index on that coloumn. And you can create a view.
Also you could use a temporary table.
From mysql docs:
You can use the TEMPORARY keyword when creating a table. A TEMPORARY
table is visible only to the current connection, and is dropped
automatically when the connection is closed. This means that two
different connections can use the same temporary table name without
conflicting with each other or with an existing non-TEMPORARY table of
the same name. (The existing table is hidden until the temporary table
is dropped.)
As a final thought, I can say that if you're performing the same query over and over again, you should rethink your model diagram, maybe do some denormalization.
What will happen when you perform select from table that has no companyId :) You can create views however, and select from them instead
Related
I'm experiencing huge performance problem in one legacy application.
There is a search form where user can search records with given value.
A result row contains 10 columns. Then a SP returns any row which contains in any column that value.
This SP uses 8 Tables and some of them have about million records. Every minute I get a new record. This SP conducts paging as well.
Execution of this SP takes sometimes around 40 seconds.
What I did was, I created a new table and put there all records by using a query from this SP, but without conditions.
When there is a new update or update in one of source table I use a trigger and update this new "cache" table.
Now waiting for results from this new table takes only 1-3 seconds.
Has someone experience with something like this?
One of my colleagues said I better use view, but then every time I will be making JOINS.
What do you think? Is there another way?
Often times temporary tables can help you resolve performance issues. One approach might be to collect only the records that you need to consider into temporary tables and then create your final select statement from the temporary tables joined to any other tables that you're not filtering.
As an example, let's say one of the fields you are searching for is field1 in table1. Start by inserting into table #table1 only records that have the value of field1 you are looking for:
select PrimaryKeyTable1, Field1, Field2, Field3, etc...
into #table1
from table1
where Field1 = 'Whatever you are looking for'
This should be pretty fast even for a big tables, especially if you have an index on Field1. You do this for every table with search fields to collect all the records that have relevant records you are searching.
Then you also need to be sure to insert any records into your temporary tables that might have foreign key references to any of your other temporary tables. So let's say you also built a table #table2 with the above method that has a foreign key to table1 called PrimaryKeyTable1. You would insert those records like:
Insert into #table1
(PrimaryKeyTable1, Field1, Field2, Field3, etc...)
select table1.PrimaryKeyTable1, table1.Field1, table1.Field2, table1.Field3, etc...
from table1
join #table2
on table1.PrimaryKeyTable1 = table2.PrimaryKeyTable1
where table1.PrimaryKeyTable1 not in
(Select PrimaryKeyTable1 from #table1)
Now you will also have any records in #table1 that match to a record in #table2 that contain records that match the search criteria. You do this for all your temporary tables that have relevant foreign keys. The order that you do the inserts matters; be sure that you don't reference any temporary tables until after the last insert statement while collecting the foreign key referenced records.
Then you can simply do your final select statement, replacing the actual tables with the temporary tables you have built and eliminating all the filters that search your field data. Depending on the structure of your query there might be other optimizations, but that is the general idea.
If you've already explored all of your indexing options and this still doesn't help, MS SQL Server has "Change Tracking" features that maybe be of use to you in building your cache table. You enable the database for change tracking and configure which tables you wish to track. SQL Server then creates change records on every update, insert, delete on a table and then lets you query for changes to records that have been made since the last time you checked. This is very useful for syncing changes and is more efficient than using triggers. It's also easier to manage than making your own tracking tables. This has been a feature since SQL Server 2005.
How to: Use SQL Server Change Tracking
Change tracking only captures the primary keys of the tables and let's you query which fields might have been modified. Then you can query the tables join on those keys to get the current data. If you want it to capture the data also you can use Change Capture, but it requires more overhead and at least SQL Server 2008 enterprise edition.
Change Data Capture
Your solution is a robust way of doing what is called in Microsoft SQL Server "an indexed view" or "materialized view" in Oracle.
Basically you are correct - it's faster to navigate single indexed table then a dozen ones which are updated constantly.
You should really try creating an indexed view (some start here https://technet.microsoft.com/en-us/library/dd171921(v=sql.100).aspx) and it will probably solve all your performance issues.
You can use schema binding View and create cluster index on view.it will store your view data physically.but after creating schema binding view you can not alter your table.
I have to handle database that is read (and only read) by third party software I cannot verify. It has a table which stores partial copy of other table.
Would it be safe to replace this table with view?
Besides the view restrictions, when using views in MySQL you have to be aware of one very important issue: The performance of WHERE statements suffers greatly:
For example:
SELECT column_a FROM table_n WHERE column_a="some value";
is quick assuming an index is in place on column_a.
Now create a view:
CREATE VIEW column_a_view AS SELECT column_a FROM table_n;
SELECT * FROM column_a_view b WHERE b.column_a="some value";
can lead to a full table scan since MySQL (at least until 5.6) did not always recognize the fact that it can use the index.
So especially if you are working with large tables, it can be more beneficial to create "copies" of the related data and replace that data once per time interval then working with views.
Is it any faster to add or drop multiple columns in one query, rather than executing a query for each column? For example, is this:
ALTER TABLE t2 DROP COLUMN c, DROP COLUMN d;
any faster than this?
ALTER TABLE t2 DROP COLUMN c;
ALTER TABLE t2 DROP COLUMN d;
Yes, it should be faster to run a single ALTER TABLE statement than two.
In my experience (with InnoDB on 5.1 and 5.5), MySQL doesn't seem to modify the table "in place". MySQL actually creates a new table, as copy of the old table with the specified modifications.
Two separate statements would require MySQL to do that copy operation twice.
With a single statement, you give MySQL the opportunity to make all the changes with just one copy operation. (I don't know the details of the MySQL internals, but it's possible that MySQL actually does the copy two times.)
Other database engines (MyISAM et al.) may get processed differently.
I believe the InnoDB plugin and/or newer versions of InnoDB in the MySQL (>5.5) have some algorithms other than the "copy" method, at least for some changes, which allow for the table to still be available while the ALTER TABLE is running (for read queries). But I don't know all the details.
Yes, it's faster. You only have to make one call to the database API, and it only has to parse the query once.
However, for ALTER TABLE queries, performance usually isn't a concern. You shouldn't be doing this frequently, only when you redesign your schema.
But if your question were about UPDATE queries, for instance, it would probably be significant. E.g. you should do:
UPDATE table
SET col1 = foo, col2 = bar
WHERE <condition>;
rather than
UPDATE table
SET col1 = foo
WHERE <condition>;
UPDATE table
SET col2 = bar
WHERE <condition>;
Here is a chunk of the SQL I'm using for a Perl-based web application. I have a number of requests and each has a number of accessions, and each has a status. This chunk of code is there to update the table for every accession_analysis that shares all these fields for each accession in a request.
UPDATE accession_analysis
SET analysis_id = ? ,
reference_id = ? ,
status = ? ,
extra_parameters = ?
WHERE analysis_id = ?
AND reference_id = ?
AND status = ?
AND extra_parameters = ?
and accession_id is (
SELECT accesion_id
FROM accessions
where request_id = ?
)
I have changed the tables so that there's a status table for accession_analysis, so when I update, I update both accession_analysis and accession_analysis_status, which has status, status_text and the id of the accession_analysis, which is a not null auto_increment variable.
I have no strong idea about how to modify this code to allow this. My first pass grabbed all the accessions and looped through them, then filtered for all the fields, then updated. I didn't like that because I had many connections with short SQL commands, which I understood to be bad, but I can't help but think the only way to really do this is to go back to the loop in Perl holding two simpler SQL statements.
Is there a way to do this in SQL that, with my relative SQL inexperience, I'm just not seeing?
The answer depends on which DBMS you're using. The easiest way is to create a trigger on one table that provides the logic of updating the other table. (For any DB newbies -- a trigger is procedural code attached to a table at the DBMS (not application) layer that runs in response to an insert, update or delete on the table.). A similar, slightly less desirable method is to put the logic in a stored procedure and execute that instead of the update statement you're now using.
If the DBMS you're using doesn't support either of these mechanisms, then there isn't a good way to do what you're after while guaranteeing transactional integrity. However if the problem you're solving can tolerate a timing difference in the two tables' updates (i.e. The data in one of the tables is only used at predetermined times, like reporting or some type of batched operation) you could write to one table (live) and create a separate process that runs when needed (later) to update the second table using data from the first table. The correctness of allowing data to be updated at different times becomes a large and immovable design assumption, however.
If this is mostly about connection speed, then one option you have is to write a stored procedure that handles the "double update or insert" transparently. See the manual for stored procedures:
http://dev.mysql.com/doc/refman/5.5/en/create-procedure.html
Otherwise, You probably cannot do it in one statement, see the MySQL INSERT syntax:
http://dev.mysql.com/doc/refman/5.5/en/insert.html
The UPDATE syntax allows for multi-table updates (not in combination with INSERT, though):
http://dev.mysql.com/doc/refman/5.5/en/update.html
Each table needs its own INSERT / UPDATE in the query.
In fact, even if you create a view by JOINing multiple tables, when you INSERT into the view, you can only INSERT with fields belonging to one of the tables at a time.
The modifications made by the INSERT statement cannot affect more than one of the base tables referenced in the FROM clause of the view. For example, an INSERT into a multitable view must use a column_list that references only columns from one base table. For more information about updatable views, see CREATE VIEW.
Inserting data into multiple tables through an sql view (MySQL)
INSERT (SQL Server)
Same is true of UPDATE
The modifications made by the UPDATE statement cannot affect more than one of the base tables referenced in the FROM clause of the view. For more information on updatable views, see CREATE VIEW.
However, you can have multiple INSERTs or UPDATEs per query or stored procedure.
What is the purpose of a temporary table like in the following statement? How is it different than a regular table?
CREATE TEMPORARY TABLE tmptable
SELECT A.* FROM batchinfo_2009 AS A, calibration_2009 AS B
WHERE A.reporttime LIKE '%2010%'
AND A.rowid = B.rowid;
Temp tables are kept only for the duration of your session with the sever. Once the connection's severed for any reason, the table's automatically dropped. They're also only visible to the current user, so multiple users can use the same temporary table name without conflict.
Temporary table ceases to exist when connection is closed. So, its purpose is for instance to hold temporary result set that has to be worked on, before it will be used.
Temporary tables are mostly used to store query results that need further processing, for instance if the result needs to be queried or refined again or is going to be used at different occasions by your application. Usually the data stored in a temporary database contains information from several regular tables (like in your example).
Temporary tables are deleted automatically when the current database session is terminated.
Support for temporary tables exists to allow procedural paradigms in a set-based 4GL, either because the coder has not switched their 3GL mindset to the new paradigm or to work around a performance or syntax issue (perceived or otherwise).