I would like to know the the difference between the materialized view and normal table in mysql. Though mysql dont support materialized view but we can use flexview
currently we are processing a call detail records( cdr files) and saving the records in the table (say table A) and then we have created another table( say table B ) which will have the records from table A but grouped by records. so table B will have computed records of table A eg group by country or city.
Table A keeps on growing at the enormous size as it is saving all the cdr files
And Table B is saving all the results we got from table A using a group by queries
Now my question how the materialized view can be implemented in this scenario and how it makes a difference when compared to table B in above scenario as far as the query performance is concerned.
In terms of comparing query performance, a SELECT from table B vs. a SELECT from a "flewview materialized view", there will be no difference, because what flexview does is create an actual table, and maintain the contents of that table.
What is probably going to be different is how the "materialized view" table is populated. It's very likely going to be different than how you populate and maintain table B.
Depending on how flexview maintains the table, when insert/update/delete operations are occurring, there might be an impact to concurrency. i.e. a query being blocked from performing a read when the "refresh" process has rows locked.
But as far as a query against the materialized view, it won't be any different than a query pf a regular table.
Related
I have two big tables for example:
'tbl_items' and 'tbl_items_transactions'
First table keeping some items metadata which may have 20 (varchar) columns with millions rows... and second table keeping each transaction of first table.
for example if a user insert new record to tbl_items then automatically a new record will be adding to tbl_items_transactions with same data plus date, username and transaction type to keep each row history.
so in the above scenario two tables have same columns but tbl_items_transactions have 3 extra columns date, username, transaction_type to keep each tbl_items history
now assume we have 1000 users that wants to Insert, Update, Delete tbl_items records with a web application. so these two tables scale very soon (maybe billion rows in tbl_items_transactions)
I have tried MySQL, MariaDB, PostgreSQL... they are very good but when table scale and millions rows inserted they are slow when run some select queries on tbl_items_transactions... but sometimes PostgreSQL is faster than MySQL or MariaDB
now I think I'm doing wrong things... If you was me... do you use MariaDB or PostgreSQL or somthing like that and structure your database like what I did?
Your setup is wrong.
You should not duplicate the columns from tbl_items in tbl_items_transactions, rather you should have a foreign key in the latter table pointing to the former.
That way data integrity is preserved, and tbl_items_transactions will be much smaller. This technique is called normalization.
To speed up queries when the table get large, define indexes on them that match the WHERE and JOIN conditions.
I am currently using mysql
I have two tables called person and zim_list_id both tables has over 2 million rows
I want to update person table using zim_list_id table
the query I am using is
update person p JOIN zim_list_id z on p.person_id = z.person_id
set p.office_name = z.`Office Name`;
I have also created index on zim_list_id table and person table , the query I executed was
create index idx_person_office_name on person(`Office_name`);
create index idx_zim_list_id_office_name on zim_list_id(`Office name`);
the query execution is taking very long. is there any way to reduce the execution time?
The indexes on Office Name do nothing at all for this query. All you've done with those indexes is make inserts and updates slower, as now the database has to update the index any time that column changes.
What you really need, if you don't already have them, are indexes on the person_id field in those tables, to make the join more efficient.
You might also consider adding Office_Name as a second column on the zim_list_id table's index, as this will allow the database to fullfill that part of the query entirely from the index. But I wouldn't do that until I had checked the results after setting the plain person_id indexes first.
Finally, I'm curious how much memory is in that server (especially relative to the total size of the database), how much of it is available in your MySql buffer_pool_size setting, and what other work that server might be doing... there could always be an environmental factor as well.
I'm using SQL Server 2008.
I have two tables like this:
OrderItems
OrderItemID
InventoryItemID
and :
InventoryItems
InventoryItemID
ItemCode
My query plan shows a lot of time getting sucked into joining the InventoryItemIDs to get the ItemCode for every order item.
I already have a nonclustered index that includes both columns on both tables, but would like to make it even faster -- can I "import" the relational ItemCode into the OrderItems table's index?
No you cannot INCLUDE columns from another table. However, one, 'outside the box' suggestion would be to create an Indexed View that joins OrderItems and InventoryItems. You can setup your clustered index on the view in such a way to achieve fastest performance and it would appear that both orders and inventory items are in the same "table", without the need to do a join. The result would be a somewhat denormalized view of the data.
Of course, there are a number of restrictions on an Indexed View, but an inner join is permitted. I have used Index Views quite frequently and they can be tremendously useful. There is a insert / update / delete performance consideration here, because each of the operations will require an update to the view. But, it might be worth it.
I don't have my server up and running yet, so unfortunately I can't test yet, but I was hoping you could shed some light on a question I have.
So, Table A has an inverse one-to-many relationship with Table B, so would it be better to store the ID's of Table B in Table A search by ID, or would it be better/faster to query Table B for all results where it's Table A ID is equal to my Table A's ID?
Basically Search (Search for row based on ID) vs Query (Grab all rows that have a certain value).
As long as the column containing the ID in B is a (foreign) key. It is most certainly faster. Storing a non-scalars (ie. lists) in columns in a database is generally a bad idea.
I have a MySQL MYISAM table (say tbl) consisting of 2 unsigned int fields, say, f1 and f2. There is an index on f2 and the table is very large (approximately 320,000,000+ rows). I update this table periodically (with approximately 100,000 new rows a week), and, in order to be able to search this table without doing an ORDER BY (which would be very time consuming in real-time queries), I physically ORDER the table according to the way in which I want to retrieve its rows.
So, I perform an ALTER TABLE tbl ORDER BY f1 DESC. (I know I have enough physical space on the server for a copy of the table.) I have read that during this operation, a temporary table is created and SELECT statements are not affected on the current rows.
However, I have experienced that this is not the case, and SELECT statements on the table that occur at the same time with the ALTER table are getting blocked and do not terminate. After the ALTER TABLE tbl completes (about 40 minutes on the production server), the SELECT statements on tbl start executing fine again.
Is there any reason why the "ALTER table tbl ORDER BY f1 DESC" seems to be blocking other clients from querying tbl?
Altering a table will always grab a lock on the table, preventing SELECTs from running.
I'll admin that I didn't even know you could do that with an ALTER TABLE.
What are you trying to get from the table? For example, all records in a given range? 320 million rows is not a trivial number. I'll give you my gut reactions:
Switch to InnoDB (allows #2, also gives transactions, but without #2 may hurt performance)
Partition the table (makes it act like a number of slightly smaller tables)
Consider a redesign, such as having a "working set" table and a "historical" table, basically manually partitioning. If you usually look for recently inserted data, this (along with partitioning) will help a lot. If your lookups are evenly distributed, this probably won't make a difference.
Consider adding a new column you could use in conjunction to narrow down selects (so instead of searching on date, search on date and customer ID)
Since I don't know what you're storing, some of these (such as #4) may not apply.
There are some other things you could try. OPTIMIZE TABLE may help you but take less time, but I doubt it. I think internally it's implemented as a dump/reload, at least on the InnoDB side.