I want to create a query result page for a simple search, and i don't know , should i use views in my db, would it be better if i would write a query into my code with the same syntax like i would create my view.
What is the better solution for merging 7 tables, when i want to build a search module for my site witch has lots of users and pageloads?
(I'm searching in more tables at the same time)
you would be better off using a plain query with joins, instead of a view. VIEWS in MySQL are not optimized. be sure to have your tables properly indexed on the fields being used in the joins
If you always use all 7 tables, i think you should use views. Be aware that mysql changes your original query when creating the view so its always good practice to save your query elsewhere.
Also, remember you can tweak mysql's query cache env var so that it stores more data, therefore making your queries respond faster. However, I would suggest that you used some other method for caching like memcached. The paying version of mysql supports memcached natively, but Im sure you can implement it in the application layer no problem.
Good luck!
Related
I have a Laravel web app that's using a VueJS front-end and MySQL as the RDBMS. I currently have a table that is 23.8gb and contains 8m+ rows and it's growing every second.
When querying this data, I'm joining it to 4 other tables so the entire dataset is humongous.
I'm currently only pulling and displaying 1000 rows as I don't need anymore than that. VueJS is showing the data in a table and there are 13 filter options for the user to select from to filter the data ranging from date, name, status, etc.
Using Eloquent and having MySQL indexes in place, I've managed to get the query time down to a respectable time but I need this section of the app to be as responsive as possible.
Some of the where clauses that kick off from the filters are taking 13 seconds to execute which I feel is too long.
I've been doing some reading and thinking maybe MongoDB or Redis may be an option but have very little experience with either.
For this particular scenario, what do you think would be the best option to maximise read performance?
If I were to use MongoDB, I wouldn't migrate the current data... I'd basically have a second database that contains all the new data. This app hasn't gone into production yet and in most use cases, only the last 30 days worth of data will be required but the option to query old data is still required hence keeping both MySQL and MongoDB.
Any feedback will be appreciated.
Try to use elasticsearch. It will speed up the read process.
Try converting the query into a stored procedure. You can execute the stored procedure like this..
DB::select('exec stored_procedure("Param1", "param2",..)');
or
DB::select('exec stored_procedure(?,?,..)',array($Param1,$param2));
Try this for without parameters
DB::select('EXEC stored_procedure')
Try using EXPLAIN to optimise the performance.
How to optimise MySQL queries based on EXPLAIN plan
I am developing with Codeigniter and when it gets to complicated database queries
I am using
$this->db->query('my complicated query');
then cast to array of object using $query->result();
and so far it's very good and useful
Now my question is
what if I want to create mysql view and select from it? Will
$this->db->from('mysql_view')
take the mysql view as it's a table or not?
And if I do that will be any difference in performance are views faster than normal database query?
What would be best practice with Codeigniter and MYSQL database dealing with complicated queries as I understand that ActiveRecord is just query builder and as some tests it's even a little slower
Thanks in advance for your advise
MySQL views are queried the same way as tables, on a side note, you can't have a table and a view share the same name.
Depends on the query you use in the view, views can be internally cached so in the long run - yes, they are faster.
Best practice in this case is to use whatever you find easy to use for yourself and your team, I personally stick to using $this->db->query(); as I find it's easier to change a simple query of this kind to have some advanced functionality like sub-queries or other things that are hard and/or impossible to do with CI query builder. My advice would be to stick to one way of queries - if you use ->query(), then use them everywhere, if you use a query builder, then use it wherever it is possible to achieve the result using it.
Initially I thought this would be a stupid question, but now I am inspired by the following question.
Background: I have a lot of data in MySQL, but MySQL's spatial support is terrible. Ideally I would like to migrate everything to Postgres, but converting from MySQL to Postgres is a massive ball of hurt (I've already wasted close to a week struggling with it). Now I am thinking, if only I could maintain only the spatial portion in Pg, do the spatial queries in Pg, then use those row ids to query non-spatial data from MySQL.
I am a Perl DBI person. My question is thus -- can I create a single database handle that actually allows querying by JOINing a table from Pg with a table from MySQL, assuming they have a common id column?
No, you will need to query both separately and combine the data at the application layer. See a more informed answer here:
How to create linked server MySQL
No, I don't think you could do it that way. You would have to query the data separately and combine the results in your code. I believe there are no REAL RDB's that can do what you want.
We've got a MySQL table in which rows are never updated, but instead new rows are added and the old ones marked obsolete. Think Rails' acts_as_paranoid, but for every update.
To make working with Rails sane, we've got a view which selects only the rows which are "current". That makes a much better "table" for our ActiveRecord model.
The snag: our indexes aren't being used anymore.
Queries on the view don't use the underlying tables' indexes. You can't add an index to a view. Without indexes, the app is unbearably slow.
The only solution we've come up with is to build a materialized view, but that's a pain in MySQL because they're not natively supported.
Is there a better way to do this?
Since MySQL executes the query underlying the view, it should still use the indexes on the query that composes the view. Do an explain on the query that you used to create the view and post here if it's not indexing.
I'd like to setup one instance of MySQL to flat-out reject certain types of queries. For instance, any JOINs not using an index should just fail and die and show up on the application stack trace, instead of running slow and showing up on the slow_query_log with no easy way to tie it back to the actual test case that caused it.
Also, I'd like to disallow "*" (as in "SELECT * FROM ...") and have that throw essentially a syntax error. Anything which is questionable or dangerous from a MySQL performance perspective should just cause an error.
Is this possible? Other than hacking up MySQL internals... is there an easy way?
If you really want to control what users/programmers do via SQL, you have to put a layer between MySQL and your code that restricts access, like an ORM that only allows for certain tables to be accessed, and only certain queries. You can then also check to make sure the tables have indexes, etc.
You won't be able to know for sure if a query uses an index or not though. That's decided by the query optimizer layer in the database and the logic can get quite complex.
Impossible.
What you could do to make things work better, is createing views optimized by you and give the users only access to these views. Now you're sure the relevent SELECT's will use indexes.
But they can still destroy performance, just do a crazy JOIN on some views and performance is gone.
As far as I'm aware there's nothing baked into MySQL that provides this functionality, but any answer of "Impossible", or similar, is incorrect. If you really want to do this then you could always download the source and add the functionality yourself, unfortunately this would certainly class as "hacking up the MySQL internals".