I'm working on a quite big magento: it will have 50 different shops (1 magento install, 1 admin to rule them all) for start, this number is expected to raise in the future, and a catalog of more than 1k products. This catalog will be shared by all shops.
I'm concerned about the server requirements I need for this to run smoothly. So far this is what I've found to get the most of it:
Caching: using magento's cache with APC, MySQL's querys
use FastCGI instead of mod_php
database clustering: I don't think it will be necesary for 1k products, what do you think?
using Zend Server
Are there other thing I can do in order to improve magento's performance? I'd like to know all I need from the beginning so I can find the right server.
thanks in advance.
Make sure also to use block-level caching for the sites. Beyond this, one of the suggestions that I've seen implemented is to change dynamic blocks (such as blocks that grab product data dynamically) over to statically defined HTML if they don't change often.
Once you've coded a site, tune it using YSlow and Firebug to make sure that as many files as possible are cached, and that the page size is minimized. Minimizing the number of HTTP requests to Apache will increase the capacity of your server.
Finally, enable the flat catalog and flat category functions in Magento. This will force Magento to use fewer joins when retrieving catalog data, so your database load will go down and speed will increase considerably.
Hope that helps!
Thanks,
Joe
In testing, I noticed amazing improvements using an Amazon instance running ubuntu php-fpm REST and nginx. The only reason I didn't go there with our recent Magento upgrade is the host I'm on still works ok and I really don't want to be sysadmin for my site again.
Also, did you know there is http://magento.stackexchange.com ? :D
Related
I hvae django app that needs to be extremely fast, and it works good for now.
So my question is, is it better to put django app on one server and mysql on another server, or on one server both?
I ask because of communication between then.
I use digitalocean, and both are on one server.
It depends how well the application is written.
Poorly written django will generate a lot of queries so maybe it's beneficial to have it on the same server. Well written Django should leverage the database to do the heavy lifting, in which case its better to have it on a separate server, so the server can be tuned for a database. (In general having a separate database server is the way to go).
The best thing to do would be to add Django debug toolbar to your application and see if it is generating a lot of queries or not, and tune the application from there.
You have couple of options but let's stick to these two.
One server for everything
Good for setting up an application quickly, as it is the simplest setup possible, but it offers little in the way of scalability and component isolation.
There are a lot of pros, it's fast, simple to work with. It does not meet latency problems. From cons: you cannot horizontally scale.
Server for web application and server for database.
First of all, I would recommend to use Postgres, since the latest version (9.6) can now work on multiple cores, which makes it way faster than mysql.
It is good for setting up an application quickly, but keeps application and database from fighting over the same system resources.
From pros it does not fight over resources (RAM / CPU / I/O).
It may also increase security by removing database from DMZ.
From cons, it is harder to setup and when high-latency is going on, the queries might take longer to execute.
To sum up. I would use first option for small and medium applications which does not require a lot of requests.
I would consider moving DB to another server/servers, whenever the application hosts thousands of users per day.
I am working on a Rails application and was using SQLite during development and the speed had been very fast.
I have started to use a remote MySQL database hosted by Amazon and am getting very slow query times. Besides trying to optimize the remote database, is there anything on the Rails side of things I can do?
Local database access vs. remote will show a significant difference in speed. Since you've not provided any specifics I can't zero in on the issue but I can make a suggestion:
Try caching your queries and views as much as possible. This will reduce the amount of queries you need to do. This works well especially for static data like menus.
Optimization is the key. Make sure you eliminate as many unnecessary queries as you can, and those queries you make only request the fields you need using the select method.
Profile the various components involved. The database server itself is one of them. The network latency is another. While for the second one probably there is little you can do, probably you can tweak alot the first part. Starting from profiling the queries and going to tweaking the server itself.
Knowing where to look for will help you start with the best approach. As for caching, always keep that in mind, but that can prove to be quite problematic depending on the nature of your application.
So Have a web application that has 10-12 pages with many POST/ GET DB Calls. We usually have a apache crash/other problem when site traffic results to 1000 or so (concurrent users) which is very small number, we have updated server with good RAM and resources. When our system admin guy do load testing on blitz and other custom script and is suggesting to move away from Apache. Some things does not make sense to me. Like Apache is not too bad to handle few thousand of concurrent users considering we have cloudflare for caching. Here is what he suggested:
replacement of Apache+mod_fcgi with Nginx+php-fpm which can make the server handle much more users, and then test it.
or
2. For testing: Need 10-20 servers to run a scenario from. Basically, what is needed is a more complex blitz.io analogue. create one server, which takes all those hours, then just clone it in the cloud and pay for about 1 hour of testing multiplied by the number of servers needed.
Once again there are many DB calls anf HT access. ALso what makes Nginx better than apache in this case?
I would check this comparison first. Basically, nginx is event based, so it's able to handle more requests concurrently. However, as the MySQL DB seems to be the choke point here, it's very possible that nginx wouldn't solve all your problems. Perhaps moving to a NoSQL kind of database, that's better at scaling horizontally, would help (if that's feasible).
my site is very slow and i disabled the bulk of modules, so I think it was my hosting, first I upped the memory limit from 90M to 256M which helped with "500 errors" but the site is still very slow. I am thinking it may be the database settings, I am changing them with the module "drupal tweaks" for convenience.
Define "slow". Is your page slow to appear? If so, is it taking a while to load the page's HTML, or is much of the time spent trying to load CSS, JS, images, etc.? The "Net" panel in Firebug can help here, as can wget.
If the page itself is slow, is the slowness encountered at the network, Web server, PHP, or database level? Network slowness is easy to diagnose; try a ping. Either the network or the web server can show faults through the simple test of just throwing a big, static file somewhere on the server and then downloading it. PHP or DB slowness will show up when you look at the page with the devel module, but DB slowness can generally also be seen as long-running (as opposed to long-sleeping!) processes in the output of SHOW PROCESSLIST.
Alternately, give NewRelic a try. It will automate several of these steps, and give you fancy graphs, to boot.
Lastly: Consider getting a better hosting provider, ideally one with a great deal of Drupal experience (like BlackMesh or Acquia). At a great host, fixing this issue (if it arose in the first place) would be a collaborative effort between the hosting provider and the customer, rather than something you have to figure out on your own. It doesn't take many (billable) hours spent debugging hosting before you've paid for better hosting than you're getting now.
If you have a staging or development copy of the site, try installing the Devel module, which has an option for logging and appending stats about each query to the bottom of each Drupal page. That should help you track down the offending query or queries.
Here are some optimisation tips you might try.
First it is possible that your cache table should be messed up. Try to log in your phpmyadmin and try to flush all your cache tables (don't delete the tables just empty them).
Check inside the MySql installation folders config file alternative like: my-large.ini, my-huge.ini and try to rename them and copy them as bin/my.ini they are out of the box optimisations for your mySQL database.
You never mentioned the Drupal version you use. Drupal 7 has a brand new database API build on top of PHP PDO objects.
Here is a link that will help you a lot: http://drupal.org/node/310075
Rather than writing a SQL query in raw text and calling a helper function to execute the query, you can create a "query" object, call methods on it to add fields, filters, sorts, and other query elements, then call the query object's "execute" method.
Others
On my website PHP memory limit is 512Mb, 215Mb however should be ok for a production website.
Also why don't you enable APC logging?
Sometimes masive scripts could slow down your pages. in Performance, check your option: agregate CSS and JS file.
you might consider Drupal Pressflow over Drupal.
Boost up your website by installing Boost module: http://drupal.org/project/boost
Cache your Views
Other advanced techniques could include:
Cacherouter ( a module that replace the default Drupal cache).
Using Elysia Cron to reduce cron impact
Eventually switch to a superior hosting package or change your hosting provider
Regards...
Drupal site performance is obvious slow, but if its slow to a extreme level then you can go with other options that improved its performance.
1) You can install the memcache module on server,that is very fast in terms of site caching. Refer: How to install memcache
2) Change the PHP ini settings or mysql conf.
3) Check the custom modules if have, because execution of frequent queries create deadlock.
4) Check the mysql table storage engine & I will recommend the InnoDB which will provide row level locking.
I have a web that currently runs off one Mediatemple VPS. I'm now at the stage were the site is getting bogged down with scaling issues and I need to move to a better setup.
Is this a sensible setup:
php on one server
mysql database on another
Cloud Files CDN used to server images, javascript and css
My main thinking is to put the MySQL database on its own server away from the rest of the files as it seems to be causing most of the problems.
Try grabbing the lowest hanging fruit by tackling the scaling problem that are easiest to resolve and don't involve hardware solutions. Identify the bottleneck that has the greatest effect on performance. If its your SQL server that is running slow and being funky, read the sql logs, and do some googling :).
Just make sure that your hardware is the problem and not your application.