If you're hosting a blog on a shared server and you're concerned about page loading time, would it at least theoretically be better to host the images in an Amazon S3 bucket and then just link to them?
As Google apparently takes page loading into consideration, will this possibly improve search rankings?
I am not sure about search rankings but it really improves your page load speed. It commonly known as CDN. You can use this http://wordpress.org/extend/plugins/w3-total-cache/ and it has an option to select a cdn that includes the amazon server. So managing images between your wordpress and the amazon server is pretty easy by using this plugin. Thanks
Yes Good Idea. It will improve your web site performance by moving your media files from your main web server. This could be as simple as creating a sub-domain that points to a host that serves your media files.
Amazon S3 also provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web.
Generally, software developers use Amazon S3 in their applications that need the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites
Related
I am working on a project that is consisted of multiple web applications communicating through some messaging system. Those applications need to be looking at some assets like images and so. Is storing images in MySQL database as blobs a good idea? or should I be using another tool.
Notes:
I cannot use a hosting server because it's not guaranteed to have internet connection where the applications are deployed.
Applications are to be deployed on different machines.
While you can store images in a database, you must carefully consider the size of each image and the number of images to be stored. Both of these will affect the optimal server memory size needed for both performance reasons and to backup your application.
You also indicated that an Internet connection is not guaranteed where the applications are to be deployed. This would make storage of images on a content delivery network (CDN) impractical.
As you plan how "multiple web applications communicate through some messaging system", that might suggest some methods of sharing images between hosts.
I tried to set up a micro-blog web site with simple function.
And in the future, I would set some API for the mobile app.
The main feature is simple. People can register, post blog , tag articles, and comments.
Currently, I am using laravel framework + Mysql + Apache and host on VPS.
(Hardwere spec is HD:160 GB,CPU:8core,RAM:8 GB.)
The database tables are basic,including user,comments,article,tags,and tags pivot table.
Everything works fine.
But
I had a little concern about scalability and performance.
Since I have no any expereince about scale a web site.
Could someone give me some key concepts of what should I concern if the users numbers increase to 10,000 ~100,000?
I am OK with change my host platform or even change the framework and database at the begining.
All I try to avoid is that the web site might be crash after deploy a period of time.
The update and transfer would be a disaster.Thanks
Look into scalable cloud hosting, such as digital ocean or amazon, where you can scale your capacity as you grow.
These companies allow you to start small with a "slice" of a server and as you grow you can grow into multiple servers. Load balancing is usually done on their end so all you need to do is focus on your application
My partner and I are trying to start a website hosted in cloud. It has pretty heavy ajax traffic and the backend handles money transactions so we need ACID in some of the DB tables.
Currently everything is running off a single server. Some of the AJAX traffic are cached in text files.
Question:
What's the best way to scale the database server? I thought about moving mysql to separate instances and do master-master duplication. However this seems tough and I heard I might lose ACID properties even with InnoDB? Is Amazon RDS a good solution?
The web server is relatively stateless except for some custom log files and the ajax cache files. What's a good way to scale to multiple web servers? I guess the custom log files can be moved to a reliable shared file system or DB but not sure what to do about the AJAX cache file coherency across multiple servers. (I dont care about losing /var/log/* if web server dies)
For performance it might be cheaper to go with larger instance with more cores and memory but eventually I would need redundancy so wondering what's the best way to do this cheaply.
thanks
take a look at this post. there is plenty of presentations on the net discussing scalability. few things i suggest to keep in mind:
plan early for the data sharding [even if you are not going to do it immediately]
try using mechanisms like memcached to limit number of queries sent to the database
prepare to serve static content from other domain, in the longer run - from ngin-x-alike server and later CDN
redundancy - depends on your needs. is 'read-only' mode acceptable for your site? if so - go with mysql replication + rsync of static files and in case of failover have your site work in that mode till you recover the master node. if you need high availability - then take a look either at drbd replication [at least for mysql] or setup with automated promotion of slave server to become master node.
you might find following interesting:
http://yoshinorimatsunobu.blogspot.com/2011/08/mysql-mha-support-for-multi-master.html
http://mysqlperformanceblog.com
http://highscalability.com
http://google.com - search for scalability, lamp, failover... there are tones of case studies and horror stories from the trench lines :-]
Another option is using a scaleable platform such as Amazon Web Services. You can start out with a micro instance and configure load balancing to fire up more instances as needed.
Once you determine average resource requirements you can then resize your image to larger or smaller depending on your needs.
http://aws.amazon.com
http://tuts.pinehead.tv/2011/06/26/creating-an-amazon-ec2-instance-with-linux-lamp-stack/
http://tuts.pinehead.tv/2011/09/11/how-to-use-amazon-rds-relation-database-service-to-host-mysql/
Amazon allows you to either load balance or change instance size based off demand.
I have developed a social networking site using the elgg framework and I am hosting it on amazon cloud (Amazon EC2, the free tier micro instance service) and thus develop a benchmark for it.I am creating around 200 columns describing each user (most of them dummy) and after that I should create around a million users with each users profile updated with some data.This is done to reflect the image of big data.When hosted on cloud we should measure the cloud's performance based on a query and an update action for all users. The problem is how to create so many users? Which tool would be optimal to choose? Done this, I should also consider storage on a file system(HDFS) and do the same with some modifications (The output should be a row and the input should be an unstructured data).
For elgg framework we are using mysql as backend. I have no idea how to start with it. Any suggestions would be really helpful.
Thank you.
I had to perform a similar task recently and came up with this script.. maybe it can help you
I have a client asking this for a requirement and haven't done this before, what does he mean by web service infrastructure?
That phrase encompasses a wide variety of technical aspects. Your infrastructure is all of the components that make up the systems that run a web business or application, including hardware. So it refers to your server and network setup, your bandwidth and connections in and out, your database setup, backup solutions, web server software, code deployment methods, and anything else used to successfully run a web business with high reliability and uptime and low error and bug incidents.
In order to make such a thing scalable, you have to architect all these components together into something that will work smoothly with growth over time. A scalable architecture should be flexible enough to handle sudden traffic spikes.
Methods used to facilitate scalability include replicated databases, clustered web servers, load balancers, RAID disk striping, and network switching. Your code has to take much of this into account.
It's a tough service to provide.
First thing that comes to mind was the Enterprise service bus.
He probably means some sort of "infrastructure" to run a lot of complex interacting web services.
Either an enterprise application that you call via a web service that can run on many instances of a web application server, or run a single instance that are very nicely multithreaded and scale to many CPUs, or deploying loads of different webservices that all talk to each other, often via message queues, until you have something that breaks all the time and requires a huge team of people to maintain. Might as well throw in a load of virtual machines to have a virtualised, scalable, re-deployable web service infrastructure (i.e., loads of tomcats or jbosses in linux VMs ready to deply as needed, one app per VM).
Then there is physical scalability. Is there enough CPU power for your needs? Is there enough bandwidth between physical nodes to send all these messages and SOAP transactions between machines? Is there enough storage? Is the storage available on a fast, low latency interconnect? Is the database nicely fed with CPU power, bandwidth, a disc system that doesn't lag. Is there a database backup. How about when a single machine can't handle the load of a particular function - then you need load balancers, although these are good for redundancy and software updates whilst remaining live as well.
Is there a site backup? Or are you scaling globally - will there be multiple data centres around the globe? Do you have redundant links to the internet from each data centre? What happens when a site goes down? How is data replicated between sites, to reduce inter-site communications, and how do these data caches and pushes work?
And so on and so forth. But your client probably just wants a web service that can be load balanced without thrashing (i.e., two or more instances can share data/sessions/etc, depends on the application really), with easy database configuration and backup. Ease of deployment is desirable, so make the install simple. Or even provide a Linux VM for them to add to their VM infrastructure. Talk to their sysadmin to see what they currently do.
This phrase is often used as a marketing term from companies who sell some part of what they'll call a "scalable web services infrastructure".
Try to find out from the client exactly what they need. Do they have existing web services? Do they have existing business logic they've decided to expose as web services? Do they have customers who are asking to be able to access your client's systems through web services?
Does your client even know what a web service is?