I am in the process of moving our companies MediaWiki from a single server to a clustered environment. The existing file based session storage was fine with the single server, but clearly not for the cluster.
To address this I'm looking to use one of our existing MySQL database servers to handle session management but the only article I've come across is for a new MediaWiki installation.
I set $wgSessionHandler in LocalSettings.php but that had no effect.
Anyone have advice/experience with this?
This might not be the answer you're looking for, but I was just facing this issue myself.
After trying to Do The Right Thing™ for some hours, I finally gave in and just put the sessions on shared storage.
So, if you can afford to, performance wise, and have some shared storage available or can easily create some, I can only recommend just pointing PHP's session.save_path to shared storage and save yourself the trouble.
It's the easy way out. ;-)
Related
I identified an issue with an infrastructure I created on the Google Cloud Platform and would like to ask the community for help.
A charge was made that I did not expect, as theoretically it would be almost impossible for me to pass the limits of the free tier. But I noticed that my database is huge, with 1gb and growing, and there are some very heavy buckets.
My database is managed by a Django APP, and accessing the tool's admin panel there are only 2 records in production. There were several backups and things like that, but it wasn't me.
Can anyone give me a guide on how to solve this?
I would assume that you manage the database yourself, i.e. it is not Cloud SQL. Databases pre-allocate files in larger chunks in order to be efficient on writes. You can check this yourself - write additional 100k records, most likely size will not change.
Go to Cloud SQL and it will say how many sql instances you have. If you see the "create instance" window that means that you don't have any Google managed SQL instances and the one we're talking about is a standalone one.
You can also check Deployment Manger to see if you deployed one from the marketplace or it's a standalone VM with MySQL installed manually.
I made a simple experiment and deployed (from Marketplace) a MySQL 5.6 on top of Ubuntu 14.04 LTS. Initial DB size was over 110MB - but there are some records in it initially (schema, permissions etc) so it's not "empty" at all.
It is still weird that your DB is 1GB already. Try to deploy new one (if your use case permits that). Maybe this is some old DB that was used for some purpose and all the content deleted afterwards but some "trash" still remains and takes a lot of space.
Or - it may well be exactly as NeatNerd said - that DB will allocate some space due to performance issues and optimisation.
If I have more details I can give you better explanation.
The project is an open source IDE like system to edit code, alter images,, run code, etc. for students in school.
It contains an upload of Archive/Folders which characteristics are:
Upload Archive, contains up to 30 files (txt,js,xul,xml,php, gif/png/jpg), av.size in total 500kb
Users can edit and fork
all the files are editable through Codemirror (like used on jsfiddle)
all the Gif/Png/Jpg are replaceable (10 - 40 per archive)
we expect daily at least 1'000 new uploads/forks with an average of 20 files, total-size min. 500 mb
Our enironment:
PHP
most likely MySQL database
Linux
To consider:
We don't require Searching through the folders/files on global scope
the User saved Data is as it is, never any changes from our side necessary
State-of-Development:
Ready besides the storage question and all their dependencies
Would you advise on SQL or a simple Filesystem?
Before starting the project we were 100% sure using MySQL, but with the added feature of Image modification and a growing database (atm 80/k files,2GB) we struggle. Also reading here let us hesitate too.
Advantages of MYSQL are surely the easy maintainment and simpler future restructure of the system.
Though it will get a huge database fast to search within.
By using a global php entryfile to read the Filesystem based on URL parameters, the searching can be ommitted and straight go displaying the fetched directory with its content.
We are not experienced in managing large data and rely on experience of people who faced such a situtation already.
Rather than just vote for Database or Filesystem please consider your own tips to make this environment more efficient to run (eg indexing, structure, tables, etc..) or elaborate your decision.
very thankful for any tips of you
btw, the project will be hosted at git
I believe that the best option to go is the file system taking into consideration your requirements. The pros of this choice are
Better performance of your database since the table will only hold a link to your file system where the file is stored.
It is easier to backup your database.
You have the possibility of storing the files in different locations by defining rules. This gives you a flexibility in managing your storage.
When we faced a similar problem for storing attachments in a service desk application, we have chosen to go with the filesystem. Till now everything is working as expected.
I hope that my answer is helpful to you.
We're running MySQL 5.1 on CentOS 5 and I need to securely wipe data. Simply issuing a DELETE query isn't an option, we need to comply with DoD file deletion standards. This will be done on a live production server without taking MySQL down. Short of taking the server down and using a secure deletion utility on the DB files is there a way to do this?
Update
The data sanitization will be done once per database when we remove some of the tables. We don't need to delete data continuously. CPU time isn't an issue, these servers are nowhere near capacity.
If you need a really secure open source database, you could take a look at Security Enhanced PostgreSQL running on SELinux. A very aggresive vacuum strategy can assure your data gets overwritten quickly. Strong encryption can be of help as well, pgcrypto has some fine PGP functions.
Not as far as I know, secure deletion requires the CPU to do a bit of work, especially DoD standard which I believe is 3 passes of inflating 1's and 0's. You can, however, encrypt the harddrive. Given that a user would need phsyical access and a password for the CentOS to recover the data. As long as you routinely monitory access logs for suspicious activity on the server, this should be "secure".
While searching found this article: Six Steps to Secure Sensitive Data in MySQL
Short of that though, I do not think a DoD standard wipe is viable or even possible without taking the server down.
EDIT
One thing I found is this software: data wiper. If there is a linux comparable version of that, that might work "wipes unused disk space". But again this may take a major performance toll on your server, so may be advisable to run at night at a set time and I do not know what the re-precautions (if any) of doing this too often to a harddrive.
One other resource is this forum thread. It talks about wiping unused space etc. From that thread one resource stands out in particular: secure_deletion toolkit - sfill. The man page should be helpful.
If it's on a disk, you could just use: http://lambda-diode.com/software/wipe/
I have several databases hosted on a shared server, and a local testing server which I use for development.
I would like to keep both set of databases somewhat synchronized (more or less daily).
So far, my ideas to solve the problem seem very clumsy. Anyway, for reference, here is what I have considered so far:
Make a database dump from online databases, trash local databases, and recreate the databases from the dump. It's a lot of work and requires a lot of download time (which guarantees I won't do it as much as I would like it to be done)
Write a small web service to access the new data, and write a small application locally to communicate with said web service, download the newest data, and update the local databases.
Both solutions sound like a lot of work for a problem that is probably already solved a zillion times over. Or maybe it's even an existing feature which I completely overlooked.
Is there an easy way to keep databases more or less in synch? Ideally something that I can set up once, schedule and forget about.
I am using MySQL 5 (MyISAM) databases on both servers.
=============
Edit: I had a look at replication, but it seems that I can't go that route because the shared hosting does not give me enough control on the server itself (I got most permissions on my databases, but not on the MySQL server itself)
I only need to keep the data synchronized, nothing else. Is there any other solution that doesn't require full control on the server?
Edit 2:
Sorry, I forgot to mention I am running on a LAMP stack on the shared server, so Windows-only solutions won't work.
I am surprised to see that there is no obvious off-the-shelves solution for this problem.
Have you considered replication? It's not to be trifled with but may be what you want. See here for more details... http://dev.mysql.com/doc/refman/5.0/en/replication-configuration.html
Take a look at Microsoft Sync Framework - you will need to code in .net, but it can resolve your issues.
http://msdn.microsoft.com/en-in/sync/default(en-us).aspx
Here is a sample for SQL server, but it can be adapted to mysql as well using ado.net provider for Mysql.
http://code.msdn.microsoft.com/sync/Release/ProjectReleases.aspx?ReleaseId=4835
You will need the additional tables for change tracking and anchors (keeping track of last synchronization) for this to work, in your mysql database, but you wont need full control as long as you can access the db.
Replication would have simpler :), but this might just work in your case.
I have a website using cPanel on a dedicated account, I would like to be able to automatically sync the website to a second hosting company or perhaps to a local (in house ) server.
Basically this is a type of replication. The website is database driven (MySQL), so Ideally it would sync everything (content, database, emails etc.) , but most importantly is syncing the website files and its database.
I'm not so much looking for a fail-over solution as an automatic replication solution, so if the primary site (Server) goes off-line, I can manually bring up the replicated site quickly.
I'm familiar with tools like unison and rsync, but most of these only sync file(s) and do not do too well with open database connections.
Don't use one tool when two is better; Use rsync for files, but use replication for MySQL.
If, for some reason, you don't want to use replication, you might want to consider using DRBD. This is of course only applicable if you're running Linux. DRBD is now part of the main kernel (since version 2.6.33).
And yes - I am aware of at least one large enterprise deployment of DRBD which is used, among other things, store MySQL database files. In fact, MySQL website even has relevant page on this topic.
You might also want to Google for articles against DRBD/MySQL combination; I remember reading few posts of that.