Can someone intercept it like man in the middle or something like that?
What's the drill when you want to backup your database with all the user info onto your hard drive?
Should I encrypt it? My phpMyAdmin runs under SSL so I guess connection is encrypted.
I assume that you are interested in moving your production database into your local PC for development purposes, and that you are concerned about protecting your end users from prying eyes. You highlight a number of valid concerns about how the data might be intercepted, but there is something that you seem to not realize: Exporting your production database to your developer machine is in-and-of-itself a breach of your application's security, even if nobody else knows about it. Your users expect that their data remain hidden from everyone, including you as a (presumably well-intentioned) developer! Using their personal data for your own purposes--no matter how honorable you may think your purposes are--is a violation of those expectations. With the exception of creating backups solely for the purpose of recovering from catastrophic data-loss events (failing hard drives in production, botched patches, and whatnot), you should never be sending raw dumps of your production data anywhere. (And sure, when you do create those backups, you should probably encrypt them.)
Getting back to the assumption that you want to protect your end-users' data, your best bet against malicious entities is to implement data masking on sensitive data. This is where you export the relational contents of your schemata with data that won't compromise the identities or intent of your actual, real-world users. Essentially, you replace names and e-mail addresses with spoof identities, and anything else that could be classified as "confidential" (which you will need to determine on your own) is similarly redacted and/or replaced with fake data.
The advantages to data-masking should be immediately apparent. Even if (god forbid!) someone were to intercept your backup and try to deduce information about your users from it, all they would end up with is a set of fictional data, which cannot be used to infer anything about your actual users. Of course, if they did intercept your dump somehow, they could easily reverse-engineer your schema, and that would be a trouble in-and-of-itself, but at least they won't immediately have access to the private details of your userbase. That said, there are a number of reasonably secure ways to transmit data these days, such as FTP over SSL (aka FTPS). (Note that FTPS is not the same as SFTP!)
Related
I am trying to improve the security a a MySQL database that will contain sensitive data. I am struggling to get to grips with some terminology. Can somebody let me know if I have understood the situation correctly:
Encryption at rest - it seems like I can enable this on a table level. All data in the table in encrypted using a key. If somebody got hold of a backup file or gained physical access to the server then the data would be protected. This assumes, of course, that the key is stored elsewhere.
AES_ENCRYPT - when inserting/updating data into my table I can use AES_ENCRYPT('data', 'password'). When querying the data via a SELECT I use AES_DECRYPT
Assuming I was just using encryption at rest then do I need to do anything different in my PHP code to query the data? Does my PHP code need to send the key to the database via my PDO request? Or can I use my normal code for querying the database and the decryption is handled automatically?
Or have I misunderstood what encryption at rest does and I need to use AES_ENCRYPT instead/as well
Encryption at rest
Encryption at rest is the data in the database when it is not being used/accessed or updated. Encryption on the move is things like TLS where the data (from the database) is transported from server to server to browser, to server, to browser, etc. TLS is perfectly good in most situations if it's handled carefully and approached with an attitude that you need to do more than the bare minimum to actually make it realisitically secure.
A typical example is people put on a TLS certificate from LetsEncrypt on their domain and think that suddenly all their stuff is safe; but they don't encrypt their sessions or their cookies so leaving a massive potential hole in their defences.
Do not use MySQL's built in encryption system.
I can not stress this enough; the built in encryption system in MySQL is not suitable for actual secure data protection.
Please read my answer to a very similar question here as to the details (I don't want to simply copy/paste).
Ok, then, because you insist.... here:
I have always understood NOT TO USE MySQL's built in encryption fuctionality because the point of encryption of data at rest (in the SQL) is that if the server is compromised, the data is not at [as much] risk.
The problem with the MySQL built in functionality is that it doesn't apply to when the data is passed to and from the "at rest" state, so the plaintext of any data can be recorded in MySQL logs (and elsewhere on the storage system, such as query lookups are not encrypted so you can from numerous lookups and their count results deduce column values) before/as it is encrypted. You can read more about this here.
Regarding encryption, you should use some tried and tested library like defuse/php-encryption.
From what I've read in my own research on this topic, the link provided by Magnus to defuse/php-encryption is one of the best ways of preventing MySQL ever causing you to compromise your data, by never letting the MySQL program/server ever see the plaintext value of your data.
-- Answer as posted May 7th 2017.
Also Bill Karwin's answer to the same question gives some valuable additional insights:
+1 to Martin's answer, but I'll add some info for what it's worth.
MySQL 5.7 has implemented encryption at rest for InnoDB tablespaces (https://dev.mysql.com/doc/refman/5.7/en/innodb-tablespace-encryption.html).
MySQL 8.0 will reportedly also implement encryption at rest for InnoDB redo log and undo log files (https://dev.mysql.com/doc/refman/8.0/en/innodb-tablespace-encryption.html).
This still leaves unencrypted the query logs and the binary log. We'll have to wait for some future version of MySQL for that.
Why does it take so long? The head of the security engineering for MySQL said at a bird-of-feather session at the Percona Live conference last month [April 2017] that they are being very careful to implement encryption right. This means implementing features for encryption, but also key security and key rotation, and other usage. It's very complex to get this right, and they don't want to implement something that will become deprecated and make everyone's encrypted databases invalid.
-- Answer as posted May 7th 2017.
Closing Point:
Security is complex. If you want to do it properly and have a confidence in your protective onion skins then you need to do a lot of things (see bullets below); but the first thing you need to do is:
Define Who you are protecting against
Seriously. You need different strategies against someone who wants to steal your plaintext names and addresses versus someone who wants to take over your server versus someone who simply wants to trash the data just because. It is a myth that you can protect against everyone all of the time, by concept this is impossible*; so you need to define the most likely agressors and then work out how best to mitigate their advances.
Sepcifically to MySQL, some clear recommendations:
Keep the SQL and the PHP on the same server. Do not remote access to the MySQL data.
Exclude external access to the SQL (so it's localhost only)
Obfuscate your table names and column names; if someone break into your data and you have HDTBJ^BTUETHNUYT under the column username then they know that this garble is probably a username so they have a very good start in trying to break your encryption.
IMPORTANT: Really lock down your table access; set up lots of MySQL users, each with only the bare minimum privilieges to do what they need; you want a user to read the table (only) and only read certain tables; users to write to certain tables but have no access to other tables. It's seperation of concern so that if any one user on the MySQL is compromised; you've not automatically lost every piece of data in there.
Use PHP encrpytion services . Store Encryption keys in a completely separate place; for example have another server you use solely for backup that you can access solely for reaching out to grab the encryption keys, therefore if your PHP/MySQL server is compromised you have some room to cut off and lock down the Key server so thay you can limit the damage. If the key server also has backups then really you're not too badly compromised (situation dependant).
Set up lots of watchers and email informers to tell you exactly when certain processes are running and which server users (not people but programs) are doing what. So you can see why an unexpected process starts to run at 5am to try and measure the size of the MySQL tables. WTF?
There is a lot of potential to have your MySQL AES_ENCRYPT'ed data "sniffed" even if it is not at rest in the DB, but if the website gets compromised (or worse, the PHP code is insecure) then timing attacks can work out data contents by timing query lookups and data packet returns.
Security is a black hole; at some point or another you're going to think "Sod this, I've done enough". No one ever has total security, some very dedicated organisations have enough security. You need to work out how far you're willing to walk before you've gone the distance.
* Why impossible? Because to protect your data from all threats, all of the time, it would need to be unreadable, unusable, like a hash. A hash is protected from everyone, all of the time. But a hash can never be un-hashed.
I'm already using salted hashing to store passwords in my database, which means that I should be immune to rainbow table attacks.
I had a thought, though: what if someone does get hold of my database? It contains the users' email addresses. I can't really hash these, because I'll be using them to send notification emails, etc..
Should I encrypt them?
Bruce Schneier has a good response to this kind of problem.
Cryptography is not the solution to your security problems. It might be part of the solution, or it might be part of the problem. In many situations, cryptography starts out by making the problem worse, and it isn't at all clear that using cryptography is an improvement.
Essentially encrypting your emails in the database 'just in case' is not really making the database more secure. Where are the keys stored for the database? What file permissions are used for these keys? Is the database accesable publically? Why? What kind of account restrictions are in place for these accounts? Where is the machine stored, who has physical access to this box? What about remote login/ssh access etc. etc. etc.
So I guess you can encrypt the emails if you want, but if that is the extent of the security of the system then it really isn't doing much, and would actually make the job of maintaining the database harder.
Of course this could be part of an extensive security policy for your system - if so then great!
I'm not saying that it is a bad idea - But why have a lock on the door from Deadlocks'R'us which cost $5000 when they can cut through the plywood around the door? Or come in through the window which you left open? Or even worse they find the key which was left under the doormat. Security of a system is only as good as the weakest link. If they have root access then they can pretty much do what they want.
Steve Morgan makes a good point that even if they cannot understand the email addresses, they can still do a lot of harm (which could be mitigated if they only had SELECT access)
Its also important to know what your reasons are for storing the email address at all. I might have gone a bit overboard with this answer, but my point is do you really need to store an email address for an account? The most secure data is data that doesn't exist.
I realize this is a dead topic, but I agree with Arjan's logic behind this. There are a few things I'd like to point out:
Someone can retrieve data from your database without retrieving your source code (i.e. SQL injection, third-party db's). With this in mind, it's reasonable to consider using an encryption with a key. Albeit, this is only an added measure of security, not the security...this is for someone who wants to keep the email more private than plaintext,
In the off chance something is overlooked during an update, or an attacker manages to retrieve the emails.
IMO: If you plan on encrypting an email, store a salted hash of it as well. Then you can use the hash for validation, and spare the overhead of constantly using encryption to find a massive string of data. Then have a separate private function to retrieve and decrypt your emails when you need to use one.
In common with most security requirements, you need to understand the level of threat.
What damage can be done if the email addresses are compromised?
What's the chance of it happening?
The damage done if email addresses are REPLACED could be much greater than if they're EXPOSED. Especially if you're, for example, using the email address to verify password resets to a secure system.
The chance of the passwords being either replaced or exposed is much reduced if you hash them, but it depends what other controls you have in place.
I would say it depends on the application of your database.
The biggest problem is, where do you store the encryption key? Because if the hacker has excess to anything more than your DB, all your efforts are probably wasted. (Remember, your application will need that encryption key to decrypt and encrypt so eventually the hacker will find the encryption key and used encryption scheme).
Pro:
A leak of your DB only will not expose the e-mail addresses.
Cons:
Encryption means performance loss.
Allot of database actions will be harder if not impossible.
Don't accidentally conflate encryption with obfuscation. We commonly obfuscate emails to prevent spam. Lots of web sites will have "webmaster _at_ mysite.com" to slow down crawlers from parsing the email address as a potential spam target. That should be done in the HTML templates -- there's no value to doing this in persistent database storage.
We don't encrypt anything unless we need to keep it secret during transmission. When and where will your data being transmitted?
The SQL statements are transmitted from client to server; is that on the same box or over a secure connection?
If your server is compromised, you have an unintentional transmission. If you're worried about this, then you should perhaps be securing your server. You have external threats as well as internal threats. Are ALL users (external and internal) properly authenticated and authorized?
During backups you have an intentional transmission to backup media; is this done using a secure backup strategy that encrypts as it goes?
Both SQL Server and Oracle (and I believe also others DBs) support encryption of data at the database level. If you want to encrypt something why don't simply abstract the access to the data that could be encrypted on the database server side and left the user choose if use the encrypted data (in this case the SQL command will be different) or not. If the user want to user encrypted data then it can configure the database server and all the maintenance work connected with key management is made using standard DBA tool, made from the DB vendor and not from you.
A copy of my answer at What is the best and safest way to store user email addresses in the database?, just for the sake of the search...
In general I agree with others saying it's not worth the effort. However, I disagree that anyone who can access your database can probably also get your keys. That's certainly not true for SQL Injection, and may not be true for backup copies that are somehow lost or forgotten about. And I feel an email address is a personal detail, so I wouldn't care about spam but about the personal consequences when the addresses are revealed.
Of course, when you're afraid of SQL Injection then you should make sure such injection is prohibited. And backup copies should be encrypted themselves.
Still, for some online communities the members might definitely not want others to know that they are a member (like related to mental healthcare, financial help, medical and sexual advice, adult entertainment, politics, ...). In those cases, storing as few personal details as possible and encrypting those that are required (note that database-level encryption does not prevent the details from showing using SQL Injection), might not be such a bad idea. Again: treat an email address as such personal detail.
For many sites the above is probably not the case, and you should focus on prohibiting SELECT * FROM through SQL Injection, and making sure visitors cannot somehow get to someone else's personal profile or order information by changing the URL.
It's worth to encrypt data in Databases, it's not making it a bit more difficult but way more difficult when its encrypted in the right way so stop philosophy and encrypt the sensitive data ;)
#Roo
I somewhat agree to what you are saying but isn't it worth encrypting the data just to make it a little more difficult for someone to get it?
With your reasoning, it would be useless to have locks or alarms in your house, because they can also easily be compromised.
My reply:
I would say that if you have sensitive data that you don't want to fall in the wrong hands, you should probably do it as hard as you can for a hacker to get it, even if it's not 100% fool proof.
I miss one important answer here.
When you have European users, you have to comply with the GDPR rules.
Email addresses are considered personal data meaning Art.5 does apply on email addresses.
processed in a manner that ensures appropriate security of the
personal data, including protection against unauthorised or unlawful
processing and against accidental loss, destruction or damage, using
appropriate technical or organisational measures (‘integrity and
confidentiality’).
Of course this does not say that you must encrypt email addresses. But by encrypting the data you do protect it from snooping employees. And protect yourself as a developer from requests to make a manual change in the database.
You really have to weigh your worst case senario of someone obtaining those email addresses, the likelihood of someone obtaining them, and your extra effort/time needed to impliement the change.
There are many questions on StackOverflow about simple, database-less login systems. I was about to suggest a salted hash approach on a recent one, when I thought: "does it really make sense to do that?".
I have been storing salted hashes on databases for years, and I understand why it's more secure: if the database is compromised, the information it contains won't allow anyone to log into my system (unlike if I were storing plain text passwords in the db).
But in a setup that does not involve a database, does hashing+salting offer any security benefits? The only reason I can think of is, if an attacker gains read-only access to my server-side code, it won't be possible to figure out any passwords. Is this a likely scenario? Because as soon as the attacker gains write access to the files, he can do anything.
So my question is: when setting up very simple, database-less login systems, should passwords be salted/hashed, or just stored as plain-text?
Yes, it still provides a benefit to hash and salt them. If the script's sourcecode is leaked people could otherwise simply use the hardcoded password or google for the hash and possibly find the input value. With a salted hash neither is possible.
I think the question is answered for you if you can figure out the answer to, "is my source code significantly less likely to be read by an attacker, than is a database?".
I would suggest that it is not -- perhaps your source is somewhat less likely to leak, depending how things are backed up etc. Even so I doubt that it's so much less likely to leak that you can neglect the risk, given that you do not neglect the same risk for databases. The reason that passwords in database should be salted/hashed isn't that there's some special property of databases that means attackers can view their contents[*], it's that attackers can get a look at all kinds of things, one way or another.
In fact source code might even be more likely to leak than a database, given that anyone working on the system might need access to the source, whereas not everyone working on a system necessarily needs access to the contents of the live DB. Not that I think your developers are dishonest (if they are, you have worse problems than the password leaking), just that the logistics around sharing source might introduce more (or just different) ways it can accidentally leak, than the logistics around backing up a DB.
Personally, in your situation I would create a small file on the server containing the hashed/salted password and approximately nothing else. Users installing different instances of the app can generate their own versions of this file, containing their own password, separate from the actual application code. They should lock it down with the same write-access restrictions as they do the source code.
Whether you call this file "a read-only database" or "part of the server code" doesn't affect how easy it is for an attacker to view it, although it might affect whether you refer to the password as "hard-coded".
[*] of course there are potential flaws that are special to particular databases, SQL injection attacks or whatever. Those are not the decisive reason why passwords in databases should be salted and hashed.
Well, as Steve Jessop already outlined. Source code can leak or is more likely to get in some hands. If you hardcode a password (what I understnand) then why not store it as a datastructure of two parts => the salt used and the hashed password. You know it but it never appears in source code. This is also what people do with DB connection strings or similar. Encrypt it with the key lying in the variable beneath it. Thus it never appears right in source code. maybe not even in memory dump unless just crossed.
I'm creating a service that will gather user data such as username, name, email, login password. How do I securely store this data? One thing I was thinking is store it encrypted in the DB so that if anyone gets access to the DB they won't be able to use the data.
But that arises two issues - #1 - much much slower search of the DB and #2 the encryption key will still be stored in the Perl script that will encrypt the data submitted by the user so if someone gets access to the whole server they'll still be able to decrypt all data.
So I was wondering what do services such as Twitter, Facebook, Hotmail, etc. use to securely store data?
BTW: as I said I'm working with Perl/MySQL.
Thank you all nice people!
You should try and avoid encrypting everything in the database. At the very least you will have to figure out which fields are being used for searching, relations, sorting etc and leave those unencrypted. You have to consider layers of encryption based on who you are hiding the data from. It would also help if you are clear about the differences between one way hashing (MD5, SHA), symmetric encryption and asymmetric encryption.
Encrypting passwords in the database makes sense because it hides the passwords from the dba or anyone with the ability to query the db directly. Actually, passwords are not encrypted, they are hashed. This way you can never recover a password, you can only generate a new one.
Data that needs to be recovered will be encrypted, so that it can be decrypted. There are various algorithms for doing that and they have different attributes of strength and cost (in terms of processing time and memory). You should choose one which suits your purpose.
The thing here is that encryption and decryption need to be done in the application and not in the database. The reason for this is that at some point there might be an open network between the database and the application servers and any data transmitted between the application and the database could be sniffed.
In deciding between symmetric and asymmetric encryption, there are two factors. One is for how long does this information need to be kept secure. If it is a bid on an auction that is going to end in 2 days, you may use symmetric encryption, because it ideally will not matter if people figure out things after that. Credit card numbers typically do not have to be kept safe. If you are storing the expiry date and the ccv numbers of the card then those have to be kept safe, typically for years. In that case you have to use asymmetric encryption.
In encrypting the whole file system it is not the performance degradation that you have worry about mainly. As far as I have seen, it requires a person to key in the correct password when OS is booting, and that requires physical access, and persons who can be trusted to know the password staying as close to the servers as the SLA requires. In this it is like setting a bios password or a grub password. If you indeed encrypt your file system, make sure to verify this or find a way around it.
Others have pointed out that encryption will incur a slowdown. How much will depend on lots of different factors, so you'll need to benchmark your code.
But in reality the most important thing to protect is probably the password since if that is compromised then the users' accounts on other services could be compromised as well since people tend to reuse the same username/email/password combos.
Fortunately passwords are easy to secure. Use a 1-way hash like SHA1 with a salt (to protect against rainbox tables) and never store the actual password in your DB. Store the salted-hash. Then when the user logs in you can check the pw they give you against the hashed one to make sure it matches without ever having to store what their pw really is.
See my answer here. The gist of it is that you can add security, but there is no 100% foolproof way to avoid a compromise of (some of) your user information if your web application is completely broken.
Encrypting the filesystem on which the database is stored minimizes many other types of vulnerabilities. Encrypting data in the database protects you against those and also against a user breaking your DB password. Encrypting data in the database with a hardware token provides the best protection but is inconvenient.
You have a few options:
You can encrypt the data in the middle tier
You can encrypt the database
You will encounter slower searches either way. In both cases, the encryption key is not included in the backup file set.
there's this interesting problem i can not solve myself. I will be very glad, if you help me.
Here's it:
there are many client applications that send data records to one MySQL server.
Few data records are not very important, but the whole database is. (You can imagine it is facebook DB :) )
Is there any way to ensure that
data from DB won't be used by anyone but true owner
DB will preserve essential features such as sorting etc.
assuming that attacker can mysteriously gain full access to server?
You can't simply encrypt data client-side and store it encrypted, since client application is wide-spread and attacker can get key from it.
Maybe adding some layers between application and DB, or combining encryption methods client- and server-side (using mysql built-in methods) will help?
As long as the database needs to start up and run unattended you can't hide the keys from a compromised root account (= 'mysterious full access'). Anywhere the database could possibly store the master key(s), the root will also have access. No amount of business layers or combination of client-server encryption will ever circumvent this simple fact. You can obfuscate it till the day after but if the prize is worth then root can get it.
One alternative is to require a manually assisted start up process, ie. a human enters the master key password during the server boot (or hardware module PIN), but this is extremely hard to maintain in real world, it requires a highly trusted employee to be on pager call to log in and start the database whenever there is downtime.
Solutions like TPM offer protection against physical loss of the server, but not against a compromised root.
Your root is as important as the database master key(s), so you must protect your root with the same care as the keys. This means setting up operating procedures, screening who has access to root, rotating the root password and so on and so forth. The moment someone gains 'mysteriously full access' the game is pretty much lost.
I pretty much agree with Remus Rusanu's answer.
Maintaining good security is hard, but you can always pay attention to what you do. When ever you access sensitive information carefully verify your query and make sure it cannot be spoofed or exploited to gain access to information which shouldn't be accessible by given client.
If you can roll out physical access to the box by the attacker then there are several things you can do to harden your security. First of all I'd configure ssh access only to only allow connections from specific IP or IP range (and of course no root access). You can also do that that on your firewall. This would mean that the weakest link is your server (the application which receives data/requests from clients, could be web-server and whatever scripts you use). Now you "just" have to make sure that no one can exploit your server. There are a lot more things you could do to harden your system, but it think it would be more appropriate to ask on ServerFault.
If you're worried about physical access to the PC, there isn't really much you can do and most stuff has already been mentioned in Remus answer.
There's also another option. This is by far the most ineffective method from speed and ease to develop viewpoint, but it would partly protect you from any kind of an attack on your server (including physical). It's actually quite simple, but a bit hard to implement - only store the encrypted data in the database and handle all encryption/decryption client-side using javascript or flash. Only the client will have the key and data will always be transfered over the wire and stored in encrypted format. The biggest drawback is that once client forgets the key there's no way back, the data is inaccessible.
Of course it's all matter of time, money and effort - with enough of these anything can be broken.
I've no idea if such a thing exists in MySql, but row-level-versioning in Oracle enables you to define access rights on row-level IN the database: so that means, regardless of what tool is being used to access the data, the user only ever sees the same selection as determined by his/her credentials.
So if my username/role is only allowed to see data limited by some WHERE clause, that can appended to each and every SELECT that appears in the database, regardless of whether it comes from a web app, a SQL querying tool, or whatever.
I will use a 2nd layer and a firwall between them.
so you have firewall ---- web server --- firewall -- 2nd layer server --- firewll --- db
it will be wise to use different platfroms between layers, it all depends how important is the data.
anyway - the web server should have no access to DB.
about preserving sort - if you use a file encrypotion mechisim - it will only protect you from Hard drive theaft.
if you encrypt the data it self, and if you do it smartly (storing the keys in a separate place) you will not loose sorting as you will look for the encryoted entry and not the real one- but now you have another thing to protect....