I'm already using salted hashing to store passwords in my database, which means that I should be immune to rainbow table attacks.
I had a thought, though: what if someone does get hold of my database? It contains the users' email addresses. I can't really hash these, because I'll be using them to send notification emails, etc..
Should I encrypt them?
Bruce Schneier has a good response to this kind of problem.
Cryptography is not the solution to your security problems. It might be part of the solution, or it might be part of the problem. In many situations, cryptography starts out by making the problem worse, and it isn't at all clear that using cryptography is an improvement.
Essentially encrypting your emails in the database 'just in case' is not really making the database more secure. Where are the keys stored for the database? What file permissions are used for these keys? Is the database accesable publically? Why? What kind of account restrictions are in place for these accounts? Where is the machine stored, who has physical access to this box? What about remote login/ssh access etc. etc. etc.
So I guess you can encrypt the emails if you want, but if that is the extent of the security of the system then it really isn't doing much, and would actually make the job of maintaining the database harder.
Of course this could be part of an extensive security policy for your system - if so then great!
I'm not saying that it is a bad idea - But why have a lock on the door from Deadlocks'R'us which cost $5000 when they can cut through the plywood around the door? Or come in through the window which you left open? Or even worse they find the key which was left under the doormat. Security of a system is only as good as the weakest link. If they have root access then they can pretty much do what they want.
Steve Morgan makes a good point that even if they cannot understand the email addresses, they can still do a lot of harm (which could be mitigated if they only had SELECT access)
Its also important to know what your reasons are for storing the email address at all. I might have gone a bit overboard with this answer, but my point is do you really need to store an email address for an account? The most secure data is data that doesn't exist.
I realize this is a dead topic, but I agree with Arjan's logic behind this. There are a few things I'd like to point out:
Someone can retrieve data from your database without retrieving your source code (i.e. SQL injection, third-party db's). With this in mind, it's reasonable to consider using an encryption with a key. Albeit, this is only an added measure of security, not the security...this is for someone who wants to keep the email more private than plaintext,
In the off chance something is overlooked during an update, or an attacker manages to retrieve the emails.
IMO: If you plan on encrypting an email, store a salted hash of it as well. Then you can use the hash for validation, and spare the overhead of constantly using encryption to find a massive string of data. Then have a separate private function to retrieve and decrypt your emails when you need to use one.
In common with most security requirements, you need to understand the level of threat.
What damage can be done if the email addresses are compromised?
What's the chance of it happening?
The damage done if email addresses are REPLACED could be much greater than if they're EXPOSED. Especially if you're, for example, using the email address to verify password resets to a secure system.
The chance of the passwords being either replaced or exposed is much reduced if you hash them, but it depends what other controls you have in place.
I would say it depends on the application of your database.
The biggest problem is, where do you store the encryption key? Because if the hacker has excess to anything more than your DB, all your efforts are probably wasted. (Remember, your application will need that encryption key to decrypt and encrypt so eventually the hacker will find the encryption key and used encryption scheme).
Pro:
A leak of your DB only will not expose the e-mail addresses.
Cons:
Encryption means performance loss.
Allot of database actions will be harder if not impossible.
Don't accidentally conflate encryption with obfuscation. We commonly obfuscate emails to prevent spam. Lots of web sites will have "webmaster _at_ mysite.com" to slow down crawlers from parsing the email address as a potential spam target. That should be done in the HTML templates -- there's no value to doing this in persistent database storage.
We don't encrypt anything unless we need to keep it secret during transmission. When and where will your data being transmitted?
The SQL statements are transmitted from client to server; is that on the same box or over a secure connection?
If your server is compromised, you have an unintentional transmission. If you're worried about this, then you should perhaps be securing your server. You have external threats as well as internal threats. Are ALL users (external and internal) properly authenticated and authorized?
During backups you have an intentional transmission to backup media; is this done using a secure backup strategy that encrypts as it goes?
Both SQL Server and Oracle (and I believe also others DBs) support encryption of data at the database level. If you want to encrypt something why don't simply abstract the access to the data that could be encrypted on the database server side and left the user choose if use the encrypted data (in this case the SQL command will be different) or not. If the user want to user encrypted data then it can configure the database server and all the maintenance work connected with key management is made using standard DBA tool, made from the DB vendor and not from you.
A copy of my answer at What is the best and safest way to store user email addresses in the database?, just for the sake of the search...
In general I agree with others saying it's not worth the effort. However, I disagree that anyone who can access your database can probably also get your keys. That's certainly not true for SQL Injection, and may not be true for backup copies that are somehow lost or forgotten about. And I feel an email address is a personal detail, so I wouldn't care about spam but about the personal consequences when the addresses are revealed.
Of course, when you're afraid of SQL Injection then you should make sure such injection is prohibited. And backup copies should be encrypted themselves.
Still, for some online communities the members might definitely not want others to know that they are a member (like related to mental healthcare, financial help, medical and sexual advice, adult entertainment, politics, ...). In those cases, storing as few personal details as possible and encrypting those that are required (note that database-level encryption does not prevent the details from showing using SQL Injection), might not be such a bad idea. Again: treat an email address as such personal detail.
For many sites the above is probably not the case, and you should focus on prohibiting SELECT * FROM through SQL Injection, and making sure visitors cannot somehow get to someone else's personal profile or order information by changing the URL.
It's worth to encrypt data in Databases, it's not making it a bit more difficult but way more difficult when its encrypted in the right way so stop philosophy and encrypt the sensitive data ;)
#Roo
I somewhat agree to what you are saying but isn't it worth encrypting the data just to make it a little more difficult for someone to get it?
With your reasoning, it would be useless to have locks or alarms in your house, because they can also easily be compromised.
My reply:
I would say that if you have sensitive data that you don't want to fall in the wrong hands, you should probably do it as hard as you can for a hacker to get it, even if it's not 100% fool proof.
I miss one important answer here.
When you have European users, you have to comply with the GDPR rules.
Email addresses are considered personal data meaning Art.5 does apply on email addresses.
processed in a manner that ensures appropriate security of the
personal data, including protection against unauthorised or unlawful
processing and against accidental loss, destruction or damage, using
appropriate technical or organisational measures (‘integrity and
confidentiality’).
Of course this does not say that you must encrypt email addresses. But by encrypting the data you do protect it from snooping employees. And protect yourself as a developer from requests to make a manual change in the database.
You really have to weigh your worst case senario of someone obtaining those email addresses, the likelihood of someone obtaining them, and your extra effort/time needed to impliement the change.
Related
I'm building a website, where users are going to store sensitive data. People who have access to the DB should not be able to view this data, but I can't use hashing functions, as the users will need to view the data they have stored. How should I go about this?
TL;DR: Encrypting columns of databases won't help much.
Best practice: figure out your threat model before you spend time and money securing your system. If you build complex security measures without a clear idea of your threat model, you'll trick yourself into a false sense of safety.
And, encrypting columns in a DBMS is a complex security measure.
What is your threat model? What attacks will you get? By whom? How will attacks damage you?
Your most likely outsider attack comes from cybercriminals breaking into your web servers to steal your users' information to commit identity theft (Equifax), blackmail (Ashley Madison), or espionage (US Government human resources database).
If you encrypt some columns in your DBMS and your web users need to be able to view and update those columns, your web servers will have to know the encryption and decryption keys. If a criminal pwns your web server, he will have your keys. Therefore he will have access to the encrypted columns of your dbms. And, he'll have a big signpost saying LOOK! Here's the secret stuff!
There are plenty of other imaginable outsider attacks, of course. Somebody could break through your firewall and hit your database directly. Somebody could get into a cache and grab cached sensitive data. Somebody could guess your web app's administrator password. Or, steal a bulk upload file.
Your proposed design imagines an insider attack. People who already have DBMS access credentials must be prevented from seeing certain columns in certain tables. What will they do with that information? You didn't say. What's the threat?
Stopping short of encryption, you can do these things to keep your insiders from violating your users' confidentiality.
Get the sensitive data out of your system entirely. For example, if you're handling credit cards, work with stripe.com or braintree.com. They'll hold your secrets for you, and they have excellent cybersecurity teams.
Sort out whether you can trust your insiders. Investigate prospective employees, etc.
Establish clear security policies. For example, "We never look at the credit_card table unless we have a specific need to do so." If you're handling health care data in the US, you already have HIPAA guidelines. Get your insiders to understand and agree to your guidelines.
Sack insiders who violate these policies intentionally.
Build mechanisms to help enforce policies. Issue each insider his or her own username/password pair to access the DBMS. Use selective GRANT operations at the table and column level to allow and disallow viewing of data. For example,
GRANT SELECT (name, address) ON person TO username#'%';
lets username see the name and address columns, but not the taxpayer_id column in the person table. Read this. https://dev.mysql.com/doc/refman/5.7/en/grant.html#grant-column-privileges
Spend your time and money on good firewalls protecting your DBMS machines. Study up on OWASP and follow those practices. Spend time and money running penetration tests of your web app and fixing the problems. Spend them on vetting and training your insiders. These things slow down attackers more effectively than the imagined magic bullet of encrypted columns.
There's the old joke about the two guys and the bear.
Bear: Roar.
Adam: Uh oh, I don't know if we can run faster than this bear.
Bill: I just have to run faster than you.
That's a good way to handle security for your small web site. Make it hard enough to crack that the bad guys will attack somebody else.
If you're running a big web site with a large number of sensitive records (I'm looking at you, Equifax) this isn't good enough.
I am making an application where there is some PII (personally identifiable information) being stored, such as the Date of Birth along with the name and a few other fields. I want to encrypt the data with AES_ENCRYPT using CBC on MySQL, however there is an administrator who needs to see the information, but if I encrypt the user's information, the administrator won't be able to see. Should I encrypt the PII with the username and have the administrator not be able to see and encrypt the username with the username, or should I leave the username unencrypted but encrypt everything else with the username, leaving some security risks, but allowing the administrator to see the information. What should I do?
Generally speaking, encrypting data at rest in DBMSs with symmetric encryption like AES is both unnecessary and insufficient to protect it from attack.
Why?
For one thing, the weak link in a system like this is typically the web server, not the MySQL server machine. But the web server must contain the necessary key to both encrypt and decrypt the data. If an attacker pwns your web server, he takes possession of your key and your algorithms to generate keys, and therefore he pwns your sensitive data. He also has a really robust way to figure out what data you think is sensitive.
It's much less likely the attacker will be able to compromise your MySQL server directly; in situations involving PII those machines ordinarily live behind firewalls.
For another thing: your administrator. He will need to have possession of the key, and administrators are juicy targets for attackers. (I'm not saying he's a black hat, just that his white hat has a big red target on it.)
Another thing: it may give you a false sense of security.
(Notice that passwords are not encrypted: they are hashed. It's simple to compare a password to a hashed password and say "match" or "no match." But it's much harder to take a hashed password and recover the original one. So the argument that passwords are "encrypted" isn't relevant to this situation.)
I seriously suggest you spend your time and money not on encrypting your PII at rest as you suggest. Instead spend your resources to make sure your web site is secure, and making sure your MySQL server is safely behind a robust firewall and invisible to the public internet.
Look at the OWASP Top Ten vulnerabilities. Spend time dealing with them on your site. Use a penetration test tool like burp scan or zap scan to try to break in.
Securing your web site is like the old joke about the two guys and the bear.
Bear: roar.
Joe: I hope I can outrun the bear.
Mike: I just have to outrun you.
In cybersecurity, you don't want to be Joe.
Can someone intercept it like man in the middle or something like that?
What's the drill when you want to backup your database with all the user info onto your hard drive?
Should I encrypt it? My phpMyAdmin runs under SSL so I guess connection is encrypted.
I assume that you are interested in moving your production database into your local PC for development purposes, and that you are concerned about protecting your end users from prying eyes. You highlight a number of valid concerns about how the data might be intercepted, but there is something that you seem to not realize: Exporting your production database to your developer machine is in-and-of-itself a breach of your application's security, even if nobody else knows about it. Your users expect that their data remain hidden from everyone, including you as a (presumably well-intentioned) developer! Using their personal data for your own purposes--no matter how honorable you may think your purposes are--is a violation of those expectations. With the exception of creating backups solely for the purpose of recovering from catastrophic data-loss events (failing hard drives in production, botched patches, and whatnot), you should never be sending raw dumps of your production data anywhere. (And sure, when you do create those backups, you should probably encrypt them.)
Getting back to the assumption that you want to protect your end-users' data, your best bet against malicious entities is to implement data masking on sensitive data. This is where you export the relational contents of your schemata with data that won't compromise the identities or intent of your actual, real-world users. Essentially, you replace names and e-mail addresses with spoof identities, and anything else that could be classified as "confidential" (which you will need to determine on your own) is similarly redacted and/or replaced with fake data.
The advantages to data-masking should be immediately apparent. Even if (god forbid!) someone were to intercept your backup and try to deduce information about your users from it, all they would end up with is a set of fictional data, which cannot be used to infer anything about your actual users. Of course, if they did intercept your dump somehow, they could easily reverse-engineer your schema, and that would be a trouble in-and-of-itself, but at least they won't immediately have access to the private details of your userbase. That said, there are a number of reasonably secure ways to transmit data these days, such as FTP over SSL (aka FTPS). (Note that FTPS is not the same as SFTP!)
I'm creating a service that will gather user data such as username, name, email, login password. How do I securely store this data? One thing I was thinking is store it encrypted in the DB so that if anyone gets access to the DB they won't be able to use the data.
But that arises two issues - #1 - much much slower search of the DB and #2 the encryption key will still be stored in the Perl script that will encrypt the data submitted by the user so if someone gets access to the whole server they'll still be able to decrypt all data.
So I was wondering what do services such as Twitter, Facebook, Hotmail, etc. use to securely store data?
BTW: as I said I'm working with Perl/MySQL.
Thank you all nice people!
You should try and avoid encrypting everything in the database. At the very least you will have to figure out which fields are being used for searching, relations, sorting etc and leave those unencrypted. You have to consider layers of encryption based on who you are hiding the data from. It would also help if you are clear about the differences between one way hashing (MD5, SHA), symmetric encryption and asymmetric encryption.
Encrypting passwords in the database makes sense because it hides the passwords from the dba or anyone with the ability to query the db directly. Actually, passwords are not encrypted, they are hashed. This way you can never recover a password, you can only generate a new one.
Data that needs to be recovered will be encrypted, so that it can be decrypted. There are various algorithms for doing that and they have different attributes of strength and cost (in terms of processing time and memory). You should choose one which suits your purpose.
The thing here is that encryption and decryption need to be done in the application and not in the database. The reason for this is that at some point there might be an open network between the database and the application servers and any data transmitted between the application and the database could be sniffed.
In deciding between symmetric and asymmetric encryption, there are two factors. One is for how long does this information need to be kept secure. If it is a bid on an auction that is going to end in 2 days, you may use symmetric encryption, because it ideally will not matter if people figure out things after that. Credit card numbers typically do not have to be kept safe. If you are storing the expiry date and the ccv numbers of the card then those have to be kept safe, typically for years. In that case you have to use asymmetric encryption.
In encrypting the whole file system it is not the performance degradation that you have worry about mainly. As far as I have seen, it requires a person to key in the correct password when OS is booting, and that requires physical access, and persons who can be trusted to know the password staying as close to the servers as the SLA requires. In this it is like setting a bios password or a grub password. If you indeed encrypt your file system, make sure to verify this or find a way around it.
Others have pointed out that encryption will incur a slowdown. How much will depend on lots of different factors, so you'll need to benchmark your code.
But in reality the most important thing to protect is probably the password since if that is compromised then the users' accounts on other services could be compromised as well since people tend to reuse the same username/email/password combos.
Fortunately passwords are easy to secure. Use a 1-way hash like SHA1 with a salt (to protect against rainbox tables) and never store the actual password in your DB. Store the salted-hash. Then when the user logs in you can check the pw they give you against the hashed one to make sure it matches without ever having to store what their pw really is.
See my answer here. The gist of it is that you can add security, but there is no 100% foolproof way to avoid a compromise of (some of) your user information if your web application is completely broken.
Encrypting the filesystem on which the database is stored minimizes many other types of vulnerabilities. Encrypting data in the database protects you against those and also against a user breaking your DB password. Encrypting data in the database with a hardware token provides the best protection but is inconvenient.
You have a few options:
You can encrypt the data in the middle tier
You can encrypt the database
You will encounter slower searches either way. In both cases, the encryption key is not included in the backup file set.
there's this interesting problem i can not solve myself. I will be very glad, if you help me.
Here's it:
there are many client applications that send data records to one MySQL server.
Few data records are not very important, but the whole database is. (You can imagine it is facebook DB :) )
Is there any way to ensure that
data from DB won't be used by anyone but true owner
DB will preserve essential features such as sorting etc.
assuming that attacker can mysteriously gain full access to server?
You can't simply encrypt data client-side and store it encrypted, since client application is wide-spread and attacker can get key from it.
Maybe adding some layers between application and DB, or combining encryption methods client- and server-side (using mysql built-in methods) will help?
As long as the database needs to start up and run unattended you can't hide the keys from a compromised root account (= 'mysterious full access'). Anywhere the database could possibly store the master key(s), the root will also have access. No amount of business layers or combination of client-server encryption will ever circumvent this simple fact. You can obfuscate it till the day after but if the prize is worth then root can get it.
One alternative is to require a manually assisted start up process, ie. a human enters the master key password during the server boot (or hardware module PIN), but this is extremely hard to maintain in real world, it requires a highly trusted employee to be on pager call to log in and start the database whenever there is downtime.
Solutions like TPM offer protection against physical loss of the server, but not against a compromised root.
Your root is as important as the database master key(s), so you must protect your root with the same care as the keys. This means setting up operating procedures, screening who has access to root, rotating the root password and so on and so forth. The moment someone gains 'mysteriously full access' the game is pretty much lost.
I pretty much agree with Remus Rusanu's answer.
Maintaining good security is hard, but you can always pay attention to what you do. When ever you access sensitive information carefully verify your query and make sure it cannot be spoofed or exploited to gain access to information which shouldn't be accessible by given client.
If you can roll out physical access to the box by the attacker then there are several things you can do to harden your security. First of all I'd configure ssh access only to only allow connections from specific IP or IP range (and of course no root access). You can also do that that on your firewall. This would mean that the weakest link is your server (the application which receives data/requests from clients, could be web-server and whatever scripts you use). Now you "just" have to make sure that no one can exploit your server. There are a lot more things you could do to harden your system, but it think it would be more appropriate to ask on ServerFault.
If you're worried about physical access to the PC, there isn't really much you can do and most stuff has already been mentioned in Remus answer.
There's also another option. This is by far the most ineffective method from speed and ease to develop viewpoint, but it would partly protect you from any kind of an attack on your server (including physical). It's actually quite simple, but a bit hard to implement - only store the encrypted data in the database and handle all encryption/decryption client-side using javascript or flash. Only the client will have the key and data will always be transfered over the wire and stored in encrypted format. The biggest drawback is that once client forgets the key there's no way back, the data is inaccessible.
Of course it's all matter of time, money and effort - with enough of these anything can be broken.
I've no idea if such a thing exists in MySql, but row-level-versioning in Oracle enables you to define access rights on row-level IN the database: so that means, regardless of what tool is being used to access the data, the user only ever sees the same selection as determined by his/her credentials.
So if my username/role is only allowed to see data limited by some WHERE clause, that can appended to each and every SELECT that appears in the database, regardless of whether it comes from a web app, a SQL querying tool, or whatever.
I will use a 2nd layer and a firwall between them.
so you have firewall ---- web server --- firewall -- 2nd layer server --- firewll --- db
it will be wise to use different platfroms between layers, it all depends how important is the data.
anyway - the web server should have no access to DB.
about preserving sort - if you use a file encrypotion mechisim - it will only protect you from Hard drive theaft.
if you encrypt the data it self, and if you do it smartly (storing the keys in a separate place) you will not loose sorting as you will look for the encryoted entry and not the real one- but now you have another thing to protect....