I have read a lot of things in SO but I still cannot figure out the whole mechanism of the encryption thing...
I want my site to write data in an SQL Server. Where should the encryption function be? What does my database need?
I want a webservice of mine to get that data, process them and insert them to another SQL Server decrypted. Should I declar the same protocols there as well? How will know the encryption, keys in order to decrypt? What is the circuit like?
Sorry for being SO FADE (mods you can reject the post) but I cannot follow the logic behing the SQL encryption/decryption mechanism...
There are different points at which you need to encrypt data.
The first is "in motion". Meaning you encrypt the data while it is in transit between two machines. The common points include between the browser and web server (typically using SSL) and between the web server and database server (Kerberos is the common one for Windows machines). The purpose is to prevent someone from eavesdropping on the wire.
The second is "at rest". This means the data should be encrypted while it is in the storage facility: namely SQL server. The purpose here is to prevent the data from being decrypted should a hacker obtain a copy of the database file. The easiest mechanism by far is to utilize SQL Servers built in encryption mechanisms.
Another option for "at rest" encryption that is sometimes employed is to have the web server encrypt the data prior to sending it to the SQL server. Generally speaking this is no more effective than simply allowing the database server handle the encryption/decryption itself. However it does have several drawbacks such as making it near impossible to query the data from SQL servers tools. Further the keys have a wider distribution as every application that needs to touch the data must have access to the keys in order to work. This can be somewhat limited by using a service layer on top of SQL server to handle data management.. but, again, this doesn't provide any more security than what you have right out of the box.
Related
I am trying to improve the security a a MySQL database that will contain sensitive data. I am struggling to get to grips with some terminology. Can somebody let me know if I have understood the situation correctly:
Encryption at rest - it seems like I can enable this on a table level. All data in the table in encrypted using a key. If somebody got hold of a backup file or gained physical access to the server then the data would be protected. This assumes, of course, that the key is stored elsewhere.
AES_ENCRYPT - when inserting/updating data into my table I can use AES_ENCRYPT('data', 'password'). When querying the data via a SELECT I use AES_DECRYPT
Assuming I was just using encryption at rest then do I need to do anything different in my PHP code to query the data? Does my PHP code need to send the key to the database via my PDO request? Or can I use my normal code for querying the database and the decryption is handled automatically?
Or have I misunderstood what encryption at rest does and I need to use AES_ENCRYPT instead/as well
Encryption at rest
Encryption at rest is the data in the database when it is not being used/accessed or updated. Encryption on the move is things like TLS where the data (from the database) is transported from server to server to browser, to server, to browser, etc. TLS is perfectly good in most situations if it's handled carefully and approached with an attitude that you need to do more than the bare minimum to actually make it realisitically secure.
A typical example is people put on a TLS certificate from LetsEncrypt on their domain and think that suddenly all their stuff is safe; but they don't encrypt their sessions or their cookies so leaving a massive potential hole in their defences.
Do not use MySQL's built in encryption system.
I can not stress this enough; the built in encryption system in MySQL is not suitable for actual secure data protection.
Please read my answer to a very similar question here as to the details (I don't want to simply copy/paste).
Ok, then, because you insist.... here:
I have always understood NOT TO USE MySQL's built in encryption fuctionality because the point of encryption of data at rest (in the SQL) is that if the server is compromised, the data is not at [as much] risk.
The problem with the MySQL built in functionality is that it doesn't apply to when the data is passed to and from the "at rest" state, so the plaintext of any data can be recorded in MySQL logs (and elsewhere on the storage system, such as query lookups are not encrypted so you can from numerous lookups and their count results deduce column values) before/as it is encrypted. You can read more about this here.
Regarding encryption, you should use some tried and tested library like defuse/php-encryption.
From what I've read in my own research on this topic, the link provided by Magnus to defuse/php-encryption is one of the best ways of preventing MySQL ever causing you to compromise your data, by never letting the MySQL program/server ever see the plaintext value of your data.
-- Answer as posted May 7th 2017.
Also Bill Karwin's answer to the same question gives some valuable additional insights:
+1 to Martin's answer, but I'll add some info for what it's worth.
MySQL 5.7 has implemented encryption at rest for InnoDB tablespaces (https://dev.mysql.com/doc/refman/5.7/en/innodb-tablespace-encryption.html).
MySQL 8.0 will reportedly also implement encryption at rest for InnoDB redo log and undo log files (https://dev.mysql.com/doc/refman/8.0/en/innodb-tablespace-encryption.html).
This still leaves unencrypted the query logs and the binary log. We'll have to wait for some future version of MySQL for that.
Why does it take so long? The head of the security engineering for MySQL said at a bird-of-feather session at the Percona Live conference last month [April 2017] that they are being very careful to implement encryption right. This means implementing features for encryption, but also key security and key rotation, and other usage. It's very complex to get this right, and they don't want to implement something that will become deprecated and make everyone's encrypted databases invalid.
-- Answer as posted May 7th 2017.
Closing Point:
Security is complex. If you want to do it properly and have a confidence in your protective onion skins then you need to do a lot of things (see bullets below); but the first thing you need to do is:
Define Who you are protecting against
Seriously. You need different strategies against someone who wants to steal your plaintext names and addresses versus someone who wants to take over your server versus someone who simply wants to trash the data just because. It is a myth that you can protect against everyone all of the time, by concept this is impossible*; so you need to define the most likely agressors and then work out how best to mitigate their advances.
Sepcifically to MySQL, some clear recommendations:
Keep the SQL and the PHP on the same server. Do not remote access to the MySQL data.
Exclude external access to the SQL (so it's localhost only)
Obfuscate your table names and column names; if someone break into your data and you have HDTBJ^BTUETHNUYT under the column username then they know that this garble is probably a username so they have a very good start in trying to break your encryption.
IMPORTANT: Really lock down your table access; set up lots of MySQL users, each with only the bare minimum privilieges to do what they need; you want a user to read the table (only) and only read certain tables; users to write to certain tables but have no access to other tables. It's seperation of concern so that if any one user on the MySQL is compromised; you've not automatically lost every piece of data in there.
Use PHP encrpytion services . Store Encryption keys in a completely separate place; for example have another server you use solely for backup that you can access solely for reaching out to grab the encryption keys, therefore if your PHP/MySQL server is compromised you have some room to cut off and lock down the Key server so thay you can limit the damage. If the key server also has backups then really you're not too badly compromised (situation dependant).
Set up lots of watchers and email informers to tell you exactly when certain processes are running and which server users (not people but programs) are doing what. So you can see why an unexpected process starts to run at 5am to try and measure the size of the MySQL tables. WTF?
There is a lot of potential to have your MySQL AES_ENCRYPT'ed data "sniffed" even if it is not at rest in the DB, but if the website gets compromised (or worse, the PHP code is insecure) then timing attacks can work out data contents by timing query lookups and data packet returns.
Security is a black hole; at some point or another you're going to think "Sod this, I've done enough". No one ever has total security, some very dedicated organisations have enough security. You need to work out how far you're willing to walk before you've gone the distance.
* Why impossible? Because to protect your data from all threats, all of the time, it would need to be unreadable, unusable, like a hash. A hash is protected from everyone, all of the time. But a hash can never be un-hashed.
I am trying to improve the security a a MySQL database that will contain sensitive data. I am struggling to get to grips with some terminology. Can somebody let me know if I have understood the situation correctly:
Encryption at rest - it seems like I can enable this on a table level. All data in the table in encrypted using a key. If somebody got hold of a backup file or gained physical access to the server then the data would be protected. This assumes, of course, that the key is stored elsewhere.
AES_ENCRYPT - when inserting/updating data into my table I can use AES_ENCRYPT('data', 'password'). When querying the data via a SELECT I use AES_DECRYPT
Assuming I was just using encryption at rest then do I need to do anything different in my PHP code to query the data? Does my PHP code need to send the key to the database via my PDO request? Or can I use my normal code for querying the database and the decryption is handled automatically?
Or have I misunderstood what encryption at rest does and I need to use AES_ENCRYPT instead/as well
Encryption at rest
Encryption at rest is the data in the database when it is not being used/accessed or updated. Encryption on the move is things like TLS where the data (from the database) is transported from server to server to browser, to server, to browser, etc. TLS is perfectly good in most situations if it's handled carefully and approached with an attitude that you need to do more than the bare minimum to actually make it realisitically secure.
A typical example is people put on a TLS certificate from LetsEncrypt on their domain and think that suddenly all their stuff is safe; but they don't encrypt their sessions or their cookies so leaving a massive potential hole in their defences.
Do not use MySQL's built in encryption system.
I can not stress this enough; the built in encryption system in MySQL is not suitable for actual secure data protection.
Please read my answer to a very similar question here as to the details (I don't want to simply copy/paste).
Ok, then, because you insist.... here:
I have always understood NOT TO USE MySQL's built in encryption fuctionality because the point of encryption of data at rest (in the SQL) is that if the server is compromised, the data is not at [as much] risk.
The problem with the MySQL built in functionality is that it doesn't apply to when the data is passed to and from the "at rest" state, so the plaintext of any data can be recorded in MySQL logs (and elsewhere on the storage system, such as query lookups are not encrypted so you can from numerous lookups and their count results deduce column values) before/as it is encrypted. You can read more about this here.
Regarding encryption, you should use some tried and tested library like defuse/php-encryption.
From what I've read in my own research on this topic, the link provided by Magnus to defuse/php-encryption is one of the best ways of preventing MySQL ever causing you to compromise your data, by never letting the MySQL program/server ever see the plaintext value of your data.
-- Answer as posted May 7th 2017.
Also Bill Karwin's answer to the same question gives some valuable additional insights:
+1 to Martin's answer, but I'll add some info for what it's worth.
MySQL 5.7 has implemented encryption at rest for InnoDB tablespaces (https://dev.mysql.com/doc/refman/5.7/en/innodb-tablespace-encryption.html).
MySQL 8.0 will reportedly also implement encryption at rest for InnoDB redo log and undo log files (https://dev.mysql.com/doc/refman/8.0/en/innodb-tablespace-encryption.html).
This still leaves unencrypted the query logs and the binary log. We'll have to wait for some future version of MySQL for that.
Why does it take so long? The head of the security engineering for MySQL said at a bird-of-feather session at the Percona Live conference last month [April 2017] that they are being very careful to implement encryption right. This means implementing features for encryption, but also key security and key rotation, and other usage. It's very complex to get this right, and they don't want to implement something that will become deprecated and make everyone's encrypted databases invalid.
-- Answer as posted May 7th 2017.
Closing Point:
Security is complex. If you want to do it properly and have a confidence in your protective onion skins then you need to do a lot of things (see bullets below); but the first thing you need to do is:
Define Who you are protecting against
Seriously. You need different strategies against someone who wants to steal your plaintext names and addresses versus someone who wants to take over your server versus someone who simply wants to trash the data just because. It is a myth that you can protect against everyone all of the time, by concept this is impossible*; so you need to define the most likely agressors and then work out how best to mitigate their advances.
Sepcifically to MySQL, some clear recommendations:
Keep the SQL and the PHP on the same server. Do not remote access to the MySQL data.
Exclude external access to the SQL (so it's localhost only)
Obfuscate your table names and column names; if someone break into your data and you have HDTBJ^BTUETHNUYT under the column username then they know that this garble is probably a username so they have a very good start in trying to break your encryption.
IMPORTANT: Really lock down your table access; set up lots of MySQL users, each with only the bare minimum privilieges to do what they need; you want a user to read the table (only) and only read certain tables; users to write to certain tables but have no access to other tables. It's seperation of concern so that if any one user on the MySQL is compromised; you've not automatically lost every piece of data in there.
Use PHP encrpytion services . Store Encryption keys in a completely separate place; for example have another server you use solely for backup that you can access solely for reaching out to grab the encryption keys, therefore if your PHP/MySQL server is compromised you have some room to cut off and lock down the Key server so thay you can limit the damage. If the key server also has backups then really you're not too badly compromised (situation dependant).
Set up lots of watchers and email informers to tell you exactly when certain processes are running and which server users (not people but programs) are doing what. So you can see why an unexpected process starts to run at 5am to try and measure the size of the MySQL tables. WTF?
There is a lot of potential to have your MySQL AES_ENCRYPT'ed data "sniffed" even if it is not at rest in the DB, but if the website gets compromised (or worse, the PHP code is insecure) then timing attacks can work out data contents by timing query lookups and data packet returns.
Security is a black hole; at some point or another you're going to think "Sod this, I've done enough". No one ever has total security, some very dedicated organisations have enough security. You need to work out how far you're willing to walk before you've gone the distance.
* Why impossible? Because to protect your data from all threats, all of the time, it would need to be unreadable, unusable, like a hash. A hash is protected from everyone, all of the time. But a hash can never be un-hashed.
I have a database along with REST API for clients to access the data. For performance and other reasons, I need to move the application along with the data to the client's physical server. Is there a way for me to encrypt the data in the database, so the only way the client can get access to it is through API that I expose, and not by cracking MySql and getting at raw data. I do not want the client to see the data stored on my DB, as I feel they will steal it or share it. What can I do to accomplish that?
One idea:
Is it possible to implement some form of one-way encryption, where its based on the lookup value provided in api.
e.g. api lookup by email, that is then gets one-way encrypted compared in the DB for match, and returns a record. This way if they happen to look at my database, the can not see list of emails, all they see is data that is something similar /etc/passwd file.
No.
From the 10 Immutable Laws of Security
Law #3: If a bad guy has unrestricted physical access to your computer, it's not your computer anymore
What you want is fundamentally impossible, without caveats. Always and everywhere.
I work at a small company and I am trying to figure out a solution for storing sensitive data of multiple clients in Microsoft SQL server. Actually, I feel like this is a general database question and it is not specific to MSSQL.
Until now we have been using a proprietary database where the client data is stored as db files (flat files) in the client’s root directories in the file system. So the operating system permissions guarantee that the application used by client X can never fetch data from client Y’s database. Please note that there is no database server/instance/engine here…
However, for my project I want to use SQL database. But the security folks are expressing concerns over putting data of different clients on a single database.
One option is to create separate database instances for different clients. However, I am not sure if this idea is scalable.
So my questions are:
1) Is there any mechanism in MSSQL that enables you to store databases ‘separately’ in different files used by the SQL server?
2) Let’s say I have only one database instance where I have databases of client X and client Y. How can I make sure that client X’s requests can never (accidentally) get misdirected to client Y’s database? I do not want to rely on some parameter in my code to determine which database to fetch from! :)
So, is there any solid authentication scheme to guarantee that my queries could not be misdirected to fetch from an incorrect client table?
I think this is a very common problem and there has to be a good solution for this. What are other companies doing?
Please let me know if there are any good articles to read up on this.
Different databases are always stored in different files in SQL Server so you don't even have to do anything special for this. However, NTFS permissions will not help you in this case as the clients aren't ever accessing the files directly on disk.
One possible solution in SQL Server is to create separate sets of Windows user IDs and map those to separate SQL Logins for each customer. You could then only assign those logins access to the appropriate databases. For example, if you were hosting web sites for client X and client Y, you would set up the connection string(s) in the web.config for client X's web site to use the appropriate login(s) for client X's database. Vice versa for client Y. This guarantees that no matter what (barring a hard-coded login), the code from client X's site will never access client Y's database.
You can have 32,000 databases on a single instance of SQL server and having separate databases enables a number of improved serviceability scenarios (such as restoring a single customer's DB in case of a data problem without affecting all of your other customers).
http://technet.microsoft.com/en-us/library/ms143432.aspx
We have recently implemented Transparent Data Encryption in SQL Server 2008 for local databases on our developers laptops to keep them protected in the case a laptop is stolen or lost. This works fine.
Now we are trying to figure out a way to have the certificate expire everyday, forcing an automated process (a script at logon maybe) to go out to a network path and grab a new certificate with an expiration for a day later. This would ensure that if something unforeseen happened, the data would not be usable the next day.
I also looked into using a Cryptographic provider but there doesn't appear to be any "providers" out there. Maybe I'm wrong.
I am open to suggestions. If there is a better way please let me know. Thanks!
Short answer: No
Long answer: Once a message (piece of data) is encrypted, that same key will decrypt the same encrypted message, regardless of what time the decryption algorithm is applied. If the key is changed every day, the data must be decrypted with the old key and re-encrypted with the new. If this process doesn't occur (i.e. someone stops the piece of code that performs the re encryption from running), the old key will still work. Even if you do create a cryptographic provider to check the date, someone else can create a new provider to perform the decryption without first checking the date.
T address the question rather than the motivation. If you set up a Microsoft CA with a derived template (Set to expire for a day) and also allow autoenrollment on that certificate template. You could then set your SQL machine to be part of a OU within the Directory that uses autoenrolment (Technet will give you resources on this requires the use of goup policy). That way when the certificate expires the machine will automagically request a new one.
http://windowsitpro.com/article/articleid/40948/windows-server-2003-pki-certificate-autoenrollment.html
Mark
Not true! There are options available for SQL Server 2008 encryption. Check out the database encryption solutions here at TownsendSecurity.com. Townsend's Alliance AES Encryption is a NIST-certified solution that would put you into compliance with the regulations around health care, credit cards, and banking. Also see the white paper on Alliance AES Encryption.
Businesses with sensitive data in database applications
want to encrypt the data in order to secure it from loss.
Protecting sensitive data increases customer trust and
loyalty, reduces legal liability, and helps meet regulatory
requirements for data security. Examples of databases
that might contain sensitive information are Oracle
Database, IBM DB2, Microsoft SQL Server, MySQL,
and Microsoft Access. Regardless of the disk or folder
encryption technology that might be used, the actual
data should be encrypted to prevent loss
Full disclosure: I'm an intern at Townsend Security.
Without additional detail I fail to understand how your TDE setup will protect data in case it is lost or stolen.
If you are not using full disk encryption (via Bitlocker, Truecrypt, etc) then I as an attacker in physical possession of your hardware can easily reset the local admin password, boot up the laptop and access the SQL Server instance with the local admin credentials. At that point I am a sysadmin on the database server and am able to extract any data I want or to turn off TDE.
In addition since all of the encryption keys and certificates are stored locally it is relatively easy for an attacker in physical possession of the device to gain access to them. TDE is only meaningful for data protection when you physically separate the Database Encryption Key protectors (stored in the master database) from the encrypted database.
If you are using full disk encryption than the usage of TDE is not providing any additional deterrent to an attacker and is only adversely affecting system performance of your developers laptops.
You're right - what you want is a cryptographic provider, and you're right that there's none out there yet.
If you're going to the PASS Summit in November, talk to JC Cannon from Microsoft. He's doing a session on compliance, and he's the head of the SQL Server Compliance group. He's tied into the vendors that are currently working on building cryptographic providers, and he may be able to talk to you about vendor names. Right now they haven't come out publicly to announce who's doing it yet.