Edited based on comments.
My company is developing a system and is in the early design stages. I am the lead on the project. Some of the information to be stored in the system is highly confidential. The system must protect this information such that only authorized users have the ability to see this information. In fact, it is such that we want to protect it from ourselves even. The database to be used will probably be mySQL but does not have to be. I am aware that the data can be encrypted in the database. I am also aware that the software will have to be able to retrieve this data from the database for authorized users. I do not want any developers, including myself, to be able to decrypt and view this information -- in production -- but they do need to be able to view it during development(we will disable encryption during development).
The software will be a cloud based offering but will have an in house hosting option as well.
We can have a software based encryption and I realize that if software can get at the data then it's accessible - but I want this as tight as possible so that a developer cannot (including myself) view this data, but signed in, authorized users, can view it.
I'm stuck in my own thinking box about how to do this so I'm seeking advice as to how to secure this.
The client side will need a decryption key to decrypt the data. The software will have to allow access to this key for authorized users. The users cannot hold this key as the "client" devices could be virtually anything. Now I know I have no control over what the end users who have access do with the data, and that's not by concern. I want them to be able to store this information and have it secure in such a way that I cannot see it! I also do not want to have the ability to make myself authorized to see it - but to have that ability rest in the hands of a user side trusted administrator.
The data can be binary data. I do not want to encrypt the entire database, nor do I want to ever need to re-encrypt data because it can be huge amounts of data.
I have read posts about using GnuPG to handle this with public key for encryption and decryption via any one of a set of multiple private keys. This might be feasable for this but I'm looking for multiple solutions. My "issue" with this is the pass phrase protecting the private key and issueance of these private keys. I really don't want to user having to key in the passphrase associated with the private key every time that they login to the system or try to view the encrypted/protected data, but I don't want to store this passphrase anywhere a developer can see it either!
I understand about PKI but it doesn't offer protection from developers who can access the master decryption key which is shared using a public/private keypair.
I've edited the question with a proposed solution, please punch holes in it.
Development Language is C#. Microsoft supports asymetric keys which is what we want to use.
An X509 Certificate will be utilzied for the encryption keys. The public key will be stored in the database and used for all encryption. The private key will be stored in the database as well but encrypted using a passphrase entered by a trusted administrator. (This is the weak link in my mind, more on this later).
The administrator will create users who have access to the encrypted data. The users will be issued a copy of the private decryption key via the administrator and this key will be associated with their user account and encrypted in the system using their initial pass phrase.
When the user first logs in and is prompted to change their pass phrase, the private key will be re-encrypted using their new pass phrase. Their pass phrase is not stored in the database, only a hash for validation. Their keyed pass phrase will be encrypted using a timestamp or other data and will be stored on their local computer not on the server or in the servers identifable memory except when it is used for decryption of the private key. A software algorithm will be created to retrieve the decryption key for the encrypted master decryption key associated with this user.
Thus users can be created and revoked at the will of the administrator. Users can have their pas phrases reset - but in that case a new decryption key would need to be issued because the old one would not be decryptable any longer.
The one flaw I see, and I'm open to input, is if the administrator forgets their pass phrase and can no longer issue private keys. We could write code that given a users password, recovers the private key and encrypts it using the administrators new password when they enter it.
Another weak link is user pass phrase. If that becomes compromised then a developer could use that to obtain a copy of the private decription key and decrypt data.
Also - if for some reason an administrator needs to be let go we have the same issue as above where we don't have a copy of the private key accessible to share but we could get one using a users pass phrase.
Any advice on overcoming the perceived weaknesses?
We cannot tolerate exposure of the master decryption private key - if it is exposed then the entire database needs to be re-encrypted using a new keypair.
Thank you for your comments.
If you need to protect the data from yourselves, the only option is end-to-end encryption. User has a private key and it never leaves their computer. Devs/sysadmins never have private keys and can never decrypt.
This would require you make your client software source available to users so they can see that you are not leaking their info.
Use USB smart cards for easier/safer key management. They key is generated on the smart card and never leaves it. The smart card does the decryption on your apps behalf.
This make searching harder, but you can fix some of this by storing hashes along w the encrypted values, and searching on them.
Update, answering your questions in your comment:
Web based is not an option. You need a thick client
There is no "master key"
Please clarify "authorized user". Authorized by whom? Please provide an example.
Another option would be to skip encryption entirely and rely solely on access control. Use a db like postgresql, ms sql, or oracle that does row-level security. Don't give access to any one DBA. Give part of password to one dba, another part to another. DBA's need to be in same room to access the server.
Either way, you should be familiar with https://en.wikipedia.org/wiki/Secret_sharing
If you were to condider SQL Server instead of mysql, then the most secure design I can think of that meets your requirements is to require SQL Server Enterprise Edition an Extensible Key Management system that uses a Hardware Security Module. SQL Server Extensible Key Management enables the encryption keys that protect the database files to be stored in an off-box device such as a smartcard, USB device, or EKM/HSM module. EKMs are only supported in Enterprise edition. An HSM will allow for the delegation and isolation of managing the encryption keys and functions under a group other than both developers and DBAs, if that is your intent. Other features I would suggest in the design include the following options to complete end to end transport\session encryption:
Installing a domain or CA certificate on the SQL Server and enabling Force Protocol Encryption or enabling IPSec on the host of the instance
Installing a domain or CA certificate on all IIS websites or web services and forcing encryption in the connection strings and https
The benefits of this design over SQL Server Always Encrypted are that the encryption is protected by a single system that can be monitored and defended with more focus as opposed to numerous clients that increase exposure to theft of the private key from one of the users key stores. Also, key rotation for EKMs should be easy for those who are managing the system. It is also possible with an EKM to require that more than one person be involved in generating keys.
Related
I have already read through Saving credit card information in MySQL database? and Storing Credit Card Information.
I'm aware that storing credit card information requires PCI compliance, which is not an easy task.
That is not what this question is about. My question is the following:
What is a secure way to encrypt user credit cards? The simplest and easiest that comes to mind is using a private key and encrypting CC's with that. This doesn't seem very secure because the key has to be stored on the server, and if an attacker can get my database, they can probably get the key too.
What I'd like to be able to do is encrypt every CC using that users password as part of the encryption process. If anyone gets the database, they can't decrypt anything because the passwords are stored as salted hashes. This would work great for transactional purchases - the user clicks "Buy," types in their password as a confirmation, I decrypt their CC and make the charge. Their password is only in memory for the duration of the request, and is never written to disk.
Unfortunately this won't work for what I'm trying to build - a service which charges a recurring fee (say, once a month), regardless of whether or not the user is logged in when I need to make the charge.
Given this scenario, is there a secure way of storing user CC's?
As you need to be able to decrypt, there's always the possibility that the encryption keys leak and you'll lose everything. So you'll never get to absolute security, but you can make it harder for attackers to get to the data.
Nobody but you can really judge what level of security (or obscurity) you should have. This is most likely a function of size of database, visibility etc.
For leaks, unfortunately you'll have to assume that everything leaks and sooner or later (e.g. with brute force attacks on weak passwords) you haven't gained too much when they get out.
Given the last credit card leak scandals - the worst ones had the 3-digit (CVV) number saved with the regular credit card number, which credit card companies explicitly forbid (that's why you'll always have to give it again even if someone has your credit card information on file)
If you don't want to assume the responsibility for holding and processing this kind of data, a good way to go is with an external payment service - let them do the processing and just assert to you that the payment has been processed. You'd have to pay them for their services, but you'd also have to pay for implementing your own solution and for taking the risk.
If you use the password as the salt for the CC encryption, it would be a very effective way of securing the information, however, they would never be able to change their password... If it is changed, then the encrypted data is lost. The bottom line for securing the encryption key is to make it as difficult as possible to find... essentially the more steps you use to hide the key, the harder it is for them to find it... which means it is harder for you to use and program for it. There is no magic bullet at this time to protect everything. (Invent a secure way to keep the key and you will be rich)
As for the CVV number, it cannot be stored as previously mentioned. With each transaction the cc processing company will give the merchant a Reference Number which is then used in each reoccurring payment. This means if the original transaction required the CVV number, then logic will dictate that the recurring payment will also be authorized by the same user who put it in on the first transaction. Therefore, the reoccurring payments will not need the CVV to maintain the same level of security.
You could essentially use multiple servers. Encrypt the cc with a key, but keep that key on a separate encryption server, the is only accessible by a master username and password for windows or whatever OS you're using. This way you're securing your key, setting up a services on the encyrption service to run the card through the encryption and then submit it to the database.
Use php's private/public openssl functions when a user makes a purchase you use the data in memory to make the purchase then you store the information using a public key to encrypt it.
To process billing monthly you decrypt the data using the private key that could be manually punched in or stored in code. If you want to store the ssl key in code and not have to remember it or get it everytime. I would encrypt the key using a salt stored in the configuration variables + buy a yubi key and generate a 32 character password + my own password on top of it. Store the yubikey in a safe place (A safe lol). When you need to process credit cards do it with a script that runs in the background and runs all billing at once. To change the password would require you decrypt all cards and re-encrypt them using the new private/public key, or you may just decrypt and re-encrypt the private key ssl.
Magic :)
You require the card information to be reversibly encrypted. The decryption information has to come from somewhere. You've said the data cannot come from the user, and you don't want it stored at the server, so it must be on separate equipment that is presumably more secure. And if you have the ability to recall that information, so does an attacker who has compromised your system. So presumably the decryption information is not retrieved at the vulnerable host during decryption.
Perhaps consider a third-party service that you can encrypt and send information to, perhaps one that specializes in PCI compliance. It might be able to decrypt the credit card information when you send it a second time and apply a charge, or it might actually store the card information for later use. It might even perform recurring transactions for you.
http://www.authorize.net/solutions/merchantsolutions/merchantservices/automatedrecurringbilling/
I just Googled that, I don't recommend them. But it's an example.
Encrypt the CC information twice. First, encrypt the credit card data based off the user's password (+ salt). Then encrypt the output of that with the server's key.
To access the information, you thus require the user's password (i.e. decrypt using server's key, then decrypt based off password). If the database and server key are compromised, the information still isn't exposed without attacking the user's password first.
It's important that the user's password is for the internal encryption - this allows you to re-encrypt when you change server encryption keys.
When the user changes their password, you also re-encrypt the data. If the user resets their password, then the CC information should be erased (and is lost anyway, as it can't be unencrypted).
I have very sensitive data of mine in the database available in the public domain. Though I have secured it to prevent sql injections, I little bit worried about the data. Can I able to encrypt the entire database using any advance encryption algorithms. Is that a correct way of securing the data? Any suggestions?
Database: MySql
I'm not sure to understand how a database in the public domain (this has some very specific legal meaning) can contain sensitive data.
What you might perhaps do is encrypt some columns of your database, that is have the encryption and decryption happening in your client application, on some columns of each rows.
First of all, only data encryption is not the answer.
You should know and control "Who can access and accessed data".
Although you encrypt data itself, attacker can get the encrypted data.
To prevent it, you should control the access privilege or authority.
Also, you should know who accessed encrypted data. Because in these days, data can be stolen by not only external users, but also internal users.
You should encrypt data, control privilege and audit who accessed.
In those days, those three(3) technology have been developed in each way.
Most companies have bought each solutions.
But for personal user, it is hard to buy it and build those environment itself.
I would you like to review MyDiamo that provides those three(3) functions.
Goal:
I'd like to allow users to create questions and collect information from other users in a custom web app (PHP/MySQL in a shored hosting environment) and protect the data collected.
Background:
The default questions that all users answer are general enough that they can not be construed as personally identifiable information (PII), thus limiting my liability to protect it, but users who create their own questions will likely ask for PII which then becomes a liability.
What I would like to do is protect this information in such a way that if either the hosting account or the database were compromised (or both!), the PII would not be recoverable without a significant amount of work, and even then, only a small portion would theoretically be recoverable.
Proposed solution:
Assuming MySQL's built-in AES_ENCRYPT()/AES_DECRYPT() functions are used to encrypt the PII table, the passphrase would need to be stored in the hosting account, so if the hosting account were compromised, the data could easily be read.
Since the users' passwords are well protected (hashed with salt), I'm thinking of capturing their plaintext password during authentication, encrypting it, and storing it in the PHP session until the user logs out.
A public/private key combo will be created for each user with the private key being password protected with the user's password + salt.
Then, when PII data based on that user's custom questions are added to the DB, the user's public key would be used to encrypt the PII that they collected through the app. When the data is read (only when the user is logged in), the data would be unencrypted with the user's private key (which is unlocked with their password + salt).
The benefits I see are:
in the worst case scenario, where servers completely compromised, app code is read to find encryption keys, PHP session files are decrypted to find user's passwords, then entries in the PII table associated with that user are decrypted, then only the PII collected from questions of currently logged-in users could be recovered. Any users not logged in would be safe.
even the DBA or similar wouldn't be able to read the PII.
The drawbacks I see are:
user passwords are stored in a recoverable form while they are logged in.
users who forget their passwords would loose access to their data.
each relatively small bit of data will take up much more space in the DB due to encryption.
My question: Is there a better way to do this?
I see a number problems with this design from a security perspective. First of all passwords must never be encrypted, this is a vulnerability identified by CWE-257.
Further more MySQL's AES_ENCRYPT() is complete garbage for more than one reason. It uses EBC mode, and here is a good example of why this is crap:
Original Image:
EBC Mode (which is what mysql's AES_ENCRYPT() uses):
But if the database where compromised the attacker is going to defeat AES_ENCRYPT() by enabling the query log.
Using the user's password for encryption should be avoided, you should be using a crypgoraphic nonce. If you do use a password make sure you use a String2Key funciton. You must also use CBC or CMAC mode with a random iv. I don't really see how asymmetric cryptography can help. Asymmetric cryptography is very slow, memory intensive. They data that it protects is made less secure when the attacker controls the message because you can compare cipher text messages. This is why an random IV is important, and in the asymmetric world you don't have this level of protection.
Key Generation should look something like:
$key=string2key($base_nonce.$salt.$user_password)
Make sure the output of your string2key function is the same size as your keyspace. So aes 128 needs a 128bit key. Each password should have its own $salt, and the $base is a cryptographic nonce stored in textfile. (An attacker would have to read this file before he can crack the key, if this value is large like 128 bits then its a moot point.) Each message needs its own $iv and this value must also be a cryptographic nonce (similar to a salt). I would generate the $salt,$iv and $base_nonce from /dev/urandom. The IV can be stored in plain text in a column in your database along with the cipher text.
From a legal standpoint even if you build a secure cryptogrpahic system you still have problems with insider threats and if the server is completely compromised, all of the data will still be compromised. This really isn't an engineering problem.
The best defense against a legal threat is a strong Terms and Conditions written by a skilled lawyer.
One concern I'd have is the following. The "any users not logged in would be safe" part is too optimistic. By protecting the private key with the user's password, you open yourself up to various password brute-forcing attacks. Not just for current sessions, but for all. One effective one is to simply enumerate the top 100 common passwords, say, and just try them out against all users. Attacker is bound to uncover some keys. (I'm assuming that you're either storing per-user random salt with the user record, which the attacker can see, or else that you have a secret salt that the attacker was able to obtain through the compromise.)
I have a web based (perl/MySQL) CRM system, and I need a section for HR to add details about disciplinary actions and salary.
All this information that we store in the database needs to be encrypted so that we developers can't see it.
I was thinking about using AES encryption, but what do I use as the key? If I use the HR Manager's password then if she forgets her password, we lose all HR information. If she changes her password, then we have to decrypt all information and re-encrypt with the new password, which seems inefficient, and dangerous, and could go horrifically wrong if there's an error half way through the process.
I had the idea that I could have an encryption key that encrypts all the information, and use the HR manager's password to encrypt the key. Then she can change her password all she likes and we'll only need to re-encrypt the key. (And without the HR Manager's password, the data is secure)
But then there's still the problem of multi-user access to the encrypted data.
I could keep a 'plaintext' copy of the key off site, and encrypt it with each new HR person's password. But then I know the master key, which doesn't seem ideal.
Has anyone tried this before, and succeeded?
GnuPG allows documents to be encrypted using multiple public keys, and decrypted using any one of the corresponding private keys. In this way, you could allow data to be encrypted using the public keys of the everyone in the HR department. Decryption could be performed by any one having one of the private keys. Decryption would require both the private key and the passphrase protecting the key to be known to the system. The private keys could be held within the system, and the passphrase solicited from the user.
The data would probably get quite bloated by GnuPG using lots of keys: it has to create a session key for the payload and then encrypt that key using each of the public keys. The encrypted keys are stored alongside the data.
The weak parts of the system are that the private keys need to be available to the system (ie. not under the control of the user), and the passphrase will have to pass through the system, and so could be compromised (ie. logged, stolen) by dodgy code. Ultimately, the raw data passes through the system too, so dodgy code could compromise that without worrying about the keys. Good code review and release control will be essential to maintain security.
You are best avoiding using MySQL's built in encryption functions: these get logged in the replication, slow, or query logs, and can be visible in the processlist - and so anyone having access to the logs and processlist have access to the data.
Why not just limit access to the database or table in general. That seems much easier. If the developer has access to query the production, there is no way to prevent them from seeing the data b/c at the end of the day, the UI has to decrypt / display the data anwyays.
In the experience I've had, the amount of work it takes to achieve the "developers cannot see production data at all" is immense and nearly imposible. At the end of the day, if the developers have to support the system, it will be difficult to achieve. If you have to debug a production problem, then it's impossible not to give some developers access to production data. The alternative is to create a large number of levels and groups of support, backups, test data, etc..
It can work, but it's not as easy as business owners may think.
Another approach is to use a single system-wide key stored in the database - perhaps with a unique id so that new keys can be added periodically. Using Counter Mode, the standard MySQL AES encryption can be used without directly exposing the cleartext to the database, and the size of the encrypted data will be exactly the same as the size of the cleartext. A sketch of the algorithm:
The application generates a unique initial counter value for the record. This might be based on some unique attribute of the record, or you could generate and store a unique value for this purpose.
The application generates a stream of counter blocks for the record based on the initial counter value. The counter stream must be the same size or up to 1 block larger than the cleartext.
The application determines which key to use. If keys are being periodically rotated, then the most recent one should be used.
The counter stream is sent to the database to be encrypted: something like
select aes_encrypt( 'counter', key ) from hrkeys where key_id = 'id';
The resulting encrypted counter value is trimmed to the length of the cleartext, and XORed with the cleartext to produce the encrypted text.
The encrypted text is stored.
Decryption is exactly the same process applied to the encrypted text.
The advantages are that the cleartext never goes any where near the database, and so the administrators cannot see the sensitive data. However, you are then left with the problem of preventing your adminstrators from accessing the encrypted counter values or the keys. The first can be achieved by using SSL connections between your application and database for the encryption operations. The second can be mitigated with access control, ensuring that the keys never appear in the database dumps, storing the keys in in-memory tables so that access control cannot be subverted by restarting the database with "skip-grants". Ultimately, the only way to eliminate this threat is to use a tamper-proof device (HSM) for performing encryption. The higher the security you require, the less likely you will be able to store the keys in the database.
See Wikipedia - Counter Mode
I am just thinking out loud.
This seems to call for a public/private key mechanism. The information would be stored encrypted with the HR public key and would only be viewable by someone in possession of the associated private key.
This, to me, seems to rule out a web based interface to view these confidential data (entering them via the web interface is certainly feasible).
Given that individuals come and go, tying the keys to a specific person's account seems infeasible. Instead, one must handle key distribution separately and have a mechanism for someone to change the keypair used (and re-encrypt the database — again without the use of a web interface) in case the current HR manager is replaced with someone else. Of course, nothing would prevent the HR manager from dumping all the data before leaving while before the keys are replaced.
I'm not sure how feasible this is currently, or what current stable DB systems have support for this, but alternate authentication mechanisms at the database level may help. For example Drizzle, a refactoring of the MySQL code base, supports (or aims to?) completely pluggable authentication, allowing no auth, server housed auth, or auth through PAM or some other mechanism, meaning you can use LDAP.
If you had different levels of access based on the database connection, and the application login also specified what you could actually access in the database, you could theoretically build a system where it wasn't possible to access the confidential database info unless using an account with specific access rights, regardless of the privilege escalation attempts in the application itself.
As long as the people setting user account access rights can be trusted or themselves are OK to see the confidential information, this should be fairly secure.
P.S. It might be useful to use a generic DB connection for "regular" application information, but when an attempt to access confidential information is made, then the specific DB connection is attempted. This allows for a few DB connections to handle most requests, assuming the majority of users aren't viewing confidential info. Otherwise, a separate DB connection per user may become burdensome to the DB.
We have recently implemented Transparent Data Encryption in SQL Server 2008 for local databases on our developers laptops to keep them protected in the case a laptop is stolen or lost. This works fine.
Now we are trying to figure out a way to have the certificate expire everyday, forcing an automated process (a script at logon maybe) to go out to a network path and grab a new certificate with an expiration for a day later. This would ensure that if something unforeseen happened, the data would not be usable the next day.
I also looked into using a Cryptographic provider but there doesn't appear to be any "providers" out there. Maybe I'm wrong.
I am open to suggestions. If there is a better way please let me know. Thanks!
Short answer: No
Long answer: Once a message (piece of data) is encrypted, that same key will decrypt the same encrypted message, regardless of what time the decryption algorithm is applied. If the key is changed every day, the data must be decrypted with the old key and re-encrypted with the new. If this process doesn't occur (i.e. someone stops the piece of code that performs the re encryption from running), the old key will still work. Even if you do create a cryptographic provider to check the date, someone else can create a new provider to perform the decryption without first checking the date.
T address the question rather than the motivation. If you set up a Microsoft CA with a derived template (Set to expire for a day) and also allow autoenrollment on that certificate template. You could then set your SQL machine to be part of a OU within the Directory that uses autoenrolment (Technet will give you resources on this requires the use of goup policy). That way when the certificate expires the machine will automagically request a new one.
http://windowsitpro.com/article/articleid/40948/windows-server-2003-pki-certificate-autoenrollment.html
Mark
Not true! There are options available for SQL Server 2008 encryption. Check out the database encryption solutions here at TownsendSecurity.com. Townsend's Alliance AES Encryption is a NIST-certified solution that would put you into compliance with the regulations around health care, credit cards, and banking. Also see the white paper on Alliance AES Encryption.
Businesses with sensitive data in database applications
want to encrypt the data in order to secure it from loss.
Protecting sensitive data increases customer trust and
loyalty, reduces legal liability, and helps meet regulatory
requirements for data security. Examples of databases
that might contain sensitive information are Oracle
Database, IBM DB2, Microsoft SQL Server, MySQL,
and Microsoft Access. Regardless of the disk or folder
encryption technology that might be used, the actual
data should be encrypted to prevent loss
Full disclosure: I'm an intern at Townsend Security.
Without additional detail I fail to understand how your TDE setup will protect data in case it is lost or stolen.
If you are not using full disk encryption (via Bitlocker, Truecrypt, etc) then I as an attacker in physical possession of your hardware can easily reset the local admin password, boot up the laptop and access the SQL Server instance with the local admin credentials. At that point I am a sysadmin on the database server and am able to extract any data I want or to turn off TDE.
In addition since all of the encryption keys and certificates are stored locally it is relatively easy for an attacker in physical possession of the device to gain access to them. TDE is only meaningful for data protection when you physically separate the Database Encryption Key protectors (stored in the master database) from the encrypted database.
If you are using full disk encryption than the usage of TDE is not providing any additional deterrent to an attacker and is only adversely affecting system performance of your developers laptops.
You're right - what you want is a cryptographic provider, and you're right that there's none out there yet.
If you're going to the PASS Summit in November, talk to JC Cannon from Microsoft. He's doing a session on compliance, and he's the head of the SQL Server Compliance group. He's tied into the vendors that are currently working on building cryptographic providers, and he may be able to talk to you about vendor names. Right now they haven't come out publicly to announce who's doing it yet.