I'm looking to migrate a table of user data to my shiny new MongoDB rig. Having trouble wrapping my head around how to handle a column of passwords. I'm using MySQL's PASSWORD() function to store the passwords. When the thing was constructed I didn't see any reason to ever need to reverse the encryption, so I didn't see any harm in using the PASSWORD() function. But now I can't transfer the passwords as is, because (as far as I know) I can't use PASSWORD() the same way in MongoDB to check the validity of the password.
Any ideas?
You have several options:
Option 1: Lazy migration.
Keep both your MySQL and MongoDB servers online and connected. When a user attempts to log in, check the password against MongoDB. If it fails (ie, a password was never set), then check it against MySQL. If it succeeds, then hash the password and store it in the MongoDB document.
Downsides: Your MySQL server has to stay online forever (or at least until all your users migrate).
Upsides: You can easily replace the MySQL password format with your own format (ie, bcrypt hashes or whatnot). You don't have to have any knowledge of how MySQL is hashing passwords internally.
Option 2: Figure out how the MySQL password() function works, and replicate it clientside.
According to Simulating MySql's password() encryption using .NET or MS SQL the algorithm MySQL versions 4.1 and greater use for PASSWORD() is "*" + sha1(sha1("password")), more or less. If you're running an older version, you'll need to find out what the hashing algorithm is, and use it instead. You can just take your password, double SHA1-hash it, prepend an asterisk, and check to see if that value matches what's in the DB.
Downsides: The exact algorithm depends on the version of MySQL you're running, so you might have to do a bit of digging depending on your MySQL version. You're still stuck using the MySQL password format in your MongoDB documents (though you could do a lazy upgrade, with a procedure similar to what was described in option 1).
Upsides: You can do the migration once and then take your MySQL server offline.
Related
I am trying to improve the security a a MySQL database that will contain sensitive data. I am struggling to get to grips with some terminology. Can somebody let me know if I have understood the situation correctly:
Encryption at rest - it seems like I can enable this on a table level. All data in the table in encrypted using a key. If somebody got hold of a backup file or gained physical access to the server then the data would be protected. This assumes, of course, that the key is stored elsewhere.
AES_ENCRYPT - when inserting/updating data into my table I can use AES_ENCRYPT('data', 'password'). When querying the data via a SELECT I use AES_DECRYPT
Assuming I was just using encryption at rest then do I need to do anything different in my PHP code to query the data? Does my PHP code need to send the key to the database via my PDO request? Or can I use my normal code for querying the database and the decryption is handled automatically?
Or have I misunderstood what encryption at rest does and I need to use AES_ENCRYPT instead/as well
Encryption at rest
Encryption at rest is the data in the database when it is not being used/accessed or updated. Encryption on the move is things like TLS where the data (from the database) is transported from server to server to browser, to server, to browser, etc. TLS is perfectly good in most situations if it's handled carefully and approached with an attitude that you need to do more than the bare minimum to actually make it realisitically secure.
A typical example is people put on a TLS certificate from LetsEncrypt on their domain and think that suddenly all their stuff is safe; but they don't encrypt their sessions or their cookies so leaving a massive potential hole in their defences.
Do not use MySQL's built in encryption system.
I can not stress this enough; the built in encryption system in MySQL is not suitable for actual secure data protection.
Please read my answer to a very similar question here as to the details (I don't want to simply copy/paste).
Ok, then, because you insist.... here:
I have always understood NOT TO USE MySQL's built in encryption fuctionality because the point of encryption of data at rest (in the SQL) is that if the server is compromised, the data is not at [as much] risk.
The problem with the MySQL built in functionality is that it doesn't apply to when the data is passed to and from the "at rest" state, so the plaintext of any data can be recorded in MySQL logs (and elsewhere on the storage system, such as query lookups are not encrypted so you can from numerous lookups and their count results deduce column values) before/as it is encrypted. You can read more about this here.
Regarding encryption, you should use some tried and tested library like defuse/php-encryption.
From what I've read in my own research on this topic, the link provided by Magnus to defuse/php-encryption is one of the best ways of preventing MySQL ever causing you to compromise your data, by never letting the MySQL program/server ever see the plaintext value of your data.
-- Answer as posted May 7th 2017.
Also Bill Karwin's answer to the same question gives some valuable additional insights:
+1 to Martin's answer, but I'll add some info for what it's worth.
MySQL 5.7 has implemented encryption at rest for InnoDB tablespaces (https://dev.mysql.com/doc/refman/5.7/en/innodb-tablespace-encryption.html).
MySQL 8.0 will reportedly also implement encryption at rest for InnoDB redo log and undo log files (https://dev.mysql.com/doc/refman/8.0/en/innodb-tablespace-encryption.html).
This still leaves unencrypted the query logs and the binary log. We'll have to wait for some future version of MySQL for that.
Why does it take so long? The head of the security engineering for MySQL said at a bird-of-feather session at the Percona Live conference last month [April 2017] that they are being very careful to implement encryption right. This means implementing features for encryption, but also key security and key rotation, and other usage. It's very complex to get this right, and they don't want to implement something that will become deprecated and make everyone's encrypted databases invalid.
-- Answer as posted May 7th 2017.
Closing Point:
Security is complex. If you want to do it properly and have a confidence in your protective onion skins then you need to do a lot of things (see bullets below); but the first thing you need to do is:
Define Who you are protecting against
Seriously. You need different strategies against someone who wants to steal your plaintext names and addresses versus someone who wants to take over your server versus someone who simply wants to trash the data just because. It is a myth that you can protect against everyone all of the time, by concept this is impossible*; so you need to define the most likely agressors and then work out how best to mitigate their advances.
Sepcifically to MySQL, some clear recommendations:
Keep the SQL and the PHP on the same server. Do not remote access to the MySQL data.
Exclude external access to the SQL (so it's localhost only)
Obfuscate your table names and column names; if someone break into your data and you have HDTBJ^BTUETHNUYT under the column username then they know that this garble is probably a username so they have a very good start in trying to break your encryption.
IMPORTANT: Really lock down your table access; set up lots of MySQL users, each with only the bare minimum privilieges to do what they need; you want a user to read the table (only) and only read certain tables; users to write to certain tables but have no access to other tables. It's seperation of concern so that if any one user on the MySQL is compromised; you've not automatically lost every piece of data in there.
Use PHP encrpytion services . Store Encryption keys in a completely separate place; for example have another server you use solely for backup that you can access solely for reaching out to grab the encryption keys, therefore if your PHP/MySQL server is compromised you have some room to cut off and lock down the Key server so thay you can limit the damage. If the key server also has backups then really you're not too badly compromised (situation dependant).
Set up lots of watchers and email informers to tell you exactly when certain processes are running and which server users (not people but programs) are doing what. So you can see why an unexpected process starts to run at 5am to try and measure the size of the MySQL tables. WTF?
There is a lot of potential to have your MySQL AES_ENCRYPT'ed data "sniffed" even if it is not at rest in the DB, but if the website gets compromised (or worse, the PHP code is insecure) then timing attacks can work out data contents by timing query lookups and data packet returns.
Security is a black hole; at some point or another you're going to think "Sod this, I've done enough". No one ever has total security, some very dedicated organisations have enough security. You need to work out how far you're willing to walk before you've gone the distance.
* Why impossible? Because to protect your data from all threats, all of the time, it would need to be unreadable, unusable, like a hash. A hash is protected from everyone, all of the time. But a hash can never be un-hashed.
I am creating a database and I need to store user passwords. I am already using bcrypt to hash the passwords on the client side, but I have read that only hashing on the client side makes the hash essentially equivalent to a password as far as the database is concerned. I'd like to hash the passwords (which are now hashes) again before they are stored in the database. Do I have to use a method native to MySQL like SHA2(pwd), or is there a way to use bcrypt on the server?
Bcyrpt is a good call here, but you should be doing the hashing on the server end, not the client. The client can't know all the information it needs to produce a hash you can verify is correct, only the server has that information.
What you need to do is pass through the password securely, such as over HTTPS, and hash it there in your application layer. MySQL alone does not have the functions necessary to do proper password hashing. SHA2 is completely inadequate, it's a high-speed hash by design which makes it immediately unsuitable. Password hashing algorithms are deliberately slow to make brute-forcing passwords painfully expensive.
What is the equivalent function in MSSQL Server for PASSWORD() function of MYSQL? I have to migrate MySQL queries into MSSQL Server.
Below is the sample query of my source code:-
Select * from users where username = 'demo' and pasword = PASSWORD('demo');
this function is being used for creating user account creating as well as authentication.
Actually the function PASSWORD() of MySql should not have been used to hash passwords. From the MySql docu:
PASSWORD() is used by the authentication system in MySQL Server; you
should not use it in your own applications.
The problem is, that it uses an unsalted and fast hashing algorithm. This doesn't give you much protection, instead use a hash algorithm with a cost factor, like BCrypt SCrypt or PBKDF2. MySql does not offer such an algorithm, so you should calculate the hash in your development language.
I would strongly recommend to switch to a more safe algorithm, for backwards compatibility you can have a look at "defines" answer, he tried to implement the MySql behaviour:
MySQL Hashing Function Implementation
In MySQL documentation for PASSWORD function:
The PASSWORD() function is used by the authentication system in MySQL
Server; you should not use it in your own applications. For that
purpose, consider MD5() or SHA1() instead.
Why we shouldn't use this function in our application?
A few reasons I can think of
It's a fast hash (SHA1 I believe) which isn't a good property for password hashes.
They might change what hash it uses in a future version of MySQL, breaking your application. They've already done this once, hence the OLD_PASSWORD() function.
It doesn't naturally use a salt (although you could use a salt with it if you wanted to by appending it to the password before calling the PASSWORD function)
It's non-standard SQL, so if you ever need to port your app to another platform you'll need to come up with a replacement
Is storing a password in the DB using MySQL's password function just as bad as this?
http://money.cnn.com/2012/06/06/technology/linkedin-password-hack/?source=linkedin
The problem with SHA-1 is that it translates the same text the same way each time. So if your password is "password" and your friend's password is also "password," they will be hashed exactly the same way. That makes reversing the process to uncover the original password significantly easier.
I know it says SHA-1, but obviously any unsalted one way hash would have the same issue.
Is storing a password in the DB using MySQL's password function just as bad as this?
Yes.
Generally speaking you want to use a method that includes a salt, preferably unique for each user, and is slow to run to prevent brute force cracking. Bcrypt is the currently recommended way to go when storing passwords because it is intentionally (relatively) slow to create.
MySQL documentation says that you shouldn't be using the PASSWORD() function in your own application:
The PASSWORD() function is used by the authentication system in MySQL
Server; you should not use it in your own applications.
Internally, MySQL's PASSWORD() function utilizes SHA1(2), that's SHA1 twice. However, it doesn't utilize a salt. So, yes, it's still vulnerable to rainbow table attacks.