I'm using MySQL v5.7.17.
I'm trying to decrypt data I encrypted via Ruby using the aes-256-gcm algorithm.
So far I have this:
SELECT AES_DECRYPT(UNHEX(#encrypted_account_number), #key, #encrypted_account_number_iv);
Which would work, however is returning NULL because the encryption mode is does not match with what I used to encrypt.
I did some research and apparently MySQL does not support aes-256-gcm.
Is that true? If so, is there any way around it?
BTW - this is the command I'm using to switch between encryption modes:
SET ##session.block_encryption_mode = 'aes-256-ctr';
Thanks a lot!
mysql currently does not support ctr and ctr based modes like gcm, so I fear you will have to do this outside of your dbms.
http://dev.mysql.com/doc/refman/5.7/en/server-system-variables.html#sysvar_block_encryption_mode
Related
I am in the process of migrating databases from sqlite to mysql. Now that I've migrated the data to mysql, I'm not able to use my sqlalchemy code (in Python3) to access it in the new mysql db. I was under the impression that sqlalchemy syntax was database agnostic (i.e. the same syntax would work for accessing sqlite and mysql), but this appears not to be the case. So my question is: Is it absolutely required to use a DBAPI in addition to Sqlalchemy to read the data? Do I have to edit all of my sqlalchemy code to now read mysql?
The documentation says: The MySQL dialect uses mysql-python as the default DBAPI. There are many MySQL DBAPIs available, including MySQL-connector-python and OurSQL, which I think means that I DO need a DBAPI.
My old code with sqlite successfully worked like this with sqlite:
engine = create_engine('sqlite:///pmids_info.db')
def connection():
conn = engine.connect()
return conn
def load_tables():
metadata = MetaData(bind=engine) #init metadata. will be empty
metadata.reflect(engine) #retrieve db info for metadata (tables, columns, types)
inputPapers = Table('inputPapers', metadata)
return inputPapers
inputPapers = load_tables()
def db_inputPapers_retrieval(user_input):
result = engine.execute("select title, author, journal, pubdate, url from inputPapers where pmid = :0", [user_input])
for row in result:
title = row['title']
author = row['author']
journal = row['journal']
pubdate = row['pubdate']
url = row['url']
apa = str(author+' ('+pubdate+'). '+title+'. '+journal+'. Retrieved from '+url)
return apa
This worked fine and dandy. So then I tried to update it to work with the mysql db like this:
engine = create_engine('mysql://snarkshark#localhost/pmids_info')
At first when I tried to run my sample code like this, it complained because I didn't have MySqlDB. Some googling around informed me that MySqlDB does NOT work for Python 3. So then I tried pip installing pymysql and changing my engine statement to
engine = create_engine('mysql+pymysql://snarkshark#localhost/pmids_info')
which also ends up giving me various syntax errors when I try to adjust things.
So what I want to know, is if there is any way I can get my current syntax to work with mysql? Since the syntax is from sqlalchemy, I thought it would work perfectly for the exact same data in mysql that was previously in sqlite. Will I have to go through and update ALL of my db functions to use the syntax of the DBAPI?
This will sound like a dumb answer, but you'll need to change all the places where you're using database-specific behavior. SQLAlchemy does not guarantee that anything you do with it is portable across all backends. It leaks some abstractions on purpose to allow you to do things that are only available on certain backends. What you're doing is like using Python because it's cross-platform, then doing a bunch of os.fork()s everywhere, and then being surprised that it doesn't work on Windows.
For your specific case, at a minimum, you need to wrap all your raw SQL in text() so that you're not affected by the supported paramstyle of the DBAPI. However, there are still subtle differences between different dialects of SQL, so you'll need to use the SQLAlchemy SQL expression language instead of raw SQL if you want portability. After all that, you'll still need to be careful not to use backend-specific features in the SQL expression language.
I'm using Cakephp 3 using sqlserver as datasource server. I am sure there's no problem with my database connection.. as home.ctp prompts that I am connected to my database.. and I'm as well using migrations plugin to create my tables.. it seems like there is no problem working with these tools. but after I bake my MVC, I only got page full of errors..
for example
$bin\cake bake all tests
there are no errors I found and MVC are in its specific folder, testController.php, testTable, etc.
and in browsers
localhost:8765\tests
but all I got is page of different errors.. Im seeing
Error: SQLSTATE[42000]: [Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near the keyword 'desc'.
SELECT * FROM (SELECT Tests.id AS [Tests__id], Tests.desc AS [Tests__desc], (ROW_NUMBER() OVER (ORDER BY (SELECT NULL))) AS [_cake_page_rownum_] FROM tests Tests) _cake_paging_ WHERE _cake_paging_._cake_page_rownum_ <= :c0
and more errors on the left side.
I assume this is because of controllers with wrong queries or queries generated by bake is for mysql only. I just wanna know how to deal with this. is there a setting I forgot to do? please advice. I am new to Cakephp, and English is not my native language, sorry if I can't explain my question properly. thanks in advance.
As already mentioned by Vishal Gajjar in the comments, you are using the reserved keyword desc for your column name, hence the error, it's not bakes fault, it's yours.
In order to be able to use such reserved words, the column name needs to be quoted properly, however CakePHP 3 doesn't auto-quote by default anymore, as it's an expensive operation.
If you insist on using reserved words, enable identifier quoting via the quoteIdentifiers option in your app.php config, or enable it manually using the autoQuoting() (enableAutoQuoting() as of CakePHP 3.4) method of the DB driver.
See also
Cookbook > Database Access & ORM > Database Basics > Identifier Quoting
Cookbook > 3.x Migration Guide > New ORM Upgrade Guide > Identifier Quoting Disabled by Default
API > \Cake\Database\Driver::autoQuoting()
API > \Cake\Database\Driver::enableAutoQuoting()
You can use this code before problematic query:
$this->Tests->connection()->driver()->autoQuoting(true);
and when you are finished you can turn auto quoting off:
$this->Tests->connection()->driver()->autoQuoting(false);
So bad performance would be only on problematic query.
Use this :
SELECT * FROM (SELECT Tests.id AS [Tests__id], Tests.[desc] AS [Tests__desc],
(ROW_NUMBER() OVER (ORDER BY (SELECT NULL))) AS [_cake_page_rownum_] FROM tests Tests) _cake_paging_
WHERE _cake_paging_._cake_page_rownum_ <= :c0
If you do use a keyword, use it in square braces [ ]
I have a portion of a MySQL query which is designed to save a password using SHA512-CRYPT:
SELECT ENCRYPT('firstpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16)))
I need to port the application to use PostgreSQL, and as such, the statement is not compatible.
My attempt in PostgreSQL is like so:
SELECT CRYPT('firstpassword'::text, CONCAT('$6$', SUBSTRING(ENCODE(DIGEST(RANDOM()::text , 'sha1'), 'hex') FROM '.{16}$')))
When tested in component parts, each of these implementations appear identical, but as completed statements, the output differs.
I have found that the CONCAT statements (that generate the salt) appear to provide identical output.
If I try comparing the output of CRYPT or ENCRYPT using simple plaintext words, the output is identical. However, if I combine it with the output of a salt, the output differs;
MySQL:
SELECT ENCRYPT( 'firstpassword', '$6$ae73a5ca7d3e5b11' )
Produces: $6$ae73a5ca7d3e5b11$v/RbcEEx4VR37VMUF6gBnPNo2ptSyU...
PostgreSQL:
SELECT CRYPT('firstpassword'::text, '$6$ae73a5ca7d3e5b11'::text)
Produces: $6eTK2KpfoaQM
Can someone explain why these statements are diverging or suggest a better way to implement this MySQL query?
They diverge because they use different encryption algorithms.
The first thing you want to do is to get very familiar with the pg_crypto documentation. See http://www.postgresql.org/docs/9.2/static/pgcrypto.html
My suspicion is that you probably want to switch from something like encrypt/crypt to the use of SHA-2 for the actual password hashing so you can specifically control the algorithms and ensure compatibility on both sides.
SHA512-CRYPT is not supported by PostgreSQL core or the pgcrypto extension shipped with the main sources. The crypt function shown in the question comes from pgcrypto and does only support bf, md5, des and xdes (see "supported algorithm" in the documentation).
But you may use the shacrypt extension:
test=# create extension shacrypt ; -- (after installing from the sources)
CREATE EXTENSION
test=# select sha512_crypt('firstpassword', '$6$ae73a5ca7d3e5b11' );
sha512_crypt
------------------------------------------------------------------------------------------------------------
$6$ae73a5ca7d3e5b11$v/RbcEEx4VR37VMUF6gBnPNo2ptSyU3ys1sg6i8hhBrcfBeLY6hpsfvXR67bwwjMainpPEaLkYV6eO0ow0xVH.
which gives the same result as MySQL's SELECT ENCRYPT( 'firstpassword', '$6$ae73a5ca7d3e5b11' )
I have a complicated query and since I need that my module work on both mysql and postgres, I need to write two version of it.
Unfortunately, I don't know how I can check if the db I use is mysql or postgres, to know which query use. Do you know if a function can return this value?
As #kordirko says, one option is to query the server version: SELECT version(); will work on both MySQL and PostgreSQL, though not most other database engines.
Parsing version strings is always a bit fragile though, and MySQL returns just a version number like 5.5.32 wheras PostgreSQL returns something like PostgreSQL 9.4devel on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.7.2 20121109 (Red Hat 4.7.2-8), 64-bit. What do you do if you're connecting to a PostgreSQL-compatible database like EnterpriseDB Postgres Plus, or a MySQL-compatible database?
It's much safer to use the Drupal function for the purpose, DatabaseConnection::databaseType. This avoids a query round-trip to the DB, will work on databases that won't understand/accept SELECT version(), and will avoid the need to parse version strings.
You'll find this bug report useful; it suggests that the correct usage is Database::getConnection()->databaseType().
(I've never even used Drupal, I just searched for this).
As long as the abstract DatabaseConnection class extends PDO class, you can invoking pdo methods in order to know the current database driver.
For instance:
$conn = Database::getConnection();
print $conn->getAttribute($conn::ATTR_DRIVER_NAME); #returns mysql, pgsql...
There is a second way to do it using DatabaseConnection::driver():
print $conn->driver();
or DatabaseConnection::databaseType();
print $conn->databaseType();
Note that DatabaseConnection::driver() and DatabaseConnection::databaseType() are similar functions but not equals!
The return value from DatabaseConnection::driver() method depends on the implementation and other factors.
in the Drupal Database API page:
database.inc abstract public DatabaseConnection::driver()
This is not necessarily the same as the type of the database itself. For instance, there could be two MySQL drivers, mysql and mysql_mock. This function would return different values for each, but both would return "mysql" for databaseType().
In the most cases you just gonna want to use only
$conn->getAttribute($conn::ATTR_DRIVER_NAME)
or $conn->databaseType()
If you want get more specific properties, you should take advantage the PHP ReflectionClass features:
$conn = Database::getConnection();
$ref = new ReflectionClass($conn);
#ref->getProperties, ref->getConstants $ref->isAbstract...
Reference:
PDO::getAttribute
PDO::ATTR_DRIVER_NAME
Drupal Database API
Drupal Base Database API class
I am using the RJDBC package to connect to a MySQL (Maria DB) database in R on a Windows 7 machine and I am trying a statement like
select a as b
from table
but the column will always continue to be named "a" in the data frame.
This works normally with RODBC and RMySQL but doesn't work with RJDBC. Unfortunately, I have to use RJDBC as this is the only package that has no problem with the encoding of chinese, hebrew and so on letters (set names and so on don't seem to work with RODBC and RMySQL).
Has anybody experienced this problem?
I have run into the same frustrating issue. Sometimes the AS keyword would have its intended effect, but other times it wouldn't. I was unable to identify the conditions to make it work correctly.
Short Answer: (Thanks to Simon Urbanek (package maintainer for RJDBC), Yev, and Sebastien! See the Long Answer.) One thing that you may try is to open your JDBC connection using ?useOldAliasMetadataBehavior=true in your connection string. Example:
drv <- JDBC("com.mysql.jdbc.Driver", "C:/JDBC/mysql-connector-java-5.1.18-bin.jar", identifier.quote="`")
conn <- dbConnect(drv, "jdbc:mysql://server/schema?useOldAliasMetadataBehavior=true", "username", "password")
query <- "SELECT `a` AS `b` FROM table"
result <- dbGetQuery(conn, query)
dbDisconnect(conn)
This ended up working for me! See more details, including caveats, in the Long Answer.
Long Answer: I tried all sorts of stuff, including making views, changing queries, using JOIN statements, NOT using JOIN statements, using ORDER BY and GROUP BY statements, etc. I was never able to figure out why some of my queries were able to rename columns and others weren't.
I contacted the package maintainer (Simon Urbanek.) Here is what he said:
In the vast majority of cases this is an issue in the JBDC driver, because there is really not much RJDBC can do other than to call the driver.
He then recommended that I make sure I had the most recent JDBC driver for MySQL. I did have the most recent version. However, it got me thinking "maybe it IS a bug with the JDBC driver." So, I searched Google for: mysql jdbc driver bug alias.
The top result for this query was an entry at bugs.mysql.com. Yev, using MySQL 5.1.22, says that when he upgraded from driver version 5.0.4 to 5.1.5, his column aliases stopped working. Asked if it was a bug.
Sebastien replied, "No, it's not a bug! It's a documented change of behavior in all subsequent versions of the driver." and suggested using ?useOldAliasMetadataBehavior=true, citing documentation for the JDBC driver.
Caveat Lector: The documentation for the JDBC driver states that
useColumnNamesInFindColumn is preferred over useOldAliasMetadataBehavior unless you need the specific behavior that it provides with respect to ResultSetMetadata.
I haven't had the time to fully research what this means. In other words, I don't know what all of the ramifications are of using useOldAliasMetadataBehavior=true are. Use at your own risk. Does someone else have more information?
I don't know RJDBC, but in some cases when it is necessary to give permanent aliases to columns without renaming them, you can use VIEWs
CREATE OR REPLACE VIEW v_table AS
SELECT a AS b
FROM table
... and then ...
SELECT b FROM v_table
There is a separate function in the ResultSetMetaData interface for retrieving the column label vs the column name:
String getColumnLabel(int column) throws SQLException;
Gets the designated column's suggested title for use in printouts and
displays. The suggested title is usually specified by the SQL AS
clause. If a SQL AS is not specified, the value returned
fromgetColumnLabel will be the same as the value returned by the
getColumnName method.
Using getColumnLabel should resolve this issue (if not, check that your JDBC driver is following this spec).
e.g.
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
while(rs.next()) {
for (int i = 1; i < columnCount + 1; i++) {
String label = rsmd.getColumnLabel(i);
System.out.println(rs.getString(label));
}
}
This is the work around we use for R and SAP HANA via RJDBC:
names(result)[1]<-"b"
It's not the nicest work around, but since Aaron's solution does work for us, we went with this "solution".