Entity Framework converts StartsWith to MySQL's Locate, MySQL's Locate doesn't use index - mysql

I'm using Entity Framework with MySQL, and my Linq Query:
db.Persons.Where(x => x.Surname.StartsWith("Zyw")).ToList();
..is producing the SQL:
SELECT PersonId, Forename, Surname
FROM Person
WHERE (LOCATE('Zyw', Surname)) = 1
...and it would seem that this doesn't make use of the index on Surname.
If LOCATE is replaced with the equivalent LIKE, the query speedily returns the required results. As it is it takes all afternoon.
Why is Entity Framework and its connecting drivers opting for this wierd LOCATE function / how can I make it use LIKE instead / why is MySQL making a poor index decision for the LOCATE function / how can I make it better?
Update:
I'm afraid I was guilty of over simplifying my code for this post, the Linq producing the error is in fact:
var target = "Zyw";
db.Persons.Where(x => x.Surname.StartsWith(target)).ToList();
If target term is hard coded, the SQL generated does indeed use LIKE, but with a variable term the SQL changes to use LOCATE.
This is all using the latest generally available MySQL for Windows as delivered by MySQL Installer 5.6.15.
Update:
A couple more notes to go with the bounty; am using:
Visual Studio 2010
EntityFramework 6.0.2
MySQL Installer 5.6.15,
which in turn gives:
MySql.Data 6.7.4
MySql.Data.Entities 6.7.4
The Entity Framework code is generated database first style.
I've also tried it with the latest connector from Nuget (MySql.Data 6.8.3) and the problem is still there.

It's likely your problem is caused by:
You are using an older connector with the bug.
You have a special case (using a variable to hold the .Contains search) described as a bug here
Does your case fall into any of those?

This looks like a regression of MySQL bug #64935 to me.
I can confirm that, using the same builds of EF6 and MySQL Connector, I'm getting the same SQL generated too:
context.stoppoints.Where(sp => sp.derivedName.StartsWith(stopName));
...logs as:
SELECT
`Extent1`.`primaryCode`,
...
`Extent1`.`stop_timezone`
FROM `stoppoints` AS `Extent1`
WHERE (LOCATE(#p__linq__0, `Extent1`.`derivedName`)) = 1
Entity Framework: 6.0.2
MySQL Connector.Net: 6.8.3
I have reported this as a MySQL bug regression.

Related

SSIS ODBC Simba - Error when access table list on ODBC Source \ Destination

I'm using Simba ODBC to create a connection with Google Big Query and using SSIS (Visual Studio 2019) to read and write information on Big Query. The connection works fine and when I use the ODBC Source with the query option, I'm able to get data from Big Query and used it inside SSIS. But when I use the list of tables, I get an error as below:
Exception of HRESULT: 0xC0014020
Error in Data Flow Task[ODBC Source [100]]: SQLSTATE: 42000, Message: [Simba][BigQuery] (70) Invalid query: Invalid dataset ID ""TEST"". Dataset IDs must be alphanumeric (plus underscores and dashes) and must be at most 1024 characters long.
I believe that this happens because the list of tables appear between ("), instead of (`).
Print of table list
The same happens when I use the ODBC Destination. Is there a way to change the format in which the table list appears ?
Obs.: On the Visual Studio 2015 this table list comes with (`) and I can connect with big query just fine.
I can see that the tool is sending "TEST" as the dataset, however, depending on if Visual Studio is using StandardSQL or LegacySQL, the dataset should be specified as:
# LegacySQL
FROM [myproject:TEST.TABLE_TEST]
# StandardSQL
FROM `myproject:TEST.TABLE_TEST`
I was wondering if Visual Studio accepts a custom query or can be parameterized to remove the quotes. If this doesn't help, could you please share the query that cause the error? I understand that there is a query option (I'm not familiar with Visual Studio) and it is not clear for me the exact moment when the tool returns the error, screenshot without sensitive information would be appreciated.
UPDATE:
You can review the following checkpoints that could help to verify that the Simba driver is correctly set up and it is not the cause of the reported error:
Installation. Check that you are using the last version of the driver. The last version usually contains improvements on the driver.
ODBC Configuration. For example, the Step 13 of the link you will be able to see a drop-down list with the datasets available and select one as the default. If you don't have issues is this step, then the issue could be in the tool that uses the ODBC connection.
Language Dialect. In here you case change between StandardSQL or LegacySQL as needed, for example, you can force your tool to use LegacySQL and use the characters [ and ] that I explained above.
Connection String. If your tool allows to use a string with the connection, you might want to use it and explicitly indicating the default Dataset (among other driver options).

CakePHP 3 new ORM MySQL keyword errors [duplicate]

I'm using Cakephp 3 using sqlserver as datasource server. I am sure there's no problem with my database connection.. as home.ctp prompts that I am connected to my database.. and I'm as well using migrations plugin to create my tables.. it seems like there is no problem working with these tools. but after I bake my MVC, I only got page full of errors..
for example
$bin\cake bake all tests
there are no errors I found and MVC are in its specific folder, testController.php, testTable, etc.
and in browsers
localhost:8765\tests
but all I got is page of different errors.. Im seeing
Error: SQLSTATE[42000]: [Microsoft][SQL Server Native Client 11.0][SQL Server]Incorrect syntax near the keyword 'desc'.
SELECT * FROM (SELECT Tests.id AS [Tests__id], Tests.desc AS [Tests__desc], (ROW_NUMBER() OVER (ORDER BY (SELECT NULL))) AS [_cake_page_rownum_] FROM tests Tests) _cake_paging_ WHERE _cake_paging_._cake_page_rownum_ <= :c0
and more errors on the left side.
I assume this is because of controllers with wrong queries or queries generated by bake is for mysql only. I just wanna know how to deal with this. is there a setting I forgot to do? please advice. I am new to Cakephp, and English is not my native language, sorry if I can't explain my question properly. thanks in advance.
As already mentioned by Vishal Gajjar in the comments, you are using the reserved keyword desc for your column name, hence the error, it's not bakes fault, it's yours.
In order to be able to use such reserved words, the column name needs to be quoted properly, however CakePHP 3 doesn't auto-quote by default anymore, as it's an expensive operation.
If you insist on using reserved words, enable identifier quoting via the quoteIdentifiers option in your app.php config, or enable it manually using the autoQuoting() (enableAutoQuoting() as of CakePHP 3.4) method of the DB driver.
See also
Cookbook > Database Access & ORM > Database Basics > Identifier Quoting
Cookbook > 3.x Migration Guide > New ORM Upgrade Guide > Identifier Quoting Disabled by Default
API > \Cake\Database\Driver::autoQuoting()
API > \Cake\Database\Driver::enableAutoQuoting()
You can use this code before problematic query:
$this->Tests->connection()->driver()->autoQuoting(true);
and when you are finished you can turn auto quoting off:
$this->Tests->connection()->driver()->autoQuoting(false);
So bad performance would be only on problematic query.
Use this :
SELECT * FROM (SELECT Tests.id AS [Tests__id], Tests.[desc] AS [Tests__desc],
(ROW_NUMBER() OVER (ORDER BY (SELECT NULL))) AS [_cake_page_rownum_] FROM tests Tests) _cake_paging_
WHERE _cake_paging_._cake_page_rownum_ <= :c0
If you do use a keyword, use it in square braces [ ]

MySQL Connector/J v5.x upgrade: query now returning byte[] instead of String

I just updated the JDBC driver for my application from
mysql-connector-java-3.1.12-bin.jar
to
mysql-connector-java-5.1.34-bin.jar.
With the v3.x driver, this kind of a query works:
select concat("<a href>", count(sakila.payment.payment_id), "</a>")
from sakila.payment;
But now with the new v5.x driver, the query only works with a cast().
select cast(concat("<a href>", count(sakila.payment.payment_id), "</a>")
as char(30)) from sakila.payment;
Is there any property in the MySQL database I can change?
I don't want to change hundreds of queries like that.
I suspect that you will have to bite the bullet and update your code. There is a bug report here that seems to match your circumstances and the status of that bug report is "Won't fix". The response from the developers ([4 Apr 2007 17:43] Reggie Burnett) was:
This is something that we can't really fix. Let me explain.
MySQL has several issues when it comes to reporting whether a result if binary or not. This was very bad on MySQL versions prior to 5.0 but it's still a problem even today. The SQL you reported is returned by MySQL as binary when it obviously is not. The connector can't know for sure. With 5.0.5 and 5.0.6, we tried to make a "best guess" but that code caused more problems than it solved, so with 5.0.7 we have rolled it out. Your SQL will return string properly with 5.0.7, but that doesn't mean it's fixed. In fact, it returns string because we are ignoring the binary flag so that means you could generate valid SQL that should return binary and 5.0.7 will return string.
Until the server is fixed, the connector just can't always do the right thing. I hope this has cleared it up somewhat.

Drupal : How can I know if the db is mysql or postgres

I have a complicated query and since I need that my module work on both mysql and postgres, I need to write two version of it.
Unfortunately, I don't know how I can check if the db I use is mysql or postgres, to know which query use. Do you know if a function can return this value?
As #kordirko says, one option is to query the server version: SELECT version(); will work on both MySQL and PostgreSQL, though not most other database engines.
Parsing version strings is always a bit fragile though, and MySQL returns just a version number like 5.5.32 wheras PostgreSQL returns something like PostgreSQL 9.4devel on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.7.2 20121109 (Red Hat 4.7.2-8), 64-bit. What do you do if you're connecting to a PostgreSQL-compatible database like EnterpriseDB Postgres Plus, or a MySQL-compatible database?
It's much safer to use the Drupal function for the purpose, DatabaseConnection::databaseType. This avoids a query round-trip to the DB, will work on databases that won't understand/accept SELECT version(), and will avoid the need to parse version strings.
You'll find this bug report useful; it suggests that the correct usage is Database::getConnection()->databaseType().
(I've never even used Drupal, I just searched for this).
As long as the abstract DatabaseConnection class extends PDO class, you can invoking pdo methods in order to know the current database driver.
For instance:
$conn = Database::getConnection();
print $conn->getAttribute($conn::ATTR_DRIVER_NAME); #returns mysql, pgsql...
There is a second way to do it using DatabaseConnection::driver():
print $conn->driver();
or DatabaseConnection::databaseType();
print $conn->databaseType();
Note that DatabaseConnection::driver() and DatabaseConnection::databaseType() are similar functions but not equals!
The return value from DatabaseConnection::driver() method depends on the implementation and other factors.
in the Drupal Database API page:
database.inc abstract public DatabaseConnection::driver()
This is not necessarily the same as the type of the database itself. For instance, there could be two MySQL drivers, mysql and mysql_mock. This function would return different values for each, but both would return "mysql" for databaseType().
In the most cases you just gonna want to use only
$conn->getAttribute($conn::ATTR_DRIVER_NAME)
or $conn->databaseType()
If you want get more specific properties, you should take advantage the PHP ReflectionClass features:
$conn = Database::getConnection();
$ref = new ReflectionClass($conn);
#ref->getProperties, ref->getConstants $ref->isAbstract...
Reference:
PDO::getAttribute
PDO::ATTR_DRIVER_NAME
Drupal Database API
Drupal Base Database API class

Renaming columns in a MySQL select statement with R package RJDBC

I am using the RJDBC package to connect to a MySQL (Maria DB) database in R on a Windows 7 machine and I am trying a statement like
select a as b
from table
but the column will always continue to be named "a" in the data frame.
This works normally with RODBC and RMySQL but doesn't work with RJDBC. Unfortunately, I have to use RJDBC as this is the only package that has no problem with the encoding of chinese, hebrew and so on letters (set names and so on don't seem to work with RODBC and RMySQL).
Has anybody experienced this problem?
I have run into the same frustrating issue. Sometimes the AS keyword would have its intended effect, but other times it wouldn't. I was unable to identify the conditions to make it work correctly.
Short Answer: (Thanks to Simon Urbanek (package maintainer for RJDBC), Yev, and Sebastien! See the Long Answer.) One thing that you may try is to open your JDBC connection using ?useOldAliasMetadataBehavior=true in your connection string. Example:
drv <- JDBC("com.mysql.jdbc.Driver", "C:/JDBC/mysql-connector-java-5.1.18-bin.jar", identifier.quote="`")
conn <- dbConnect(drv, "jdbc:mysql://server/schema?useOldAliasMetadataBehavior=true", "username", "password")
query <- "SELECT `a` AS `b` FROM table"
result <- dbGetQuery(conn, query)
dbDisconnect(conn)
This ended up working for me! See more details, including caveats, in the Long Answer.
Long Answer: I tried all sorts of stuff, including making views, changing queries, using JOIN statements, NOT using JOIN statements, using ORDER BY and GROUP BY statements, etc. I was never able to figure out why some of my queries were able to rename columns and others weren't.
I contacted the package maintainer (Simon Urbanek.) Here is what he said:
In the vast majority of cases this is an issue in the JBDC driver, because there is really not much RJDBC can do other than to call the driver.
He then recommended that I make sure I had the most recent JDBC driver for MySQL. I did have the most recent version. However, it got me thinking "maybe it IS a bug with the JDBC driver." So, I searched Google for: mysql jdbc driver bug alias.
The top result for this query was an entry at bugs.mysql.com. Yev, using MySQL 5.1.22, says that when he upgraded from driver version 5.0.4 to 5.1.5, his column aliases stopped working. Asked if it was a bug.
Sebastien replied, "No, it's not a bug! It's a documented change of behavior in all subsequent versions of the driver." and suggested using ?useOldAliasMetadataBehavior=true, citing documentation for the JDBC driver.
Caveat Lector: The documentation for the JDBC driver states that
useColumnNamesInFindColumn is preferred over useOldAliasMetadataBehavior unless you need the specific behavior that it provides with respect to ResultSetMetadata.
I haven't had the time to fully research what this means. In other words, I don't know what all of the ramifications are of using useOldAliasMetadataBehavior=true are. Use at your own risk. Does someone else have more information?
I don't know RJDBC, but in some cases when it is necessary to give permanent aliases to columns without renaming them, you can use VIEWs
CREATE OR REPLACE VIEW v_table AS
SELECT a AS b
FROM table
... and then ...
SELECT b FROM v_table
There is a separate function in the ResultSetMetaData interface for retrieving the column label vs the column name:
String getColumnLabel(int column) throws SQLException;
Gets the designated column's suggested title for use in printouts and
displays. The suggested title is usually specified by the SQL AS
clause. If a SQL AS is not specified, the value returned
fromgetColumnLabel will be the same as the value returned by the
getColumnName method.
Using getColumnLabel should resolve this issue (if not, check that your JDBC driver is following this spec).
e.g.
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
while(rs.next()) {
for (int i = 1; i < columnCount + 1; i++) {
String label = rsmd.getColumnLabel(i);
System.out.println(rs.getString(label));
}
}
This is the work around we use for R and SAP HANA via RJDBC:
names(result)[1]<-"b"
It's not the nicest work around, but since Aaron's solution does work for us, we went with this "solution".