I'm trying to build a Data Integration job uses pass through to extract data from a view in a MySQL database.
Wev'e been using pass through a lot in the project, mostly extracting data from Redshift,
however with MySQL I was not able to do make it work properly.
It keeps complaining a table is missing even though when pass through is off, view is found and data is extracted...
tried every trick I know, starting from enabling case-sensitive DBMS object names, to manually remove single/double quotes from the statement just in case MySQL confuses confuses it with something else...
No luck.
ODBC driver is [MySQL][ODBC 5.3(a) Driver][mysqld-5.5.53].
Ran on a Windows environment.
Any idea how to solve this?
Thank you in advance.
EDIT
So, first of all, one correction (even though not that important - I extract from a view, not a table).
This is the code generated by SAS Create Table transformation, pass through enabled. I only put an asterisk instead of the full list of columns:
proc sql;
connect to ODBC
(
READBUFF=10000 DATASRC="cmp.web_api" AUTHDOMAIN="MYSQL_CMP_Auth"
);
create table work."W7ZZZKOC"n as
select
*
from connection to ODBC
(
select
V_BI_ACCOUNT.ACCOUNT_NAME,
V_BI_ACCOUNT.ACQUISITION_SOURCE__C,
V_BI_ACCOUNT.ZUORA__ACTIVE__C,
V_BI_ACCOUNT.ADDRESS_LINE_1__C,
V_BI_ACCOUNT.ADDRESS_LINE_2__C,
V_BI_ACCOUNT.ADDRESS_LINE_3__C,
V_BI_ACCOUNT.AGREEMENT_DATE,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_1__C,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_2__C,
V_BI_ACCOUNT.PERSONBIRTHDATE,
V_BI_ACCOUNT.BLOCKED_REASON__C,
V_BI_ACCOUNT.BRAND__C,
V_BI_ACCOUNT.CPN__C,
V_BI_ACCOUNT.ACCCREATEDBYID,
V_BI_ACCOUNT.ACCCREATEDDATE,
V_BI_ACCOUNT.CURRENCY_PREFERENCE__C,
V_BI_ACCOUNT.CUSTOMER_FULL_NAME__PC,
V_BI_ACCOUNT.ACCOUNTID,
V_BI_ACCOUNT.ZUORA__CUSTOMERPRIORITY__C,
V_BI_ACCOUNT.DELIVERY_SALUTATION__C,
V_BI_ACCOUNT.DISPLAY_NAME,
V_BI_ACCOUNT.PERSONEMAIL,
V_BI_ACCOUNT.EMAILKEY__C,
V_BI_ACCOUNT.FACEBOOKKEY,
V_BI_ACCOUNT.FIRSTNAME,
V_BI_ACCOUNT.GENDER__C,
V_BI_ACCOUNT.PHONE,
V_BI_ACCOUNT.ACCLASTACTIVITYDATE,
V_BI_ACCOUNT.ACCLASTMODIFIEDDATE,
V_BI_ACCOUNT.LASTNAME,
V_BI_ACCOUNT.OTHER_EMAIL__C,
V_BI_ACCOUNT.PI_TYPE__C,
V_BI_ACCOUNT.ACCPARENTID,
V_BI_ACCOUNT.POSTCODE__C,
V_BI_ACCOUNT.PRIMARY_ACCOUNT_OF_THIS_CUSTOMER,
V_BI_ACCOUNT.ACCPRIMARY__C,
V_BI_ACCOUNT.ACCREASON_FOR_STATUS__C,
V_BI_ACCOUNT.ZUORA__SLA__C,
V_BI_ACCOUNT.ZUORA__SLASERIALNUMBER__C,
V_BI_ACCOUNT.SALUTATION,
V_BI_ACCOUNT.ACCSYSTEMMODSTAMP,
V_BI_ACCOUNT.PERSONTITLE,
V_BI_ACCOUNT.ZUORA__UPSELLOPPORTUNITY__C,
V_BI_ACCOUNT.X_CODE__C,
V_BI_ACCOUNT.ZUORA__ACCOUNT_ID__C,
V_BI_ACCOUNT.ZUORA__PAYMENTMETHODID__C,
V_BI_ACCOUNT.CITY,
V_BI_ACCOUNT.ORIGINAL_CREATED_DATE,
V_BI_ACCOUNT.SOURCE_SYSTEM_ID,
V_BI_ACCOUNT.STATUS,
V_BI_ACCOUNT.ZUORA__CONTACT_ID,
V_BI_ACCOUNT.ACCISDELETED,
V_BI_ACCOUNT.BILLING_ACCOUNT_NAME,
V_BI_ACCOUNT.ACZCREATEDDATE,
V_BI_ACCOUNT.ACZSYSTEMMODSTAMP,
V_BI_ACCOUNT.ACZLASTACTIVITYDATE,
V_BI_ACCOUNT.ZUORA__ACCOUNT__C,
V_BI_ACCOUNT.ZUORA__ACCOUNTNUMBER__C,
V_BI_ACCOUNT.ZUORA__AUTOPAY__C,
V_BI_ACCOUNT.ZUORA__BALANCE__C,
V_BI_ACCOUNT.ZUORA__CREDITCARDEXPIRATION__C,
V_BI_ACCOUNT.ZUORA__CURRENCY__C,
V_BI_ACCOUNT.ZUORA__MRR__C,
V_BI_ACCOUNT.ZUORA__PAYMENTTERM__C,
V_BI_ACCOUNT.ZUORA__PURCHASEORDERNUMBER__C,
V_BI_ACCOUNT.ZUORA__LASTINVOICEDATE__C,
V_BI_ACCOUNT.COUNTRY_NAME,
V_BI_ACCOUNT.COUNTRY_CODE,
V_BI_ACCOUNT.FAVOURITE_FOOTBALL_CLUB,
V_BI_ACCOUNT.COUNTY
from
web_api.V_BI_ACCOUNT as V_BI_ACCOUNT
);
%rcSet(&sqlrc);
disconnect from ODBC;
quit;
And again, when I extract data without pass through - works successfully,
I found out the problem was a column name exceeds 32 positions.
As SAS supports up column names up to 32,
the query fails to find PRIMARY_ACCOUNT_OF_THIS_CUSTOMER as the original column name is PRIMARY_ACCOUNT_OF_THIS_CUSTOMER__C.
EDIT
One more thing I found out is, MySQL doesn't like specifying schema name nor aliases.
Therefore,
From clause to only specify table name i.e : 'from v_bi_account' rather than 'web_api.v_bi_account'
and do not use aliases i.e use 'from v_bi_account' rather than 'from v_bi_account as v_bi_account'
Thank you guys so much for your help.
I am currently working on the website that uses ADODB library. In entire website all the queries are written in UPPERCASE.
The problem is when I run the query it doesn't work because of table name which is UPPERCASE. But when I change the table name to lowercase it works.
$sql = "SELECT * FROM MEMBERS where USERNAME = '$username'";
$db = ADONewConnection('mysql');
$db->debug = true;
$db->Connect(DB_HOSTNAME, DB_USERNAME, DB_PASSWORD, DB_NAME);
$resultFriends = $db->Execute($sql);
while ($row = $resultFriends->FetchRow()) {
var_dump($row);
die;
}
Here is the error I get:
ADOConnection._Execute(SELECT * FROM MEMBERS where USERNAME = 'fury', false) % line 1012, file: adodb.inc.php
ADOConnection.Execute(SELECT * FROM MEMBERS where USERNAME = 'fury') % line 15, file: index.php
Bear in mind I don't want to change the scripts. There are 1000 files and 10000 places.
Is there any library or are there any way that I can run this queries without error?
The version for live sire was linux kernel. but the new dev site is ubuntu.
I have done this on ubuntu/ mysql CML and it didn't work.
The solution is I had to reconfigure the mySql database in AWS/rdbs
You have to modify the “lower_case_table_names” parameter for your DB Instance(s). Prior to today, the lower_case_table_names parameter was not modifiable, with a system default of zero (0) or “table names stored as specified and comparisons are case sensitive.” Beginning immediately, values of zero and one (table names are stored in lowercase and comparisons are not case sensitive) are allowed. See the MySQL documentation for more information on the lower_case_table_names parameter.
The lower_case_table_names parameter can be specified via the rds-modify-db-parameter-group API. Simply include the parameter name and specify the desired value, such as in the following example:
rds-modify-db-parameter-group example --parameters "name=lower_case_table_names, value=1, method=pending-reboot" --region us-east-1
Support for modifying parameters via the AWS Management Console is expected to be added later this year.
setting the lower_case_table_names parameter via a custom DB Parameter Group and doing so before creating an associated DB Instance. Changing the parameter for existing DB Instances could cause inconsistencies with point-in-time recovery backups and with Read Replicas.
Amazon RDS
What's the SQL command to undo:
USE db;
The syntax I see everywhere is:
USE [db] ;
implying that I can leave out the db part. Not so - this is a syntax error however (maybe just syntax errors in the SQL syntax syntax?).
edit
The programming problem this is causing is that I can't reset the environment in which subsequent commands run. I could reset my DB connection, but this seems efficient.
cmdX; // Works
vs.
cmdX;
cmdY; // May fail because command X upset some state.
cmdX should clean up after itself and put things back where it found them.
Analogously:
cd ./a
doX()
cd ../
doY() // Y expects to not be in a?
I don't think you can. The documentation doesn't say the parameter is optional. It says:
The database remains the default until the end of the session or another USE statement is issued:
So if you want to drop the default, end your session and start a new one without selecting a DB.
What programming problem is this causing for you?
The database argument is not optional.
mysql> use
ERROR:
USE must be followed by a database name
I'm not sure where you saw this command with square brackets around the argument. That is not shown at the documentation page: http://dev.mysql.com/doc/refman/5.6/en/use.html
Microsoft SQL Server uses square brackets around identifiers (as opposed to a style to indicate an optional argument), but the MS SQL documentation for USE also doesn't show it: http://msdn.microsoft.com/en-us/library/ms188366.aspx
What would it mean to "undo" a USE command? Would it be like cd - in bash, making the previous default database again the default? There is no such command in MySQL for this. It doesn't remember what was your previous default database. If you want to return, you just have to USE that database and name it explicitly.
I have an issue with Entity Framework 5.0. I'm working with Silverlight 5 and MySQL 5.6 too.
I need to set an environment MySQL variable before each connexion to the MySQL server.
E.g
SET #my_var = 'test';
Under Mysql I don't have any issues.
The following raises an EntityFrameworkException (syntax error near '#').
this.ObjectContext.CreateQuery<object>(" SET #my_var = 'test' ");
OR
this.ObjectContext.CreateQuery<object>(" CALL set_my_var('test') ");
This last method raises a MySQLException saying that a DataReader is already open and need to be closed.
this.ObjectContext.ExecuteStoreQuery<object>(" CALL set_my_var('test') ", null);
I also tried to set a MySQL system environment (no '#') with the same result every time.
Any help will be much appreciated !
Thank you.
I tried so many things that I misspelled my variable in my code.
So the following finaly worked : ctx.ExecuteStoreCommand("SET #my_var = 'test'");
I decided to leave the instruction in the method Initialize of my domain service. This method is inherited of the LinqToEntitiesDomainService class.
But you need to set Allow User Variables=True in your MySQL connection string
(ref : Is it possible to use a MySql User Defined Variable in a .NET MySqlCommand?)
You simply need to use a recent version of the MySQL Connector because older versions use the '#' mark to define SQL parameters so it could conflict with custom variables. Now it uses the '?' mark : http://dev.mysql.com/doc/refman/5.0/es/connector-net-examples-mysqlcommand.html
My library was already up to date (6.6.5).
Thank you for the help !
Since your statement is not a query (i.e. does not return any result) you should use ExecuteStoreCommand. Something like this should work:
ctx.ExecuteStoreCommand("SET #my_var = 'test'")
Some days ago I asked a question about my problem and I was advised to use CONCAT_WS function. I am using CONCAT_WS on my local mysql database and it is working perfectly. But it is not working on server(application hosted) and generate the following error.
FUNCTION test.CONCAT_WS does not exist
Here test in error string is my database name on server.
My query is like this:
SELECT * FROM patient WHERE CONCAT_WS (',', LastName,FirstName,BirthDate ) NOT IN ('Abdul,Quddus,2000-09-30','Wasim,Akram,1993-09-12');
Can someone tell me the problem or suggest me another solution asked in linked question above ?
Thanks
The easiest way to fix it is by removing the whitespace between the function name and the parenthesis, i.e. CONCAT_WS(...) instead of CONCAT_WS (...).
From the MySQL Manual:
By default, there must be no
whitespace between a function name and
the parenthesis following it. This
helps the MySQL parser distinguish
between function calls and references
to tables or columns that happen to
have the same name as a function.
...
You can tell the MySQL server to
accept spaces after function names by
starting it with the
--sql-mode=IGNORE_SPACE option.
Also, this behavior depends on the MySQL version, this is why it works on one server and doesn't work on another, quote from the "Function Name Parsing and Resolution" manual page:
The number of function names affected
by IGNORE_SPACE was reduced
significantly in MySQL 5.1.13, from
about 200 to about 30.