I'm using Simba ODBC to create a connection with Google Big Query and using SSIS (Visual Studio 2019) to read and write information on Big Query. The connection works fine and when I use the ODBC Source with the query option, I'm able to get data from Big Query and used it inside SSIS. But when I use the list of tables, I get an error as below:
Exception of HRESULT: 0xC0014020
Error in Data Flow Task[ODBC Source [100]]: SQLSTATE: 42000, Message: [Simba][BigQuery] (70) Invalid query: Invalid dataset ID ""TEST"". Dataset IDs must be alphanumeric (plus underscores and dashes) and must be at most 1024 characters long.
I believe that this happens because the list of tables appear between ("), instead of (`).
Print of table list
The same happens when I use the ODBC Destination. Is there a way to change the format in which the table list appears ?
Obs.: On the Visual Studio 2015 this table list comes with (`) and I can connect with big query just fine.
I can see that the tool is sending "TEST" as the dataset, however, depending on if Visual Studio is using StandardSQL or LegacySQL, the dataset should be specified as:
# LegacySQL
FROM [myproject:TEST.TABLE_TEST]
# StandardSQL
FROM `myproject:TEST.TABLE_TEST`
I was wondering if Visual Studio accepts a custom query or can be parameterized to remove the quotes. If this doesn't help, could you please share the query that cause the error? I understand that there is a query option (I'm not familiar with Visual Studio) and it is not clear for me the exact moment when the tool returns the error, screenshot without sensitive information would be appreciated.
UPDATE:
You can review the following checkpoints that could help to verify that the Simba driver is correctly set up and it is not the cause of the reported error:
Installation. Check that you are using the last version of the driver. The last version usually contains improvements on the driver.
ODBC Configuration. For example, the Step 13 of the link you will be able to see a drop-down list with the datasets available and select one as the default. If you don't have issues is this step, then the issue could be in the tool that uses the ODBC connection.
Language Dialect. In here you case change between StandardSQL or LegacySQL as needed, for example, you can force your tool to use LegacySQL and use the characters [ and ] that I explained above.
Connection String. If your tool allows to use a string with the connection, you might want to use it and explicitly indicating the default Dataset (among other driver options).
Related
I'm importing a perfectly working SSIS project from TFS.
I have actually a problem with all the packages that contain a data FLOW with a date importation.
I get dozens of this error :
Validation error. DFT Get Date ODBC Source CodeDate2 [63]: The OLE DB provider used by the OLE DB adapter cannot convert between types "DT_BYTES" and "DT_DBDATE" for "Date".
and when I click on the odbc source editor, I have the following message:
the metadata of the following output columnsdoes not match the metadata of the external columns with which the output columns are associated:
Output "ODBC Source Output": "Date"
Do you want to replace the metadata of the output columns with the metadata of the external columns?
the fact is that works everywhere but on my computer.
is there an ole db provider component I'm lacking of something like that?
Downgrading will work, but if that's not possible for you, then rewriting your queries may also solve your problem.
In my case I had a Postgres query returning columns of type date. I just converted them all to timestamptz using ::timestamptz. At that point the columns changed from DT_BYTES to DT_DBTIMESTAMP, which was just fine for my purposes.
It might be related to the version of Visual Studio or SSDT.
Try to install SSDT 15.8.0(SSDT previous releases), and run the package in it.
I once saw similar posts on MSDN after the release of Visual Studio 15.9.2
Import from Teradata using ODBC gives VS_NEEDSNEWMETADATA error
ODBC Progress datatype problems after updating to VS 2017 15.9
Same here, I forced the type casting it in the select and it works :
SELECT
[...]
cast(release_date as datetime) as release_date,
[...]
FROM cm_wo
I'm trying to build a Data Integration job uses pass through to extract data from a view in a MySQL database.
Wev'e been using pass through a lot in the project, mostly extracting data from Redshift,
however with MySQL I was not able to do make it work properly.
It keeps complaining a table is missing even though when pass through is off, view is found and data is extracted...
tried every trick I know, starting from enabling case-sensitive DBMS object names, to manually remove single/double quotes from the statement just in case MySQL confuses confuses it with something else...
No luck.
ODBC driver is [MySQL][ODBC 5.3(a) Driver][mysqld-5.5.53].
Ran on a Windows environment.
Any idea how to solve this?
Thank you in advance.
EDIT
So, first of all, one correction (even though not that important - I extract from a view, not a table).
This is the code generated by SAS Create Table transformation, pass through enabled. I only put an asterisk instead of the full list of columns:
proc sql;
connect to ODBC
(
READBUFF=10000 DATASRC="cmp.web_api" AUTHDOMAIN="MYSQL_CMP_Auth"
);
create table work."W7ZZZKOC"n as
select
*
from connection to ODBC
(
select
V_BI_ACCOUNT.ACCOUNT_NAME,
V_BI_ACCOUNT.ACQUISITION_SOURCE__C,
V_BI_ACCOUNT.ZUORA__ACTIVE__C,
V_BI_ACCOUNT.ADDRESS_LINE_1__C,
V_BI_ACCOUNT.ADDRESS_LINE_2__C,
V_BI_ACCOUNT.ADDRESS_LINE_3__C,
V_BI_ACCOUNT.AGREEMENT_DATE,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_1__C,
V_BI_ACCOUNT.AGREEMENT_LEGAL_CLAUSE_2__C,
V_BI_ACCOUNT.PERSONBIRTHDATE,
V_BI_ACCOUNT.BLOCKED_REASON__C,
V_BI_ACCOUNT.BRAND__C,
V_BI_ACCOUNT.CPN__C,
V_BI_ACCOUNT.ACCCREATEDBYID,
V_BI_ACCOUNT.ACCCREATEDDATE,
V_BI_ACCOUNT.CURRENCY_PREFERENCE__C,
V_BI_ACCOUNT.CUSTOMER_FULL_NAME__PC,
V_BI_ACCOUNT.ACCOUNTID,
V_BI_ACCOUNT.ZUORA__CUSTOMERPRIORITY__C,
V_BI_ACCOUNT.DELIVERY_SALUTATION__C,
V_BI_ACCOUNT.DISPLAY_NAME,
V_BI_ACCOUNT.PERSONEMAIL,
V_BI_ACCOUNT.EMAILKEY__C,
V_BI_ACCOUNT.FACEBOOKKEY,
V_BI_ACCOUNT.FIRSTNAME,
V_BI_ACCOUNT.GENDER__C,
V_BI_ACCOUNT.PHONE,
V_BI_ACCOUNT.ACCLASTACTIVITYDATE,
V_BI_ACCOUNT.ACCLASTMODIFIEDDATE,
V_BI_ACCOUNT.LASTNAME,
V_BI_ACCOUNT.OTHER_EMAIL__C,
V_BI_ACCOUNT.PI_TYPE__C,
V_BI_ACCOUNT.ACCPARENTID,
V_BI_ACCOUNT.POSTCODE__C,
V_BI_ACCOUNT.PRIMARY_ACCOUNT_OF_THIS_CUSTOMER,
V_BI_ACCOUNT.ACCPRIMARY__C,
V_BI_ACCOUNT.ACCREASON_FOR_STATUS__C,
V_BI_ACCOUNT.ZUORA__SLA__C,
V_BI_ACCOUNT.ZUORA__SLASERIALNUMBER__C,
V_BI_ACCOUNT.SALUTATION,
V_BI_ACCOUNT.ACCSYSTEMMODSTAMP,
V_BI_ACCOUNT.PERSONTITLE,
V_BI_ACCOUNT.ZUORA__UPSELLOPPORTUNITY__C,
V_BI_ACCOUNT.X_CODE__C,
V_BI_ACCOUNT.ZUORA__ACCOUNT_ID__C,
V_BI_ACCOUNT.ZUORA__PAYMENTMETHODID__C,
V_BI_ACCOUNT.CITY,
V_BI_ACCOUNT.ORIGINAL_CREATED_DATE,
V_BI_ACCOUNT.SOURCE_SYSTEM_ID,
V_BI_ACCOUNT.STATUS,
V_BI_ACCOUNT.ZUORA__CONTACT_ID,
V_BI_ACCOUNT.ACCISDELETED,
V_BI_ACCOUNT.BILLING_ACCOUNT_NAME,
V_BI_ACCOUNT.ACZCREATEDDATE,
V_BI_ACCOUNT.ACZSYSTEMMODSTAMP,
V_BI_ACCOUNT.ACZLASTACTIVITYDATE,
V_BI_ACCOUNT.ZUORA__ACCOUNT__C,
V_BI_ACCOUNT.ZUORA__ACCOUNTNUMBER__C,
V_BI_ACCOUNT.ZUORA__AUTOPAY__C,
V_BI_ACCOUNT.ZUORA__BALANCE__C,
V_BI_ACCOUNT.ZUORA__CREDITCARDEXPIRATION__C,
V_BI_ACCOUNT.ZUORA__CURRENCY__C,
V_BI_ACCOUNT.ZUORA__MRR__C,
V_BI_ACCOUNT.ZUORA__PAYMENTTERM__C,
V_BI_ACCOUNT.ZUORA__PURCHASEORDERNUMBER__C,
V_BI_ACCOUNT.ZUORA__LASTINVOICEDATE__C,
V_BI_ACCOUNT.COUNTRY_NAME,
V_BI_ACCOUNT.COUNTRY_CODE,
V_BI_ACCOUNT.FAVOURITE_FOOTBALL_CLUB,
V_BI_ACCOUNT.COUNTY
from
web_api.V_BI_ACCOUNT as V_BI_ACCOUNT
);
%rcSet(&sqlrc);
disconnect from ODBC;
quit;
And again, when I extract data without pass through - works successfully,
I found out the problem was a column name exceeds 32 positions.
As SAS supports up column names up to 32,
the query fails to find PRIMARY_ACCOUNT_OF_THIS_CUSTOMER as the original column name is PRIMARY_ACCOUNT_OF_THIS_CUSTOMER__C.
EDIT
One more thing I found out is, MySQL doesn't like specifying schema name nor aliases.
Therefore,
From clause to only specify table name i.e : 'from v_bi_account' rather than 'web_api.v_bi_account'
and do not use aliases i.e use 'from v_bi_account' rather than 'from v_bi_account as v_bi_account'
Thank you guys so much for your help.
I am trying to query a MySQL linked server using SQL Server.
The below query runs just fine.
SELECT * FROM OPENQUERY([Linked_Server], 'SELECT * FROM Table_Name')
Is it possible to run the same query without using the OpenQuery call?
Found the answer here. Now I can the three dot notation query. Thanks
http://www.sparkalyn.com/2008/12/invalid-schema-error/
Go to the provider options screenIn SQL Server 2005 you can see the list of providers in a folder above the linked server (assuming you have appropriate permissions). Right click on MSDASQL and go to properties. In SQL Server 2000, the provider options button is in the dialog box where you create the linked server.
Check the box that says “level zero only”
you can use the statement below
select * from [linkedServerName]...[databaseName.TableName]
but before executing the code above ,, you have to do some changes ..
In the SSMS
SSMS -> Expand "linked servers" Folder -> open Provider folder -> find MSDASQL and gets it's property
Then check "Level Zero Only" press Ok
Then execute the above query and Enjoy it !!!
Try like this:
SELECT * FROM [Linked_Server]...[db_name.table_name]
Working properly, however there are the problems of converting data types.
Safer and more reliable to use is OPEQUERY.
SELECT * FROM OPENQUERY([Linked_Server], 'SELECT * FROM db_name.table_name')
You should be able to simply query the linked server directly.
select * from mylinkedserver.database.schema.mytable
EDIT:
Try with the three dot notation as noted in this post:
http://www.ideaexcursion.com/2009/02/25/howto-setup-sql-server-linked-server-to-mysql/
SELECT * FROM MYSQLAPP...tables
Msg 7399, Level 16, State 1, Line 1 The OLE DB provider "MSDASQL" for
linked server "MySQLApp" reported an error. The provider did not give
any information about the error. Msg 7312, Level 16, State 1, Line 1
Invalid use of schema or catalog for OLE DB provider "MSDASQL" for
linked server "MySQLApp". A four-part name was supplied, but the
provider does not expose the necessary interfaces to use a catalog or
schema.
This “four-part name” error is due to a limitation in the
MySQL ODBC driver. You cannot switch catalogs/schemas using dotted
notation. Instead, you will have to register another DSN and Linked
Server for the different catalogs you want to access. Be sure and
follow the three-dot notation noted in the example query.
There is an important point for using this:
SELECT * FROM [Linked_Server]...[db_name.table_name]
You must go on
Linked Server -> provider-> MSDASQL:
and make sure these three options have been checked
Dynamic Parameter
Level zero only
Allow inprocess
https://www.sqlteam.com/forums/topic.asp?TOPIC_ID=153024
This solution is great for querying small tables, however it seems that it doesn't use indexes, so getting even few rows from large tables, even by field indexed on the remote server takes ages.
So - correct me if I'm wrong - for large datasets it's still better to use OPENQUERY, as the query is evaluated and optimized on the remote server, using indexes and so on.
In case anyone is still having trouble with this...I had to go into the linked server properties -> Server Option and change RPC and RPC Out to true. Then I could run with like this [linked server]...[table]
We've just upgraded from Access 2003 to Access 2010 and string comparisons are failing with an invalid procedure call error when default conditions are used. I’ve recreated two presumably related problems in a new Access 2007 format database containing only the default table, a query with the SQL below and a module containing only the code below, so I seriously doubt that this is a corruption issue.
First the following sub fails on the If Then line with Run-time error 5: Invalid procedure call or argument
Option Compare Database
Option Explicit
Sub checkStrCmp()
Dim str As String
str = "s"
If str = "s" Then
MsgBox "works"
End If
End Sub
If I change Option Compare Database to Option Compare Text the sub works as expected, but this seems like a bad idea as I may want to preform text as well as numeric comparisons inside a single sub.
I’m also getting “Invalid procedure call” errors in string comparison functions inside of SQL. The Replace function is requiring the supposedly optional compare parameter.
Select replace("foo-bar-baz", "-", "|", 1,-1);
Generates the “Invalid procedure call” error
Setting the compare parameter to any of the available values (0 -3) works as expected:
SELECT replace("foo-bar-baz", "-", "|", 1,-1, 0);
produces “foo|bar|baz”
Has anyone else seen this? Is there a setting that needs to be tweaked? Any other ideas outside of “Database corruption” which is all I’ve been able to find via Google.
TIA
apoligies for the sloppy code blocks I can't for the life of me get them to work right.
UPDATE: I should have mentioned that I'm running XP Pro sp3.
The problem seems limited to databases I create on my box. When I opened the test database I created on my box from other workstations on our network I saw the issue, but was then unable to recreate it when creating a new database as described above on those workstations. The databases I created on the two other workstations (same OS and MS Office versions installed) also worked correctly when opened on my machine. I was also unable to recreate the issue when I inserted new modules in those DBs from my machine.
In short the problem seems to only exist on databases created on my machine (and in old 2003 format databases I've converted to 2007 format on my machine). My best guess is that my install is hosed but I’d like to have some idea of how and why before I approach IT with a request to reinstall Office. I’d also like to rule out a conflict with other software on my box.
Your code modules do not all need to share the same Option Compare setting. So you could place those procedures which should use text comparisons in a module which has Option Compare Text in its Declarations section.
However, I don't understand your statement, "I may want to preform text as well as numeric comparisons inside a single sub." According to Access' help topic, the Option Compare Statement is "Used at module level to declare the default comparison method to use when string data is compared". In other words, Option Compare has no effect on the comparisons of numeric values.
Edit: Since the problem is limited to Option Compare Database for database files created on only one machine, I'll suggest you check Access' "New database sort order" setting on that machine. Change it to a choice which starts with "General" if it is set to anything else. Then create a new database and see whether you still have the problem.
The reason for this suggestion is that Option Compare Database tells Access to use the database's codepage setting for sorting. And "New database sort order" can set the codepage to the one which never gives me such troubles. However, my understanding of codepage details is pretty shallow; I never change it and don't know what the consequences of other settings would be.
I'm trying to create an SSIS 2008 Data Source View that reads from an Ingres database via the ODBC driver for Ingres. I've downloaded the Ingres 10 Community Edition to get the ODBC driver, installed it, set up the data access server and a DSN on the server running SSIS.
If I connect to the SQL Server 2008 Database Engine on the server running SSIS, I can retrieve data from Ingres over the ODBC DSN by running the following command:
SELECT *
FROM OPENROWSET( 'MSDASQL'
, 'DSN=IngresODBC;UID=testuser;PWD=testpass'
, 'SELECT * FROM iitables')
So I am quite sure that the ODBC setup is correct.
If I try the same query with SQL Server style bracketed identifier quotes, I get an error, as Ingres doesn't support this syntax.
SELECT *
FROM OPENROWSET( 'MSDASQL'
, 'DSN=IngresODBC;UID=testuser;PWD=testpass'
, 'SELECT * FROM [iitables]')
The error is "[Ingres][Ingres 10.0 ODBC Driver][Ingres 10.0]line 1, Unexpected character '['.".
What I am finding is that I get the same error when I try to add tables from Ingres to an SSIS Data Source View. The initial step of selecting the ODBC Provider works fine, and I am shown a list of tables / views to add. I then select any table, and try to add it to the view, and get "ERROR [5000A] [Ingres][Ingres 10.0 ODBC Driver][Ingres 10.0]line 3, Unexpected character '['.".
Following Ed Harper's suggestion of creating a named query also seems to be stymied. If I put into my named query the following text:
SELECT *
FROM "iitables"
I still get an error: "ERROR [5000A] [Ingres][Ingres 10.0 ODBC Driver][Ingres 10.0]line 2, Unexpected character '['".
According to the error, the query text passed by SSIS to ODBC was:
SELECT [iitables].*
FROM
(
SELECT *
FROM "iitables"
)
AS [iitables]
It seems that SSIS assumes that bracket quote characters are acceptable, when they aren't. How can I persuade it not to use them? Double quotes are acceptable.
I don't know a way to change the quoted identifier, but you may be able to get around this by creating a blank DSV (click through the DSV wizard without adding any tables) then, rather than adding the tables to the DSV directly, adding them as named queries (right-click the empty DSV and select "New Named Query".
This enables you control the text of the query yourself, and set your own identifiers.
(I'm making this suggestion based on SSIS 2005, but I think 2008 works in a similar way.)