I have created a SSIS package in order to transfer datas from ACCESS to SQL SERVER.
Source > SQL Command from "mdb" file joinning two tables
Destination > Flat table in SQL Server
I'm performing the JOIN in the source SQL Command because of the number of records in the ACCESS tables (~500k).
I tried to use SSIS join but it take ages doing the ORDERING before JOIN.
While running package in VS2010, it works great.
But after deploying and executing the package on my SQL Server 2014 the following error occurs.
No column information was returned by the SQL command.
Returned validation status "VS_NEEDSNEWMETADATA"."
I'm pretty sure my SQL command is correct (Working in VS and the preview button in the editor show me records).
I tried to disable ValidateMetadata but the same error still occurs, but at execution this time.
In the SQL Server 2014 I have other packages calling ACCESS data (But without join) and it works properly.
Thanks for your help,
Q.
ValidateMetadata is (generally) a good thing.
This error is being caused by the metadata on your source or destination (it's unclear from your question) being different.
At a guess, at least one of the columns in your SQL2014 database is of a different datatype (or length, or is nullable etc.) - there is a difference either way.
Related
We have a problem reading and loading some columns from AS/400 via OLE DB Provider for DB2 in SSIS 2015, the project is in SQL Server 2016.
We're migrating the project from an old server that is working with SQL Server 2008.
In the new one, when I preview the data in the Data Flow OLE DB Source, some columns have "System.Byte[]" as a value.
And I got the message "The metadata of the following output columns does not match thr metadata of the external columns with which the output columns are associated".
The external columns are read as DT_BYTES, while on the old server they were read as DT_STR.
The output columns are also expecting a DT_STR.
SSIS changed the metadata of the external columns, thus, making them unreadable.
I tried casting them as string, nothing works.
I tried setting the "ValidateExternalMetadata" to False, nothing.
The crazy thing is, with some other tables that have similar data types it reads them well, they have exactly the same data types as the failed ones.
There's no problem with the AS/400 itself as the old SSIS 2008 reads everything well.
So, I'm sure the problem is either with OL DB2 Provider, SSIS 2015 or SQL Server 2016.
I can't force it to read the external columns as STR with the Advanced Editor.
I also tried converting the data types in the output columns to DT_BYTES to match it, but still it didn't work.
It can't even read the columns from the beginning.
I tried a whole new package with no query, just reading the table as it is from the source, still the same problem.
I changed the Code Page to match the oiginal (1256), still nothing, then I changed it to match the new one (1252), still no success.
I'm really stuck and tried everything I could.
Any help please?
I finally solved it!
The solution is to open the connection manager, go to "All" tab, and match the Binary Codepage value (which is 0 by default) to the Host CCSID value.
I also set up the DBMS Platform to "DB2/AS400".
I have created Ole DB connection to execute different SQL task across SSIS package. Its working fine too.
In one of task where i need to do insert data into table, used SQLBulkCopy as i have dynamic tables and columns based on getting files from different sources.
SQLBulkCopy only works with SqlConnection, so i opened SqlConnection, executed SqlBulkCopy. This is also working fine.
After done with SqlBulkCopy, i have created Sql Task which updates metadata of inserted rows, for e.g. Count, Min & Max date and so on in different table.
This table is not getting updated and if i execute stored procedure from Sql Management Studio, it works as expected.
So my assumption is that, Ole DB connection is not able to get the latest data Data inserted thru SQL Connection.
I may be wrong, but not sure why i can see sql execution task successful but still table is not updated.
Am i missing anything here?
My bad.
Instead of passing data type as long (int in SQL), i was passing it as Varchar.
I was looking from last few hours and as soon as i post question here, it strikes me to check the data type.
Hope it will help somebody.
I have an SSIS package set up to export data from a SQL Server 2008 R2 table to a MySQL version of that table. The package executes however, I am getting about 1% of the rows failing to be exported.
My source connection uses the SQL statement
SELECT * FROM Table1
all of the columns are integers. An example of a row which is exported successfully is
2169,2680, 3532,NULL, 2169
compared to a row which fails
2168,2679,3532,NULL, 2168
virtually nothing different that I can ascertain.
Notably, if I change the source query to only attempt the transfer of a single failing row - ie.
SELECT * FROM Table1 WHERE ID = 2168
then the record is exported fine - it is only when part of a select which returns multiple rows that it fails. The same rows fail the export each time. I have redirected error rows to a text file which displays a -1071610801 error for the failing rows. This would apparently translate to:-
DTS_E_ADODESTERRORUPDATEROW: "An error has occurred while sending this row to destination data source."
which doesn't really add a great deal to my understanding of the issue!
I am wondering if there is a locking issue or something preventing given rows from being fetched or inserted correctly but if anyone has any ideas or suggestions on what might be causing this or even better how to go about resolving it they would be greatly appreciated. I am currently at a total loss...
Try to setup longer timeout (1 day) ot the mysql (ADO.NET) destination.
Well after much head scratching and attempting every work around that I could come up with I have finally found a solution for this.
In the end I switched out the MySQL connector for a different driver produced by devArt -dotConnect for MySql and, with a few minor exceptions (which I think I can resolve) all of my data is now exporting without error.
The driver is a paid for product unfortunately but in the end I'd have taken out a new mortgage to see all those tasks go green!
I am writing an SSIS package that has a conditional split from a SQL Server source that splits records to either be updated or inserted into a MYSQL database.
The SQL Server connection has provider .NET Provider for OldDB\SQL Server Native Client 10.0.
The MYSQL connection is a MYSQL ODBC 5.1 ADO.NET connection.
I was thinking about using the OLE DB Command branching off of the conditional split to update records but I connect use this and connect to the MYSQL database.
Does anyone know how to accomplish this task?
I would write to a staging table for updates including the PK and columns to be updated and then execute an UPDATE SQL statement using that table and the table to be updated. The alternative is to use the command for every row and that just doesn't seem to perform that well in my experience - at least compared to a nice fat batch insert and a single update command.
For that matter, I guess you could do without the conditional split altogether, write everything to a staging table and then use an UPDATE and INSERT in SQL back to back.
Probably, the following MSDN blog link might help you. I haven't tried this.
How do I UPDATE and DELETE if I don’t have an OLEDB provider?
The post suggests the following three options.
Script Component
Store the data in a Recordset
Use a custom component (like Merge destination component)
The author also had posted two other articles about MySQL prior to posting the above article.
Connecting to MySQL from SSIS
Writing to a MySQL database from SSIS
Hope that points you in the right direction.
I am trying to set up a third party MySQL as a Linked Server on SQL Server 2008.
When I try to access the data via a simple OpenQuery below I get an error.
SELECT * FROM OpenQuery(SERVERNAME, 'SELECT * FROM table')
The error message:
OLE DB provider 'MSDASQL' for linked server 'SERVERNAME' returned data that does not match expected data length for column '[MSDASQL].tablename'. The (maximum) expected data length is X, while the returned data length is Y.
NOTE:(where X > Y)
I've tried with another MySQL table that I built myself and it works fine. Therefore I assume the problem might be with the MySQL source.
EDIT: After digging a little deeper into the MySQL data I found several "invalid" dates such as zero dates. These could be a reason, however I have no way to change the data source so I must find a way to convince SQL to ignore it.
I recall something form previous versions of SQL where you could switch to "Lazy Schema Validation" to force SQL not to check this. However this option appears to be gone in 2008.
Any ideas how I could make this work?