I have a requirement to read Access DB Extract in Talend. There is a component in Talend 'tAccessInput' which is used to read Access DB tables. But it requires a connection to Access Database.
However, for my requirement i am given an extract of Access db, say MasterTables.accdb and it is not a live database connection. I need to extract the tables that are present in this Access DB Extract using Talend.
Also, i know there is an option of exporting from Access DB Extract by opening the extract and exporting the required tables, but i do not wont to do it manually.
So, is there a component/steps that can help me achieve my requirement using Talend.
As guided by #iMezouar(Thanks for the inputs), i was able to use
tAccessInput component of Talend and achieve my requirement. Below are
the steps i followed:
Step 1- Configure tAccessInput component. Set the Database field with
the path of the .accdb extract. Leave the username and password blank
if the extract is not password protected
Step 2- In the Table Name field provide the name of the table that you
want to read from your access extract
Step 3- Go to edit schema section and add the column details of the
TableName provided in the above step
Step 4- Now go to Query Type and select Guess Query. Once you have
clicked this button, it will populate the Query section with the
relevant query
Step 5- Next connect the tAccessInput to tMap if you intend to do any
processing else connect it directly to an output component. In my case
its tFileOutputDelimited and your job is ready to run to extract data
from the access dump
Step 6- If you get a Warning issue in the Run Console i.e 'Error in
the metadata of the table: table's row count in the metadata is XXX
but XXY records have been found and loaded by UCanAccess. All will
work fine, but it's better to repair your database', just open the
access dump, go to Database Tools tab and select 'Compact and Repair
Database'. Then save the file. This will remove the warning issue also.
Related
I have tried to find an answer for this elsewhere but cannot, I hope someone can help me.
I am trying to import the MySQL sample database into Oracle SQL Developer.
I have an existing database/connection I want to dump it into. I have created an empty table named classicmodels in my existing connection. Yes that name is only 1 table within the sample db, correct. Ignore the error in naming convention.
When I R-click on it and try 'import data' I cannot import a .sql file, I can only do it with XL, CSV, etc.
When I try and run a script it found on dba.stackexchange
#\path\mysqlsampledatabase.sql , I get a series of 'please provide substitution value' messages, which does not make sense to me given that I am importing a database which is built for SQL (ie what reason is there to substitute).
Pictures below:
The 'UnseenCollection' is a single table I imported as a csv file. I need to import the mysqlsampledatabase file such that it shows up the same way, I can access all tables within the sample db.
Anyone can help I would appreciate it. I need the end result to be the entire mysqlsampledatabase to populate within the 'classicmodels' node.
Thank you.
connect to MySQL
connect to Oracle
for a single MySQL table, right-click, 'Copy to Oracle'
for a few tables, select, drag and drop onto Oracle connection (requires newer version of SQL Developer)
for an entire MySQL database, use the migration project feature
How can I easily set the path for the front end to connect to the backend database. Do I have to use the linked table manager every time ?
I found a tool here but it isn't very reliable. There is no simple config to set the back end path ?
When importing a table from another MS-Access database you can choose between a 'table' and a 'linked source'. If you now choose the 'linked source' and then you select the file of the other MS-Access file (where you select the table you want to link to ) you will get the table in your front-end Access as a normal table with one new property 'linked table'.
Now you can do everything on the selected data in both Access environments. The data will be synchronised as soon as you enter one record (make a change in one line and go to the next line). You do not need to visit the linked table manager any more at all.
I'm new to Talend. I want to make a script that transform my data (using an sql query) from a database and insert them to an other one.
Could you help me please with the list of components that I should use ? any tutorial ?
Thank you
Talend has many database specific components. You will find input and output components for all databases. input components are those which enables you to read data from database and bring it to talend and output components are those which allows you to write/insert data to database.
for example if your source and target are both oracle then -
toracleinput will be for reading from oracle db - (you have to mention connection details, sql query and schema for this component).
toracleoutput will be for inserting this data back to oracle database table. Here again you have to mention connection details, target table name, type of operation etc..
finally you have to connect flow of data from input to output component and if you need any intermediate transformations, joins etc you have to user other talend components in between like tMap etc..
I have a local SQL Server instance on which I created a Linked Server connection to a DB2 database named "DB2OurDatabase." In creating the Linked Server connection, I specified a UID and PWD that I use in various query tools or applications to query "[SchemaX].[TableX]."
I seemed to have success in creating the Linked Server: A Linked Server node Object by the name of "DB2OurDatabase" was created under the Linked Server node in SSMS and when I expand it, I am able to see the of tables in the database.
When I right mouse click on the [SchemaX].[TableX] table and select
"Script Table as => Select To ==> New Window", a new query window was opened with the text
--[DB2OurDatabase].[DataCenterCityName2_DB2OurDatabase].[SchemaX].[TableX]
contains no columns that can be selected or the current user does not have permissions on that object.
GO
I don't understand how I was able to create a Linked Server that can see the table names in the database but yet apparently seem to encounter what appears to be a lack of rights to query the table even tghough I am using same credentials that I have used in Squirell SQL query tool, for example, to query the table.
In SSMS, I tried to execute this
SELECT *
FROM [DB2OurDatabase].[DataCenterCityName2_DB2OurDatabase].[SchemaX].[TableX]
Error:
Msg 7314, Level 16, State 1, Line 1
The OLE DB provider "IBMDADB2" for linked server "DB2OurDatabase]" does not contain the table ""DataCenterCityName2_DB2OurDatabase"."SchemaX"."TableX"". The table either does not exist or the current user does not have permissions on that table.
I was a little surprised that the fully qualified table name included [DataCenterCityName2_DB2OurDatabase] since I did not specify this when I set up the Linked Server connection, but the name of the DataCenter city was correct so I took this as a further sign that the Linked Server connection was successful.
Nevertheless, I also tried to execute remove this level of the fully qualified table name:
SELECT *
FROM [DB2OurDatabase].[SchemaX].[TableX]
which resulted in this error.
Msg 208, Level 16, State 1, Line 1
Invalid object name 'DB2OurDatabase.[SchemaX].[TableX]'.
What do I need to do to create a DB2 Linked Server that lets me query the tables in the DB2 database? Here's my linked server properties:
I haven't investigate what are probably multiple ways to conenct and query DB2 from Sql Server, but this worked:
SELECT * FROM OPENQUERY(DB2OurDatabase, 'SELECT * FROM SchemaX.TableX')
Obviously, you modified the actual commands by replacing object names, so it's impossible to be sure, but the problem may be caused by your use of quoted identifiers (those square brackets), which essentially makes the object names case-sensitive. DB2 will by default create object (table, schema) names in uppercase, unless they are quoted. create table MySchema.MyTable... (unquoted) on the DB2 side will create the table MYSCHEMA.MYTABLE, and referencing it later from SSMS as [MySchema].[MyTable] (using quoted identifiers) will obviously fail.
These are the 3 steps that bring me to the solution:
Download and install Microsoft OLE DB Provider for DB2 Version 6.0 (this is the latest version but by the time you read this post there might be a new version now)
From the start menu open the Data Access Tool > File > New Data Source and complete all the steps: provide the notorious credentials like Server, Port, Database, User, Password. If unsure contact your DBA. Once completed test the connection and copy the Connection String
Now on SSMS go to Server Object > Linked Servers > New Linked Server and fill up like in picture setting up in the Provider string the string you copied before from the Data Access Tool:
Done, you are good to go now,
So I have this gigantic table, containing approx 7 million records, in MS Access (*.mdb), I want to transfer it into a much more workable MySQL format, and store it on my webserver. The file itself weighs 2GB.
The problem is, since the table is so large, it won't let me export it normally (Access says the limit is 65,536 records.)
I've tried some 3rd party software but to no avail.
Can anyone recommend a clean way of doing so, without damaging the data inside?
Thanks in advance for any help.
Install an ODBC driver for MySQL, if you don't have one already. The latest version is available here: Download Connector/ODBC
Create a DSN (Data Source Name) for your MySQL server from the Windows ODBC Data Source Administrator.
Then from Access 2003, select your table in the Database Window, and choose File->Export from Access' main menu. In the "Export Table 'yourtablename' To ..." dialog, select "ODBC Databases()" from the "Save as type" drop-down list (at the bottom of the dialog). The next dialog allows you to specify the name MySQL will use for the exported table, and it defaults to the Access table name. After you click OK, you will get another dialog, "Select Data Source", where you can select your DSN for MySQL. After you click OK on that dialog, you will probably get one more asking you for user name and password. Supply them, and click OK.
Hopefully your table will then transfer without errors. However, I've never done that operation with MySQL. It has worked for me with ODBC transfers to SQL Server and PostGreSQL. So I don't see why it wouldn't work with MySQL, too.
Also I've never attempted to export 7 million records in one go. If it chokes, we'll have to figure out a work-around.
If you're using Access 2007 instead of 2003, look for a similar option starting with the Export section of the ribbon.
I suggested this approach because my impression is this export will be a one-time deal, so I think the Access UI export method would be easiest. However, you can do essentially the same operation with VBA code using the DoCmd.TransferDatabase Method with your ODBC DSN.
Yet another alternative would be to create a compatible table structure in MySQL, create a link in Access to the MySQL destination table (using your DSN again), then run an "append query" from Access:
INSERT INTO link_to_mysql_table (field1, field2, field3, etc)
SELECT field1, field2, field3, etc
FROM access_table;
The append query approach could be useful in case the export chokes on 7 million records. You could add a WHERE clause to limit the SELECT query's output record set to a manageable chunk size, and then repeat with a different WHERE to specify another chunk.
Is that 7 million value after a compact + repair? I mean, if each record is about 120 chars in length, you can fit 32 million records in 2 gigs.
Also, I not aware of a limit of exporting 65,000 records, but only in regards to Excel.
So, you can/should be able to export the data to a csv, and then use a bulk text import in mySql to pull that data in. So, try exporting the table as csv. That should work.
I mean, you could link a table via odbc if you have a good local connection to the sql server, but if not, then I would export to csv (it is VERY fast). I would then zip the file (they zip fantastic). Upload file to server, and un-zip, and then use bulk text import. So, such a zipped file is VERY small and will save huge amounts of transfer time.
You can also consider using tab delimited as mySql also can import those, but a simple text file should work just fine.
I would use pyodbc as described in
http://en.wikibooks.org/wiki/Python_Programming/Database_Programming
download python 2.7 from
http://python.org/
download
http://code.google.com/p/pyodbc/
modify the following coede to set myfile.mdb and MyTable according to your table and file
save the code in a file translate.py
import csv
mycsv = csv.writer(open('result.csv', 'wb'), delimiter=',',
quotechar='"', quoting=csv.QUOTE_MINIMAL)
import pyodbc
DBfile = 'myfile.mdb'
conn = pyodbc.connect('DRIVER={Microsoft Access Driver (*.mdb)};DBQ='+DBfile)
cursor = conn.cursor()
SQL = 'SELECT * FROM MyTable;'
for row in cursor.execute(SQL): # cursors are iterable
mycsv.writerow(row)
cursor.close()
conn.close()
run python translate.py
Install MySQL on your own system and upsize to it rather than trying to use your local server. Then run an append query from your MySQL to the server instance.