I need to create a process to import a multi tabbed excel spreadsheet into SQL Server 2008R2. Each tab will be a different table in the database. This will need to be done weekly and imports should be automated. Ideally I want to pop the spreadsheet into a folder [or have some intern do it] and have sql run a procedure that looks in this folder, and adds the data to the tables in this db. I would also like to have another table that tracks the imports and date stamps them. I really have no idea where to even start here as I'm a pretty huge noob when it comes to tsql.
There is a nice article by microsoft - http://support.microsoft.com/kb/321686 - that outlines the processes involved.
The process is simply
SELECT * INTO XLImport3 FROM OPENDATASOURCE('Microsoft.Jet.OLEDB.4.0',
'Data Source=C:\test\xltest.xls;Extended Properties=Excel 8.0')...[Customers$]
Where XLImport3 is the table you want to import into and the datasource is the excel sheet you want to import from.
If you're limited solely to TSQL, the above two answers will show you some ideas. If you have access to either Data Tools or Business Intelligence, with SSIS, you can automate it with the assumption that each sheet in the Excel workbook matches each time. With SSIS, you'll use a Data Flow task and each sheet will be imported into the table that you want. When you're ready for the file the next week, you'll drop it into the folder and run the SSIS package.
However, if the sheet names change, (for instance, one week sheets are called Cats, Dogs, Rain and the next week it's Sulfur, Fire, Hell) then this would cause the package to break. Otherwise, if only the data within the worksheet change, then this can be completely automated with SSIS.
Example article: https://www.simple-talk.com/sql/ssis/moving-data-from-excel-to-sql-server---10-steps-to-follow/
Below is the code to insert data from a csv file into a given table. I don't what the full requirements are for the project, but if I were you I would just separate each table into a different file and then just run a proc that inserts data into each of the tables.
BULK
INSERT TABLE_NAME
FROM 'c:\filename.csv'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
insert into import_history ('filename', 'import_date') values ('your_file_name', getdate())
Also, for the table that tracks imports and timestamps them, you could just insert some data into that table after each bulk insert as seen above.
Also, here's a link to tutorial on bulk inserting from a csv file that may also help: http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
Its very simple. Right click the Database in Sql Server(2008), select Tasks and select Import Data
Now change the DataSource to Microsoft Excel. Chose the path of Excel file by clicking Browse button and click Next.
Chose the Sql Server instance and chose the database to which the excel to be imported.
Select Copy data from one or more tables or views and click Next.
Now select the sheets to be imported to Sql Server.
Click Next
Now click Finish
Now the wizard imports the data from Excel to Sql Server and click Close.
Here is the table
Related
I have a current package which basically copies an Excel template sheet, writes data into this from a table in the SQL database and then sends to Excel destination (ishare drive and server drives). We have repeated rows with specific data which need to be removed from the Excel sheet before it is sent out. We have a VBA code which can be run in Excel as a macro in order to achieve this result. I am wondering how I can automate this in SSIS in the data flow?
I have a current package which basically copies a an excel template
sheet, writes data into this from a table in the sql db
You can use a Data Flow Task where you import data from the SQL Server database:
Create a connection manager :
When you click on the New Connection Manager option, an Add SSIS Connection Manager window form will open to select the connections managers from the given list.
Then select the Provider, Server Name, and Database Name (you will point to a SQL Server database you can either select the table or use a query)
If you want to remove duplicate entries you can use Drag Transformation and Connect your OLE DB Source to it. Double Click on Sort Transformation and Choose the columns to Sort. Also Check the Check Box : Remove rows with duplicate sort values and then click OK.
If you have other specific rules, you can implement a T-SQL query when importing data from the begining (when using the OLE DB Source)
Then you can link it to an Excel destination.
I have a question which I'm sure has been asked before, but I don't know the terminology for this question. (Hence, I have tried searching for an answer to this question on this, but no luck.)
I am on SQL Server Management Studio 2008. I have a table that was created by an import of a flat file. At the beginning of every month, I want to update this table with a new version of the given flat file. The headers on the flat file / table will always stay the same, and no previous records will be lost. Data on previous records may change, and new records will be included.
What is the best way for doing this each month? Right now, my solution is to delete the current table and re-create it with an import of the new flat file. (Or, I could run a truncate, and then re-import.)
One of the faster methods would be to drop all indexes, truncate, re-import, and re-create all indexes. Note that with a flat file you could automate using SSIS, or you could use a BULK INSERT for a job schedule. For instance, if the file is in the same location every month and all the delimiters and details are the same, a procedure or TSQL script that BULK INSERTs the file would work when called by a job once a month on a schedule.
BULK INSERT MonthlyTable
FROM 'C:\MonthlyFileDrop\MonthlyFile.txt'
WITH (
FIELDTERMINATOR = ','
,ROWTERMINATOR = '0x0a'
,FIRSTROW=2
)
Another approach (one that I'm not partial to) would be to insert the data into a stage table, compare what data are not in the existing table from the staging table, populate those data, then re-index the existing table and drop the staging table.
I have a table students that contains 100 Records.
Now i'm going to Run a Query
select * from Students
I Copy the Students Data and Paste it in My Excel File
After some Time i inserted another 100 Records now again i'm going to run the Query copy the Data and Paste it in Excel File
Every Time copied and Paste in Excel File.
Is there any Alternate way or Query that directly save the Data in my Excel File
Simplest way would be accessing database table by creating a data connection from excel. I assume you are using Sql server database.
In Excel go to the Data tab
Select relevant data source in the "From Other Sources" drop down
Select Sql server and give necessary conection information and select the table you wanted while navigating through.
Select "Table" as prefered view option and it will show the table in excel.
When a change happen in the database simply right click the excel table and Refresh it
I want to develop an automation in SSIS.
Problem statement :
I have an excel sheet which has a single column.
Based on the values in that column (will be included as a search parameter in the SQL query) I need to fetch 2 or more columns from SQL server database
The results are to be stored in the same Excel sheet against the data obtained for that particular column.
I already have an excel macro for the same. But, now I want to develop a package for the same.
Please guide me through the necessary steps.
I will also keep trying to obtain the solution
Create an Excel Source and link it to your file
use a lookup component to perform a SQL select to obtain the missing data
Create an Excel destination to save your target data
I am trying to update one of my SQL tables with new columns in my source CSV file. The CSV records in this file are already in this SQL table, but this SQL table is lacking some of the new columns from this CSV file.
I already added the new columns to my SQL table structure via ALTER TABLE. But now I just need to import the data from this CSV file into the new columns. How can I do this? I am trying to use SSIS and SQL Server to accomplish this, but am pretty new to Excel.
This is probably too late to solve salvationishere's problem; though I'm posting this for future readers!
You could just generate the SQL INSERT/UPDATE/etc command by parsing the csv file (a simple python script will do).
You could alternatively use this online parser:
http://www.convertcsv.com/csv-to-sql.htm
(Hoping that it'd still be available when you click!)
to generate your SQL command. The interface is extremely straight forward and it does the entire job in an awesome way.
You have several options:
If you are loading the data into a non-production system where you can edit the target tables, you could load the data into a new table, rename the old table to obsolete, and rename the new table to the old table name.
You can load the data into a staging table and then write a SQL statement to update the target table from the staging table.
You can open the CSV file in Excel and write a formula to generate an update script, drag the formula down across all rows so that you get a separate update statement for each row, and then run the separate update statements in management studio.
You can truncate the target table and update your existing ssis package that imports the file to use the new columns if you have the full history in your CSV file.
There are more options, but any of the above would probably be more than adequate solutions.