We have a nightly process that creates scrubbed copies of our databases so that our developers can use them in their dev environment. They restore the databases and then run the database deployment from a database project in Visual Studio 2010 to get updated schemas.
I'm working on an SSIS package that will copy data out of the primary database into staging tables and then into an archive for reporting. Once completed I'd like the database deployment to kick off the SSIS package so that it updates the archive database, is there any way to do this?
The source and target databases will both be in the solution. I'm thinking the source database deployments will stay the same, with no dependencies on the target or vice versa. Deployments on the target db project are where the package execution would get called right after post-deployment.
Unless I'm missing something, add a step into your nightly process to start the SSIS package. SQL Agent has a job step explicitly for starting SSIS packages but if your process isn't that, just make an operating system call to DTEXEC
dtexec.exe /file C:\RepliScrubberPackage.dtsx
Related
We have an SSIS project in Visual Studio (2019 Community) that we deploy to our SQL server (2016) using project deployment. We have a couple of hundred packages and we most often deploy single packages at a time.
We had been doing most of our development in a single Azure Git master branch since these changes were separate from most of our other development. A few changes were done in a different development branch and then that single package was deployed from there. It should not have caused a problem because that single package was new and not being used in the master at all. However, when that single package was deployed it removed a couple of new packages from our live server that had been deployed from the master branch but had not been in the development branch when it was initially created.
I thought that a single package deployment only deployed that single package you selected or the few packages you selected if you chose a few of them. It seems like in addition to deploying that package it deleted packages that were on the live server but not in the dev branch that we deployed from.
We deploy from Visual Studio by right-clicking our package or packages and then choosing Deploy Package from the context menu. That process creates an .ispac file when we are deploying to our development SQL Server. That .ispac file is copied to the live server and then run directly on the live SQL server.
This should not be deploying all packages, correct? Just those highlighted? Does it remove missing packages if I only deploy a single package at a time but the packages in my project in VS and the project in the SSIS catalog do not match?
It depends!
For 2012 and 2014, the only option for deployment to the SSISDB was "Project" deployment. The entirety of the .ispac file is applied to the SSISDB - whether package1 had changed or not.
With 2016+, you have the ability to do Project or Incremental Package deployment. In the "Review" tab of the deployment wizard, in the Summary section you can see the command line indicates /ModelType:Package That is the indication it is deploying package(s) versus the whole project. See figure 5 of
https://www.sqlservercentral.com/articles/incremental-package-deployment-%E2%80%93-a-ssis-2016-feature
If you're deploying from the .ispac, then there should be a selector in the Select Source tab (not shown in referenced article) that allows you to do full or incremental deployment but the ModelType in the Review tab will identify which approach was selected. I suspect someone did a full/project deploy.
But, the nice thing about the SSISDB is that you can use SSMS to roll the project back to the previous state so no harm, no foul, no lost packages.
I am using VS 2017 SSDT 15.9.20 to create an SSIS package. The package is originally created in Server A where SSDT is installed and the SQL server resides in Server A. So I was ale to create the package in Server A. I created a SQL job in Server A and linked directly to run the DTSX file without deploying it.
What my package does:
In server A my package will read the excel in the location C:\Users\xxx\Documents\myproj\excelfile.xls
and will create excel sheets inside C:\Users\xxx\Documents\myproj\files\ folder.
Deployment:
Now I want to deploy this package to Server B. And my package will read the excel in the location \ServerB\S:\Documents\myproj\excelfile.xls
and will create excel sheets inside \ServerB\S:\Documents\myproj\files\ folder.
My questions:
Should I deploy my project to create dtsx file? There is already a dtsx file inside my project folder. Can I not just move that file to server B and change the connection string and paths? Will it work that way?
SSIS deployent tool does not work for me from server A. It does not identify the destination servers.How can be deployment made easy? How can I change the destination paths?
Server A has both SSDT installed and SQL database resides in same server. But the target server B is a database server and does not have SSDT in it. I will schedule a job in SQL server of server B that will call my dtsx package and execute it.
I am new to SSIS. so please don't close this question and would be great if anyone can help me with these questions. Thanks!
Without knowing the version of SQL, I'll give both answers:
Should I deploy my project to create dtsx file? There is already a
dtsx file inside my project folder. Can I not just move that file to
server B and change the connection string and paths? Will it work
that way?
Versions less than SQL 2012: Copy the dtsx file from your project to the folder on the target server. Connection strings can be change in the configuration of the SQL Agent job or by using package Configurations:
https://learn.microsoft.com/en-us/sql/integration-services/lesson-5-add-ssis-package-configurations-for-the-package-deployment-model?view=sql-server-ver15
Versions greater SQL 2012: Create the ssis catalog on the target servers:
https://learn.microsoft.com/en-us/sql/integration-services/create-the-ssis-catalog?view=sql-server-2014.
Right click on the project and use the wizard to deploy to the target server. Connection strings can be modified in SQL Agent as noted above, or you can use parameters:
https://learn.microsoft.com/en-us/sql/integration-services/lesson-6-using-parameters-with-the-project-deployment-model-in-ssis?view=sql-server-ver15
The latter my seem like more work at first, but there is a myriad of benefits and it is the preferred way of doing things
SSIS deployent tool does not work for me from server A. It does not
identify the destination servers.How can be deployment made easy?
How can I change the destination paths?
Follow either deployment method noted above. Package deployment (copy files to a folder) still works in later versions, but it is less easy to manage and less things done for you like securing configurations and setting up logging.
Server A has both SSDT installed and SQL database resides in same
server. But the target server B is a database server and does not
have SSDT in it. I will schedule a job in SQL server of server B
that will call my dtsx package and execute it.
SSDT has no bearing on SSIS, but Integration Services does. SSDT is a developer tool and really should only be installed on developers machines for the purpose of creating packages and testing. It is better to not have this on the server because it encourages development to happen there and for devs to logon to the server like it's their laptop. Integration Services is a service that comes with SQL Server and is used for executing packages. You'll need to add this service to the instance if you want to execute packages from SQL Agent. This link explains that and gives guidance on installing SSDT locally:
https://learn.microsoft.com/en-us/sql/integration-services/install-windows/install-integration-services?view=sql-server-ver15
I am migrating from a local sql server 2012, to a sql server 2016 in AWS.
For the databases I will backup them up, and then restore them in the new server, I will also transfer the logins and jobs. (I have done this a few times in the past).
I am puzzled about how to migrate SSIS and SSAS… is there a script to back it all up and restore in the new server? I have never migrated neither SSIS nor SSAS, not sure what to expect.
SSIS - at the core SSIS is either files with a .dtsx extension or a zip file with a .ispac extension
The .ispac should live in the SSISDB on your SQL Server. You can export them out by simply right clicking on the project and select Export
The .dtsx files might live in msdb under sysssispackages but they might also be in the SSIS Package Store, which is a fancy location under the SQL Server installation. Or they might be anywhere on the file system. Best approach is to find the job that runs the packages and pick apart the target to discover the where.
SSAS - They're a big fine model so won't you just back that cube up {Apologies to Juvenile}. An alternative article for backing up SSAS but you will end up with an .abf file which is everything you should need for the restore. You might also take a moment to start scheduling regular backups of your AS instances because it sounds like that's never been done.
You can also fire up SSDT and reverse engineer the existing model (tabular or multidimensional) and deploy from VS.
I have installed SSIS for VS2012 (SQL Server 2012) and I am able to create basic packages that run, however I am unable to get any of the logging to work. I have configured logging for text files and for the SQL Provider however, neither will log the execution of the package which is just a few SQL statements. I have configured all of the events to log in the details and in the advanced selections. The exec SQL tasks are in a Data Flow container, which I have also enabled logging.
When I look for the table msdb.dbo.sysssislog it is there but it is empty. If I configure the file writer, it will not create a file, which I specified to be in \windows\temp to avoid any permission problems.
If you deploy your packages to SSIS Catalog, you can have access to a better logging system that is offered by SSIS2012. You can Right-Click on your project under Integration Services Catalog and Report All Executions. This is highly recommended and makes your DBA's life much easier. you can learn more about how to set this up here.
I have some task which I want to schedule and I did not work with SSIS before. I have created SSIS package using the Execute T-SQL Statement Task. It has run my procedures from DB.
But I need to run this procedures weekly using SSIS package. Is it possible and how can it be implemented?
Once this is implemented, I will send this SSIS package to the guys who are working with SQL server.
One possible option would be to use an SQL Server Agent job to run the SSIS packages.
Have a look at the following link. It explains how to create an SQL job that can execute an SSIS package on a scheduled basis. You can create a Credential and Proxy if you need to access any external resources outside of the database. If your package accesses only a database, you can possibly skip the steps that create Credential and Proxy Account.
The scheduling part is not explained in the below link but I think it should be easy enough to figure out on the SQL Server Agent Job creation dialog.
How do I create a step in my SQL Server Agent Job which will run my SSIS package?
Hope that gives you an idea.
SSIS package execution scheduling is done on the SSIS server, when the package is imported (or, on the machine you wrote the SSIS package on, if it's the Production machine - it isn't, right?)
The SSIS GUI (BIDS) offers a range of scheduling options.