I am executing SSIS packages from an application server that is different than the database server. This package will be reading and writing files. Is it better to have the potentially large input files on the DB server or the application server where the package is executed? I'm not clear on where the actual "work" takes place when I execute a package on a server other than the database server.
The Integration Services server is where the actual execution is done. Generally that means "the DB server" but it doesn't always have to be the same server that the houses the DB you are reading from/writing to.
"This package will be reading and writing files."
isnt that one of the specialities of a database server? ;)
Related
I was wondering if it is possible to create custom MySQL servers in VB.NET while working in visual studio at runtime so that if the server already exists it connects and if it isn't there, the code creates the server. I have searched for this everywhere but couldn't find anything. I would appreciate it a lot if someone guides me to the right path.
You could certainly write some .net code to start a MySQL server on your Windows box when an attempt to connect fails. You simply get a cmd.exe console with administrator privileges and give the command net start mysql.
But MySQL must already be installed on the box for that to work.
You might investigate Sqlite. It provides SQL locally to a .net program, storing your tables in a file called whatever.db. It has very similar .net API access to MySQL's Connector/Net and SQL Server's connector. It's in a NuGet package.
I don't completely understand your "custom MySQL servers" requirement. Sqlite gives you a way to use SQL in your application without connecting to a shared server. That may do what you need.
MySQL does have a CREATE SERVER statement in its SQL dialect. The purpose of this statement is to create a connection to another, remote, MySQL server. With that connection you can use the FEDERATED storage engine to access tables in the remote server. Of course, there is no way to run this CREATE SERVER statement unless your program is already connected to a MySQL server.
With respect, your "task which states to create a server at runtime" doesn't make much sense. Is there more to this requirement? What workflow needs this step? Is it part of the installation of some application software on a new box?
I have SSIS packages in SSIS catalog on SQL Server, Serv1. The packages, while executing, establish connection to SQL Server, Serv2. And they fail while acquiring connection.
I know a little bit about linked server that is to run query from one server on another the latter one has to be a linked server to former one. The above scenatrio looks the same but i didn't find any information related to it. Do i have to add Serv2 as Linked Server for Serv1?
No. Linked server scenario is used when you need to access DB from the outer server inside SQL query run on local server.
SSIS packages are built with goal that it access some DB server, possibly a remote one, fetch data from it, transform and store results somewhere - in files, DB etc. Accessing a remote SQL DB is a normal scenario.
Moreover, using Linked server in SSIS package is a bad practice. You move control of DB access from SSIS catalogue to DB server, in case of any problems - it is more difficult to trace and investigate.
In your case - SSIS packages in SSIS catalogue - check in package execution log which connection string is used to connect to the remote server. Is it an integrated authentication? Does the account under which SSIS package is executed have an access to the DB? If using SQL auth - are login and password valid?
I have created one project for SSIS and deployed that on sql server 2014. When I am running packages from sql server data tool its running fine and performing all operations, but when I am running from catlog procs [SSISDB].[CATALOG].[Create_execution] its shows run successful but I can't see any data into my staging tables. I have used configuration tables to configure connection and files path.
Any idea.
PLease check the user privileges (file system, etc.) - if you call the procedures with another user than you execute the package from within Data Tools, this might be the reason.
I have an existing SSIS package on my DB server and I have another server which is the application server which uses this DB server for its operation. This SSIS package, is being used by a job running on the DB server. I would like to add an additional step. That step is just a basic select count(*) query to get count from one table. I implemented that on my test server and gave the output path in the advanced tab as well. And i got the output in a simple text file.
My question now is, How do i send this output file to my application server instead? Because the output path of a T-SQL server seems to only take local drives. I tried giving the path of my application server(which was being used by the SSIS package already) but the output doesn't come on the text file even 'though it says the job executed successfully. I don't see the necessity to create another SSIS package for just getting a simple count info.
Use the UNC path to your application server file system.
I have my main web server and I need a seperate workhorse server, I cant get around this unfortunatly. The workhorse server generates a largeish (+100mb) SQL file which I need to import to the web server every day.
The way I see it, I have two options:
1. Option one - Allow the web server to connect to postgreSQL on the workhorse server and import the data directly from postgre running on the workhorse server
2. Option two - Export the SQL database from the workhorse server and upload it to AmazonS3. Then download the file from S3 on the web server and import it to postgreSQL server running on the web server.
My main concerns is to keep the system as simple, secure and reliable as possible. I want to limit the chances of the database getting corrupted.
From the answers I got, it looks like option 1 is the best best. To use the server as an SQL host and query it directly rather than move the SQL file.