We host multiple SQL Server 2008 databases provided by another group. Every so often, they provide a backup of a new version of one of the databases, and we run through a routine of deleting the old one, restoring the new one, and then going into the newly restored database and adding an existing SQL login as a user in that database and assigning it a standard role that exists in all of these databases.
The routine is the same, except that each database has a different name and different logical and OS names for its data and log files. My inclination was to set up an auxiliary database with a table defining the set of names associated with each database, and then create a stored procedure accepting the name of the database to be replaced and the name of the backup file as parameters. The SP would look up the associated logical and OS file names and then do the work.
This would require building the commands as strings and then exec'ing them, which is fine. However, the stored procedure, after restoring a database, would then have to USE it before it would be able to add the SQL login to the database as a user and assign it to the database role. A stored procedure can't do this.
What alternative is there for creating an automated procedure with the pieces filled in dynamically and that can operate cross-database like this?
I came up with my own solution.
Create a job to do the work, specifying that the job should be run out of the master database, and defining one Transact-SQL step for it that contains the code to be executed.
In a utility database created just for the purpose of hosting objects to be used by the job, create a table meant to contain at most one row, whose data will be the parameters for the job.
In that database, create a stored procedure that can be called with the parameters that should be stored for use by the job (including the name of the database to be replaced). The SP should validate the parameters, report any errors, and, if successful, write them to the parameter table and start the job using msdb..sp_start_job.
In the job, for any statement where the job needs to reference the database to be replaced, build the statement as a string and EXECUTE it.
For any statement that needs to be run in the database that's been re-created, doubly-quote the statement to use as an argument for the instance of sp_executesql IN THAT DATABASE, and use EXECUTE to run the whole thing:
SET #statement = #dbName + '..sp_executesql ''[statement to execute in database #dbName]''';
EXEC (#statement);
Configure the job to write output to a log file.
Related
For the sake of simplification, let's say I have 2 databases with data, db_data_1 and db_data_2 which have the same set of tables and I have a 3rd database where my stored procedures are defined, say db_sp. Let's say my stored procedure is called my_sp
When I try to call db_sp.my_sp from either db_data_1 or db_data_2, I get an error saying that the tables referenced in db_sp.my_sp don't exist.
How can I have db_sp.my_sp query the tables in the calling database vs the database where my_sp is defined (namely db_sp)
Thanks.
You must qualify the table names in your query with the database name in the stored procedure. SELECT col FROM db_data_1.tbl instead of SELECT col FROM tbl, for example.
The documentation says this:
USE statements within stored routines are not permitted. When a routine is invoked, an implicit USE db_name is performed (and undone when the routine terminates). The causes the routine to have the given default database while it executes. References to objects in databases other than the routine default database should be qualified with the appropriate database name.
Why is this so? It seems like a big pain in the xxx neck.
A big use of stored code is the hiding of data from unprivileged users. You can GRANT MySQL users access to stored procedures without granting access to the underlying tables. This restriction ties the tables to the procedures.
A user who has privileges only in the test database shouldn't be able to do this sort of thing.
USE production;
CALL test.get_all_user_private_data();
And, if you're USEing one database and you run stored code that's in a second database, it gets the data from that second database.
Your solution is to consider your stored code (procedures, functions) to be part of the schema definition for each database. They go along with your other data definition operations like CREATE TABLE. Don't try to put them in their own "code library" database, but put them in each database where they're needed.
I am very new to mysql and I have I a situation where I need to update all my stored procedure in all my database. for example I have 10 database just say:
client_1,client_2,client_3,.....client_10.
Every database have same stored procedure just say:
proc_1,proc_2,proc_3,proc_4.
So if I made any changes to one of my stored procedure then it should get updated in all other database So that I don't have to do it manually.
I know the similar question have been asked but I am looking for some different approach. So what I want is some kind of mysql query or something like that in which we will pass the name of the database like:
client_1, client_3, client_8
and changes will only made to this databases.
I am using heidiSql- 10.2 with MySQL 5.6.
Thanks.
I am not entirely sure what you are wanting to do but I think you want something like this. First save the definition of your stored procedure to a file. Make sure it doesn't contain an schema references like client1.tableA. You want it to be able to run in any copy of your schema correctly. Be sure to follow the syntax rules defined by MySQL
Defining Stored Programs
Then once the stored procedure is saved you can use the mysql command line to run it for each client you want to update.
You would first connect to the database server using the mysql command line. Then issue a USE command to activate the first client database. Then run the script using the SOURCE command. See MySQL Batch Commands Then repeat for each client.
USE client1;
source c:\temp\storedProcedure.sql
USE client2;
source c:\temp\storedProcedure.sql
If this is not exactly what you needed hopefully it gives you some ideas to get you what you need.
Note that you could do the connection to the database and execute these commands via batch file instead of manually if you wanted to.
There are no statements in MySQL that create/drop/alter multiple procedures at once. You can only change one procedure at a time.
You can write an SQL script that includes a series of statements. But it's up to you to write that script.
You may write some script in Python (or other favorite language) that outputs the SQL script.
I don't know HeidiSQL, but I doubt it has any facility to apply the same change to many procedures. Nor does any other MySQL client that I'm aware of.
I am implementing a SSIS package and currently trying to do the following.
Truncate the destination table
Fetch the data by executing the stored procedure and insert it into the destination table.
I have created an Execute SQL task to address step 1 and dataflow with oledb source and oledb destination to address the second point. It been working successfully so far but isn't working for one my stored procedure that uses temp tables.
When I edit the oledb source and click the preview button, I get the error no column returned
I know that SSIS has an issue with generating column while executing stored procedures that depend on temp tables. I have converted the stored proc to use temporary table variables and its now able to return columns in SSIS when I do a preview. The only downside is that the stored procedure is taking longer time to execute. Its taking 1 hour 15 mins as compared to 15 mins while using temp tables.
I did see a suggestion to use SET FMTONLY before executing the stored procedure as an alternate solution to changing to temp table variables but that didn't seem to work as I am getting syntax or permission denied error.
Could somebody tell me a solution to my problem which does not compromise on the performance.
Sounds like you've already read all the approaches to using Temp tables in SSIS, including the IF 1=0... trick? If you haven't seen that one yet, google it.
You say that using Table Variables causes your stored procedure to take about 5 times longer than using Temp Tables. The most likely reason for that is that you are indexing your temp tables but not your table variables. If you didn't know that table variables can be indexed, they can. You might try that.
Finally, a solution that you haven't mentioned is that you can replace your temporary table with a real table that gets truncated when you're done using it.
Short comment:
Try EXEC WITH RESULT SETS and specify the metadata yourself for a proc with temp tables; or use the Script Component as a source and specify the Output columns yourself.
Long comment:
Technically speaking, it is the driver/database you are using in SSIS that would decide the behavior when working with temp tables.
Metadata is an important factor when using SSIS's pipeline components. By metadata, I mean the names of the columns, their data types etc that a pipeline component uses. When designing a data flow, someone/something should provide this metadata to the components that require it.
In most cases, SSIS automatically retreives the metadata. Components that do not connect to a external data source, like Conditional Split etc, get their metadata from the other components they are connected to. For the pipeline components that connect to a external data source (like Oledb source, oledb destination, Lookup etc.), SSIS provides a mechanism to get this metadata without human involvement. This mechanism involves the driver connecting to the database and retrieving the metadata of the output. If the driver/database is capable of returning the metadata, then that metadata is used. If the driver/database is incapable, then you get the errors you are seeing. The rest of my comments are based on the assumption that you are using a SQL Server database in your question.
When working with a SQL Server database in SSIS, typically, we use the native client drivers provided by Microsoft. When trying to get the metadata, these drivers try to get the metadata without actually executing the SQL Statement (actual execution can have side effects; and also, might take more than a few seconds/minutes/hours; and you dont want side effects and long wait times during package design time.) So to get the metadata, the driver relies on the metadata of the actual objects used in the sql command. If the command uses a physical table or view, SQL Server already has the metadata available and can supply it to the driver. If it is a temp table, SQL Server does not have the metadata until it can create the temp table. If using FMT ONLY option, you can use it in such a way to create the temp tables, but avoid any heavy processing/side affects and thus be able to retrieve metadata without penalties. Post 2012, these native client drivers rely on some newer functionality to retrieve metadata than the drivers before 2012. In 2012 and after, the driver uses the sp_describe_first_result_set proc to retrieve metadata. So, whether you can get metadata or not is determined by the ability of the sp_describe_first_result_set proc.
So while SSIS can automatically get the metadata (because of the driver/database), it does not automatically get the metadata in some cases (again because of the driver/database). In cases involving the second scenario, some other process (typically a human) can help the driver infer metadata or provide the metadata to the component directly.
To help the driver, in case of SQL Server 2012 and after, you can use the WITH RESULTSETS clause to specify the output metadata. When this clause is present, the driver will use it and doesnt try to query the metadata from system objects; and thus avoid the error which you would otherwise get. If you are using the drivers that came with SQL Server 2008, you can use FMT ONLY. This option is at the driver/database level.
Another option could be to use a Script Component as the Source and in the Output columns, you can specify the columns/metadata. SSIS would not try to retrieve metadata from the datasource in this case, but would rely on the definitions you provided in the Output section of the Script Component.
As you can see, both options involve a human (or some other process) specifying the metadata instead of SSIS trying to retrieve the metadata in an automated fashion. I would prefer the first option if working with SQL Server and the second option if working with databases like MySql.
I am working on an SSIS ETL and I wanna know if there is a possibility to execute a MySQL Stored Procedure.
Here is what I want to do : From an SQL Server Database, I want to get Information by an ETL (SSIS) and send them to a MySQL Database (by a stored procedure)
Here is what I have done so far : I get my data from SQL Server Database and tranform them.
Here where I am stuck : I don't know how to execute an existing stored procedure on the MySQL Server Database (my destination)
Here is my ETL (DATA FLOW) diagram :
I also add an OLEDB provider on the server and add my destination source (MySQL Database) but I don't know what I need to do in my ETL to execute the stored procedure.
I can provide more information if necessary.
Thanks in advance
I am not sure if you mean that you want to execute a stored procedure by passing in every row in memory in SSIS that enters through your multicast. If that is the case, that may be possible with SSIS, but I have never done it.
why not just send the rows to two separate tables in MySQL from the multicast, then go back to the control flow tab and add two execute sql tasks, one for each stored procedure. Change each stored procedure to run on the tables instead of individual rows.
You would probably get a performance boost as well from switching to a set based operation instead of row by row.
I have postponed writing SQL code for university, and now that I want to start learning it, I have no idea how to.
In C I'd define headers and begin with coding main, but in SQL classes all I have is a plain example
CREATE TABLE Sailors(
sid INTEGER,
sname VARCHAR(30) NOT NULL,
rating INTEGER DEFAULT 0,
age REAL DEFAULT 18
)
and some commands for using the table I created.
My questions are: How is a correct script supposed to look? How do I run it to create a database? (MySQL) How do I use MySQL to run scripts and where do I type commands in real time to do stuff I haven't scripted?
I just can't wrap my head around it. All tutorials I've seen use a terminal I can't find, or another I did find and I can't use because I get errors using any command (can't create file in some directory and some modules report errors so it shuts down)
The following is a very vague description of Mysql to get an idea of it:
Mysql (or SQL) is separated in 3 types of language:
DML : Data Manipulation Language
DDL : Data Definition Language
DCL : Data Control Language
Read about them to find out which kind of command belongs where.
You will find out that you almost exclusively need DML and DDL to work with Data in MySQL. While DCL is mostly used to keep the database running, control user privileges , etc.
Also when running code there will be only one command of your script executed at a time without a possibility to point somewhere else in your script.
Loops and Cursors can be used , but have to be stored in a special form of script called stored procedure. Usually you execute your code in a sequence without a code based relation between the different commands (the relation comes from the context of the commands).
Get Data into your Database:
(Consider installing the community edition for MySQL if you have problems running MySQL correctly)
To get Data into your Database , you should import data from files into your database. The MySQL-GUIs available (Workbench, Toad, Navicat, HeidSQL...) usually provide an Import Wizard that makes it easy to import Data from all kind of Files (txt, Excel, Database Files ..).
You can create an excel spreadsheet and import it into your database for example.
here is a picture of the Workbench SQL Editor:
https://dev.mysql.com/doc/workbench/en/images/wb-getting-started-tutorial-adding-data-movies.png
Workbench (or any other GUI) will be your IDE. Getting into it will answer many of your questions.
Regarding the correct script:
A complete MySQL command is called a query.
A Query is defined by a ; at the end (default) .
A chain of MySQL commands is called a script.
Therefore, a correct script consists of correct querys.
To solve more complex problems, use stored procedures in MySQL (this should come close to your usage of the word script).
some MYSQL commands you will have to be familiar with:
select
update
insert into
delete
create table
drop table
alter table
You have a lot to read. But make sure that your Database is running and you have some data in it to test code. As you already have programming experience, you should understand this really fast with the right setup.