Not sure I am referring to this correctly, but we seem to have an "orphaned" or rogue Integrated Service Catalog, SSISDB, with a database SSISDB on an azure managed instance
We were testing SSIS options a few months ago including creating it in data factory. I know that's when it was created but the Azure-SSIS integration runtime is not in the data factory that is listed in
SELECT [property_name]
,[property_value]
FROM [SSISDB].[catalog].[catalog_properties]
The data factory is there but there is no Integration Runtime or any reference to SSISDB in there.
How do we delete the catalog?
Do we have to delete the entire data factory to remove this?
Edit: I have already dropped the SSISDB database to see if it helps, it didn't so restored it. the database itself is not the problem.
Answering this so if someone else has the same issue
I renamed the database, had several errors trying to reconnect, for some reason the server tries connecting to SSISDB. But I finally got in & the catalog was gone. So though probably dropping DB might work in normal circumstances, for some reason on our Azure managed Instance dropping the db did not remove the catalog. Maybe it would have after a while, I don't know, but when I initially dropped it, the catalog didn't go anywhere for hours. and kept getting login errors for explicitly specified database 'SSISDB', I thought it broke something so I restored it.
So if dropping the database doesn't fix it, restore it, rename it wait a few minutes for SSIS catalog to drop, then drop the database.
Related
First time posting a question, please forgive if I don't have enough information.
I have a tiered BIML Script that has the following tiers:
10-Connection – create the connection nodes
20- Model – loop through the connections to build database, table nodes
30-Create/Drop Staging Tables – This is included as the package/s need to be run prior to the remainder of the creation process.
30- Flat File – loop through the table object to create flat file formats and connections
40-Packages – loop through the table objects and create extract and load packages
45-Project Params & Connections– attaches the project params and connections (using named connections and GUIDs from 10-Connections). Project params manually created in SSIS.
The process successfully connects to the source SQL Server database, generates the Create/Drop Staging Tables packages with correct metadata, and will create the extract packages successfully the first time.
Upon a second attempt to process the same BIML scripts with no changes made to the files, the process fails with “Object reference not set to an instance of an object.” & “Unable to Query on Connection” on the OleDBSource Node.
The BIML files generated in preview and output debugging have valid queries and source metadata that indicate a positive connection and proper model. I have used emitted queries in SSMS without error. When I move the BIML files to a new project, the process is successful the first time and fails subsequently.
I have tried the following:
Connection Managers
Delete project connection managers prior to package re-generation
GUIDs annotated and used in PackageProject and Packages Nodes.
Delay Validation/Validate External Metadata – I have tried with both true and false on Package, DFT and OleDBSource
Project
Delete .proj files from directory
Direct PackageProject to new ProjectSubpath
I also tried simply hard coding the BimlScript to simplify and remove any variables with the same result.
The most maddening point is that the metadata and queries all indicate the process can connect and query this exact table and it functions, but only on initial creation. Adding or re-generating during testing fails. Anyone ever come across this before?
Great thanks and shout out to cathrine-wilhelmsen, billinkc, whose posts and tutorials have been very helpful. Any and all help would be greatly appreciated.
I changed the driver from SQLNCLI11 to SQLOLEDB with no changes to code. I tested different drivers after seeing a few example connection strings with different drivers.
I wish I could explain why.
Been tasked with moving a code first database from MSSQL to MySQL. After a few hours of kung fu, I was able to get the asp.net core project to properly deploy all migrations to mysql. Now I need to migrate the data inside of the existing mssql tables. I saw posts mentioning MySQL Migration Toolkit but that appears to be old. Also attempted to do so with MySQL Workbench and DBLoad's Data Loader but haven't had any luck.
Table structure is pretty simple with incremental integer keys + the usual crap with asp.net core identify framework (GUID). Just need to keep that consistent during the migration. What is the best way to migrate the data now that the table structure is setup in MySQL? Any recommendations would be greatly appreciated!
Update: Some more details...
I attempted a direct migration of the database from MSSQL to MySQL using MySQL WorkBench and DBLoader. But failed on the ASP.net Identity tables big time plus other issues. .net core api took a huge dump in multiple places so that idea is out.
From that point, I migrated the api controller over to mysql and then had to fix a myriad of issues related to mssql fks being too long and a few other issues.
So at this point, the controller works on mysql. I just need to dump all of the data into MySQL and keep the FKs consistent.
I have had a few thoughts with it such as CSV export>import and/or trying a few other things. Any recommendations?
You can use Data Export wizard built-in with SqlServer Management Studio (task -> Export Data)
or use SSIS package to migrate data.
Tried MySQL Workbench, DBLoad from DBLoad.com etc to migrate the data directly and they all failed.
So ended up finding a "solution"...
First off, I modified the ASP.net Core project with:
//services.AddDbContext<ApplicationDbContext>(options =>
// options.UseSqlServer(
// Configuration.GetConnectionString("DefaultConnection")));
services.AddDbContext<ApplicationDbContext>(options =>
options.UseMySql(
Configuration.GetConnectionString("DefaultConnection")));
Then went to Package Manager Console and ran: update-database
This created all of the tables in MySQL.
Then opened up Microsoft SQL Management Studio. Right clicked on the database Tasks > Generate Scripts. Saved everything to one file with Advanced option Schema and Data selected.
Then opened the db script in Notepad++ and applied the following edits using Replace with Extended Search Mode enabled:
GO -> blank
[dbo]. -> blank
[ -> blank
] -> blank
)\r\n -> );\r\n
\r\n' -> '
DateTime2 -> DATETIME
After edits were made in Notepad++, I removed all of the SET, ALTER and CREATE related stuff from the text file and then copied all of the insert lines into MySQL Workbench in order to ensure foreign keys were already populated before that table's data was inserted. Thank goodness there were no Stored Procedures to deal with! What a pain in the tucus!
Also on a side note, the app is hosted on Azure. Spent a couple of hours fighting the API not connecting to the database. This was not apparent at first as the app was fake throwing a 404 error to Postman. When I first attempted to wire up the API controllers to the MySQL DB, I entered the database connection string into Azure's App Service Configuration. It didn't work at all even though running the app locally worked fine. Ended up finding another post on this here site mentioning to get rid of the database connection string out of the App Service > Configuration window. Worked like a champ after that. All of the data with its auto incremented keys linked up without issue.
I am very pleased with the results and hope I never have to go through this process again! It is always a nice feeling to know an app now runs on a completely open source infrastructure. Hope this helps someone. Good luck.
I've started using the SQLcomplete refactoring tool for SQL Server Management Studio. It appears that every time I add a new database to my instance it doesn't update the SQLcomplete tool with the new schema for that database.
Anybody know how to get around that so it works?
Have you tried pressing Ctrl+Shift+R? This refreshes the cached schema, and can be done any time there is a schema change--which includes adding new databases.
I have been working on a huge ETL project with 150+ tables and during the design I had to make a major change on destination column names and data types for a couple of tables.
My problem is that I can't get SSIS to see the new schema for the tables I changed. So I would like to know how can I get SSIS to refresh this schema? I find it kind of ridiculous that there no way to tell SSIS to update the metadata from database schema, especially for database migration.
Recreating the project from scratch is out of question because I already spent some hours on it. Also changing manually the 400+ columns I changed is also not an option.
What about using the Advanced Editor and pressing the Refresh button on the left side below?
Following my previous auto-answer, I finally found what was preventing the metadata from being refreshed.
When I originally modified my database, I actually executed another script that was making a DROP on the table and then a CREATE TABLE to recreate the table from scratch. There, SSIS was never able to detect changes and I had to do all the things in my other answer.
Later today I had to make some minor modification and this time I opted for an ALTER TABLE. Magically, this time SSIS detected all the changes even notifying me to refresh columns from the advanced editor, which worked fine.
So basically all these issues has been caused by my poor knowledge about DBA and its best practices.
I found a way to fix it but that was a bit tricky.
Even thought I was completely removing any references of the table from my packages, I was always getting the old metadata.
I still don't have a clear fix but here is what I did to fix it:
Removed any reference to concerned source and destination tables
Deleted obj and bin folder from the project folder
Saved, closed and then reopened the project
Created new data flow from scratch and updated metadata was finally there
Don't know where those informations were cached but I suspect that the obj folder keeps a cached copy of your packages or that Visual Studio keeps metadata in memory which is freed when you close it. Anyway, following these steps should fix it.
Im creating a database via a "code first" application, the sql server contains no databases.
The application runs fine, creates the database and seeds the data i have defined in my initializer.
a service i have running tries to add some data to the database for the first time. i get the error:
The model backing the 'yyyContext' context has changed since the database was created. Either manually delete/update the database, or call Database.SetInitializer with an IDatabaseInitializer instance. For example, the DropCreateDatabaseIfModelChanges strategy will automatically delete and recreate the database, and optionally seed it with new data.
There should be no changes since the database was created and when then service runs.
I'm running EF4.1, and the database doesn't exist so unlike questions with similar titles:
Database.SetInitializer<YourContext>(null);
Isn't the solution for me.
Any ideas about what could be wrong are welcome.
Doh! moment, turns out the service wasnt using the same connectionstring as the other app.
the reason the databases didnt look the same must be because earlier in development i started the main app with no connectionstring aswell, so it provisioned a local instance database for itself to use.
Then later when i was trying to use the service, it was trying to access the same database from earlier, and the model changed significantly since then.
I pointed the connectionstring to the correct database and everything worked from there.