I've perused the threads here on migration from SQL 2000 to SQL 2008 but haven't really run into my question, so here we go with another one.
I'm building a strategy to move specific SQL 2000 databases to a new SQL 2008 R2 instance. My question comes with regards to the best method for transferring the schema and data. One way I know of is to do the quick 'n' dirty detach - copy - attach method, which should work so long as I've done my homework wrt compatibility and code and such.
What if, though, I wrote the schema and logins via script and then copied the data via SSIS? I'm thinking of trying that so I can more easily integrate some of my test cases into the package (error handling and whatnot). What would I be setting myself up for if I did this?
Since you are moving the data between servers or instances, I would recommend moving the data via data flows. If you don't expect to run the code more than once, then you can let the wizard generate your code for this move. However, when I did this once 2+ years ago, the wizard code generated combined execute sql tasks that combined many "create table" commands into one task and created a few data flow tasks that had multiple source and destinations in them to insert data in the destination. This was good to get up and running, but it was inadequate when I wanted to refresh the tables one more time after I modified the schema of the new target tables. If you expect to run the refresh more than once, then you may want to take the time to create the target schema first and then manually create the data flows.
Once you have moved the data, then you can enable full-text search on the new server. I don't believe you will need to have this enabled on your first load.
One reason I recommend against the detach-attach method for migration is that you bring all the dirty laundry from the 2000 database to the 2008 R2 database. If you had too lax security on the 2000 server or many ancient users that shouldn't exist, it could be easier to clean this up by starting from scratch. If you use the detach-attach method, then you have to worry about users.
Related
What’s the best practice for integrating SQL Server with Active Directory (AD)?
NB. I’m using SQL Server 2016
Crux of the issue: I'm using SSRS 2016 and have several reports that need to be filtered based on the user accessing the reports. Originally I created a table of users that would need to access the reports. Then in the report builder I passed the UserID as a parameter within the query so that the resulting dataset would be limited to the data the user needed to see.
The problem this created is that the User table would have to be maintained, and Active Directories are dynamic. Now that I have some time to develop a better option, I’d like to link the LDAP data with SQL Server.
I’m wondering what the best practice for doing this is.
One way I pursued this was through an SSIS package ADO.Net connection. Then convert the data. Then load it into a table. Then schedule a job to run the package however often I needed it. This was problematic because for whatever reason I couldn’t get the data conversion process to work.
The second way I’ve been approaching this is to create a linked server instance for the AD. My research has indicated that I’ll need to create a function that overcomes the string limitation of the xp_sprintf Function. Then leverage temp tables and loop through LDAP data to get around the 1000 record limitation from the AD. I've been able to accomplish all this.
At this point though, there appears to be some other issues.
This ultimately increases the code necessary in the views for my reports which may make it harder for other database users to update if & when the time comes. To the point that I'd need to abandon the views and create stored procedures for the reports to pull from.
This also increases transaction counts beyond the SQL Server to include LDAP every time a user accesses a report.
So to resolve that I could wrap the original query of the LDAP data to create a table and then create a job to run that stored procedure every so often.
Either option solves the problem of maintaining the users table which is good, but it isn't perfect because AD changes can take place at any time.
Which option is better here?
If the SSIS package is the better route, I’m curious as to why that is the better route. I’m not opposed to going back and figuring out what it is I’m missing on the SSIS package to make it work.
Are there additional options I should consider if I want to get the most up-to-date Active Directory listing?
Thanks.
I'm writing an intranet application (in a LAMP environment) that uses data from sections of an MSSQL 2012 database (used by another much larger application).
As I see it my options are to:
Directly query the database from the application.
Create a web service
Use Microsoft SQL Server Integration Services to have the data
automatically integrated into my applications database
I'm sure the best solution here would be using SSIS, however I've not done this before am on a deadline - so if that's the case could someone let me know
a) With my limited experience in that area would I be able to set that up, and
b) What are the pros and cons of the above options?
Any other suggestions outside of the options I've thought of would also be appreciated
Options:
Directly query the database from the application.
Upside:
Never any stale data
Downside:
Your application now contains specific code and is tied that
application
If you are in the common situation where the business
buys another application containing the same master data, you now
need special code to connect to two applications
Vendor might not like it
Might be performance impacts on source application
Use Windows Task Scheduler / SQL Agent to run a script or SSIS to replicate data at x minute intervals or so.
Upside:
Your application is only tied to your local copy of the database, which you can customise as required. If your source app gets moved to the cloud or something then you don't need to make application changes, just integration changes
If another source application appears with the same type of master data, you can now replicate that into your local DB rather than making application changes to connect to 2 databases.
Downside:
Possibility of stale data
Even worse: possibility of stale data without users realising it, with subsequent loss of confidence in the application
Another component to maintain
If you write a batch script, .Net app or SSIS, they are all pieces of logic that needs to be scheduled to run
Another option is to replicate the database using differential replication if your source database is Oracle or SQL, you can use replication to replicate it into another database.
You need to consider where you will be in a few years. The data copy method probably gives you more flexibility to adapt to changes in the source system as you only need to change your integration, not your whole app if something drastic changes with your source system.
You also need to consider: will you ever be asked to propogate changes back the other way, i.e. update data in your local copy and have it pushed back to the source systems.
I am trying to build a database in sql server that replicates exact data present in tables in oracle production database. The database in sql server will be used for reporting and for analysis. I want every new or updated data in oracle tables to be present in sql server tables in around 1 hour time span. Does sql server integration services helps on this? is there any tool that does this i.e. it makes sure that data present in oracle table and sql server table is always same( neglecting the 1 hour lag?)......
There are two things you could look into: replication and SSIS. SQL Server replication allows you to replicate data from Oracle to MSSQL so that would be one way to handle the data copy. On the other hand, if you plan on doing data transformations, mappings etc. then you might want to use SSIS because it's a full ETL tool.
One important question is how you can identify new data in Oracle, because that may determine at least the first part of your solution. And you then have to decide what transformations are necessary once you've copied the data into SQL Server; perhaps you will need to run some stored procedures to clean the data and put it into reporting tables. Since your reporting system is a different platform from the source, you will need to handle data type transformations at some point, whatever solution you choose.
Your question is quite general, and it isn't really possible to say what you should do without a lot more detail about your environment, your requirements, your resources and so on. I suggest that you try to break down your task into smaller ones, and then you should be able to ask more specific questions.
We are currently having an OLTP sql server 2005 database for our project. We are planning to build a separate reporting database(de-normalized) so that we can take the load off from our OLTP DB. I'm not quite sure which is the best approach to sync these databases. We are not looking for a real-time system though. Is SSIS a good option? I'm completely new to SSIS, so not sure about the feasibility. Kindly provide your inputs.
Everyone has there own opinion of SSIS. But I have used it for years for datamarts and my current environment which is a full BI installation. I personally love its capabilities to move data and it still is holding the world record for moving 1.13 terabytes in under 30 minutes.
As for setup we use log shipping from our transactional DB to populate a 2nd box. Then use SSIS to de-normalize and warehouse the data. The community for SSIS is also very large and there are tons of free training and helpful resources online.
We build our data warehouse using SSIS from which we run reports. Its a big learning curve and the errors it throws aren't particularly useful, and it helps to be good at SQL, rather than treating it as a 'row by row transfer' - what I mean is you should be creating set based queries in sql command tasks rather than using lots of SSIS component and dataflow tasks.
Understand that every warehouse is difference and you need to decide how to do it best. This link may give you some good idea's.
How we implement ours (we have a postgres backend and use PGNP provider, and making use of linked servers could make your life easier ):
First of all you need to have a time-stamp column in each table so you can when it was last changed.
Then write a query that selects the data that has changed since you last ran the package (using an audit table would help) and get that data into a staging table. We run this as a dataflow task as (using postgres) we don't have any other choice, although you may be able to make use of a normal reference to another database (dbname.schemaname.tablename or somthing like that) or use a linked server query. Either way the idea is the same. You end up with data that has change since your query.
We then update (based on id) the data that already exists then insert the new data (by left joining the table to find out what doesn't already exist in the current warehouse).
So now we have one denormalised table that show in this case jobs per day. From this we calculate other tables based on aggregated values from this one.
Hope that helps, here are some good links that I found useful:
Choosing .Net or SSIS
SSIS Talk
Package Configurations
Improving the Performance of the Data Flow
Trnsformations
Custom Logging / Good Blog
I am SQL Server developer and the current assignment is little different than what I have done in past. I found Stack Overflow very promising for my problem. I am working on the SQL Server 2005 database for the internal application for my client and the client also got the public facing web application with MySQL database. I do not have any details about this web application, but I got the assignment to update the MySQL database (on public domain) from the SQL Server database (internal domain) on daily basis as auto process. How can I achieve this through the SQL Server?
You might want to try Pentaho Data integrator.
http://wiki.pentaho.com/display/EAI/Latest+Pentaho+Data+Integration+%28aka+Kettle%29+Documentation
The product would allow you to speak to both data technologies. (MSSQL+MySQL) You will find the product similar to DTS. You may be able to construct your solution will little to no code.
SSIS will do this just fine. The hard part is determining how you want to transform the data from one structure to the other (I assume they are not exactly alike in terms of table design.)
But basically you create a dataflow task, connect to the SQL Server for the source data and use a query to define what data you are going to copy, then you do any transformations needed to make the data fit into the MySQL structure and connect to a MySQL destination.
Repeat this process for mulitple data sets you want to send to differnt places.
Once the SSIS pacakge is done, set up configurations so that you can run the package on the production server (you will want to test development to development of course!) then schedule the package to run at an appropriate time.
Depending on how different the two databases are and how much data you need to move, this can be a relatively simple process or very complicated.