I am trying to use Microsoft Sync Framework for syncing 2 SQL Server 2005 database (server and client). There are multiple tables in the database with lots of foreign key relation between them. I am using SyncOrchestrator to synchronize the two databases.
string clientConnectionString = "<connection string>";
string serverConnectionString = "<connection string>";
SqlSyncProvider localProvider
= ConfigureClientProvider(clientConnectionString);
SqlSyncProvider remoteProvider
= ConfigureServerProvider(serverConnectionString);
SyncOrchestrator orchestrator = new SyncOrchestrator();
orchestrator.LocalProvider = localProvider;
orchestrator.RemoteProvider = remoteProvider;
orchestrator.Direction = SyncDirectionOrder.Download;
In the function ConfigureClientProvider and ConfigureServerProvider I am initializing connection and checking if scope doesn't exits then create it:
public static SqlSyncProvider ConfigureClientSyncProvider()
{
SqlSyncProvider provider = new SqlSyncProvider();
provider.Connection = new SqlConnection(Configs.ConnectionString);
DbSyncScopeDescription scopeDesc = new DbSyncScopeDescription("Test1");
SqlSyncScopeProvisioning serverConfig = new SqlSyncScopeProvisioning();
if (!serverConfig.ScopeExists("Test1", (System.Data.SqlClient.SqlConnection)provider.Connection))
{
scopeDesc.Tables.Add(SqlSyncDescriptionBuilder.GetDescriptionForTable
("Employees", (SqlConnection)provider.Connection));
scopeDesc.Tables.Add(SqlSyncDescriptionBuilder.GetDescriptionForTable
("Profiles", (SqlConnection)provider.Connection));
scopeDesc.Tables.Add(SqlSyncDescriptionBuilder.GetDescriptionForTable
("Department", (SqlConnection)provider.Connection));
serverConfig.PopulateFromScopeDescription(scopeDesc);
serverConfig.SetCreateTableDefault(DbSyncCreationOption.Skip);
serverConfig.Apply((System.Data.SqlClient.SqlConnection)provider.Connection);
}
return provider;
}
Now when I try to run sync its works fine for updated data but I got foreign key issues while there are any inserts or deletes in the database. e.g.
The INSERT statement conflicted with
the FOREIGN KEY constraint
"FK_Employees_Departments". The
conflict occurred in database
"Sync_Client", table
"dbo.Departments", column
'DepartmentID'.
If I do some change in order of tables then I am able to resolve one case of another case arises because of deletion.
The DELETE statement conflicted with
the REFERENCE constraint
"FK_Employees_Departments". The
conflict occurred in database
"Sync_Client", table "dbo.Employees",
column 'DepartmentID'.
Does anyone have any idea how this can be fixed. What I think the sync framework is not able to property executing changes in correct order. This order depending on several factor like foreign key relations, type of command e.g. insert, update etc. I am really stuck here. Early help will be appreciated.
This is an old question now, but since there's no real answer:
Sync requires you to list tables in each scope in insert order, so that all Foreign Key parents are in place before any Foreign Key children are inserted. Sync will automatically reverse that order on delete.
This is all fine and dandy, but if you have a database where for whatever reason the data in your parent or child tables is stored on different servers based on some independent piece of information, so that the parent and child might have different sync rules, you've overstepped any automatic processing that's available.
In this case, where the normal sync filters are built against the primary key information in your BASE tables, you will need to force the filters to use the primary key information in the TRACKING tables instead. There is now some content about this on social.msdn.microsoft.com.
Related
I have already existing DataBase inside SQL server 2008 around (300 tables) so I faced the following problems when i try to create an entity framework .edmx file:
If I specify to map all the 300 tables at once, the visual Studio will hang.
So I decide to include only the tables which I currently need, then the mapping will work fine. But if after that I add a new table to them, a Foreign key error will occur. So I have to delete the existing models and add them again with the new table. So the FK error will be removed.
So can anyone advice on how to overcomes these problems?
The error I usually got when adding a new table to the .edmx file will look something similar to.
Problem in mapping fragments starting at lines 2186, 2265:Foreign key constraint 'SDOrgPostalAddr_FK2' from table SDOrgPostalAddr (POSTALADDR_ID) to table AaaPostalAddress (POSTALADDR_ID):: Insufficient mapping: Foreign key must be mapped to some AssociationSet or EntitySets participating in a foreign key association on the conceptual side.
while if I added the table from the beginning then no error will be displayed.
Regards
I am trying to restore a DB using an SQL script, but things foreign key constraints get in the way
I am taking a MySQL DB and bringing it over to PostgreSQL.
Since the MySQL create table syntax ended up being quite different, I took another PostgreSQL DB with the same schema, but different data and restored the schema only, from that.
In other words, I now have a database with tables, constraints, sequences and all that shnaz but no data inside.
So, it's is time to restore data.
I take a backup of the MySQL DB with phpMyAdmin (data only) as an SQL script (pgAdmin does not seem to accept zip or gzip files for some reason) and run the SQL script.
Now, this is where the problems start to happen, it's only natural, I am going from MySQL to PostgreSQL, so syntax errors are bound to happen.
But, there are other non syntax related problems to, like this one:
ERROR: insert or update on table "_account" violates foreign key constraint "fk_1_account"
DETAIL: Key (accountid)=(2) is not present in table "_entity".
So, yeah, basically, a foreign constraint exists, the query is trying to insert data into the _account table, but the corresponding data has not been inserted into the _entity table yet.
How do I get around that? Is there a way to make pgAdmin3/PostgreSQL disable ALL OF the constraints, insert the data, and then re-enable the constraints?
A syntax related error I encountered, was this one:
INSERT INTO _accounttype_seq (id) VALUES (11);
The PostgreSQL equivalent of that statement (if I am correct) is
ALTER SEQUENCE _accounttype_seq INCREMENT BY 11;
But, it's a bit of a pain to run through the whole script and change all 200+ Sequence insert statements. So, I am being lazy here, but is there an easier way to deal with the sequences as well?
Or, do you guys have any suggestions for a different set of tools to make this easier?
Thanks for your time, have a good day.
Do not try to get around the foreign key constraints. That is the way to make sure the data is bad.
First look at the constraints and make sure you are inserting to the tables in the correct order. If _entity is parent of "_account, then it should be populated first.
Next you need to have the script move any failing records to an exception table. Then you can look at them and see what the data integrity issues is and if you need to throw the records away permanently or try to figure out what the missing parent value should be. If it is critical data such as orders where the customer no longer exists (possible in any system that didn't have correct fks to begin with) and you must keep the record and cannot determine what the parent value should have been, you can create an 'Unknown" record in the customer table and assign all bad orders to that customer id.
And manually changing the alter sequences shouldn't take long even if it is boring. There wil be plently of other things you need to handle manually in a conversion of this type.
I would try to find a data import tool for PostgreSQL - I live in SQL server world where I would use SSIS but you need the equivalent of SSIS for the PostgreSQL world.
Aparently the foreign keys weren't actually enforced in MySQL (maybe because of using MyISAM) or the generated SQL just does it in the wrong order.
If it's "only" the wrong order, I see two possible solutions:
edit the generated script and either move all FK definitions to the end of the script
Edit the definition of each FK constraint and set them all to initially deferred. Then run the script as one single transaction with only on commit at the very end.
Edit (because this is too much to be put as a comment)
Using SET CONSTRAINTS ALL DEFERRED will only work if the constraints have been created with the option DEFERRABLE.
To run everything in one single transaction, you have to make sure you have turned autocommit off. Then simply run the INSERTs and at the very end issue a COMMIT. A ; will only commit if you have autocommit on.
If you want to be independent of the autocommit setting, then start your script with [BEGIN][1] and make sure there is only a single COMMIT at the very end.
BEGIN DEFERRABLE
INSERT INTO table_one ... ;
INSERT INTO table_two ... ;
.....
COMMIT;
Finally reached data migration part of my Project and now trying to move data from MySQL to SQL Server.
SQL Server has new schema (mapping is not always one to one).
I am trying to use SSIS for the conversion, which I started learning today morning.
We have customer and customer location table in MySQL and equivalent table in SQL Server. In SQL server all my tables now have surrogate key column (GUID) and I am creating the same in Script Component.
Also note that I do have a primary key in current mysql tables.
What I am looking for is how I can add child records to customer location table with newly created guid as parent key.
I see that SSIS have Foreach loop container, is this of any use here.
if not another possibility that I can think of is create two Data Flow Task and [somehow] just before the master data is sent to Destination Component [Table] on primary dataflow task , add a variable with newly created GUID and another with old PrimaryID, which will be used to create source for DataTask Flow for child records.
May be to simplyfy , this can also be done once datatask for master is complete and then datatask for child reads this master data and inserts child records from MySQL to SQL Server table. This would though mean that I have to load all my parent table records back into memory.
I know this is all too confusing and it is mainly because I am very confused :-(, to bear with me and if you want more information let me know.
I have been through may links that i found through google search but none of them really explains( or I was not able to uderstand) how the process is carried out.
Please advise
regards,
Mar
** Edit 1**
after further searching and refining key words i found this link in SO and going through it to see if it can be used in my scenario
How to load parent child data found in EDI 823 lockbox file using SSIS?
OK here is what I would do. Put the my sql data into staging tables in sql server that have identity columns set up and an extra column for the eventual GUID which will start out as null. Now your records have a primary key.
Next comes the sneaky trick. Pick a required field (we use last_name) and instead of the real data insert the value form the id field in the staging table. Now you havea record that has both the guid and the id in it. Update the guid field in the staging table by joing to it on the ID and the required field you picked out. Now update the last_name field with the real data.
To avoid the sneaky trick and if this is only a onetime upload, add a column to your tables that contains the staging table id. Again you can use this to get the guid for inserting to related tables. Then when you are done, drop the extra column.
You are aware that there are performance issues involved with using GUIDs? Make sure not to make them the clustered index (as the PK they will be by default unless you specify differntly) and use newsequentialid() to populate them. Why are you using GUIDs? If an identity would work, it is usually better to use it.
How can I add a constraint that references a foreign column from another database?
Some time ago I read that it can be done with linked server and others say with triggers. What's the preferred way of doing this if that's possible at all?
Thanks!
Linked servers will not work.
FK's must point to local tables.
The preferred way is not to do this, though you can pull a few hacks to make it happen.
For example you could have triggers cause the reference to be created/checked but I wouldn't consider that equivalent to a FK constraint.
Related question:
Can you have a Foreign Key onto a View of a Linked Server table in SQLServer 2k5?
Use an instead of trigger (you may or may not need linked servers depending on whether the other database is on a differnt server). Make sure the trigger can handle multiple row inserts/updates/deletes. I'd also suggest moving the records that fail the check to an exception table.
Foreign keys cannot go across database boundaries. If you try to do this, you'll get:
Msg 1763, Level 16, State 0, Line 1
Cross-database foreign key
references are not supported. Foreign key ***
Msg 1750, Level 16, State 0, Line 1
Could not create constraint. See previous errors.
If you need to enforce some kind of relationship between two separate databases, then yes - you might need a linked server (if that second database is on a second server), and possibly triggers - but all of those things will be very hard to get right, very inefficient and very error prone.
One way you might be able to do this would be data replication - replicate the table you want to reference into your source database, and then establish a foreign key relationship with that replicated table. But that will never be quite "live" and "real-time" - there will also be a bit of a lag in the data replication.
I get this error when insert a new album:
{System.Data.SqlClient.SqlException:
INSERT statement conflicted with COLUMN FOREIGN KEY
constraint 'FK_ChannelAlbum_Group'.
The conflict occurred in database 'Stamper', table 'Channel', column 'ID'.
I don't know what is going on because sometime I insert the new album to the context the error will occur but Album table has no Channel ID columns only Album_Channel table. This insert does not influence the album_channel and channel table at all. Why is there a conflict.
I realized that after I have created the new album after that i try inserting a album_channel data I have an error so I stop debugger and try to fix the problem. Once i have work that out, I go an insert a new album once more but it gives me this error always. So I close my visual studio and reopen it in order to work.....
I am not sure is it a good way of having singleton style of creating the context e.g.
I have write a context as class and this context only created once, to prevent it to create too many times.
Hard to answer without a diagram. It looks like you have a foreign key in the table you're inserting which is not a valid primary key in the table it is referencing. If you don't believe you have one, then you could try checking your relationships and possibly regenerating your L2S entities. Maybe even try to insert into that table manually and see if you get the same error.
Context wrapped in a singleton is a bad idea for web applications as it isn't thread safe. You could look into the 'unit of work' pattern or try caching the context instance.