I'm teaching myself Delphi database programming using a MySQL database. I'm trying to add a record from a nested ClientDataSet with the link between master and detail tables an autoincrement field in the master table. I found a question/answer pair that appears to answer my question at: Inserting records with autoincrementing primary keys
The thing I don't understand is setting the required flag in the Query. I can't figure out how to do that as I'm too inexperienced, nor do I understand why it is necessary.
Similar to the question linked above, I have a
(SQLConnection->TSQLDataSet->DataSetProvider->ClientDataSet using dbexpress.
| |->LinkDataSource
->TSQLDataSet2->LinkDataSource
I load data into my nested ClientDataSet fine, so the component links to create the nested structure work. After loading the master/detail tables into the nested dataset, the following code gives an error.
MasterCDS1.Append;
MasterCDS1.FieldByName('TLNo').Required := False;
MasterSDS.FieldByName('TLNo').Required := False; { Error: Field 'TLNo' not found }
MasterCDS1.FieldByName('TLNo').ProviderFlags := [pfInWhere, pfInKey];
{ ... Populate Master table Fields}
MasterCDS1.Post;
MasterCDS1.ApplyUpdates(0);
TLNo is the field linking the tables and part of the primary key of the master table, and part of the primary key of the detail table. The third line where I try to set the TSQLDataSet generates the error shown in the comment. MasterSDS is where I put my 'Select * from master' query. MasterCDS learns the Schema from this query and that the field TLNo is a required field in both master and detail MySQL tables. That third line of code is my "interpretation" of what Mr Uwe Raabe said to do. Clearly I did this wrong. Can someone provide a code example so that this Delphi noob won't misinterpret the instructions? Thanks in advance.
The only reason I can imagine for the error you describe is that MasterSDS is not open when you execute that third line. "Field not found" raises either when the field does not exist in the table or the dataset (i.e. query in this case) is not open and has no static fields defined.
This leads to another point I want to mention: place the Required and the ProviderFlags settings in the AfterOpen event of the corresponding dataset. There is no need to repeat these settings whenever you append a record. If you work with static fields you can even do these settings in the Object Inspector.
For being a starter I suggest you always use static fields which can be adjusted inside the IDE. This will simplify things significantly.
Related
MySQL 5.7
I know squat about databases but I do have a custom program that uses the databases I have. If I want to use another account (twitter) with my program, I need to make all the datasets. I've done this a few times before. It always takes me a while to figure out, but I get there.
Yet this time I'm stuck with a read-only dataset.
What I usually do is I make a new schema with a proper name.
I then go to one of my other schemas, export what I need ('sources' and 'entries') and import them to my newly made schema.
That should work and it looked like it worked again this time. BUT there is a difference... It's read-only and I am way to green to tackle this on my own. I'll try to investigate a bit more myself, but some help sure would be appreciated : ) The warning is: "read only: the table has no unique row identifier (primary key or a NOT NULL unique index)": I couldn't find any results with this exact warning so I thought to post it here :)
Thank you!
Devvie
See my comments above on how to solve it.
TL;DL - if you are stuck with a read only schema after importing your data, you might want to tick the correct checkboxes for your schemas. Select schema, select Wrench Icon and match the table to be the same as the schema's table you imported from.
Thank you all.
I have a table in the database with the following columns: ID, Name, Txt. We are using Linq To Sql to implement our DAL. In there another collegue added two extra columns so in the code the same table results: ID, Name, Txt, NameTemp, TxtTemp.
These two "fake" tables are used in different parts of the code in LINQ joins and analyzing with SQL Profiler the parsed SQL query takes the "real" columns and everything works properly.
Now I need to make an INSERT using that table, but I get an exception since also the fake columns are used in the statement.
Since I cannot add the two fake columns in the DB(since unuseful there), is there a way in which I could make an insert with Linq omitting these two columns?
I think i know where you're getting at. You should be able to add properties to a partial linq class no problem, only thing is that if you try and use a linq query against these "fake" columns, you'll get an exception when linqtosql tries to reference a column that doesn't exist in the database. I've been through this before - i wanted to be able to select columns that don't exist in the database (but do in the linq2sql dbml class) and have linq2sql translate the columns into what they really are in the database. Only problem is that there's no real easy way to do this - you can add attributes to the "fake" properties so that linq2sql thinks that NameTmp and TxtTmp are in fact Name and Txt in the sql world, only problem is that when it comes to inserting a record, the translated sql specifies the same column twice (which SQL doesn't like and throws an exception).
You can mark the column with IsDbGenerated = true - that'll let you insert records without getting the double column problem, but you can't update a record without linqtosql complaining that you can't update a computed column. I guess you can use a sproc to get around this perhaps?
I logged a bug with Microsoft a while back, which they'll never fix. The info here might help you get what you need -
http://social.msdn.microsoft.com/Forums/eu/linqtosql/thread/5691e0ad-ad67-47ea-ae2c-9432e4e4bd46
https://connect.microsoft.com/VisualStudio/feedback/details/526402/linq2sql-doesnt-like-it-when-you-wrap-column-properties-with-properties-in-an-interface
LINQ is not for inserting data, but for querying only - Language INtegrated Query. Use ADO.NET for inserting the data.
(Leaving the first part to remind my stupidity)
Check ScottGu. The classes generated are partial (mentioned here), so you can put your 2 properties into the editable part and since they won't have any mapping attribute defined, they won't be mapped nor persisted.
I am a bit rusty with mysql and trying to jump in again..So sorry if this is too easy of a question.
I basically created a data model that has a table called "Master" with required fields of a name and an IDcode and a then a "Details" table with a foreign key of IDcode.
Now here's where its getting tricky..I am entering:
INSERT INTO Details (Name, UpdateDate) Values (name, updateDate)
I get an error: saying IDcode on details doesn't have a default value..so I add one then it complains that Field 'Master_IDcode' doesn't have a default value
It all makes sense but I'm wondering if there's any easy way to do what I am trying to do. I want to add data into details and if no IDcode exists, I want to add an entry into the master table. The problem is I have to first add the name to the fund Master..wait for a unique ID to be generated(for IDcode) then figure that out and add it to my query when I enter the master data. As you can imagine the queries are going to probably get quite long since I have many tables.
Is there an easier way? where everytime I add something it searches by name if a foreign key exists and if not it adds it on all the tables that its linked to? Is there a standard way people do this? I can't imagine with all the complex databases out there people have not figured out a more easier way.
Sorry if this question doesn't make sense. I can add more information if needed.
p.s. this maybe a different question but I have heard of Django for python and that it helps creates queries..would it help my situation?
Thanks so much in advance :-)
(decided to expand on the comments above and put it into an answer)
I suggest creating a set of staging tables in your database (one for each data set/file).
Then use LOAD DATA INFILE (or insert the rows in batches) into those staging tables.
Make sure you drop indexes before the load, and re-create what you need after the data is loaded.
You can then make a single pass over the staging table to create the missing master records. For example, let's say that one of your staging table contains a country code that should be used as a masterID. You could add the master record by doing something along the lines of:
insert
into master_table(country_code)
select distinct s.country_code
from staging_table s
left join master_table m on(s.country_code = m.country_code)
where m.country_code is null;
Then you can proceed and insert the rows into the "real" tables, knowing that all detail rows references a valid master record.
If you need to get reference information along with the data (such as translating some code) you can do this with a simple join. Also, if you want to filter rows by some other table this is now also very easy.
insert
into real_table_x(
key
,colA
,colB
,colC
,computed_column_not_present_in_staging_table
,understandableCode
)
select x.key
,x.colA
,x.colB
,x.colC
,(x.colA + x.colB) / x.colC
,c.understandableCode
from staging_table_x x
join code_translation c on(x.strange_code = c.strange_code);
This approach is a very efficient one and it scales very nicely. Variations of the above are commonly used in the ETL part of data warehouses to load massive amounts of data.
One caveat with MySQL is that it doesn't support hash joins, which is a join mechanism very suitable to fully join two tables. MySQL uses nested loops instead, which mean that you need to index the join columns very carefully.
InnoDB tables with their clustering feature on the primary key can help to make this a bit more efficient.
One last point. When you have the staging data inside the database, it is easy to add some analysis of the data and put aside "bad" rows in a separate table. You can then inspect the data using SQL instead of wading through csv files in yuor editor.
I don't think there's one-step way to do this.
What I do is issue a
INSERT IGNORE (..) values (..)
to the master table, wich will either create the row if it doesn't exist, or do nothing, and then issue a
SELECT id FROM master where someUniqueAttribute = ..
The other option would be stored procedures/triggers, but they are still pretty new in MySQL and I doubt wether this would help performance.
I have a relatively simple Django app, with quite heavy usage that is responsible for quite some concurrency in the db operations.
I have a model Post with a m2m to a Tag model.
A single line in my code, p.add(t) is repeatedly causing mysql exceptions (where p is a Post instance and t is a Tag instance.)
IntegrityError: (1062, "Duplicate entry '329051-1827414' for key 'post_id'")
When this is raised I can manually run this p.add(t) successfully, so it must have to do with some peculiar state that the db/app are in at the time of normal execution. It happens about once every 1000 tag-adding attempts, without any pattern that I can detect (i.e both numbers in the "329051-1827414" pair of the example change)
A CHECK TABLE in mysql on the relevant table shows that they are all seemingly OK.
Any ideas?
Usually you see errors like that when trying to add to an intermediate table if the row being added duplicates the unique-together constraint for the FK's. I'm guessing that in the example you provided "329051" is a Post id and "1827414" is a Tag id.
Normally in Django you can call the add() method repeatedly to add the same instance and Django takes care of everything for you. I'm assuming the model manager maintains some state to help it determine if each add() represents a new or existing row and if the row appears to be new it attempts an insert.
That in itself doesn't explain why you're getting the error. You mention "is responsible for quite some concurrency in the db operations.". Without knowing what that means, I'm guessing that you could be getting a race condition where multiple thread/processes are attempting to add the same new tag around the same time and both are attempting inserts.
I think I'm seeing a similar problem in my app - If I send two identical requests to add a m2m relation (e.g. tag in my case as well), I get that error because the m2m table has a unique constraint on (user, tag). I'm guessing the server is processing the .add functions at the same time.
if not already in database:
# Both invocations reach here because the next line takes some time to process.
create m2m row
I don't know how that can be remedied.
I have a question related to this one. I don't want to do a calculation (aggregation), but I need to get display values from an association. In my C# code, I can directly reference the value, because the foreign key constraint made Linq generate all the necessary wiring.
When I specify the IQueryable as the Gridview datasource property, and reference something that is not a column of the primary entity in the result set, I get an error that the column does not exist.
As a newbie to Linq, I am guessing the assignment implicitely converts the IQueryable to a list, and the associations are lost.
My question is, what is a good way to do this?
I assume that I can work around this by writing a parallel query returning an anonymous type that contains all the columns that I need for the gridview. It seems that by doing that I would hold data in memory redundantly that I already have. Can I query the in-memory data structures on the fly when assigning the data source? Or is there a more direct solution?
The gridview is supposed to display the physician's medical group associations, and the name of the association is in a lookup table.
IQueryable<Physician> ph =
from phys in db.Physicians
//from name in phys.PhysicianNames.DefaultIfEmpty()
//from lic in phys.PhysicianLicenseNums.DefaultIfEmpty()
//from addr in phys.PhysicianAddresses.DefaultIfEmpty()
//from npi in phys.PhysicianNPIs.DefaultIfEmpty()
//from assoc in phys.PhysicianMedGroups.DefaultIfEmpty()
where phys.BQID == bqid
select phys;
(source: heeroz.com)
So, based on Denis' answer, I removed all the unneeded stuff from my query. I figured that I may not be asking the right question to begin with.
Anyways, the page shows a physician's data. I want to display all medical group affiliations in a grid (and let the user insert, edit, and update affiliations). I now realize that I don't need to explicitly join in these other tables - Linq does that for me. I can access the license number, which is in a separate table, by referencing it through the chain of child associations.
I cannot reference the medical group name in the gridview, which brings me back to my question:
AffiliationGrid.DataSource = ph.First().PhysicianMedGroups;
This does not work, because med_group_print_name is not accessible for the GridView:
A field or property with the name 'med_group_print_name' was not found on the
selected data source.
Again, bear with me, if it is all too obvious that I don't understand Linq ... because I don't.
Your query seems strange. You should try to simply display
ph = from phys in db.Physicians
where phys.BQID == bqid
select phys;
in your grid. That should work.
Also, why the calls to Load()? If the DataContext is not disposed when the grid is binding, you should not need it.
If you still have issues, can you please post the error message you get, that would help...
Part 2
The problem is that you have the name is effectively not in the PhysMedGroup. You need to navigate one level down to the MedGroupLookup to access the name, since it is a property of that class.
Depending on the technology you are using (it seems to be either WinForms or Web Forms), you will need to configure your data-binding to access MedGroupLookup.med_group_print_name.