I have a view ObjectDisplay that is composed of two relevant tables: Object and State. State represents the state of an Object, and the view pulls some of the details from the most recent State for each Object.
On the page that is displaying this information, a user can enter some comments, which creates a new State. After creating the new State, I immediately pull the Object from ObjectDisplay and send it back to be dropped into a partial view and replace the Object in the grid on the page.
// Add new State.
db.States.Add(new State()
{
ObjectId = objectId,
Comments = comments,
UserName = username
});
// Save the changes (executes all of the above).
db.SaveChanges();
// Return the new Object information.
return db.Objects.Single(c => c.ObjectId == objectId);
According to my db trace, the Single call occurs about 70 ms after the SaveChanges call, and it occurs on the same SPID.
Now for the issue: The database defaults the value of RecordDate in State to GETUTCDATE() - I don't provide the date myself. What I'm seeing is that the Object returned has the State's RecordDate of the old State and the Comments of the new State information of the old State. I am seeing that the Object returned has the old State's information. When I refresh the page, all the correct information is there, but the wrong information is returned in the initial call from the database/EF.
So.. what could be wrong? Could the view not be updating quickly enough? Could something be going on with EF? I don't really know where to start looking.
If you've previously loaded the same Object entity in the same DbContext, EF will return the cached instance with the stale values, and ignore the values returned from SQL.
The simplest solution is to reload the entity before returning it:
var result = db.Objects.Single(c => c.ObjectId == objectId);
db.Entry(result).Reload();
return result;
This is indeed odd. In SQL Server views are not persisted by default and therefore show changes in the underlying data right away. You can create a clustered index on a view with effectively persists the query, but in that case the data is updated synchronously, so you should see the change right away.
If you are working with snapshot isolation level your changes might not be visible to other SPIDs right away, but as you are on the same SPID and do not use snapshot isolation, this cant be the culprit either.
The only thing left at this point is the application layer. Are you actually using the result of the Single call higher up in the call stack or does that get lost somewhere. I assume that a refresh of the page uses a different code path, which would explain why it is working there.
Related
Thanks in advance for attempting to asssist me with this issue.
I'm using CakePHP 2 (2.10.22).
I have a system which creates applications. Each application that gets created has a unique application number. The MySQL database column that stores this application number is set to 'Not null' and 'Unique'. I'm using CakePHP to get the last used application number from the database to then build the next application number for the new application that needs to be created. The process that I have written works without any problem when a single request is received at a given point in time. The problem arises when two requests are received to create an application at the exact same time. The behaviour that I have observed is that the the request that gets picked up first gets the last application number - e.g. ABC001233 and assigns ABC001234 as the application number for the new application it needs to create. It successfully saves this application into the database. The second request which is running concurrently also gets ABC001233 as the last application number and tries to create a new application with ABC001234 as the application number. The MySQL database returns an error saying that the application number is not unique. I then put the second request to sleep for 2 seconds by which time the first application has successfully saved to the database. I then re-attempt the application creation process which first gets the last application number which should be ABC001234 but instead each database read keeps returning ABC001233 even though the first request has long been completed. Both requests have transactions in the controller. What I have noticed is that when I remove these transactions, the process works correctly where for the second request after the first attempt fails, the second attempt works correctly as the system correctly gets ABC001234 as the last application number and assigns ABC001235 as the new application number. I want to know what I need to be doing so as to ensure the process works correctly even with the transaction directives in the controller.
Please find below some basic information on how the code is structured -
Database
The last application number is ABC001233
Controller file
function create_application(){
$db_source->begin(); //The process works correctly if I remove this line.
$result = $Application->create_new();
if($result === true){
$db_source->commit();
)else{
$db_source->rollback();
}
}
Application model file
function get_new_application_number(){
$application_record = $this->find('first',[
'order'=>[
$this->name.'.application_number DESC'
],
'fields'=>[
$this->name.'.application_number'
]
]);
$old_application_number = $application_record[$this->name]['application_number'];
$new_application_number = $old_application_number+1;
return $new_application_number;
}
The above is where I feel the problem originates. For the first request that gets picked up, this find correctly finds that ABC001233 is the last application number and this function then returns ABC001234 as the next application number. For the second request, it also picks up ABC001233 as the last application number but will fail when it tries to save ABC001234 as the application number as the first request has already saved an application with that number. As a part of the second attempt for the second request (which occurs because of the do/while loop) this find is requested again, but instead of returning ABC001234 as the last application number (per the successfuly save of the first request), it keeps returning ABC001233 resulting in a failure to correctly save. If I remove the transaction from the controller, this then works correctly where it will return ABC001234 in the second attempt. I couldn't find any documentation as to why that is and what can be done about the same and is where I need some assistance. Thank you!
function create_new(){
$new_application_number = $this->get_new_application_number();
$save_attempts = 0;
do{
$save_exception = false;
try{
$result = $this->save([$this->name=>['application_number'=>$new_application_number]], [
'atomic'=>false
]);
}catch(Exception $e){
$save_exception = true;
sleep(2);
$new_application_number = $this->get_new_application_number();
}
}while($save_exception === true && $save_attempts++<5);
return !$save_exception;
}
You just have to lock the row with the previous number in a transaction using SELECT ... FOR UPDATE. It's much better than the whole table lock as said in the comments.
According to documentation https://book.cakephp.org/2/en/models/retrieving-your-data.html you just have to add 'lock' => true to get_new_application_number function:
function get_new_application_number(){
$application_record = $this->find('first',[
'order'=>[
$this->name.'.application_number DESC'
],
'fields'=>[
$this->name.'.application_number'
],
'lock'=>true
]);
$old_application_number = $application_record[$this->name]['application_number'];
$new_application_number = $old_application_number+1;
return $new_application_number;
}
How does it work:
The second transaction will wait on that request while the first transaction is ended.
P.S. According to documentation lock option was added in the 2.10.0 version of CakePHP.
Can anyone advise on the following problem:
I have a custom get_or_create method, which checks multiple fields and does some fancy stuff upon creation:
def fancy_get_or_create(name):
object = self.fancy_get(name)
if not object:
object = self.fancy_create(name)
return object
def fancy_get(name):
return self.filter(Q(name=name) | Q(alias=name)).first()
def fancy_create(name):
name = self.some_preprocessing(name)
return self.create(name=name, alias=name)
There's a race condition, where one request will check to see if the object exists, find nothing, and start creating it. Before that request finishes creating the object, another request comes in looking for the same object, finds, nothing, and begins creating the new object. This request will fail because the database has some uniqueness constraints (the previous request had just created the object, so the second request will fail).
Is there any way to prevent request 2 from querying the database until request 1 has finished? I was reading about transaction management and it did not seem like the solution, since the issue is not partial updates (which would suggest an atomic transaction), but rather the need to make the second request wait until the first has finished.
Thanks!
Update:
Here's what I went with:
try:
return self.fancy_get(name) or self.fancy_create(name)
except IntegrityError:
return self.fancy_get(name)
There are two viable solutions:
Use a mutex so only one process can access the fancy_get_or_create
function at the same time.
Capture the error thrown by the database and do something instead: ignore
that create, update the row instead of creating it, throw an
exception, etc.
Edit: another solution might be doing an INSERT IGNORE instead of just an INSERT. https://dev.mysql.com/doc/refman/5.1/en/insert.html
I'm having a strange issue.
I created a model observer for my user model. The model observer is being run at 'saving'. when I dump the object at the very end of the user model to be displayed ( this is just before it saves.. according to laravel docs ) it displays all the attributes set correctly for the object, I've even seen an error that showed the correct attributes as set and being inserted into my database table. However, after the save has been completed and I query the database, two of the fields are not saved into the table.
There is no code written by myself sitting between the point where I dumped the attributes to check that they had been set and the save operation to the database. so I have no idea what could be causing this to happen. All the names are set correctly, and like I said, the attributes show as being inserted into the database, they just never end up being saved, I receive no error messages and only two out of ten attributes aren't being saved.
In my searches I have found many posts detailing that the $fillable property should be set, or issues relating to a problem with variables being misnamed or unset, however because I already have the specific attributes not being saved specified in the $fillable array, on top of the fact that they print out exactly as expected pre save, I don't believe those issues are related to the problem I am experiencing.
to save I'm calling:
User::create(Input::all());
and then the observer that handles the data looks like this:
class UserObserver {
# a common key between the city and state tables, helps to identify correct city
$statefp = State::where('id',$user->state_id)->pluck('statefp');
# trailing zeros is a function that takes the first parameter and adds zeros to make sure
# that in this case for example, the dates will be two characters with a trailing zero,
# based on the number specified in the second parameter
$user->birth_date = $user->year.'-'.$user->trailingZeros( $user->month, 2 ).'-'.$user->trailingZeros( $user->day, 2 );
if(empty($user->city)){
$user->city_id = $user->defaultCity;
}
$user->city_id = City::where( 'statefp', $statefp )->where('name', ucfirst($user->city_id))->pluck('id');
# if the user input zip code is different then suggested zip code then find location data
# on the input zip code input by the user from the geocodes table
if( $user->zip !== $user->defaultZip ){
$latlon = Geocode::where('zip', $user->zip)->first();
$user->latitude = $latlon['latitude'];
$user->longitude = $latlon['longitude'];
}
unset($user->day);
unset($user->month);
unset($user->year);
unset($user->defaultZip);
unset($user->defaultCity);
}
that is the code for the two values that aren't being set, when I run
dd($user);
all the variables are set correctly, and show up in the mysql insert attempt screen with correct values, but they do not persist past that point.. it seems to me that possibly mysql is rejecting the values for the city_id and the birth_date. However, I cannot understand why, or whether it is a problem with Laravel or mysql.
since I was calling
User::create();
I figured I'd try to have my observer listen to:
creating();
I'm not sure why it only effected the date and city variables, but changing the function to listen at creating() instead of saving() seems to have solved my problem.
I'm a bit confused in regards to how EF's dbContext works.
If I do something like _context.Persons.Add(_person) (assuming person is a valid entity), if I then (before calling _context.SaveChanges()) query Persons, will the person I just added be included in the results?
For example:
Person _person = new Person() {Firstname = "Bill", Lastname = "Snerdly"};
_context.Persons.Add(_person);
var _personList = _context.Persons.Where(p => p.Lastname.StartsWith("Sne"));
Whenever I try this, it seems as though the context loses track of the fact that I've added this new person to the context.
What confuses me is that if I edit an existing person and attach the person and set the state to modified, querying the context seems to keep track of the changes that were made and returns them in the results. For example:
//Assuming that Person 5 exists with the name William Snerdly
Person _person = new Person() {Id = 5, Firstname = "Bill", Lastname = "Snerdly"};
_context.Persons.Attach(_person);
_context.Entry(_person).State = System.Data.EntityState.Modified;
var _personList = _context.Persons.Where(p => p.Lastname.StartsWith("Sne"));
In this case, it seems like the person with the id of 5 will show up in the list with the name Bill instead of William. IOW, the context queried the data but retained the changes while in the first scenario, the context queried the data but ignored any added items. It just seems a bit inconsistant.
Am I understanding this correctly or am I missing something?
Thanks for your help with this.
No, as it does not yet exist in the database. It will, however, be accessible through the ObjectStateManager of the ObjectContext, or alternatively, if you're using the DbContext/DbSet wrappers, through the .Local property of the DbSet.
In the case of the edit, you're seeing the ORM's first level cache at work. The query is executed against the database (and so compares against the values in there - your example would get even weirder if you modified the Lastname in the context, but still get the result from the query looking for the unmodified Lastname), but when its results are processed, first the ID of the returned entity is checked, and since the entity with that ID is already present in the context, you get that instance back. This is the default "AppendOnly" mode of operation.
I don't know what you want to do, but I had to understand all that when I wanted to validate my changes according to rules that needed to use the values of both loaded and unread entities. I ended up starting a transaction, saving the changes with the "None" options, doing my validation queries againt the database (which then contained the "merged" view of the data), and the rolling back the transaction if the data was invalid, or accepting the changes and committing the transaction otherwise.
I ran into an interesting problem while using DLINQ. When I instantiate an entity, calling .SubmitChanges() on the DataContext will insert a new row into the database - without having ever called .Insert[All]OnSubmit(...).
//Code sample:
Data.NetServices _netServices = new Data.NetServices(_connString);
Data.ProductOption[] test = new Data.ProductOption[]
{
new Data.ProductOption
{
Name="TEST1",
//Notice the assignment here
ProductOptionCategory=_netServices.ProductOptionCategory.First(poc => poc.Name == "laminate")
}
};
_netServices.SubmitChanges();
Running the code above will insert a new row in the database. I noticed this effect while writing an app to parse an XML file and populate some tables. I noticed there were 1000+ inserts when I was only expecting around 50 or so - then I finally isolated this behavior.
How can I prevent these objects from being persisted implicitly?
Thanks,
-Charles
Think of the relationship as having two sides. When you set one side of the relationship the other side needs to be updated so in the case above as well as setting the ProductOptionCategory it is effectively adding the new object to the ProductOptions relationship on the laminate ProductOptionCategory side.
The work-around is as you have already discovered and to set the underlying foreign key instead so LINQ to SQL will not track the objects in the usual way and require implicit indication it should persist the object.
Of course the best solution for performance would be to determine from the source data which objects you don't want to add and never create the instance in the first place.