I have a simple "account" document with the following view:
function (doc, meta) {
if ( doc.entityType === 'account' && doc.status != "DELETED" ) {
emit(meta.id, null);
}
}
Sometimes, (quite often), after adding an entry I get duplicate entries when query the db using that view. These entries are identical including their meta data.
Once the view retrieve a duplicate entry it will always return it as duplicate (the issues is consistently).
When checking the entries in the bucket I can see only one entry of this id thus the duplication is only in the view.
Once I create another view or delete and recreate this view, all duplicated entries are gone.
I'm using Couchbase 3.0.0 Enterprise Edition (build-1118)
This was a bug in version 3.0.0 that has since been fixed: https://issues.couchbase.com/browse/MB-12110
Upgrade to the latest version (3.0.2 as of this writing) and that should solve the problem.
Related
I have a feed table that contains id, body, created_at fields. When I send Post() on postman after Delete() method the id for the feed table auto_increments as if a record has not been deleted. I am unsure how to rectify this, I am using MySql database, nestjs and TypeORM for the backend.
feed controller.ts
#Controller("feed")
export class FeedController {
constructor(private feedService: FeedService) {}
#Post()
createNewPost(#Body() feedPost: HomeFeedDto): Observable<HomeFeedDto> {
return this.feedService.createPost(feedPost);
}
#Get()
allPosts(): Observable<HomeFeedDto[]> {
return this.feedService.getAllPosts();
}
//api delete method
#Delete(":id")
// delete home feed post by id
deleteFeedPost(#Param("id") id: number): Observable<DeleteResult> {
return this.feedService.deletePost(id);
}
}
This is just the way that auto incrementing columns work in a database. Once a record has been created that uses a particular id value it can never be used again, even if the record that owned it was deleted.
What would you expect to happen in the case where there were many records? If the current incrementing id was 1000 and then you deleted the record with id = 1 would you expect that the next time you inserted a record it would be given id = 1 again instead of id = 1001?
There are lots of practical reasons why re-using a previously issued id would be very bad for business logic especially if anyone who is a consumer of your API has a cached version of the old record.
If you really want to achieve this behavior you would have to look at writing custom functions either inside of the database or your API which check to see if any ids are missing from sequence and then manually assign your own IDs instead of letting the database do it. I would highly recommend you don't do this though as the behavior you're seeing is designed like that for a reason.
I'm working on a project where users could upload excel files into a MySQL database. Those files are the main source of our data as they come directly from the contractors working with the company. They contain a large number of rows (23000 on average for each file) and 100 columns for each row!
The problem I am facing currently is that the same file could be changed by someone (either the contractor or the company) and when re-uploading it, my system should detect changes, update the actual data, and save the action (The fact that the cell went from a value to another value :: oldValue -> newValue) so we can go back and run a versions comparison (e.g 3 re-uploads === 3 versions). (oldValue Version1 VS newValue Version5)
I developed a tiny mechanism for saving the changes => I have a table to save Imports data (each time a user import a file a new row will be inserted in this table) and another table for saving the actual changes
Versioning data
I save the id of the row that have some changes, as well as the id and the table where the actual data was modified (Uploading a file results in a insertion in multiple tables, so whenever a change occurs, I need to know in which table that happened). I also save the new value and the old value which is gonna help me with restoring the "archives data".
To restore a version : SELECT * FROM 'Archive' WHERE idImport = ${versionNumber}
To restore a version for one row : SELECT * FROM 'Archive' WHERE idImport = ${versionNumber} and rowId = ${rowId}
To restore all version for one row : SELECT * FROM 'Archive' WHERE rowId = ${rowId}
To restore version for one table : SELECT * FROM 'Archine' WHERE tableName = ${table}
Etc.
Now with this structure, I'm struggling to restore a version or to run a comparaison between two versions, which makes think that I've came up with a wrong approach since it makes it hard to do the job! I am trying to know if anyone had done this before or what a good approach would look like?
Cases when things get really messy :
The rows that have changed in a version might not have changed in the other version (I am working on a time machine to search in other versions when this happens)
The rows have changed in both versions but not the same fields. (Say we have a user table, the data of the user with id 15 have changed in 2nd and 5th upload, great! Now for the second version only the name was changed, but for the fifth version his address was changed! When comparing these two versions, we will run into a problem constrcuting our data array. name went from "some"-> NULL (Name was never null. No name changes in 5th version) and address went from NULL -> "some' is which obviously wrong).
My actual approach (php)
<?php
//Join records sets and Compare them
foreach ($firstRecord as $frecord) {
//Retrieve first record fields that have changed
$fFields = $frecord->fieldName;
//Check if the same record have changed in the second version as well
$sId = array_search($frecord->idRecord, $secondRecord);
if($sId) {
$srecord = $secondRecord[$sId];
//Retrieve straversee fields that have changed
$sFields = $srecord->fieldName;
//Compare the two records fields
foreach ($fFields as $fField) {
$sfId = array_search($fField, $sFields);
//The same field for the same record was changed in both version (perfect case)
if($sfId) {
$sField = $sFields[$sfId];
$deltaRow[$fField]["oldValue"] = $frecord->deltaValue;
$deltaRow[$fField]["newValue"] = $srecord->deltaValue;
//Delete the checked field from the second version traversee to avoid re-checking
unset($sField[$sfId]);
}
//The changed field in V1 was not found in V2 -> Lookup for a value
else {
$deltaRow[$fField]["oldValue"] = $frecord->deltaValue;
$deltaRow[$fField]["newValue"] = $this->valueLookUp();
}
}
$dataArray[] = $deltaRow;
//Delete the checked record from the second version set to avoid re-checking
unset($secondRecord[$srecord]);
}
I don't know how to deal with that, as I said I m working on a value lookup algorithm so when no data found in a version I will try to find it in the versions between theses two so I can construct my data array. I would be very happy if anyone could give some hints, ideas, improvements so I can go futher with that.
Thank you!
Is there a way to store database modifications with a versioning feature (for eventual versions comparaison [sic!])?
What constitutes versioning depends on the database itself and how you make use of it.
As far as a relational database is concerned (e.g. MariaDB), this boils down to the so called Normal Form which is in numbers.
On Database Normalization: 5th Normal Form and Beyond you can find the following guidance:
Beyond 5th normal form you enter the heady realms of domain key normal form, a kind of theoretical ideal. Its practical use to a database designer os [sic!] similar to that of infinity to a bookkeeper - i.e. it exists in theory but is not going to be used in practice. Even the most demanding owner is not going to expect that of the bookkeeper!
One strategy to step into these realms is to reach the 5th normal form first (do this just in theory, by going through all the normal forms, and study database normalization).
Additionally you can construe versioning outside and additional to the database itself, e.g. by creating your own versioning system. Reading about what you can do with normalization will help you to find better ways to decide on how to structure and handle the database data for your versioning needs.
However, as written it depends on what you want and need. So no straight forward "code" answer can be given to such a general question.
although I have searched in various resources I cannot understand how to correctly insert the ‘status’ column, I will explain better.
I have two sql tables:
From the gestionepc form using dropdown, I can select the numerazionecolumn of thenumerazionitable and so far everything works without problems. However, I need to insert the “status” column on thenumerazionitable so that if I use a record in thenumerazionicolumn, itsstatus` must automatically change to “Not active” as it is already used. For my project, it is a requirement derived from the fact that I have various groups of user permissions and various authorizations.
In practice, I'm very confused about how to insert the status column (type of column, default value, storage also on the MySQL database and initialize it) and on how to make it work through the code (perhaps using afterSave and beforeUpdate).
I read Active Record from Guide Yii2 but I don’t understand.
I modify table numerazioni in this mode:
And I try this in model Numerazioni
const STATUS_INDISPONIBILE = 'Indisponibile';
const STATUS_DISPONIBILE = 'Disponibile';
public function setStatusnumerazione()
{
if (\app\models\Gestionepc::find()->where(!isEmpty('numerazioni_id'))) {
$this->statusnumerazione = self::STATUS_INDISPONIBILE;
}
else {
$this->statusnumerazione = self::STATUS_DISPONIBILE;
}
}
But not working. Thanks in advance.
So here's my problem:
I have an article submission form with an optional image upload field.
When the user submits the form - this is roughly what happens:
if($this->view->form->isValid($_POST){
$db->beginTransaction();
try{
// save content of POST to Article table
if(!$this->_saveArticle($_POST)){
return;
}
// resize and save image using ID generated by previous condition
if(!$this->_saveImage($_FILES){
$db->rollback();
return;
}
// update record if image successfully generated
if(!$this->_updateArticle(){
$db->rollback();
}
$db->commit();
}
}catch (Exception $e){
$db->rollback()
}
All Models are saved using mappers, which automate "UPSERT" functionality by checking for the existence of a surrogate key
public function save($Model){
if(!is_null($Model->id_article){
$Mapper->insert($Model->getFields());
return;
}
$Mapper->update($Model->getFields(),$Model->getIdentity());
}
The article table has a composite UNIQUE index of ID,Title and URL. In addition, I'm generating a UID that gets added to the ID field of the Model prior to insert (instead of auto-incrementing)
When I try to execute this, it runs fine for the first article inserted into the table - but subsequent calls (with radically different input) triggers a DUPLICATE KEY error. MySQL throws back the ID generated in condition 1 (_saveArticle) and complains that the key already exists...
I've dumped out the Model fields (and the condition state - i.e. insert | update) and they proceed as expected (pseudo):
inserting!
id = null
title = something
content = something
image = null
updating!
id = 1234123412341234
title = something
content = something else
image = 1234123412341234.jpg
This row data is not present in the database.
I figure this could be one of a few things:
1: I'm loading a secondary DB adapter on user login, allowing them to interface with several sites from one login - this might be confusing the transaction somehow
2: It's a bug of some description in the Zend transaction implementation (possibly triggered by 1)
3: I need to replace the save() with an INSERT ... ON DUPLICATE
4: I should restructure the submission process, or generate a name for the image that isn't dependent on the UID of the previously inserted row.
Still hunting, but I was wondering if anyone else has encountered this kind of issue or could point me in the direction of a solution
best SWK
OK - just for the record, this is entirely possible. The problem was in my application architecture. I was catching Exceptions in my Mapper classes that were handling persistence - and then querying them to return boolean states and thus interrupt the process. This was in turn breaking the try/catch loop which was preventing the insert/update from working correctly.
To summarise - Yes - you CAN insert and update the same row in a single transaction. I've ticked community wiki to cancel rep out
I have a windows application. I am trying to insert a record through a DataContext. It has Unique identifier in the table. Even I am executing a trigger after insertion. So I am making a select query in the end of the trigger to get the auto generator number and to avoid auto-sync error. As it's a windows application I can keep the Context for longtime. When I create a new object ( for example order) and do the same previous operation, upon SubmitChanges operation, it shows cannot have duplicate key. Why can't I use this same Context to Insert the second record? Or do I need to create a new Context to insert a new Record?(Does this Unit of work Concept comes here?). Creating new Context is bad idea as I need to load all data again..
Any thought?
Some code sample to explain my situation:
CallCenterLogObjCotext = (CallCenterLogObjCotext == null ? (new CallcenterLogContext) : (CallCenterLogObjCotext));
CallDetail newCallDetailsOpenTicket = new CallDetail();
newCallDetailsOpenTicket.CallPurpose = (from callpuposelist in CallCenterLogObjCotext.CallPurposes
where callpuposelist.CallPurposeID == ((CallPurpose)(cbcallpurpose.SelectedItem)).CallPurposeID
select callpuposelist).FirstOrDefault();
Lots of settings like this ...
CallCenterLogObjCotext.CallDetails.InsertOnSubmit(newCallDetailsOpenTicket);
CallCenterLogObjCotext.SubmitChanges();
As I mentioned above, this is a click on Open Ticket button on windows form. I change the values of fname, lname and all in the textboxes available on that form and clicked the same button.
So it will call the same method again. I get the below specified error:
System.Data.Linq.DuplicateKeyException: Cannot add an entity with a key that is already in use.
You can insert more than one row with the same context object, see http://weblogs.asp.net/scottgu/archive/2007/05/19/using-linq-to-sql-part-1.aspx, http://msdn.microsoft.com/en-us/library/bb425822.aspx, and other numerous online examples. The duplicate key issue could be a linq to sql configuration issue, or a database integrity error, i.e. such as if you have a natural primary key on a table and are trying to insert a row with the same natural primary key more than once.