mysql insert vs update performance - mysql

Question

all:
I have a table to record the number of some requests on some dimensions every ten minutes. Here is my table:
CREATE TABLE IF NOT EXISTS `mydb`.`realtime_bid_traffic` (
`id` BIGINT(20) NOT NULL AUTO_INCREMENT COMMENT '',
`owner_id` BIGINT(20) NOT NULL COMMENT '',
`log_time` DATETIME NOT NULL COMMENT '',
`bid_num` BIGINT(10) NOT NULL DEFAULT 0 COMMENT '',
`v_bid_num` BIGINT(10) NOT NULL DEFAULT 0 COMMENT '',
PRIMARY KEY (`id`) COMMENT '',
UNIQUE INDEX `dim_key` USING BTREE (`owner_id` ASC, `log_time` ASC) COMMENT '')
ENGINE = InnoDB;
As you can see, id is an auto increment big integer without any particular meaning. owner_id and log_time is the dimension key while bid_num and v_bid_num is what to be updated. Limited by the business logic it's impossible for me to collect all data before inserting into database, i.e. I may have to insert into database where owner_id=10 and log_time='2015-11-11 11:00:00' two times. Since the table may be quite large (millions of rows) and need to be updated constantly, I have two options:
Insert or update on duplicate key. In this way for each dimension
there will only one row but it involves updates and in order to
improve performance I have built unique key for owner_id and
log_time.
Just insert. In this case I'll remove the unique key for
owner_id and log_time and just insert into database. Since id is the
primary key it will never duplicate, but it may increase table rows
significantly.
I have no idea which may be better from the view of performance.

Answer 1

This is a bit long for a comment.
If you only care about inserting into the table, then the second option is generally faster. Under most circumstances, inserting a new row is faster than a check-for-duplicates-and-insert/update approach. Even as the table grows really big, this remains true. This will remain true as long as the indexes fit into memory.
However, often data has other uses than merely being put into a table. For many querying purposes, not having duplicates might significantly help queries. If you are querying by user_id/log_time (as suggested by the index), then handling the duplicates on the querying side should be trivial -- two rows versus one row has minimal impact and order by id desc limit 1 takes very few resources on two rows.
(Hmmm, I suppose there is an edge case where inserting into a table with billions of rows with an index would be slower than inserting into a table with 10 rows while checking for duplicates, because the index update would be slower than the check-for-duplicates query. However, your use-case is sufficiently far from this situation because you are only talking about 2 duplicates per row.)

Answer 2

Plan A
PRIMARY KEY(id),
UNIQUE(owner_id, log_time)
Every insert must check both keys for dups; this slows down inserts.
Plan B
PRIMARY KEY(id),
INDEX(owner_id, log_time)
This requires that your SELECT code do some type of GROUP BY and aggregation.
Plan C
PRIMARY KEY(owner_id, log_time)
and no id. Why do you have id, anyway? While Plans A and B are always inserting into the data at the "end" of the table (because of AUTO_INCREMENT), Plan C will have multiple "hot spots", one per owner_id. This is OK.
Plan D
INDEX(id),
PRIMARY KEY(owner_id, log_time)
If Plan C is not acceptable, Plan D lets you keep id. No, an AUTO_INCREMENT does not have to be the PRIMARY KEY. IODKU is needed.
Which?
All but Plan B need IODKU (Insert on duplicate key update). But I don't see this as a serious drawback.
Plans C and D probably improve performance of SELECTs, especially if you select by one owner_id.
I prefer the Plans in this order: C, D, B, A. You pick, based on the constraints you can/cannot live with.