Database removal of old entries - mysql

My question is more of a high level question than technology specific.
The problem I have is I am inserting into a database a list of users corresponding to a machine. When I do so I have no idea what users were previously but I want to remove users who are in the database that correspond to that machine but no longer are current.
The list that I have is the current list of users for that machine. Any ideas?
Solution 1:
Query DB to get a list of users corresponding to a machine.
Compare against current list and build a remove list and an add list.
Execute Remove
Execute Add
Drawbacks to this solution is that we execute 3 store procedure statements.
Positive is we don't have to change the database.
Solution 2:
Add a user timestamp field to the table.
Run add all users while updating the timestamp field.
Execute new stored procedure to remove all users corresponding
to machine whose updated is less than now.
Drawback is we add a field to each row.
Positive is this is likely only 2 queries.

the easiest solution from my perspective is to alter the table so that you add a datetime or timestamp that defaults to the current time, and on update updates to the current time. That way, when the table is altered this update occurs. Then build a job to run every X period (daily, weekly, monthly, etc) that removes users who have not been updated in some period.
This gives you the most control over your database, and the most accurate data after a short period.

Keep it simple. Why not:
-- Add any new machines
INSERT [Table] (MACHINE, [USER])
SELECT MACHINE, [USER] FROM Query Q WHERE NOT EXISTS(SELECT * FROM [Table] T WHERE T.MACHINE = Q.MACHINE)
-- Delete any removed machines
DELETE [Table] WHERE Machine NOT IN (SELECT Machine FROM Query)
-- Update any changed machines
UPDATE
A
SET
A.CoName = B.CoName
--SELECT *
FROM
Cust A
JOIN (SELECT CustID, CoName FROM Cust) B
ON A.CustID = B.CustID AND A.CoName <> B.CoName
Updated my answer to cover new machines and removed machines. Wrap it all up in a Stored Procedure.

Related

how to delete from mysql multiple rows depending on other rows in another table

I have a table Called phones contains all members phone numbers each member has unique id and there is another table called posts the posts table has many rows each post has its member id and another cells
I need to collect all members IDs from the posts table depending on the branches cell and delete from phones table where the members ids collected from the posts table
I tried this but it give me error message that mysql server lost the connection
delete from `phones` where `mid` in(select `uid` from `posts` where `branches` = 'contact');
Note: I'm using mysql workbench with my local server
UPDATED
this query also worked after I setup the Preferences below
Go to Edit -> Preferences -> SQL Editor and increase the parameter: DBMS connection read time out (in seconds). For instance: 86400.
Close and reopen MySQL Workbench. Kill your previously query that probably is running and run the query again.
Thanks to #scaisEdge
your query seems correct .. anyway you can try with a inner join
delete `phones`.*
from `phones`
inner join `posts` on `phones`.`mid` = `posts`.`uid`
and `posts`.`branches` = 'contact'
the inner join don't use a IN clause so you can bypass the involved limitation
if the error persist try increment the read time out
Go to Edit -> Preferences -> SQL Editor and icrease the parameter: DBMS connection read time out (in seconds). For instance: 86400.
Close and reopen MySQL Workbench. Kill your previously query that probably is running and run the query again.

MySQL copy row from one table to another with multiple NOT IN criteria

We have an old FoxPro DB that still has active data being entered into it. I am in the process of writing a series of .bat files that will update a MySQL database for our web applications that I'm working on.
Our FoxPro databases were never set up with unique IDs or anything useful like that so I'm having to have the query look at a few different fields.
Here's my query thus far:
//traininghistory = MySQL DB
//traininghistory_test = FoxPro DB
INSERT INTO traininghistory
WHERE traininghistory_test.CLASSID NOT IN(SELECT CLASSID FROM traininghistory)
AND traininghistory_test.EMPID NOT IN(SELECT EMPID FROM traininghistory)
What I'm After is this:
I need an query that looks at the 600,000+ entries in the FoxPro DB (traininghistory_test in my code) and compares to the 600,000+ entries in the MySQL DB (traininghistory in my code) and only inserts the ones where the columns CLASSID and EMPID are new- that is, they are NOT in the traininghistory table.
Any thoughts on this (or if you know a simpler/more efficient way to execute this query in MySQL) are greatly appreciated.
One option is to use a outer join / null check:
insert into traininghistory
select values
from traininghistory_test tht
left join traininghistory th on tht.empid = th.empid
and tht.classid = th.classid
where th.empid is null
It's also worth noting, your current query may leave out records since it's not comparing empid and classid in the same records.
One way ist.
CREATE ONE UNIQUE INDEX ON THE COLUMS (CLASSID, EMPID),
THEN
INSERT IGNORE INTO traininghistory SELECT * or fieldlist FROM traininghistory_test;
Thats all

MySQL FOR UPDATE exclusive access

I have several servers hitting a common MySQL box and I need exclusive access to a table of scheduled jobs.
After some reading here and elsewhere I was led to believe SELECT...FOR UPDATE was what I wanted, but now we are (very rarely) seeing multiple servers pick up the same record.
Here's the PROC (minus the BEGIN/END stuff because it was playing hell with my formatting):
CREATE DEFINER=`root`#`%` PROCEDURE `PopScheduledJob`(OUT `JobId` varchar(36) )
SELECT ScheduledJobId INTO JobId
FROM scheduledjob
WHERE
Status = 0
AND NextRun < UTC_TIMESTAMP()
ORDER BY StartDate
LIMIT 1
FOR UPDATE;
UPDATE scheduledjob
SET Status = 2
WHERE ScheduledJobId = JobId;
So the intent here is that it should only pick up a job with Status=0, and it sets it to 1 immediately.
My hope was that this would prevent any other thread/process from accessing the same record, but now it seems that's not the case.
EDIT: forgot to mention, we have an InnoDB backing store

Query-getting last inserted row id in mysql using query

I want to get last inserted row id in mysql any one help me
Table field is
If you want to get the id just after the insertion, use LAST_INSERT_ID():
SELECT LAST_INSERT_ID();
This will return the AUTO_INCREMENT value of the last inserted element (in your current connection).
If you want to know which is the last inserted value in a determined table, use any of the queries provided in the other answers.
You can use LAST_INSERT_ID(), but you should be aware that (you not only should have the AUTO_INCREMENENT), but it does operates in at connection level. This is, it will return the last_insert_id() for the current connection and not the last_insert_id() that another connection --that might be happening meanwhile-- generated.
Example (lets say the operations arrive the database in the following order):
-Connection A : user A -> insert order (id = 1000), lets say $20
-Connection B : user B -> insert order (id = 1001), lets say $50
.... multiple orders from other users are placed
-Connection B : you retrieve last_insert_id (value 1001) and you retrieve the order amount, and update the user B profile with the total revenue generated (+$50).
-Connection A : you retrieve last_insert_id (value 1000) and you retrieve the order amount, and update the user A profile with the total revenue generated (+$20).
You should be aware that last-insert_id() operates on a connection level, therefore you want to keep the connection open until you are finished. You should not use things like do a max on the database, because on a web environment you don't have control on how many users will use your app at same time, and in which order the operations will be executed and most important you want to keep your database consistent.

mysql peformance INSERT into table SELECT for report

I am working on a mysql query for a report. The idea is to have a simple table say 'reportTable' with the values being fetched from various places. I could then use the reportTable more easily without remembering lots of joins etc and also share this table for other projects.
Should I break down the inner insert part of the query so it does
chunks at a time I will be adding probably tens of thousands of rows?
INSERT INTO reportTable
(
-- long query grabbing results from various places
SELECT var1 FROM schema1.table1
SELECT var2 FROM schema2.table1
SELECT var2 FROM schema2.table1
etc
)
This addresses your concerns that inserting data takes too long and so on. I understood it like you rebuild your table each time. So, instead of doing so, just fetch the data that is new and not already in your table. Since looking up if the data is already present in your report table might be expensive, too, just get the delta. Here's how:
Make sure that in every table you need a column like this is present:
ALTER TABLE yourTable ADD COLUMN created timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP;
The ON UPDATE clause is of course optionally, don't know if you need to keep track of changes. If so, give me a comment and I can provide you with a solution with which you can keep a history of your data.
Now you need a small table that holds some meta information.
CREATE TABLE deltameta (tablename varchar(50), LSET timestamp, CET timestamp);
LSET is short for Last Successful Extraction Time, CET for Current Extraction Time.
When you get your data it works like this:
UPDATE deltameta SET CET = CURRENT_TIMESTAMP WHERE tablename = 'theTableFromWhichYouGetData';
SELECT #varLSET := LSET, #varCET := CET FROM deltameta WHERE tablename = 'theTableFromWhichYouGetData';
INSERT INTO yourReportTable (
SELECT whatever FROM aTable WHERE created >= #varLSET AND created < #varCET
);
UPDATE deltameta SET LSET = CET WHERE tablename = 'theTableFromWhichYouGetData';
When anything goes wrong during inserting your script stops and you get the same data the next time you run it. Additionally you can work with transactions here, if you need to roll back. Again, write a comment if you need help with this.
I may be wrong, but you seem to be talking about a basic view. You can read an introduction to views here: http://techotopia.com/index.php/An_Introduction_to_MySQL_Views, and here are the mysql view docs: http://dev.mysql.com/doc/refman/5.0/en/create-view.html