I have a table that contains computer login and logoff events. Each row is a separate event with a timestamp, machine name, login or logoff event code and other details. I need to create a SQL procedure that goes through this table and locates corresponding login and logoff event and insert new rows into another table that contain the machine name, login time, logout time and duration time.
So, should I use a cursor to do this or is there a better way to go about this? The database is pretty huge so efficiency is certainly a concern. Any suggested pseudo code would be great as well.
[edit : pulled from comment]
Source table:
History (
mc_id
, hs_opcode
, hs_time
)
Existing data interpretation:
Login_Event = unique mc_id, hs_opcode = 1, and hs_time is the timestamp
Logout_Event = unique mc_id, hs_opcode = 2, and hs_time is the timestamp
First, your query will be simpler (and faster) if you can order the data in such a way that you don't need a complex subquery to pair up the rows. Since MySQL doesn't support CTE to do this on-the-fly, you'll need to create a temporary table:
CREATE TABLE history_ordered (
seq INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
hs_id INT,
mc_id VARCHAR(255),
mc_loggedinuser VARCHAR(255),
hs_time DATETIME,
hs_opcode INT
);
Then, pull and sort from your original table into the new table:
INSERT INTO history_ordered (
hs_id, mc_id, mc_loggedinuser,
hs_time, hs_opcode)
SELECT
hs_id, mc_id, mc_loggedinuser,
hs_time, hs_opcode
FROM history ORDER BY mc_id, hs_time;
You can now use this query to correlate the data:
SELECT li.mc_id,
li.mc_loggedinuser,
li.hs_time as login_time,
lo.hs_time as logout_time
FROM history_ordered AS li
JOIN history_ordered AS lo
ON lo.seq = li.seq + 1
AND li.hs_opcode = 1;
For future inserts, you can use a trigger like below to keep your duration table updated automatically:
DELIMITER $$
CREATE TRIGGER `match_login` AFTER INSERT ON `history`
FOR EACH ROW
BEGIN
IF NEW.hs_opcode = 2 THEN
DECLARE _user VARCHAR(255);
DECLARE _login DATETIME;
SELECT mc_loggedinuser, hs_time FROM history
WHERE hs_time = (
SELECT MAX(hs_time) FROM history
WHERE hs_opcode = 1
AND mc_id = NEW.mc_id
) INTO _user, _login;
INSERT INTO login_duration
SET machine = NEW.mc_id,
logout = NEW.hs_time,
user = _user,
login = _login;
END IF;
END$$
DELIMITER ;
CREATE TABLE dummy (fields you'll select data into, + additional fields as needed)
INSERT INTO dummy (columns from your source)
SELECT * FROM <all the tables where you need data for your target data set>
UPDATE dummy SET col1 = CASE WHEN this = this THEN that, etc
INSERT INTO targetTable
SELECT all columns FROM dummy
Without any code that you're working on.. it'll be hard to see if this approach will be any useful.. There may be some instances when you really need to loop through things.. and some instances when this approach can be used instead..
[EDIT: based on poster's comment]
Can you try executing this and see if you get the desired results?
INSERT INTO <your_target_table_here_with_the_three_columns_required>
SELECT li.mc_id, li.hs_time AS login_time, lo.hs_time AS logout_time
FROM
history AS li
INNER JOIN history AS lo
ON li.mc_id = lo.mc_id
AND li.hs_opcode = 1
AND lo.hs_opcode = 2
AND lo.hs_time = (
SELECT min(hs_time) AS hs_time
FROM history
WHERE hs_time > li.hs_time
AND mc_id = li.mc_id
)
Related
I'm quite new to SQL and databases.
I'm trying to make a preference table of an user.
The fields will be user_id, pref_no, prg_code.
Now if I create the table making pref_no auto_increment then it will automatically increase irrespective of the user_id.
So, my question is - Is there a way to define the table in such a way that it will be auto_incremented taking user_id into account or I have to explicitly find the last pref_no that has occurred for an user and then increment it by 1 before insertion?
Any help is appreciated. Thanks in advance.
Following what Mjh and Fahmina suggested, we can create a procedure for the insertion.
DELIMITER //
CREATE PROCEDURE test(IN u_id INT(7), p_code INT(5))
BEGIN
SELECT #pno:= MAX(pref_no) FROM temp_choice WHERE user_id = u_id;
IF #pno IS NULL THEN
SET #pno = 1;
ELSE
SET #pno = #pno + 1;
END IF;
INSERT INTO temp_choice VALUES (u_id, #pno, p_code);
END //
DELIMITER ;
Now we can easily insert data by using
CALL test(1234567, 10101);
To manage user's preference, you don't need user_id to be auto_incremented in this table, but pref_no has to be.
user_id will just be a refence (or foreign key in sql) to your user table (where user_id should be auto_incremented).
And to request preference for a given user your request would be :
SELECT * FROM [user table] INNER JOIN [pref table] ON ([user table].user_id = [pref table].user_id) WHERE [user table].user_id = ?
(replace '?' by the user_id you want)
Here is the scenario: I have tables stored in my monitoring application database using IDs as part of the table name. For instance, for 10 monitored devices, the device logs are stored in tables for each device with the device ID as part of the name, like this:
Device ID Table Name
1 device_logs.log_1
2 device_logs.log_2
I want to be able to :
select * from all device log tables where ID IN (a-list-of-IDs)
I reference this type of information a lot, and it would be easier to do it in a quick query and possibly a report. For some small list of devices, a union query works, but after about 4-5 devices, that gets too long. Programmatically, I can do this in python with string substitution, but how do you do it in MySQL as a query?
Adding a code segment I am trying to get to work, but struggling with the syntax:
drop table if exists tmp_logs;
create temporary table tmp_logs
(
device_name varchar(30) not null,
date datetime,
message varchar (255)
)
engine=innodb;
drop procedure if exists load_tmp_log_data;
delimiter #
create procedure load_tmp_log_data()
begin
declare vid int unsigned;
truncate table tmp_logs;
start transaction;
while vid in(4976,4956) do
insert into tmp_logs values
( SELECT
dev.device,
date,
message
FROM device_logs.logs_vid
inner join master_dev.legend_device dev on dev.id=vid
where date >= '2014-7-1'
and message not like '%HDMI%'
and message not like '%DVI%'
and message not like '%SIP%'
and message not like '%Completed%'
and message not like '%Template%'
and message not like '%collection%'
and message not like '%Cache%'
and message not like '%Disconnect%'
and message not like '%Row removed%'
and message not like '%detailed discovery%'
and message not like '%restarted%'
and message not like '%Auto Answer%'
);
end while;
commit;
end #
delimiter ;
call load_tmp_log_data();
select * from tmp_logs order by device_name;
You cannot dynamically specify the user name as a query in regular SQL. You could use a prepare statement with dynamic SQL.
An alternative is to set up the query as:
select l.*
from ((select l.*
from device_logs.log_1 l
where device_id = 1 and id in (list of ids)
) union all
(select l.*
from device_logs.log_2 l
where device_id = 2 and id in (list of ids)
) . . .
) l;
You need to repeat the conditions in each subquery -- that makes them more efficient. And, use union all instead of union to avoid duplication.
Fundamentally, though, having tables of the same structure is often a sign of poor database design. It would be much better to have a single table with a column specifying the id. Then your query would be really easy:
select l.*
from logs l
where id in (list of ids);
You could generate such a table by changing the application that creates the tables. Or you could generate such a table by using an insert trigger on each of the subtables. Or, if the data can be a day or so out of date, run a job that re-creates the table every night.
I've got a SQL 2008 R2 table defined like this:
CREATE TABLE [dbo].[Search_Name](
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[Name] [nvarchar](300) NULL),
CONSTRAINT [PK_Search_Name] PRIMARY KEY CLUSTERED ([Id] ASC))
Performance querying the Name field using CONTAINS and FREETEXT works well.
However, I'm trying to keep the values of my Name column unique. Searching for an existing entry in the Name column is unbelievably slow for a large number of names (usually batches of 1,000), even with an index on the Name field. Query plans indicate I'm using the index as expected.
To search for an existing value, my query looks like this:
SELECT TOP 1 Id, Name from Search_Name where Name = 'My Name Value'
I've tried duplicating the Name column to another column and searching on the new column, but the net effect was the same.
At this point, I'm thinking I must be mis-using this feature.
Should I just stop trying to prevent duplication? I'm using a linking table to join these search name values to the underlying data. It seems somehow 'dirty' to just store a whole bunch of duplicate values...
...or is there faster way to take a list of 1,000 names and see which ones are already stored in the database?
The first change to make is to get the entire list to SQL Server at one time. Regardless of how you add the names to the existing table, doing it as a set operation will make a big difference in performance.
Passing the List as a table-valued parameter (TVP) is a clean way to handle it. Have a look here for an example. You can still use an OUTPUT clause to track which rows did or didn't make the cut, for example:
-- Some sample existing names.
declare #Search_Name as Table ( Id Int Identity, Name VarChar(32) );
insert into #Search_Name ( Name ) values ( 'Bob' ), ( 'Carol' ), ( 'Ted' ), ( 'Alice' );
select * from #Search_Name;
-- Some (prospective) new names.
declare #New_Names as Table ( Name VarChar(32) );
insert into #New_Names ( Name ) values ( 'Ralph' ), ( 'Alice' ), ( 'Ed' ), ( 'Trixie' );
select * from #New_Names;
-- Add the unique new names.
declare #Inserted as Table ( Id Int, Name VarChar(32) );
insert into #Search_Name
output inserted.Id, inserted.Name into #Inserted
select New.Name
from #New_Names as New left outer join
#Search_Name as Old on Old.Name = New.Name
where Old.Id is NULL;
-- Results.
select * from #Search_Name;
-- The names that were added and their id's.
select * from #Inserted;
-- The names that were not added.
select New.Name
from #New_Names as New left outer join
#Inserted as I on I.Name = New.Name
where I.Id is NULL;
Alternatively, you could use a MERGE statement and OUTPUT the names that were added, those that weren't, or both.
SELECT LAST_INSERT_ID() as id FROM table1
Why does this query sometimes return the last inserted id of another table other than table1?
I call it in Node.js (db-mysql plugin) and I can only do queries.
LAST_INSERT_ID() can only tell you the ID of the most recently auto-generated ID for that entire database connection, not for each individual table, which is also why the query should only read SELECT LAST_INSERT_ID() - without specifying a table.
As soon as you fire off another INSERT query on that connection, it gets overwritten. If you want the generated ID when you insert to some table, you must run SELECT LAST_INSERT_ID() immediately after doing that (or use some API function which does this for you).
If you want the newest ID currently in an arbitrary table, you have to do a SELECT MAX(id) on that table, where id is the name of your ID column. However, this is not necessarily the most recently generated ID, in case that row has been deleted, nor is it necessarily one generated from your connection, in case another connection manages to perform an INSERT between your own INSERT and your selection of the ID.
(For the record, your query actually returns N rows containing the most recently generated ID on that database connection, where N is the number of rows in table1.)
SELECT id FROM tableName ORDER BY id DESC LIMIT 1
I usually select the auto-incremented ID field, order by the field descending and limit results to 1. For example, in a wordpress database I can get the last ID of the wp_options table by doing:
SELECT option_id FROM wp_options ORDER BY option_id DESC LIMIT 1;
Hope that helps.
Edit - It may make sense to lock the table to avoid updates to the table which may result in an incorrect ID returned.
LOCK TABLES wp_options READ;
SELECT option_id FROM wp_options ORDER BY option_id DESC LIMIT 1;
Try this. This is working
select (auto_increment-1) as lastId
from information_schema.tables
where table_name = 'tableName'
and table_schema = 'dbName'
Most easy way:
select max(id) from table_name;
I only use auto_increment in MySQL or identity(1,1) in SQL Server if I know I'll never care about the generated id.
select last_insert_id() is the easy way out, but dangerous.
A way to handle correlative ids is to store them in a util table, something like:
create table correlatives(
last_correlative_used int not null,
table_identifier varchar(5) not null unique
);
You can also create a stored procedure to generate and return the next id of X table
drop procedure if exists next_correlative;
DELIMITER //
create procedure next_correlative(
in in_table_identifier varchar(5)
)
BEGIN
declare next_correlative int default 1;
select last_correlative_used+1 into next_correlative from correlatives where table_identifier = in_table_identifier;
update correlatives set last_correlative_used = next_correlative where table_identifier = in_table_identifier;
select next_correlative from dual;
END //
DELIMITER ;
To use it
call next_correlative('SALES');
This allows you to reserve ids before inserting a record. Sometimes you want to display the next id in a form before completing the insertion and helps to isolate it from other calls.
Here's a test script to mess around with:
create database testids;
use testids;
create table correlatives(
last_correlative_used int not null,
table_identifier varchar(5) not null unique
);
insert into correlatives values(1, 'SALES');
drop procedure if exists next_correlative;
DELIMITER //
create procedure next_correlative(
in in_table_identifier varchar(5)
)
BEGIN
declare next_correlative int default 1;
select last_correlative_used+1 into next_correlative from correlatives where table_identifier = in_table_identifier;
update correlatives set last_correlative_used = next_correlative where table_identifier = in_table_identifier;
select next_correlative from dual;
END //
DELIMITER ;
call next_correlative('SALES');
If you want to use these workarounds:
SELECT id FROM tableName ORDER BY id DESC LIMIT 1
SELECT MAX(id) FROM tableName
It's recommended to use a where clause after inserting rows. Without this you are going to have inconsistency issues.
in my table inv_id is auto increment
for my purpose this is worked
select `inv_id` from `tbl_invoice`ORDER BY `inv_id` DESC LIMIT 1;
I have this stored procedure, which has been working for a little while
CREATE PROCEDURE `ProjectFromTemplate`(
IN TemplateID INT,
IN NewProjectID INT
)
BEGIN
INSERT INTO Project_Stage
(
ProjectID,
StageID,
StageIndex,
Time
)
(
SELECT
NewProjectID,
StageID,
StageIndex,
Time
FROM Project_Stage
WHERE ProjectID = TemplateID
);
END
But I decided to add another column to the table, but I didn't update the procedure accordingly. I would like the procedure to be able to handle any new rows that I add. I need something like this pseudo-SQL
CREATE PROCEDURE `ProjectFromTemplate`(
IN TemplateID INT,
IN NewProjectID INT
)
BEGIN
INSERT INTO Project_Stage
(
ProjectID,
*
)
(
SELECT
NewProjectID,
*
FROM Project_Stage
WHERE ProjectID = TemplateID
);
END
Is there a way to express something like 'all the subsequent rows' in MySQL?
Leave out the column clause in your INSERT to minimize the code changes, but you have to spell out the rest of the columns from Project_Stage unless it doesn't contain a ProjectID column. If it doesn't, then your good.
Beware: you might still need to recompile after altering the table, though (I'm not sure how MySQL handles schema changes and compiled procs).
CREATE PROCEDURE `ProjectFromTemplate`(
IN TemplateID INT,
IN NewProjectID INT
)
BEGIN
INSERT INTO Project_Stage
SELECT
NewProjectID,
...
the rest of the Project_Stage columns you want go here
...
FROM Project_Stage
WHERE ProjectID = TemplateID
);
END