Related
My database is composed by individual job contracts. I am updating some information to enhance the quality of the data. More precisely, I am updating information regarding workers' residence codes. In the following image I am showing an example of my database in the following image (the .csv version could be found here).
While variables are explained here below.
id -----------> "Primary key" [indexed]
worker_id ----> "Id associated ot each individual/worker" [indexed]
dt_start -----> "Starting date of the job contract"
dt_end -------> "End date of the job contract"
cod_res ------> "Old residence code"
cod_res_rev --> "New residence code"
id_lag -------> "Previous id, if the 'worker_id' is the same" [indexed]
id_lead ------> "Subsequent id, it the 'worker_id' is the same" [indexed]
As you can notice, the column cod_res_rev is characterized is full of NULL values. This is because the reconstruction of the variable cod_res_rev with the updated residence values it was based solely on specific contracts (those for which the worker had had an actual change of residence - but this is redundant for the purposes of my question). Therefore, my goal is to fill each NULL value of the column cod_res_rev with the previous one, if not missing, until the next non-empty value is reached and continue like this for each worker. The result should be something like this.
I attempted to achieve my goal through the following procedure.
-- The loop is performed based on the maximum number of entries per worker in the database identified through the table 'max_count'.
drop table if exists max_count;
create table max_count
as select worker_id, count(*) n
from ml_arm
group by worker_id;
alter table max_count add unique index (worker_id);
DROP PROCEDURE IF EXISTS doiterate;
delimiter //
CREATE PROCEDURE doiterate()
BEGIN
DECLARE total INT unsigned DEFAULT 0;
WHILE total <= (select MAX(n) from max_count) DO
update ml_arm a
left outer join ml_arm b on a.id_lag = b.id
set a.cod_res_rev =
case
when a.cod_res_rev is NULL and a.worker_id = b.worker_id and b.cod_res_rev is not NULL
then b.cod_res_rev
else a.cod_res_rev
end;
SET total = total + 1;
END WHILE;
END//
delimiter ;
CALL doiterate();
However, I do not believe this is the optimal way to update my table. In fact, by database if composed by about 25 million of rows and the value from select MAX(n) from max_count is about 4,000. I kindly ask you for any suggestion on faster approaches to update my table. I am using MySQL 8.0.22. Thank you in advance.
Eventually, here below there is a command to create a sample table of my database with a bunch of entries.
drop table if exists ml_arm;
create table ml_arm (
id MEDIUMINT NOT NULL AUTO_INCREMENT,
worker_id int,
dt_start date,
dt_end date,
cod_res varchar(50),
cod_res_rev varchar(50),
id_lag int,
id_lead int,
PRIMARY KEY (id)
);
insert into
ml_arm(id, worker_id, dt_start, dt_end, cod_res, cod_res_rev, id_lag, id_lead)
values
('12', '20', '2014-05-02', '2014-07-08', '', 'I040', NULL, '13'),
('13', '20', '2017-01-14', '2017-01-31', '', NULL, '12', '14'),
('14', '20', '2017-11-06', '2017-12-15', 'I040', NULL, '13', NULL),
('20', '29', '2014-11-24', '2017-02-11', '', 'N.D.', NULL, NULL),
('21', '42', '2016-01-22', '2016-05-05', 'G582', 'G582', NULL, NULL),
('23', '45', '2013-08-07', '2014-04-06', 'G582', 'G582', NULL, '24'),
('24', '45', '2014-05-07', '2014-05-10', 'G582', NULL, '23', NULL),
('25', '48', '2012-08-11', '2012-08-31', 'G582', 'G582', NULL, '26'),
('26', '48', '2013-08-10', '2013-08-31', 'G582', NULL, '25', NULL),
('53', '71', '2016-12-01', '2017-05-31', '', 'N.D.', NULL, '54'),
('54', '71', '2017-06-01', '2020-05-29', '', NULL, '53', '55'),
('55', '71', '2020-06-01', '2099-01-01', '', NULL, '54', NULL)
;
By following Gordon Linoff's answer on this thread it seems to work:
UPDATE ml_arm t1
JOIN (
SELECT id, #s:=IF(cod_res_rev IS NULL, #s, cod_res_rev) cod_res_rev
FROM (SELECT * FROM ml_arm ORDER BY id) r,
(SELECT #s:= NULL) t
) t2
ON t1.id = t2.id
SET t1.cod_res_rev = t2.cod_res_rev;
I have an database with alot of users (schema's) for example:
user_1
user_2
user_3
...
user_303
All of those got the same tables, for example:
CREATE TABLE `messages` (
`id` int(11) NOT NULL,
`content` text COLLATE utf8mb3_unicode_ci NOT NULL,
`date` datetime NOT NULL,
`viewed` int(11) NOT NULL,
`forId` int(11) NOT NULL,
`fromId` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3 COLLATE=utf8mb3_unicode_ci;
INSERT INTO `messages` (`id`, `content`, `date`, `viewed`, `forId`, `fromId`) VALUES
(1, 'Hello World', '2020-06-04 14:49:17', 1, 2106, 1842),
(2, 'Hi there', '2020-06-04 14:49:39', 1, 2106, 1842),
(3, 'test 1', '2022-01-03 11:40:43', 1, 3006, 3006),
(4, 'Test 2', '2022-01-20 12:01:52', 1, 1842, 1842);
What I want is a query for example:
USE user_1, user_2, user_3;
SELECT * FROM `messages` WHERE `content` LIKE `%Hi%`;
I don't know if this is possible as a SQL Query, an other option is to write a small PHP code with a for each loop but than I want a command so I get an list of all users: user_1 till user_303
The users are not from 1 till 303 there are some users deleted, to it can be that user_200 doesn't exist any more.
Hope someone here can help me out
You can use the following to write the query you want.
USE information_schema;
SELECT concat("SELECT * FROM ", table_schema,".",table_name, " UNION ALL ")
FROM tables WHERE table_name = 'messages';
You will obtain something like this;
SELECT * FROM base.messages UNION ALL
SELECT * FROM c.messages UNION ALL
You can then run this query to obtain what you want.
In Oracle you can concatenate only 2 arguments, so you have to use nesting to concatenate more than two arguments.
Also you should use ALL_TABLES instead of information_schema
SELECT
concat('SELECT MESSAGE FROM ', concat(OWNER, concat('.', concat(TABLE_NAME, ' UNION ALL '))))
FROM
ALL_TABLES
WHERE
OWNER LIKE 'user_%';
Don't forget to delete the last UNION ALL from result.
CREATE TABLE #T ( DateColumn DATE, SomeColA VARCHAR (20), Temp INT,
Attribute1 VARCHAR (10), Attribute2 VARCHAR (10) )
INSERT INTO #T VALUES ('20180101', 'A', 8, 'D', NULL)
INSERT INTO #T VALUES ('20180201', 'B', 10, NULL, 'A')
INSERT INTO #T VALUES ('20180301', 'B', 12, NULL, NULL)
INSERT INTO #T VALUES ('20180401', 'A', 14, 'D', 'C')
INSERT INTO #T VALUES ('20180501', 'A', 15, 'E', 'Y')
SELECT DateColumn,
SomeColA,
Temp,
Attribute1,
Attribute2
-- INTO #NewTable
FROM #T FOR JSON AUTO
Now I've read on several pages that this is not possible. What those pages do not include however, is how this does work.
Must be quite simple, but I can't find it.
Thanks a lot!
JSON
SQL Server (starting with 2016)
https://learn.microsoft.com/en-us/sql/relational-databases/json/json-data-sql-server?view=sql-server-2017
or upper db compatibility level to 130
Store a query as a JSON object use CTE, for example
;WITH x(a) as
(
SELECT DateColumn,
SomeColA,
Temp,
Attribute1,
Attribute2
FROM #T FOR JSON AUTO
)
SELECT a INTO #b FROM x
I just do this and it gives me json result.
;WITH x(a) as
(
SELECT
PatientId,
LName
FROM tblpatientmaster FOR JSON AUTO
)
--select * from x
SELECT a INTO #temp FROM x
select * from #temp
[{"PatientId":1,"LName":"Doe"},{"PatientId":2,"LName":"Doe"},{"PatientId":5,"LName":"Kirk"},{"PatientId":6,"LName":"Doe"},{"PatientId":7,"LName":"Dasddsa"},{"PatientId":8,"LName":"Azelia"},{"PatientId":9,"LName":"Patient"},{"PatientId":10,"LName":"Smith"},{"PatientId":11,"LName":"Brothers"},{"PatientId":12,"LName":"TtEE"},{"PatientId":13,"LName":"Tes"},{"PatientId":14,"LName":"Hernandez"},{"PatientId":15,"LName":"Velazquez"},{"PatientId":16,"LName":"Spock"},{"PatientId":17,"LName":"Patient"},{"PatientId":18,"LName":"Doe"},{"PatientId":20,"LName":"Doe"},{"PatientId":22,"LName":"Doe"},{"PatientId":23,"LName":"Doe"},{"PatientId":24,"LName":"Doe"}]
Remove the '#' in the Name Table
Example:
CREATE TABLE #T ( DateColumn DATE, SomeColA VARCHAR (20), Temp INT,
Attribute1 VARCHAR (10), Attribute2 VARCHAR (10) )
Live Demo:
http://sqlfiddle.com/#!18/2d6f9/2
Output:
[{"DateColumn":"2018-01-01","SomeColA":"A","Temp":8,"Attribute1":"D"},
{"DateColumn":"2018-02-01","SomeColA":"B","Temp":10,"Attribute2":"A"},
{"DateColumn":"2018-03-01","SomeColA":"B","Temp":12},{"DateColumn":"2018-04-01","SomeColA":"A","Temp":14,"Attribute1":"D","Attribute2":"C"},
{"DateColumn":"2018-05-01","SomeColA":"A","Temp":15,"Attribute1":"E","Attribute2":"Y"}]
The following code reads:
--Code Created by Michael Berkowski
create table dvd (
dvd_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
);
INSERT INTO dvd VALUES (1),(2),(3),(4);
CREATE TABLE dvd_price (
dvd_price_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
dvd_id INT NOT NULL,
rental_amount INT,
time_rented_for VARCHAR(10)
);
INSERT INTO dvd_price (dvd_id, rental_amount, time_rented_for)
SELECT
dvd_id,
2 AS rental_amount,
rental_period
FROM
dvd
CROSS JOIN (
-- This is where I'm having issues
SELECT (CASE dvd.dvd_id
WHEN dvd.dvd_id = 1
THEN '1-Day'
ELSE '3-Day'
END) AS rental_period
) rental_periods
Why can I not do a CASE statement after the CROSS JOIN and how would I fix this?
I get the error, "Unknown table 'dvd' in field list:", what is a better way of writing this?
Try this way:
INSERT INTO dvd_price (dvd_id, rental_amount, time_rented_for)
SELECT
dvd_id,
2 AS rental_amount,
CASE dvd.dvd_id
WHEN dvd.dvd_id = 1
THEN '1-Day'
ELSE '3-Day'
END
FROM
dvd
I have a key-value table and I need to insert multiple rows for each selected id from another table.
I get the user id:
SELECT #id := id FROM user WHERE email = 'my#email.com';
I get the ids of things by user id:
SELECT #things := id FROM `things` WHERE `owner_id` = #id;
Now I need to insert multiple values into key-value table for each #things. And problem here is that I cannot use a select subquery, because I need to insert inline values, like:
INSERT INTO key_value (key, value)
VALUES (
#things, 'CUSTOM VALUE 1'
),
VALUES (
#things, 'CUSTOM VALUE 2'
);
BUT, it does not work. The last query is obviously wrong. Any help would be much appreciated.
EDIT:
Looks like I need multiple queries to do that:
SELECT #id := id FROM user WHERE email = 'startour#netron.no';
INSERT INTO key_value (`key`, `value`)
(
SELECT id, 'CUSTOM_VALUE_1'
FROM `things`
WHERE `owner_id` = #id
);
INSERT INTO key_value (`key`, `value`)
(
SELECT id, 'CUSTOM_VALUE_2'
FROM `things`
WHERE `owner_id` = #id
);
INSERT INTO key_value (key, value)
SELECT user.id as key, things.id as value FROM user
left outer join things on user.id=things.owner_id
WHERE user.email = 'my#email.com'