I've got the use case to version objects (identified by objectOwnerId and objectId group). I insert rows to ledger table with their respective hashes.
The order of the ledger table is identified by the compound PRIMARY KEY and its timestamp up to microsecond precision + additional 3 byte entropy at the end to prevent collisions (in case multiple rows gets inserted at the same microsecond).
Once data is stored I need efficient way to get the latest hash for multiple objects at once. I've came up with a query (please see end of this post) which is built from sub-selects with JOIN and GROUP BY, but it's pretty complex I think and I am looking for ways to address my problem in a simpler (if possible) way.
Is there any way for improvement?
It would've been simpler if I have PRIMARY KEY which isn't COMPOUND, in which case I could pass the max() value upwards, however that's not the case. I was also thinking if I could merge my TIMESTAMP(6) - 7 bytes with BINARY(3) - 3 bytes and store it as BINARY(10), but wasn't sure if that's easily possible.
Please find the schema, test data and SELECT queries below.
This is my table:
CREATE TABLE `ledger` (
`objectOwnerId` CHAR(10) NOT NULL,
`objectId` VARCHAR(50) NOT NULL,
`objectHash` BINARY(16) NOT NULL,
`timestamp` TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP(6),
`timestampAdditionalEntropy` BINARY(3) NOT NULL,
PRIMARY KEY (`timestamp`, `timestampAdditionalEntropy`),
UNIQUE(`objectHash`),
INDEX(`objectId`(10))
);
Let's insert some values:
INSERT INTO ledger (objectOwnerId, objectId, objectHash, timestampAdditionalEntropy) VALUES ('owneraaaaa', 'ida', unhex(substring(sha1(random_bytes(16)), 1, 32)), random_bytes(3));
INSERT INTO ledger (objectOwnerId, objectId, objectHash, timestampAdditionalEntropy) VALUES ('owneraaaaa', 'ida', unhex(substring(sha1(random_bytes(16)), 1, 32)), random_bytes(3));
INSERT INTO ledger (objectOwnerId, objectId, objectHash, timestampAdditionalEntropy) VALUES ('owneraaaab', 'idb', unhex(substring(sha1(random_bytes(16)), 1, 32)), random_bytes(3));
INSERT INTO ledger (objectOwnerId, objectId, objectHash, timestampAdditionalEntropy) VALUES ('owneraaaab', 'idb', unhex(substring(sha1(random_bytes(16)), 1, 32)), random_bytes(3));
INSERT INTO ledger (objectOwnerId, objectId, objectHash, timestampAdditionalEntropy) VALUES ('owneraaaab', 'idb', unhex(substring(sha1(random_bytes(16)), 1, 32)), random_bytes(3));
We've got this dataset:
# objectOwnerId, objectId, objectHash, timestamp, HEX(CAST(timestampAdditionalEntropy AS CHAR(6) CHARACTER SET utf8))
#'owneraaaab', 'idb', 'A8D3B63EFC6C63FD996B8D1931FBF748', '2019-05-29 11:38:12.353521', '725E3D'
#'owneraaaab', 'idb', '9B7395F9EE2F2363BA89C7FBAEDDBB54', '2019-05-29 11:38:12.352524', '8B8162'
#'owneraaaab', 'idb', '80393C5FF4492342D073B5F8B3388EC2', '2019-05-29 11:38:12.351569', 'FEAA02'
#'owneraaaaa', 'ida', '0D84F725ACAC87838C34742CA00BBEF7', '2019-05-29 11:38:12.350648', '41E425'
#'owneraaaaa', 'ida', '9A82C936A25C4648BFB75B692850841B', '2019-05-29 11:38:12.349625', '470685'
returned by this query:
select objectOwnerId, objectId, HEX(CAST(objectHash AS CHAR(32) CHARACTER SET utf8)) as objectHash, timestamp, HEX(CAST(timestampAdditionalEntropy AS CHAR(6) CHARACTER SET utf8))
from ledger
order by timestamp desc, timestampAdditionalEntropy desc;
I need to get this:
# objectOwnerId, objectId, objectHash, timestamp, HEX(CAST(s.timestampAdditionalEntropy AS CHAR(6) CHARACTER SET utf8))
#owneraaaaa, ida, 0D84F725ACAC87838C34742CA00BBEF7, 2019-05-29 11:38:12.350648, 41E425
#owneraaaab, idb, A8D3B63EFC6C63FD996B8D1931FBF748, 2019-05-29 11:38:12.353521, 725E3D
which this query can return:
select s.objectOwnerId, s.objectId, HEX(CAST(objectHash AS CHAR(32) CHARACTER SET utf8)) as objectHash, s.timestamp, HEX(CAST(s.timestampAdditionalEntropy AS CHAR(6) CHARACTER SET utf8)) from (
select s.objectOwnerId, s.objectId, s.timestamp, max(i.timestampAdditionalEntropy) as timestampAdditionalEntropy from (
select objectOwnerId, objectId, max(timestamp) as timestamp
from ledger where ((objectOwnerId = 'owneraaaaa' AND objectId = 'ida') OR (objectOwnerId = 'owneraaaab' AND objectId = 'idb'))
group by objectOwnerId, objectId
) s
JOIN ledger i on i.objectOwnerId = s.objectOwnerId and i.objectId = s.objectId and i.timestamp = s.timestamp
group by objectOwnerId, objectId, timestamp
) s
JOIN ledger i on i.objectOwnerId = s.objectOwnerId and i.objectId = s.objectId and i.timestamp = s.timestamp and i.timestampAdditionalEntropy = s.timestampAdditionalEntropy
Related
There is a table objects, which stores data on real estate objects. Me need to use a query to calculate a new field that will display the date range from Monday to Sunday, which includes the date the object was created (for example, “2020-11-16 - 2020-11-22”)
create table objects(
object_id int NOT NULL PRIMARY KEY ,
city_id int not null ,
price int ,
area_total int ,
status varchar(50) ,
class varchar(50) ,
action varchar(50) ,
date_create timestamp,
FOREIGN KEY(city_id) references avg_price_square_city(city_id)
);
Data in the table:
INSERT INTO objects (object_id, city_id, price, area_total, status, class, action, date_create)
VALUES (1, 1, 4600000, 72, 'active', 'Secondary', 'Sale', '2022-05-12 21:49:34');
INSERT INTO objects (object_id, city_id, price, area_total, status, class, action, date_create)
VALUES (2, 2, 5400000, 84, 'active', 'Secondary', 'Sale', '2022-05-19 21:49:35');
The query should display two fields: the object number and a range that includes the date it was created. How can this be done ?
P.S
I wrote this query,but he swears at the "-" sign:
SET #WeekRangeStart ='2022/05/10';
SET #WeekRangeEnd = '2022/05/17';
select object_id,#range := #WeekRangeStart '-' #WeekRangeEnd
FROM objects where #range = #WeekRangeStart and date_create between #WeekRangeStart and #WeekRangeEnd
UNION
select object_id,#range from objects where #`range` = #WeekRangeEnd;
Error:[42000][1064] You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '#WeekRangeEnd FROM objects where #range = #WeekRangeStart and date_create betwee' at line 1
I want to receive in query:
object_id #range
1 2022/05/10 - 2022/05/17
The column #range must contain the date from the "date_create"
SET #WeekRangeStart = CAST('2022/05/10' as DATE);
SET #WeekRangeEnd = CAST('2022/05/17' as DATE);
SET #range = CONCAT(#WeekRangeStart,' - ',#WeekRangeEnd) ;
-- select #range;
select
object_id,
#range
FROM objects
where DATE(date_create) between #WeekRangeStart and #WeekRangeEnd
UNION
select object_id,#range from objects
;
Gives next result:
object_id
#range
1
2022-05-10 - 2022-05-17
2
2022-05-10 - 2022-05-17
This result is the output of the SQL part that is put after the UNION. Because date_create is not between your WeekRangeStart and WeekRangeEnd.
You should take some time, and read the UNION documentation.
The variable #range is calculated before the SQL statement, because the value is a constant.
see: DBFIDDLE
NOTE: You should try to use the same dateformat everywhere, and not mix date like '2022-05-19 21:49:35' and 2022/05/10. Use - OR use /, but do not mix them...
EDIT: After the calirification "Me need to use a query to calculate a new field that will display the date range from Monday to Sunday,...":
You probably wanted to do:
SET #WeekDate = CAST('2022/05/10' as DATETIME);
SELECT
ADDDATE(#WeekDate, -(DAYOFWEEK(#WeekDate)-1) +1) as Monday,
DATE_ADD(ADDDATE(#WeekDate, -(DAYOFWEEK(#WeekDate)-1) +9), INTERVAL -1 SECOND) as Sunday;
output:
Monday
Sunday
2022-05-09 00:00:00
2022-05-16 23:59:59
I would like to do this query to include the null result in another calculated field.
SELECT ft.fecha_inicio,
ft.fecha_fin,
IF(ft.fecha_fin is NULL, now(), ft.fecha_fin) fin,
TIMEDIFF(fin,ft.fecha_inicio) total,
IF (ISNULL(ft.fecha_fin), 1, 0) as encurso
FROM fabricaciones_tiempos ft
WHERE ft.id_fabricacion = 138;
is this possible?
You cannot directly use a calculated column as part of another column calculation, but you can do it like this:
SELECT
fecha_inicio,
fecha_fin,
TIMEDIFF(fin,fecha_inicio) total,
encurso
FROM (
SELECT ft.fecha_inicio,
ft.fecha_fin,
IF(ft.fecha_fin is NULL, now(), ft.fecha_fin) fin,
IF (ISNULL(ft.fecha_fin), 1, 0) as encurso
FROM fabricaciones_tiempos ft
WHERE ft.id_fabricacion = 138
) as q;
You can't use the alias but you can either include the original formula in place or assign the value to a variable....
SELECT ft.fecha_inicio,
ft.fecha_fin,
IF(ft.fecha_fin is NULL, now(), ft.fecha_fin) fin,
TIMEDIFF(IF(ft.fecha_fin is NULL, now(), ft.fecha_fin),ft.fecha_inicio) total,
IF (ISNULL(ft.fecha_fin), 1, 0) as encurso
FROM fabricaciones_tiempos ft
WHERE ft.id_fabricacion = 138;
or....
SELECT ft.fecha_inicio,
ft.fecha_fin,
#fin:=IF(ft.fecha_fin is NULL, now(), ft.fecha_fin),
TIMEDIFF(#fin,ft.fecha_inicio) total,
IF (ISNULL(ft.fecha_fin), 1, 0) as encurso
FROM fabricaciones_tiempos ft
WHERE ft.id_fabricacion = 138;
I have this table:
CREATE TABLE stackoverflow_question (
id int NOT NULL AUTO_INCREMENT,
name varchar(255) NOT NULL,
json_ob mediumtext default null,
PRIMARY KEY (id)
);
I do some inserts:
insert into stackoverflow_question values(null, 'albert', '[{name: "albert1", qt: 2},{name: "albert2", qt: 2}]');
insert into stackoverflow_question values(null, 'barbara', '[{name: "barbara1", qt: 4},{name: "barbara2", qt: 7}]');
insert into stackoverflow_question values(null, 'paul', '[{name: "paul1", qt: 9},{name: "paul2", qt: 11}]');
Eventually, I will need to sort this table by total quantity.
in the examples above, "paul" has quantity = 20, while "barbara" has quantity = 11. And "albert" has quantity = 4.
Is it possible to create a select statement where a new field is created on the fly? Something like this:
SELECT
SUM (loop json_ob and sum all the quantity fields) AS total_quantity,
id,
name
FROM
stackoverflow_question
ORDER BY total_quantity
If json_ob is actually a valid json object then you can use JSON_TABLE() to extract the quantities and aggregate:
SELECT s.*, SUM(t.qt) total_quantity
FROM stackoverflow_question s,
JSON_TABLE(json_ob, '$[*]' COLUMNS (qt INTEGER PATH '$.qt')) t
GROUP BY s.id
ORDER BY total_quantity DESC;
See the demo.
According to jsonlint your JSON is not valid.
That's why this SQL returns an error (ERROR 3141 (22032): Invalid JSON text in argument 1 to function json_table: "Missing a name for object member." at position 2.")
SELECT
j.name, j.qt
FROM JSON_TABLE('[{name: "paul1", qt: 9},{name: "paul2", qt: 11}]',
"$[*]" COLUMNS (name varchar(20) PATH "$.name", qt int PATH "$.qt")) j ;
and this will return the values:
SELECT
j.name, j.qt
FROM JSON_TABLE('[{"name": "paul1", "qt": 9},{"name": "paul2", "qt": 11}]',
"$[*]" COLUMNS (name varchar(20) PATH "$.name", qt int PATH "$.qt")) j ;
output:
name
qt
paul1
9
paul2
11
You can convert your relaxedJSON, to JSON, using tools like : www.relaxedjson.org
How can I express the below statement as a SQL query ?
IF EXISTS (SELECT * FROM expense_history
WHERE user_id = 40
AND DATE_FORMAT(expense_history.created_date , '%Y-%m-%d') = '2018-06-02'
AND camp_id='80')
UPDATE expense_history
SET clicks = clicks + 1,
amount = amount + 1
WHERE user_id = 40
AND DATE_FORMAT(expense_history.created_date, '%Y-%m-%d') = '2018-06-02'
AND camp_id = '80'
ELSE
INSERT INTO expense_history (camp_id, created_date, amount, user_id)
VALUES ('80', '2018-06-02 12:12:12', '1', '40')
END IF;
I just want to do increment clicks and amount if is set by day, else I want to add new row.
This is very tricky in MySQL. You are storing a datetime but you want the date part to be unique.
Starting in MySQL 5.7.?, you can use computed columns for the unique constraint. Here is an example:
create table expense_history (
user_id int,
camp_id int,
amount int default 0,
clicks int default 1,
. . .
created_datetime datetime, -- note I changed the name
created_date date generated always as (date(created_datetime)),
unique (user_id, camp_id, created_datetime)
);
You can then do the work as:
INSERT INTO expense_history (camp_id, created_datetime, amount, user_id)
VALUES (80, '2018-06-02 12:12:12', 1, 40)
ON DUPLICATE KEY UPDATE
amount = COALESCE(amount + 1, 1),
clicks = COALESCE(clicks + 1, 1);
Earlier versions of MySQL don't support generated columns. Nor do they support functions on unique. But you can use a trick on a prefix index on a varchar to do what you want:
create table expense_history (
user_id int,
camp_id int,
amount int default 0,
clicks int default 1,
. . .
created_datetime varchar(19),
unique (created_datetime(10))
);
This has the same effect.
Another alternative is to store the date and the time in separate columns.
I presumed your database is mysql, because of DATE_FORMAT() function(and edited your question as to be).
So, by using such a mechanism below, you can do what you want,
provided that a COMPOSITE PRIMARY KEY for camp_id, amount, user_id columns :
SET #camp_id = 80,
#amount = 1,
#user_id = 40,
#created_date = sysdate();
INSERT INTO expense_history(camp_id,created_date,amount,user_id,clicks)
VALUES(#camp_id,#created_date,#amount,#user_id,ifnull(clicks,1))
ON DUPLICATE KEY UPDATE
amount = #amount + 1,
clicks = ifnull(clicks,0)+1;
SQL Fiddle Demo
SELECT DISTINCT CONCAT('TOTAL NUMBER OF LEGS IN A TRIP NUMBER ', T1.TNUM, ' IS NOT EQUAL TO THE LARGEST NUMBER OF LEG IN THE TRIP') AS 'ERR'
FROM TRIPLEG T1
WHERE MAX(T1.LEGNUM) = (SELECT COUNT(T2.TNUM)
FROM TRIPLEG
WHERE T1.TNUM = T2.TNUM);
with my code, i am trying to finds all trips that violate the following consistency constraint.
the last leg of each trip must be equal to the total number of legs in a trip"
CREATE TABLE TRIPLEG(
TNUM DECIMAL(10) NOT NULL,
LEGNUM DECIMAL(2) NOT NULL,
DEPARTURE VARCHAR(30) NOT NULL, DESTINATION VARCHAR(30) NOT NULL,
CONSTRAINT TRIPLEG_PKEY PRIMARY KEY (TNUM, LEGNUM),
CONSTRAINT TRIPLEG_UNIQUE UNIQUE(TNUM, DEPARTURE, DESTINATION),
CONSTRAINT TRIPLEG_FKEY1 FOREIGN KEY (TNUM) REFERENCES TRIP(TNUM) );
INSERT INTO TRIPLEG VALUES( 1, 1, 'Sydney', 'Melbourne');
INSERT INTO TRIPLEG VALUES( 1, 2, 'Melbourne', 'Hobart');
Try following;)
SELECT DISTINCT CONCAT('TOTAL NUMBER OF LEGS IN A TRIP NUMBER ', T1.TNUM, ' IS NOT EQUAL TO THE LARGEST NUMBER OF LEG IN THE TRIP') AS 'ERR'
FROM TRIPLEG T1
GROUP BY T1.TNUM
HAVING COUNT(T1.TNUM) <> MAX(T1.LEGNUM)