I Have a table that is a lookup for scoring points based on Place (P) and Number of Racers(R)
and scoring formats indicated by points_id. Two cases are shown in the table.
Sometime the points are determined directly by the values of P and N as in points_id =3
other times they are most easily determined by a simple calculation shown in the pts_calc column.
|points_id| P | N |points|pts_calc|
| 1 | 0 | 0 | NULL | pin |
| 1 |DNS| 0 | NULL | nin+1 |
| 3 | 1 | 0 |102.00| NULL |
| 3 | 2 | 0 | 98.00| NULL |
| 3 | 3 | 0 | 96.00| NULL |
| 3 | 4 | 0 | 93.00| NULL |
| 3 | 5 | 0 | 91.00| NULL |
| 3 | 6 | 0 | 89.00| NULL |
| 3 |DNF| 0 | 85.00| NULL |
I was hoping to create a function that returned the points from the three input variables.
points_id, P, N.
Below is what I tried.
CREATE FUNCTION POINTS(pid INT,pin VARCHAR(3),nin INT)
RETURNS DEC(6,2)
DETERMINISTIC
BEGIN
DECLARE pts DECIMAL(6,2);
DECLARE pcalc VARCHAR(20);
SELECT points,pts_calc INTO pts,pcalc FROM scoring_points WHERE points_id=pid AND (P=pin OR P='0') AND (N=nin or N=0);
IF(pts IS NULL) THEN
SET #s= CONCAT('SET pts = ',pcalc);
PREPARE stmt FROM #s;
EXECUTE stmt;
END IF;
RETURN pts;
END
But i got this error.
1336 - Dynamic SQL is not allowed in stored function or trigger
Further research show the Prepare statement is not allowed in functions only but procedures.
I was hoping to do something like;
SELECT SUM(Points(pid,place,numb)) FROM t1 GROUP BY racer.id
But onto plan B (tbd) unless someone has great idea.
I think you might fare better having three numeric columns instead of your pts_calc column:
cPIN - coefficient of pin term
cNIN - coefficient of nin term
cnst - constant term
Your function could then perform:
SELECT IFNULL(points, cPIN*pin + cNIN*nin + cnst) INTO pts
FROM scoring_points
WHERE ...
Depending on your needs, you might even be able to get rid of the points column by just using cnst and leaving the other two equal to 0.
Related
I'm having trouble returning a JSON representation of a many-many join. My plan was to encode the columns returned using the following JSON format
{
"dog": [
"duke"
],
"location": [
"home",
"scotland"
]
}
This format would handle duplicate keys by aggregating the results in a JSON array, howver all of my attempts at aggregating this structure so far have just removed duplicates, so the arrays only ever have a single element.
Tables
Here is a simplified table structure I've made for the purposes of explaining this query.
media
| media_id | sha256 | filepath |
| 1 | 33327AD02AD09523C66668C7674748701104CE7A9976BC3ED8BA836C74443DBC | /photos/cat.jpeg |
| 2 | 323b5e69e72ba980cd4accbdbb59c5061f28acc7c0963fee893c9a40db929070 | /photos/dog.jpeg |
| 3 | B986620404660DCA7B3DEC4EFB2DE80C0548AB0DE243B6D59DA445DE2841E474 | /photos/dog2.jpeg |
| 4 | 1be439dd87cd87087a425c760d6d8edc484f126b5447beb2203d21e09e2a8f11 | /photos/balloon.jpeg |
media_metdata_labels_has_media (for many-many joins)
| media_metadata_labels_label_id | media_media_id |
| 1 | 1 |
| 2 | 1 |
| 3 | 1 |
| 1 | 2 |
| 4 | 2 |
| 5 | 2 |
| 1 | 3 |
| 6 | 3 |
| 7 | 3 |
| 8 | 4 |
| 9 | 4 |
media_metadata_labels
| label_id | label_key | label_value |
| 2 | cat | lily |
| 4 | dog | duke |
| 6 | dog | rex |
| 1 | pet size | small |
| 3 | location | home |
| 7 | location | park |
| 8 | location | scotland |
| 9 | location | sky |
| 5 | location | studio |
My current attempt
My latest attempt at querying this data uses JSON_MERGE_PRESERVE with two arguments, the first is just an empty JSON object and the second is an invalid JSON document. It's invalid because there are duplicate keys, but I was hoping that JSON_MERGE_PRESERVE would merge them. It turns out JSON_MERGE_PRESERVE will only merge duplicates if they're not in the same JSON argument.
For example, this won't merge two keys
SET #key_one = '{}';
SET #key_two = '{"location": ["home"], "location": ["scotland"]}';
SELECT JSON_MERGE_PRESERVE(#key_one, #key_two);
-- returns {"location": ["scotland"]}
but this will
SET #key_one = '{"location": ["home"] }';
SET #key_two = '{"location": ["scotland"]}';
SELECT JSON_MERGE_PRESERVE(#key_one, #key_two);
-- returns {"location": ["home", "scotland"]}
So anyway, here's my current attempt
SELECT
m.media_id,
m.filepath,
JSON_MERGE_PRESERVE(
'{}',
CAST(
CONCAT(
'{',
GROUP_CONCAT(CONCAT('"', l.label_key, '":["', l.label_value, '"]')),
'}'
)
AS JSON)
)
as labels
FROM media AS m
LEFT JOIN media_metadata_labels_has_media AS lm ON lm.media_media_id = m.media_id
LEFT JOIN media_metadata_labels AS l ON l.label_id = lm.media_metadata_labels_label_id
GROUP BY m.media_id, m.filepath
-- HAVING JSON_CONTAINS(labels, '"location"', CONCAT('$.', '"home"')); -- this would let me filter on labels one they're in the correct JSON format
After trying different combinations of JSON_MERGE, JSON_OBJECTAGG, JSON_ARRAYAGG, CONCAT and GROUP_CONCAT this still leaves me scratching my head.
Disclaimer: Since posting this question I've started using mariadb instead of oracle MySQL. The function below should work for MySQL too, but in case it doesn't then any changes required will likely be small syntax fixes.
I solved this by creating a custom aggregation function
DELIMITER //
CREATE AGGREGATE FUNCTION JSON_LABELAGG (
json_key TEXT,
json_value TEXT
) RETURNS JSON
BEGIN
DECLARE complete_json JSON DEFAULT '{}';
DECLARE current_jsonpath TEXT;
DECLARE current_jsonpath_value_type TEXT;
DECLARE current_jsonpath_value JSON;
DECLARE CONTINUE HANDLER FOR NOT FOUND RETURN complete_json;
main_loop: LOOP
FETCH GROUP NEXT ROW;
SET current_jsonpath = CONCAT('$.', json_key); -- the jsonpath to our json_key
SET current_jsonpath_value_type = JSON_TYPE(JSON_EXTRACT(complete_json, current_jsonpath)); -- the json object type at the current path
SET current_jsonpath_value = JSON_QUERY(complete_json, current_jsonpath); -- the json value at the current path
-- if this is the first label value with this key then place it in a new array
IF (ISNULL(current_jsonpath_value_type)) THEN
SET complete_json = JSON_INSERT(complete_json, current_jsonpath, JSON_ARRAY(json_value));
ITERATE main_loop;
END IF;
-- confirm that an array is at this jsonpath, otherwise that's an exception
CASE current_jsonpath_value_type
WHEN 'ARRAY' THEN
-- check if our json_value is already within the array and don't push a duplicate if it is
IF (ISNULL(JSON_SEARCH(JSON_EXTRACT(complete_json, current_jsonpath), "one", json_value))) THEN
SET complete_json = JSON_ARRAY_APPEND(complete_json, current_jsonpath, json_value);
END IF;
ITERATE main_loop;
ELSE
SIGNAL SQLSTATE '45000'
SET MESSAGE_TEXT = 'Expected JSON label object to be an array';
END CASE;
END LOOP;
RETURN complete_json;
END //
DELIMITER ;
and editing my query to use it
SELECT
m.media_id,
m.filepath,
JSON_LABELAGG(l.label_key, l.label_value) as labels
FROM media AS m
LEFT JOIN media_metadata_labels_has_media AS lm ON lm.media_media_id = m.media_id
LEFT JOIN media_metadata_labels AS l ON l.label_id = lm.media_metadata_labels_label_id
GROUP BY m.media_id, m.filepath
Delphi 10.3.3
FireDAC: DBGrid / FDQuery / MySQL
VCL
Hi all,
I have a table with these fields
----------------------
| id | data |
----------------------
| 1 | 0=A;1=B;2=C |
| 2 | 2=Z |
| 3 | |
| 4 | 0=Y;1=X |
| 5 | |
| 6 | |
Each row of data represents only the change in the table
I would like this to be display in a DBGRID:
-----------------------
| id | C0 | C1 | C2 |
-----------------------
| 1 | A | B | C |
| 2 | A | B | Z |
| 3 | A | B | Z |
| 4 | Y | X | Z |
| 5 | Y | X | Z |
| 6 | Y | X | Z |
What I can do for now is only the following table:
-----------------------
| id | C0 | C1 | C2 |
-----------------------
| 1 | A | B | C |
| 2 | | | Z |
| 3 | | | |
| 4 | Y | X | |
| 5 | | | |
| 6 | | | |
To obtain this result, I create additional columns in the event FDQuery1.BeforeOpen
And in the event OnCreateFields, I fill each column but I don't know the previous row content,
So, how can I do to fill in the missing fields in the DBgrid?
Thanks
Franck
I think you mean OnCalcFields, rather than OnCreateFields.
What you need
is certainly possible, either server-side by deriving the necessary values from the prior
row using e.g. a SQL subquery or client-side using calculated fields. This answer is about doing it
client-side.
The problem with doing client-side calculations involving another dataset row is that
to do this you need to be able to move the dataset cursor during the OnCalcFields event. However, at the time, the DataSet will be in either dsCalcFields or dsInternalCalc state
and, while it is, you can't easily move to another row in the dataset. It is possible to do this, but
requires declaring a descendant dataset class (TMyFDQuery) so that you can access the SetTempState
necessary to do revert to the prior state after you've picked up the necessary info from the "other"
row and, if what you need involves more that one field, you need somewhere to store the values temporarily.
So doing it that way gets messy.
A much cleaner approach involves using functional similarity between FireDAC's datasets and TClientDataSets.
One of the nice features of TClientDatasSets is the ease with which you can move the dataset contents between
two CDSs simply by doing
CDS2.Data := CDS1.Data;
FireDAC datasets can do the same trick, but between any FD dataset types. So here is what I would do in your
situation:
Add an FDMemTable to your form/datamodule and copy the query data into it in the FDQuery's AfterOpen event like
this:
procedure TForm2.FDQuery1AfterOpen(DataSet: TDataSet);
begin
FDQuery1.DisableControls;
try
FDMemTable1.Data := FDQuery1.Data;
FDMemTable1.Open;
finally
FDQuery1.First;
FDQuery1.EnableControls;
end;
end;
The FDQuery1.First is to force it to re-do its calculated fields once the FDMemTable data is available
(during the initial FDQuery1.Open, it can't be, of course).
In the FDQuery's OnCalcFields event, use code like this to base the calculated fields'
values on values picked up from the prior row (if there is one of course, the first
row can't hae a "prior" row):
procedure TForm2.FDQuery1CalcFields(DataSet: TDataSet);
begin
if FDMemTable1.Active then begin
if FDMemTable1.Locate('ContactID', FDQuery1.FieldByName('ContactID').AsInteger, []) then begin
FDMemTable1.Prior;
if not FDMemTable1.Bof then begin
// Set FDQuery1's calculated fields that depend on prior row
FDQuery1.FieldByName('PriorRowID').AsInteger := FDMemTable1.FieldByName('ContactID').AsInteger;
end;
end;
end;
end;
In this example, my queried dataset has a ContactID primary key and the calculated value is simply the ContactID value from the prior row. In real life, of course, it
would be more efficient to use persistent field variables rather than keep calling FieldByName.
I suppose another possibility might be to use the CloneCursor method to obtain a lookup cursor
to access the "prior" row, but I've not tried that myself and it may not be possible anyway
(what happens about the calculated fields in the CloneCuror copy?).
Suppose I have a MySQL table that defines a collection of things, each of which is associated with either 1 or 2 owners. For example:
CREATE TABLE thing (
id INT UNSIGNED PRIMARY KEY AUTO_INCREMENT
, name CHAR(10)
, first_owner INT UNSIGNED NOT NULL
, second_owner INT UNSIGNED DEFAULT NULL
);
+----+------------+-------------+--------------+
| id | name | first_owner | second_owner |
+----+------------+-------------+--------------+
| 1 | skateboard | Joe | NULL |
| 2 | flashlight | Joe | NULL |
| 3 | drill | Joe | Erica |
| 4 | computer | Erica | NULL |
| 5 | textbook | Diane | NULL |
| 6 | cell phone | Amy | Diane |
| 7 | piano | Paul | Amy |
+----+------------+-------------+--------------+
Each distinct owner is a node of a graph, and two owners in the same row constitute an edge between their nodes. A graph drawn from the above example rows looks like this:
In this example, there are two components: Joe and Erica are one; Diane, Paul and Amy are the other.
I want to identify these components in my table, so I add another column:
ALTER TABLE thing ADD COLUMN `group` INT UNSIGNED;
How could I write an UPDATE statement that would populate this new column by uniquely identifying the connected component to which the row belongs? Here's an example of an acceptable result for the above example rows:
+----+------------+-------------+--------------+-------+
| id | name | first_owner | second_owner | group |
+----+------------+-------------+--------------+-------+
| 1 | skateboard | Joe | NULL | 1 |
| 2 | flashlight | Joe | NULL | 1 |
| 3 | drill | Joe | Erica | 1 |
| 4 | computer | Erica | NULL | 1 |
| 5 | textbook | Diane | NULL | 2 |
| 6 | cell phone | Amy | Diane | 2 |
| 7 | piano | Paul | Amy | 2 |
+----+------------+-------------+--------------+-------+
I could do this with a stored procedure, but my actual scenario involves more tables and millions of rows, so I'm hoping there's a clever way to do this without looping through cursors for a week.
This is a simplified example for the purpose of illustrating the problem. Each component is supposed to represent a "household" and most will have only 1 or 2 nodes, but those with more nodes are especially important. There isn't necessarily any strict upper limit to the size of a household.
You can consider this method of creating hierarchical queries in mysql
CREATE FUNCTION hierarchy_connect_by_parent_eq_prior_id(value INT) RETURNS INT
NOT DETERMINISTIC
READS SQL DATA
BEGIN
DECLARE _id INT;
DECLARE _parent INT;
DECLARE _next INT;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET #id = NULL;
SET _parent = #id;
SET _id = -1;
IF #id IS NULL THEN
RETURN NULL;
END IF;
LOOP
SELECT MIN(id)
INTO #id
FROM t_hierarchy
WHERE parent = _parent
AND id > _id;
IF #id IS NOT NULL OR _parent = #start_with THEN
SET #level = #level + 1;
RETURN #id;
END IF;
SET #level := #level - 1;
SELECT id, parent
INTO _id, _parent
FROM t_hierarchy
WHERE id = _parent;
END LOOP;
END
Also, a very good article on this topic Adjacency list vs. nested sets: MySQL
A very good answer to a related question
"What is the most efficient/elegant way to parse a flat table into a
tree?"
There are several ways to store tree-structured data in a relational
database. What you show in your example uses two methods:
Adjacency List (the "parent" column) and
Path Enumeration (the dotted-numbers in your name column).
Another solution is called Nested Sets, and it can be stored in the
same table too. Read "Trees and Hierarchies in SQL for Smarties" by
Joe Celko for a lot more information on these designs.
I usually prefer a design called Closure Table (aka "Adjacency
Relation") for storing tree-structured data. It requires another
table, but then querying trees is pretty easy.
Please have a look at original question for reference.
Let's say I have 3 tables (significant columns only)
Category (catId key, parentCatId)
Category_Hierarchy (catId key, parentTrail, catLevel)
Product (prodId key, catId, createdOn)
There's a reason for having a separate Category_Hierarchy table, because I'm using triggers on Category table that populate it, because MySql triggers work as they do and I can't populate columns on the same table inside triggers if I would like to use auto_increment values. For the sake of this problem this is irrelevant. These two tables are 1:1 anyway.
Category table could be:
+-------+-------------+
| catId | parentCatId |
+-------+-------------+
| 1 | NULL |
| 2 | 1 |
| 3 | 2 |
| 4 | 3 |
| 5 | 3 |
| 6 | 4 |
| ... | ... |
+-------+-------------+
Category_Hierarchy
+-------+-------------+----------+
| catId | parentTrail | catLevel |
+-------+-------------+----------+
| 1 | 1/ | 0 |
| 2 | 1/2/ | 1 |
| 3 | 1/2/3/ | 2 |
| 4 | 1/2/3/4/ | 3 |
| 5 | 1/2/3/5/ | 3 |
| 6 | 1/2/3/4/6/ | 4 |
| ... | ... | ... |
+-------+-------------+----------+
Product
+--------+-------+---------------------+
| prodId | catId | createdOn |
+--------+-------+---------------------+
| 1 | 4 | 2010-02-03 12:09:24 |
| 2 | 4 | 2010-02-03 12:09:29 |
| 3 | 3 | 2010-02-03 12:09:36 |
| 4 | 1 | 2010-02-03 12:09:39 |
| 5 | 3 | 2010-02-03 12:09:50 |
| ... | ... | ... |
+--------+-------+---------------------+
Category_Hierarchy makes it simple to get category subordinate trees like this:
select c.*
from Category c
join Category_Hierarchy h
on (h.catId = c.catId)
where h.parentTrail like '1/2/3/%'
Which would return complete subordinate tree of category 3 (that is below 2, that is below 1 which is root category) including subordinate tree root node. Excluding root node is just one more where condition.
The problem
I would like to write a stored procedure:
create procedure GetLatestProductsFromSubCategories(in catId int)
begin
/* return 10 latest products from each */
/* catId subcategory subordinate tree */
end;
This means if a certain category had 3 direct sub categories (with whatever number of nodes underneath) I would get 30 results (10 from each subordinate tree). If it had 5 sub categories I'd get 50 results.
What would be the best/fastest/most efficient way to do this? If possible I'd like to avoid cursors unless they'd work faster compared to any other solution as well as prepared statements, because this would be one of the most frequent calls to DB.
Edit
Since a picture tells 1000 words I'll try to better explain what I want using an image. Below image shows category tree. Each of these nodes can have an arbitrary number of products related to them. Products are not included in the picture.
So if I'd execute this call:
call GetLatestProductsFromSubCategories(1);
I'd like to effectively get 30 products:
10 latest products from the whole orange subtree
10 latest products from the whole blue subtree and
10 latest products from the whole green subtree
I don't want to get 10 latest products from each node under catId=1 node which would mean 320 products.
Final Solution
This solution has O(n) performance:
CREATE PROCEDURE foo(IN in_catId INT)
BEGIN
DECLARE done BOOLEAN DEFAULT FALSE;
DECLARE first_iteration BOOLEAN DEFAULT TRUE;
DECLARE current VARCHAR(255);
DECLARE categories CURSOR FOR
SELECT parentTrail
FROM category
JOIN category_hierarchy USING (catId)
WHERE parentCatId = in_catId;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = TRUE;
SET #query := '';
OPEN categories;
category_loop: LOOP
FETCH categories INTO current;
IF `done` THEN LEAVE category_loop; END IF;
IF first_iteration = TRUE THEN
SET first_iteration = FALSE;
ELSE
SET #query = CONCAT(#query, " UNION ALL ");
END IF;
SET #query = CONCAT(#query, "(SELECT product.* FROM product JOIN category_hierarchy USING (catId) WHERE parentTrail LIKE CONCAT('",current,"','%') ORDER BY createdOn DESC LIMIT 10)");
END LOOP category_loop;
CLOSE categories;
IF #query <> '' THEN
PREPARE stmt FROM #query;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END IF;
END
Edit
Due to the latest clarification, this solution was simply edited to simplify the categories cursor query.
Note: Make the VARCHAR on line 5 the appropriate size based on your parentTrail column.
I just came across this database query and wonder what exactly this query does..Please clarify ..
select * from tablename order by priority='High' DESC, priority='Medium' DESC, priority='Low" DESC;
Looks like it'll order the priority by High, Medium then Low.
Because if the order by clause was just priority DESC then it would do it alphabetical, which would give
Medium
Low
High
It basically lists all fields from the table "tablename" and ordered by priority High, Medium, Low.
So High appears first in the list, then Medium, and then finally Low
i.e.
* High
* High
* High
* Medium
* Medium
* Low
Where * is the rest of the fields in the table
Others have already explained what id does (High comes first, then Medium, then Low). I'll just add a few words about WHY that is so.
The reason is that the result of a comparison in MySQL is an integer - 1 if it's true, 0 if it's false. And you can sort by integers, so this construct works. I'm not sure this would fly on other RDBMS though.
Added: OK, a more detailed explanation. First of all, let's start with how ORDER BY works.
ORDER BY takes a comma-separated list of arguments which it evalutes for every row. Then it sorts by these arguments. So, for example, let's take the classical example:
SELECT * from MyTable ORDER BY a, b, c desc
What ORDER BY does in this case, is that it gets the full result set in memory somewhere, and for every row it evaluates the values of a, b and c. Then it sorts it all using some standard sorting algorithm (such as quicksort). When it needs to compare two rows to find out which one comes first, it first compares the values of a for both rows; if those are equal, it compares the values of b; and, if those are equal too, it finally compares the values of c. Pretty simple, right? It's what you would do too.
OK, now let's consider something trickier. Take this:
SELECT * from MyTable ORDER BY a+b, c-d
This is basically the same thing, except that before all the sorting, ORDER BY takes every row and calculates a+b and c-d and stores the results in invisible columns that it creates just for sorting. Then it just compares those values like in the previous case. In essence, ORDER BY creates a table like this:
+-------------------+-----+-----+-----+-----+-------+-------+
| Some columns here | A | B | C | D | A+B | C-D |
+-------------------+-----+-----+-----+-----+-------+-------+
| | 1 | 2 | 3 | 4 | 3 | -1 |
| | 8 | 7 | 6 | 5 | 15 | 1 |
| | ... | ... | ... | ... | ... | ... |
+-------------------+-----+-----+-----+-----+-------+-------+
And then sorts the whole thing by the last two columns, which it discards afterwards. You don't even see them it your result set.
OK, something even weirder:
SELECT * from MyTable ORDER BY CASE WHEN a=b THEN c ELSE D END
Again - before sorting is performed, ORDER BY will go through each row, calculate the value of the expression CASE WHEN a=b THEN c ELSE D END and store it in an invisible column. This expression will always evaluate to some value, or you get an exception. Then it just sorts by that column which contains simple values, not just a fancy formula.
+-------------------+-----+-----+-----+-----+-----------------------------------+
| Some columns here | A | B | C | D | CASE WHEN a=b THEN c ELSE D END |
+-------------------+-----+-----+-----+-----+-----------------------------------+
| | 1 | 2 | 3 | 4 | 4 |
| | 3 | 3 | 6 | 5 | 6 |
| | ... | ... | ... | ... | ... |
+-------------------+-----+-----+-----+-----+-----------------------------------+
Hopefully you are now comfortable with this part. If not, re-read it or ask for more examples.
Next thing is the boolean expressions. Or rather the boolean type, which for MySQL happens to be an integer. In other words SELECT 2>3 will return 0 and SELECT 2<3 will return 1. That's just it. The boolean type is an integer. And you can do integer stuff with it too. Like SELECT (2<3)+5 will return 6.
OK, now let's put all this together. Let's take your query:
select * from tablename order by priority='High' DESC, priority='Medium' DESC, priority='Low" DESC;
What happens is that ORDER BY sees a table like this:
+-------------------+----------+-----------------+-------------------+----------------+
| Some columns here | priority | priority='High' | priority='Medium' | priority='Low' |
+-------------------+----------+-----------------+-------------------+----------------+
| | Low | 0 | 0 | 1 |
| | High | 1 | 0 | 0 |
| | Medium | 0 | 1 | 0 |
| | Low | 0 | 0 | 1 |
| | High | 1 | 0 | 0 |
| | Low | 0 | 0 | 1 |
| | Medium | 0 | 1 | 0 |
| | High | 1 | 0 | 0 |
| | Medium | 0 | 1 | 0 |
| | Low | 0 | 0 | 1 |
+-------------------+----------+-----------------+-------------------+----------------+
And it then sorts by the last three invisble columns which are discarded later.
Does it make sense now?
(P.S. In reality, of course, there are no invisible columns and the whole thing is made much trickier to get good speed, using indexes if possible and other stuff. However it is much easier to understand the process like this. It's not wrong either.)