Search ontology tree by relationships in MySQL - mysql

i am working on a MySQL database with the following schema:
In which we save information from different ontologies (exmp). Some of the terms have relationships (exmp: MS:1000004, sample mass) which are indicated by relationship: or is_a:. For the sake of this question let's focus on the is_a relationships.
We now want to provide an option to search along these is_a relationships. So a function which we give MS:1000004/sample mass as input and get in return all Terms which are connected to it by an is_a relationship. But not only all terms directly connected to MS:1000004, but also all terms which are connected to the children and so on. This image describes this a bit better, where thing would be MS:1000004/sample mass and everything below is what i want as result.
Currently we are using a rather unoptimized recursive function as a stored procedure to do this:
DELIMITER $$
CREATE DEFINER=`root`#`localhost` PROCEDURE `getAllTermsByParentTerm`(IN `parentOntology` varchar(512))
BEGIN
WITH RECURSIVE previous (accession, FK_OntologyName, name, definition, xrefvaluetype, isobsolete, fk_termAccession, relationshiptype, fk_termAccession_related, depth_level) AS (
SELECT
t.accession,
t.FK_OntologyName,
t.name,
t.definition,
t.xrefvaluetype,
t.isobsolete,
trt.fk_termAccession,
trt.relationshiptype,
trt.fk_termAccession_related,
0 depth_level
FROM Term t
INNER JOIN (TermRelationship AS trt, Term AS ref) ON(
t.Accession = trt.FK_TermAccession
AND trt.FK_TermAccession_Related = ref.Accession
AND (trt.RelationshipType = `is_a` OR trt.RelationshipType = `part_of`)
AND
(
trt.FK_TermAccession_Related = ref.Accession
AND ref.Name = parentOntology
)
)
UNION All
SELECT
t2.accession,
t2.FK_OntologyName,
t2.name,
t2.definition,
t2.xrefvaluetype,
t2.isobsolete,
trt2.fk_termAccession,
trt2.relationshiptype,
trt2.fk_termAccession_related,
(previous.depth_level+1) depth_level
FROM Term t2
INNER JOIN (TermRelationship AS trt2, previous) ON(
t2.Accession = trt2.FK_TermAccession
AND trt2.FK_TermAccession_Related = previous.Accession
)
)
SELECT
t.Accession,
t.FK_OntologyName,
t.Name,
t.Definition,
t.xRefValueType,
t.IsObsolete,
p.depth_level
FROM previous p
Inner JOIN Term AS t ON (
p.Accession = t.Accession
);
END$$
DELIMITER ;
To the question: i am not that experienced with MySQL so are there any options to optimize this function, or is MySQL the wrong tool overall?
Example rows:
Term
| Accession | FK_OntologyName | Name | Definition | XRefValueType | IsObsolete |
|------------|-----------------|----------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------|------------|
| TEST:0000000 | TEST | Proteomics Standards Initiative Mass Spectrometry Vocabularies | "Proteomics Standards Initiative Mass Spectrometry Vocabularies." [PSI:MS] | | 0 |
| TEST:1000001 | TEST | sample number | "A reference number relevant to the sample under study." [PSI:MS] | value-type:xsd\:string "The allowed value-type for this CV term." | 0 |
| TEST:1000002 | TEST | sample name | "A reference string relevant to the sample under study." [PSI:MS] | value-type:xsd\:string "The allowed value-type for this CV term." | 0 |
| TEST:1000003 | TEST | sample state | "The chemical phase of a pure sample, or the state of a mixed sample." [PSI:MS] | | 0 |
| TEST:1000004 | TEST | sample mass | "Total mass of sample used." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 |
| TEST:1000005 | TEST | sample volume | "Total volume of solution used." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 |
| TEST:1000006 | TEST | sample concentration | "Concentration of sample in picomol/ul, femtomol/ul or attomol/ul solution used." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 |
| TEST:1000007 | TEST | inlet type | "The nature of the sample inlet." [PSI:MS] | | 0 |
| TEST:1000008 | TEST | ionization type | "The method by which gas phase ions are generated from the sample." [PSI:MS] | | 0 |
| TEST:1000011 | TEST | mass resolution | "Smallest mass difference between two equal magnitude peaks so that the valley between them is a specified fraction of the peak height." [PSI:MS] | value-type:xsd\:string "The allowed value-type for this CV term." | 0 |
| TEST:1000012 | TEST | resolution measurement method | "Which of the available standard measures is used to define whether two peaks are separate." [PSI:MS] | | 0 |
| TEST:1000014 | TEST | accuracy | "Accuracy is the degree of conformity of a measured mass to its actual value." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 |
| TEST:1000015 | TEST | scan rate | "Rate in Th/sec for scanning analyzers." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 |
| TEST:1000016 | TEST | scan start time | "The time that an analyzer started a scan, relative to the start of the MS run." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 |
| TEST:1000018 | TEST | scan direction | "Direction in terms of m/z of the scan for scanning analyzers (low to high, or high to low)." [PSI:MS] | | 0 |
| TEST:1000019 | TEST | scan law | "Describes the function in control of the m/z scan (for scanning instruments). Commonly the scan function is linear, but in principle any function can be used." [PSI:MS] | | 0 |
| TEST:1000021 | TEST | reflectron state | "Status of the reflectron, turned on or off." [PSI:MS] | | 0 |
| TEST:1000022 | TEST | TOF Total Path Length | "The length of the field free drift space in a time of flight mass spectrometer." [PSI:MS] | value-type:xsd\:float "The allowed value-type for this CV term." | 0 | | value-type:xsd\:int "The allowed value-type for this CV term." | 0 |
TermRelationship
| ID | FK_TermAccession | RelationshipType | FK_TermAccession_Related | FK_OntologyName |
|--------|------------------|------------------|--------------------------|-----------------|
| 0 | TEST:1000001 | is_a | TEST:0000000 | TEST |
| 1 | TEST:1000002 | is_a | TEST:0000000 | TEST |
| 2 | TEST:1000003 | is_a | TEST:1000002 | TEST |
| 3 | TEST:1000004 | is_a | TEST:1000002 | TEST |
| 4 | TEST:1000005 | is_a | TEST:1000002 | TEST |
| 5 | TEST:1000006 | is_a | TEST:1000002 | TEST |
| 6 | TEST:1000007 | is_a | TEST:1000002 | TEST |
| 7 | TEST:1000008 | is_a | TEST:1000007 | TEST |
| 8 | TEST:1000011 | is_a | TEST:1000007 | TEST |
| 9 | TEST:1000022 | is_a | TEST:0000000 | TEST |
Expected outcome is to execute the function for TEST:0000000/Proteomics Standards Initiative Mass Spectrometry Vocabularies and get all the following terms, because they are child/grandchild terms to TEST:0000000.
TEST:1000001
TEST:1000002
TEST:1000003
TEST:1000004
TEST:1000005
TEST:1000006
TEST:1000007
TEST:1000008
TEST:1000011
TEST:1000022

Some of these may help performance:
t: INDEX(Accession)
trt: INDEX(FK_TermAccession_Related, relationshiptype, FK_TermAccession)
trt: INDEX(FK_TermAccession, FK_TermAccession_Related, relationshiptype)
ref: INDEX(Accession, Name)
previous: INDEX(Accession, depth_level)
Please use JOIN...ON instead of (TermRelationship AS trt, Term AS ref)
What table is parentOntology in?
trt.FK_TermAccession_Related = ref.Accession show up redundantly.

Related

How to generate (not draw/render) sequence diagrams

I'm looking for ways to generate Sequence Diagrams out of programmed Logic. I do not mean rendering them out of e.g. text. A lot of post talk about generating, but I see that as rendering/drawing. I've searched the internet and most found textual tools (like PlantUML) or intuitive graphical payed tools. I'm not looking for either. But I want to program the message flow and let the system draw it based on possible choices. One reason for it is that the conditional if then else with 'alt' is not very useful (my opinion) if the else triggers a whole different path. It works for one different return, but it becomes very ugly soon (again my opinion). The other reason is that I'm busy developing such a generating tool myself and I'm wondering if I do not try the make something nobody is waiting for. Except the fact that it is a nice hobby project for myself. For me it made sense to make it as it Interactively creates the Message Diagrams which are very helpful during the development process or pass on knowledge. Perhaps it could even validate the logic on dead ends. The third reason is that the text becomes complex by itself to maintain (see example below). The fourth reason is that I believe in automating a process that can/should be automated as we should not be busy drawing stuff but writing logic. So is anybody aware of generating (not draw/render) tools available for sequence diagrams?
The following examples were created where the difference started at invalid/valid card, which is difficult to capture in an if/then/else. And even so other choices are made along the way.
Invalid card, choices: electronic -> chip -> invalid card -> keep goods
Merchant Customer Terminal
| | |
+-inform-amount | |
|---choose-method--->| |
|<--chooses-terminal-| |
+-enters-amount | |
|------------start-payment----------->|
| |<--show-amount--|
| +-inserts-card |
| |-method-chosen->|
| |<--card-invalid-|
| +-pay-different |
|<-----------payment-failed-----------|
+-goods-left-behind | |
+-customer-leaves | |
Valid card, choices: electronic -> chip -> valid card -> auth valid -> enough balance
Merchant Customer Terminal Secure-Intf Acc-Srv Acc-DB
| | | | | |
+-inform-amount | | | | |
|----choose-method--->| | | | |
|<--chooses-terminal--| | | | |
+-enters-amount | | | | |
|-------------start-payment------------->| | | |
| |<---show-amount---| | | |
| +-inserts-card | | | |
| |--method-chosen-->| | | |
| |<----card-valid---| | | |
| +-enter-pin | | | |
| |---validate-pin-->| | | |
| | |--sec:authorize-->| | |
| | | |--verify-login-->| |
| | | | |---get-login-details-->|
| | | | |<-----login-details----|
| | | |<-login-response-| |
| | |<--sec:auth-valid-| | |
| | |---sec:transfer-->| | |
| | | |----transfer---->| |
| | | | |------get-balance----->|
| | | | |<-----balance-info-----|
| | | | |-upd-checking-balance->|
| | | | |-upd-merchant-balance->|
| | | | |----commit-changes---->|
| | | | |<---changes-committed--|
| | | |<---transferred--| |
| | |<-sec:transferred-| | |
|<-----------payment-successful----------| | | |
+-goods-given | | | | |
| |<-------paid------| | | |
| +-customer-leaves | | | |
Textual code valid for several free drawing tools. If any changes are needed then I think it becomes difficult to maintain. I rather generate them.
title MSG-Flow for 'Merchant-flows'
participant "Merchant" as Merchant
participant "Customer" as Customer
participant "Terminal" as Terminal
participant "Secure Intf" as Secure_Intf
participant "Acc Srv" as Acc_Srv
participant "Acc DB" as Acc_DB
note left of Merchant: inform-amount
Merchant -> Customer: choose-method
Customer -> Merchant: chooses-terminal
note left of Merchant: enters-amount
Merchant -> Terminal: start-payment
Terminal -> Customer: show-amount
note right of Customer: inserts-card
Customer -> Terminal: method-chosen
Terminal -> Customer: card-valid
note right of Customer: enter-pin
Customer -> Terminal: validate-pin
Terminal -> Secure_Intf: sec:authorize
Secure_Intf -> Acc_Srv: verify-login
Acc_Srv -> Acc_DB: get-login-details
Acc_DB -> Acc_Srv: login-details
Acc_Srv -> Secure_Intf: login-response
Secure_Intf -> Terminal: sec:auth-valid
Terminal -> Secure_Intf: sec:transfer
Secure_Intf -> Acc_Srv: transfer
Acc_Srv -> Acc_DB: get-balance
Acc_DB -> Acc_Srv: balance-info
Acc_Srv -> Acc_DB: upd-checking-balance
Acc_Srv -> Acc_DB: upd-merchant-balance
Acc_Srv -> Acc_DB: commit-changes
Acc_DB -> Acc_Srv: changes-committed
Acc_Srv -> Secure_Intf: transferred
Secure_Intf -> Terminal: sec:transferred
Terminal -> Merchant: payment-successful
note left of Merchant: goods-given
Terminal -> Customer: paid
note right of Customer: customer-leaves
So is anybody aware of generating (not draw/render) tools available for sequence diagrams?
Have you tried ZenUML(https://zenuml.com)?
It can also generate sequence diagram from Java code (as an Intellij Idea plugin).

99% working behavior from mysql statement needs to be 100%

I have inherrited a DB that I've been tasked to mine for Data.
There are 2 tables that are loosely associated - atm and dslams.
The atm table contains "remotename", "rst", and "CardNumber" fields that relate to the dslams "hostname" field.
The atm table contains the port information for the dslam cards and the dslams table contains the information about the dslam card itself.
I've been tasked with printing out all the locations (dslams.name) that have a certain type of card (dslams.model="6256") and a count of all the ports on that card that have a certain level of service (atm.speed LIKE "RI_%%09" OR atm.speed LIKE "RI%%1%").
I've crafted the following statement which almost works...
SELECT distinct(dslams.name) AS Remote, Count(atm.speed) AS Customers, dslams.model
FROM dslams
LEFT JOIN atm
ON (dslams.hostname = CONCAT(atm.remotename,'-',atm.rst,'-S',atm.CardNumber)) AND (atm.speed LIKE "RI_%_%09" OR atm.speed LIKE "RI_%_%1_%")
GROUP BY dslams.name
HAVING dslams.model="6256"
ORDER BY dslams.name;
This prints out exactly what I need for all but 1 of the locations.
ie.
MariaDB [dsl]> SELECT distinct(dslams.name) AS Remote, Count(atm.speed) AS Customers, dslams.model
-> FROM dslams
-> LEFT JOIN atm
-> ON (dslams.hostname = CONCAT(atm.remotename,'-',atm.rst,'-S',atm.CardNumber)) AND (atm.speed LIKE "RI_%_%09" OR atm.speed LIKE "RI_%_%1_%")
-> GROUP BY dslams.name
-> HAVING dslams.model="6256"
-> ORDER BY dslams.name;
+---------+-----------+-------+
| Remote | Customers | model |
+---------+-----------+-------+
| ANTH-C2 | 1 | 6256 |
| BETY-C2 | 1 | 6256 |
| BHOT-C2 | 6 | 6256 |
| BNSH-C2 | 1 | 6256 |
| BUG2-C2 | 1 | 6256 |
| CCRK-C2 | 0 | 6256 |
...
| STLN-C2 | 1 | 6256 |
| SUMR-C2 | 2 | 6256 |
...
| WGRV-C2 | 0 | 6256 |
+---------+-----------+-------+
63 rows in set (0.34 sec)
For some reason there's one location that's not getting counted - STWL-C2.
MariaDB [dsl]> SELECT distinct(name), model FROM dslams WHERE model="6256" order by name;
+---------+-------+
| name | model |
+---------+-------+
| ANTH-C2 | 6256 |
| BETY-C2 | 6256 |
| BHOT-C2 | 6256 |
| BNSH-C2 | 6256 |
| BUG2-C2 | 6256 |
| CCRK-C2 | 6256 |
...
| STWL-C2 | 6256 |
...
| WGRV-C2 | 6256 |
+---------+-------+
64 rows in set (0.00 sec)
There's no difference in the tables between the STWL-C2 location and the other locations so it should print out with a count of 0.
Can anyone help me figure out why that 1 location is being missed?
Any help or direction would be appreciated as I am a rookie SQL programmer trying to understand this as best I can.
Best Regards,
Joe
Don't use HAVING dslams.model = '6256', put that in the WHERE clause. When you use HAVING, it filters after grouping. When you group by name, the result can contain the model from any row in the group, and it won't necessarily choose model = '6256'.
SELECT dslams.name AS Remote, Count(atm.speed) AS Customers, dslams.model
FROM dslams
LEFT JOIN atm
ON (dslams.hostname = CONCAT(atm.remotename,'-',atm.rst,'-S',atm.CardNumber)) AND (atm.speed LIKE "RI_%_%09" OR atm.speed LIKE "RI_%_%1_%")
WHERE dslams.model = '6256'
GROUP BY dslams.name
ORDER BY dslams.name;

How to make IN condition check for set values

Basically I have two tables
MY_CHARACTER:
_________________________________________________
| char_ID | char_name | char_class | char_rank |
|------------------------------------------------|
| 1 | Aragorn | Fighter | 99 |
| 2 | Legolas | Archer | 90 |
| 3 | Smeagle | Spy | 20 |
|________________________________________________|
and
EQUIPMENT:
_________________________________________________
| equip_ID | equip_name | equip_owner | required |
|--------------------------------------------------|
| 1 | The one ring | 3 | 99 |
| 2 | Longsword | 1 | 90 |
| 3 | Waistcloth | 3 | 10 |
| 4 | Nazguls Mask | 2 | 95 |
|__________________________________________________|
Now what I want is to make a select command using IN condition, which will give me list of equipments that the characters are not eligible to use. So in this very example, it would be Smeagle being ineligible to wear the one ring and Legolas unable to wield the nazguls mask.
My command looks something like this
SELECT equip_name, equip_owner, required
FROM EQUIPMENT WHERE required IN (SELECT char_rank MY_CHARACTER);
Now this will only print out the equipment where required=char_rank however i want to print out a select, where required equipment level is higher than char_rank instead. Any idea how? Ideally using the IN condition.
EDIT: To clear out confusion regarding where I want, basically what my command does right now is it checks if EQUIPMENT.required=MY_CHARACTER.char_rank what I want instead is that it checks EQUIPMENT.required>MY_CHARACTER.char_rank
I would do this with a join:
select c.*, equip_id as notEligible
from my_character c join
equipment e
on c.char_rank < e.required;
So if you want a list of items, that a certain character is not allowed to use you can slightly modify your SQL:
SELECT
equip_name, equip_owner, required
FROM
EQUIPMENT
WHERE
required > (SELECT char_rank FROM MY_CHARACTER WHERE char_id=?);

MySQL -> HTML Report, Styled like a Pivot Table

Ok, I'd like to start off by apologizing (profusely), since this seems to be a common question. Most of the examples seem to be somewhat similar, as well, but - for the life of me, I cannot wrap my brain around how to apply the myriad of quality responses to my specific table. And, I'm sure it's probably just the easiest thing in the world, what with all the very thorough responses/examples/links to resources with explanations/etc.
So, I suppose I'll just get right to it. The basics:
We host off-site copies of our clients' backups.
We need to know how much space they're using.
We are not at all consistent in Naming Convention, folder vs. disk per client, etc.
We need to automate a 'report', monthly, with data as follows:
-[C.Srv 01]---Size(GB)--Free(%)
Client 01 [Total] [AVG]
Server 01 109.43 25
Server 02 415.19 25
WHERE C.Srv = [Specified Cloud Server]
Clients Get a Total Size(GB) and an Average Free(%)
My MySQL table is this:
# Name DataType Length/Set Unsigned Allow NULL ZeroFill Default
1. ID INT 11 AUTO_INCREMENT
2. Client TEXT
3. Server TEXT
4. C.Srv TEXT
5. Size DECIMAL 10,2
6. Free DECIMAL 10,4
So, for Example, let's say I have this...
___ ________ ________ _________ _________ _______
ID | CLIENT | SERVER | C.SRV | SIZE | FREE
---|--------|--------|---------|---------|-------
1 | a | adc | cs_01 | 109.43 | 0.2504
2 | a | asql | cs_01 | 415.19 | 0.2504
3 | b | bdc | cs_01 | 583.91 | 0.1930
4 | b | bdev | cs_01 | 316.52 | 0.1930
5 | b | bsql | cs_01 | 1259.56 | 0.1930
6 | c | cdc | cs_01 | 355.30 | 0.7631
7 | d | ddc | cs_01 | 398.21 | 0.5808
Is it possible to get something pretty, in HTML (preferably), that has the basic structure of this...
_______ __________ ________
CS_01 | Size(GB) | Free(%)
-------|----------|--------
-a | 524.62 | 25.04%
-------|----------|--------
adc | 109.43 | 25.04%
asql | 415.19 | 25.04%
-b | 2178.88 | 19.30%
-------|----------|--------
bdc | 583.91 | 19.30%
bdev | 316.52 | 19.30%
bsql | 1259.56 | 19.30%
+c | 355.30 | 76.31%
-------|----------|--------
+d | 398.21 | 58.08%
_______|__________|________
Or, am I just S.O.L.? Format, I can mess with in CSS, or whatever (I hope), just so long as it's in that basic structure. (I don't know if it matters, but the final goal will be to collapse at the Client Level; in case that somehow factors into the approach/data-gathering.)

MySQL join - three tables, returning null where empty

I've got the following schema:
phrase (in U.S. English): translation: code_value:
---------------------- ----------------------------------------- ------------------------------
| phrase_id | phrase | | phrase_id | translation | language_cd | | code class | code value |
---------------------- ----------------------------------------- ------------------------------
| 1 | cheese | | 1 | fromage | FR | | LANGUAGE_CD | FR |
---------------------- ----------------------------------------- | LANGUAGE_CD | DE |
| LANGUAGE_CD | ES |
------------------------------
What this collection of data does is, for a given U.S. English phrase, it will give you the corresponding translation in three languages, French, German, and Spanish (at least within the context of our web app - we're not trying to be be Google Translation or anything).
What I'm trying to do is get a list of all translations for a given phrase, and if no translation into a given destination language exists, I want it to return NULL.
My query so far is:
SELECT phrase.phrase_id, phrase.string orig_phrase, code_value.code_value, translation.string as trans_phrase
FROM phrase, translation, code_value
WHERE code_value.code_class = 'LANGUAGE_CD' AND translation.phrase_id = phrase.phrase_id
ORDER BY orig_phrase;
Which returns:
-------------------------------------------------------
| phrase_id | orig_phrase | code_value | trans_phrase |
-------------------------------------------------------
| 1 | cheese | FR | fromage |
| 1 | cheese | DE | fromage |
| 1 | cheese | ES | fromage |
-------------------------------------------------------
But what I intend for it to return is:
-------------------------------------------------------
| phrase_id | orig_phrase | code_value | trans_phrase |
-------------------------------------------------------
| 1 | cheese | FR | fromage |
| 1 | cheese | DE | <NULL> |
| 1 | cheese | ES | <NULL> |
-------------------------------------------------------
I know I need a LEFT or RIGHT JOIN in there to get the NULL values back, but I can't seem to figure out exactly how to write it.
When this is all said and done, obviously we'll have the German and Spanish translations in there as well, and I need them to match up, but the purpose of the NULL values is to clearly show us into what languages we have yet to translate a given phrase.
SELECT phrase.phrase_id, phrase.string orig_phrase, code_value.code_value, translation.string as trans_phrase
FROM phrase
inner join translation on (translation.phrase_id = phrase.phrase_id)
left join code_value on (code_value.code_value=translation.language_cd)
WHERE code_value.code_class = 'LANGUAGE_CD'
ORDER BY orig_phrase;
You are right that you'll need a LEFT or a RIGHT join. This should work:
SELECT `p`.`phrase_id`, `p`.`phrase` `orig_phrase`, `c`.`code_value`, `t`.`translation` `trans_phrase`
FROM `phrase` `p`
INNER JOIN (
SELECT `code_value`
FROM `code_value`
WHERE `code_class` = 'LANGUAGE_CD'
) `c`
LEFT JOIN `translation` `t` ON `p`.`phrase_id` = `t`.`phrase_id` AND `c`.`code_value` = `t`.`language_cd`;
Hope this helps.