Ok, I'd like to start off by apologizing (profusely), since this seems to be a common question. Most of the examples seem to be somewhat similar, as well, but - for the life of me, I cannot wrap my brain around how to apply the myriad of quality responses to my specific table. And, I'm sure it's probably just the easiest thing in the world, what with all the very thorough responses/examples/links to resources with explanations/etc.
So, I suppose I'll just get right to it. The basics:
We host off-site copies of our clients' backups.
We need to know how much space they're using.
We are not at all consistent in Naming Convention, folder vs. disk per client, etc.
We need to automate a 'report', monthly, with data as follows:
-[C.Srv 01]---Size(GB)--Free(%)
Client 01 [Total] [AVG]
Server 01 109.43 25
Server 02 415.19 25
WHERE C.Srv = [Specified Cloud Server]
Clients Get a Total Size(GB) and an Average Free(%)
My MySQL table is this:
# Name DataType Length/Set Unsigned Allow NULL ZeroFill Default
1. ID INT 11 AUTO_INCREMENT
2. Client TEXT
3. Server TEXT
4. C.Srv TEXT
5. Size DECIMAL 10,2
6. Free DECIMAL 10,4
So, for Example, let's say I have this...
___ ________ ________ _________ _________ _______
ID | CLIENT | SERVER | C.SRV | SIZE | FREE
---|--------|--------|---------|---------|-------
1 | a | adc | cs_01 | 109.43 | 0.2504
2 | a | asql | cs_01 | 415.19 | 0.2504
3 | b | bdc | cs_01 | 583.91 | 0.1930
4 | b | bdev | cs_01 | 316.52 | 0.1930
5 | b | bsql | cs_01 | 1259.56 | 0.1930
6 | c | cdc | cs_01 | 355.30 | 0.7631
7 | d | ddc | cs_01 | 398.21 | 0.5808
Is it possible to get something pretty, in HTML (preferably), that has the basic structure of this...
_______ __________ ________
CS_01 | Size(GB) | Free(%)
-------|----------|--------
-a | 524.62 | 25.04%
-------|----------|--------
adc | 109.43 | 25.04%
asql | 415.19 | 25.04%
-b | 2178.88 | 19.30%
-------|----------|--------
bdc | 583.91 | 19.30%
bdev | 316.52 | 19.30%
bsql | 1259.56 | 19.30%
+c | 355.30 | 76.31%
-------|----------|--------
+d | 398.21 | 58.08%
_______|__________|________
Or, am I just S.O.L.? Format, I can mess with in CSS, or whatever (I hope), just so long as it's in that basic structure. (I don't know if it matters, but the final goal will be to collapse at the Client Level; in case that somehow factors into the approach/data-gathering.)
Related
I've recently tried to create an executable with python 2.7 which can read a MySQL database.
The database (named 'montre') regroups two tables : patient and proto_1
Here is the content of those tables :
mysql> select * from proto_1;
+----+------------+---------------------+-------------+-------------------+-----
----------+----------+
| id | Nom_Montre | Date_Heure | Temperature | Pulsion_cardiaque | Taux
_oxy_sang | Humidite |
+----+------------+---------------------+-------------+-------------------+-----
----------+----------+
| 1 | montre_1 | 2017-11-27 19:33:25 | 22.30 | NULL |
NULL | NULL |
| 2 | montre_1 | 2017-11-27 19:45:12 | 22.52 | NULL |
NULL | NULL |
+----+------------+---------------------+-------------+-------------------+-----
----------+----------+
mysql> select * from patient;
+----+-----------+--------+------+------+---------------------+------------+----
----------+
| id | nom | prenom | sexe | age | date_naissance | Nom_Montre | com
mentaires |
+----+-----------+--------+------+------+---------------------+------------+----
----------+
| 2 | RICHEMONT | Robert | M | 37 | 1980-04-05 23:43:00 | montre_3 | ess
aye2 |
| 3 | PIERRET | Mandy | F | 22 | 1995-04-05 10:43:00 | montre_4 | ess
aye3 |
| 14 | PIEKARZ | Allan | M | 22 | 1995-06-01 10:32:56 | montre_1 | Hea
lthy man |
+----+-----------+--------+------+------+---------------------+------------+----
----------+
As I'm just used to code in C (no OOP), I didn't create class in the python project (shame on me...). But I managed, in two files, to create something (with mysql.connector) which can print (on the cmd) my database and excecute sub like looking-for() etc.
Now, I want to create a GUI for users with pyqt. Unfortunately, I saw that the structure is totally different, with class etc. But okay, I tried to go throught this and I've created a GUI which allows to display the table "patient". But I didn't manage (in the datasheet of QT) to find how I can use the programs I've already created to display. Neither how to display in a tableWidget only several rows of my table patient for exemple (Using QSQL).
For example, if I want to display all the table patient, I use this line (pyQt):
self.model.setTable("patient")
For this one, I got it, but that disturb me because there is no MySQL coding requisites to display my table and so I don't know how to sort only the rows we want to see and display them. If we only want to see, for example, the ID n°2, how to display in the table:widget only Robert ?
To recap, I want to know :
If I can take the coding I've created and combine it with pyQT
How to display (tableWidget) only rows which are selected by MySQL. Is that possible ?
Please find in the URL my code for a better understanding of my problem :
https://drive.google.com/file/d/1nxufjJfF17P5hN__CBEcvrbuHF-23aHN/view?usp=sharing
I hope I was clear, thank you all for your help !
I want to build a page like shown below and all data should be retrieved from a database. Both the term, subject and sentences is retrieved from a database. Three levels of data. And under each term (eg. Spring 2017) I can pick and choose between all of these sentences.
Spring 2017
Subject1
Sentence 1
Sentence 2
Sentence 3
Subject2
Sentence 13
Sentence 12
Sentence 17
Subject3
Sentence 11
Sentence 14
Sentence 19
Autmn 2017
...
I want to present similar info from database to user, and let the user choose between all this sentences. How should i build up my database for achieving this in the best and most efficient way.
One way is:
Table 'subject' Table 'sentences'
| id | subjects | | id | subjectid | name |
| 3 | Subject1 | | 1 | 3 | Sentence 2 |
| 4 | Subject2 | | 2 | 4 | Sentence 13 |
Table 'term'
| id | term | sentenceid |
| 1 | Spring 17 | 1,2,28 |
Another way is maybe using pivot-tables, something like this:
Table 'sentences'
| id | parentid | name |
| 1 | 0 | Subject2 |
| 2 | 3 | Sentence 2 |
| 3 | 0 | Subject1 |
| 4 | 1 | Sentence 13 |
Table 'term'
| id | term | sentenceid |
| 1 | Spring 17 | 2,4,28 |
Notice: Number of terms can be many more than just two in a year.
Is it any of this structures you recommend, or any other way you think I should build my database? Is one of these more efficient? Not so demanding? Easier to adjust?
You are doing relational analysis/design:
Find all substantives/nouns of your domain. These are candidates for tables.
Find any relationships/associations between those substantives. "Has", "consists of", "belongs to", "depends on" and so on. Divide them into 1:1, 1:n, n:m associations.
look hard at the 1:1 ones and check if you can reduce two of your original tables into one.
the 1:n lead you to foreign keys in one of the tables.
the n:m give you additional association tables, possibly with their own attributes.
That's about it. I would strongly advise against optimizing for speed or space at this point. Any modem RDBMS will be totally indifferent against the number of rows you are likely to encounter in your example. All database related software (ORMs etc.) expect such a clean model. Packing ids into comma separated fields is an absolutes no-no as it defeats all mechanisms your RDBMS has to deal with such data; it makes the application harder to program; it confuses GUIs and so on.
Making weird choices in your table setup so they deviate from a clean model of your domain is the #1 cause of trouble along the way. You can optimize for performance later, if and when you actually get into trouble. Except for extreme cases (huge data sets or throughput), such optimisation primarily takes place inside the RDBMS (indexes, storage parameters, buffer management etc.) or by optimizing your queries, not by changing the tables.
If the data is hierarchical, consider representing it with a single table, with one column referencing a simple lookup for the "entry type".
Table AcademicEntry
================================
| ID | EntryTypeID | ParentAcademicEntryID | Description |
==========================================================
| 1 | 3 | 3 | Sentence 1 |
| 2 | 1 | <null> | Spring 2017 |
| 3 | 2 | 2 | Subject1 |
Table EntryType
================================
| ID | Description |
====================
| 1 | Semester |
| 2 | Subject |
| 3 | Sentence |
Start with the terms. Every term has subjects. Every subject has sentences. Then you may need the position of a subject within a term and probably the position of a sentence in a subject.
Table 'term'
id | term
---+------------
1 | Spring 2017
Table 'subject'
id | title | termid | pos
---+----------+--------+----
3 | Subject1 | 1 | 1
4 | Subject2 | 1 | 2
5 | Subject3 | 1 | 3
Table 'sentence'
id | name | subjectid | pos
---+-------------+-----------+-----
1 | Sentence 2 | 3 | 2
2 | Sentence 13 | 4 | 1
3 | Sentence 1 | 3 | 1
4 | Sentence 3 | 3 | 3
2 | Sentence 17 | 4 | 3
...
This table design Should resolve your need.
TblSeason
(
SeasonId int,
SeasonName varchar(30)
)
tblSubject
(
Subjectid int
sessionid int (fk to tblsession)
SubjectData varchar(max)
)
tblSentences
(
SentencesID INT
Subjectid int (Fk to tblSubject)
SentenceData varchar(max)
)
I have a lot of spreadsheets that pull transactional information from our ERP software into Excel using the Microsoft Query that we then perform other calculations on automatically. Recently we upgraded our ERP system, but management made the decision to leave the transactional history in the old databases to have a clean one going forward in the new system. I still need to have some "rolling 12 months" graphs, but if I use only the old database, I'm missing new data and if I use only the new, I'm missing the last 11 months data.
Is there a way that I can write a query in Excel to pull data from the old database PartTran table and merge it with the new database PartTran table without user intervention each time? For instance, I don't want my users (if possible) to have to have two queries that they copy and paste into one Excel table. The schema of the tables (at least the columns I need) are identically named and defined.
If you want to take a bit of a fun, hacky Excel approach, you could do the "copy-paste" bit FOR your users behind the scenes. Given two similar tables OLD and NEW with structures
+-----+------+-------+------------+
| id | foo | bar | date |
+-----+------+-------+------------+
| 95 | blah | $25 | 2015-06-01 |
| 96 | bork | $12 | 2015-07-01 |
| 97 | bump | $200 | 2015-08-01 |
| 98 | fizz | | 2015-09-01 |
| 99 | buzz | $50 | 2015-10-01 |
| 100 | char | ($1) | 2015-11-01 |
| 101 | mope | | 2015-12-01 |
+-----+------+-------+------------+
and
+----+-----+-------+------------+------+---------+
| id | foo | bar | date | fizz | buzz |
+----+-----+-------+------------+------+---------+
| 1 | cat | ($10) | 2016-01-01 | 285B | 1110111 |
| 2 | dog | $25 | 2016-02-01 | 27F5 | 1110100 |
| 3 | ant | $100 | 2016-03-01 | 1F91 | 1001111 |
+----+-----+-------+------------+------+---------+
... you can union together the data for these two datasets with some prudent excel wizardry as below:
Your UNION table ( named using alt+j+t+a ) should have the following items:
New natural ID
DataSet pointer ( name of old or new table )
Derived ID from original dataset
Columns of data you want from Old & New DataSets
example:
+---------+------------+------------+----+------+-----+------------+------+------+
| UnionId | SourceName | SourceRank | id | foo | bar | date | fizz | buzz |
+---------+------------+------------+----+------+-----+------------+------+------+
| 1 | OLD | | | | | | | |
| 2 | NEW | | | | | | | |
+---------+------------+------------+----+------+-----+------------+------+------+
You will then make judicious use of Indirect() and VlookUp() to derive the lookup id and column targets. Sample code below
SourceRank - helper column
=COUNTIFS([SourceName],[#SourceName],[UnionId],"<="&[#UnionId])
id - the id from the original DataSet
=SMALL(INDIRECT([#SourceName]&"[id]"),[#SourceRank])
Everything else is just VlookUp madness!! Although I've taken the liberty of copying the sample code below for reference
foo =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[foo]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
bar =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[bar]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
date =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[date]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
fizz =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[fizz]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
buzz =VLOOKUP([#id],INDIRECT([#SourceName]),MATCH(UNION[[#Headers],[fizz]],INDIRECT([#SourceName]&"[#Headers]"),0),0)
Output
You'll likely want to make prudent use of If() and/or IfError() to help your users ignore the new column references to the old table and those rows that do not yet have data. Without that, however, you'll end up with something like the below.
This is both ready to accept & read new inputs to both OLD and NEW DataSets and is sortable to get rid of those pesky placeholder rows...
Hope this helps! Happy coding!
I have a query I am trying to build which I want to dose some natural language searching. I am unsure of the best way to do this in mysql. I believe mysql has some cool natural language stuff that I can use.
I have two tables which I have shown below.
1. transaction_category...
+--------------------+--------------------+-------------------+----------+
| tran_category_code | tran_category_desc | tran_category_seq | btn_type |
+--------------------+--------------------+-------------------+----------+
| CarParking | Car Parking | 2 | default |
| Electricity | Electricity | 1 | default |
| Groceries | Groceries | 4 | default |
| HealthInsurance | Health Insurance | 5 | default |
| Other | Other | 7 | default |
| Petrol | Petrol | 3 | default |
| Phone | Phone | 6 | default |
+--------------------+--------------------+-------------------+----------+
2. transaction_category_keyword...
+---------------------------------+------------------------------+--------------------+
| transaction_category_keyword_id | transaction_category_keyword | tran_category_code |
+---------------------------------+------------------------------+--------------------+
| 6 | Telstra | Phone |
| 7 | Park | CarParking |
| 8 | Coles | Groceries |
| 9 | Bp Connect | Petrol |
| 10 | Bupa | HealthInsurance |
+---------------------------------+------------------------------+--------------------+
My query is below and that returns the results I want but I was just wondering if anyone could give me advice on whether this could be improved using mysql's natural language functions. This would help me because the search is very simple now but I will be building on it a lot soon.
SELECT
tck.transaction_category_keyword_id,
tck.transaction_category_keyword,
tck.tran_category_code
FROM transaction_category tc, transaction_category_keyword tck
WHERE tc.tran_category_code = tck.tran_category_code
AND 'Coles Menai Syd Au' like '%' ||UPPER(tck.transaction_category_keyword) || '%'
+---------------------------------+------------------------------+--------------------+
| transaction_category_keyword_id | transaction_category_keyword | tran_category_code |
+---------------------------------+------------------------------+--------------------+
| 7 | Park | CarParking |
| 8 | Coles | Groceries |
| 10 | Bupa | HealthInsurance |
| 9 | Bp Connect | Petrol |
| 6 | Telstra | Phone |
+---------------------------------+------------------------------+--------------------+
thanks
In general, if you have a wildcard at both the beginning and end of your search field, then your searches are going to be fairly slow on any non-trivial table sizes, as the field will have to be searched starting from every index.
You would definitely benefit from full text search and match as you are searching for bags of words (and their relative frequencies in the index), rather than a specific string within some other field. I assume you have read the docs at http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html. There are a number of subtleties you need to understand such as stop words, boolean search, query expansion, etc. The comments on these pages are very good as they have the accumulated knowledge of people who have been there before and experimented.
It is also worth reading about tf-idf which is how MySQL (and many other full-text searches) work internally, see the docs, wich basically ranks a search according to a combination of how rare a word is in all documents and how many times is occurs in a particular document.
I can't give you any more focused examples, or performance metrics, as your question is asking will full text outperform a double wildcard like search, to which the answer is a pretty much unqualified yes.
CAVEAT: Always worth mentioning, given the differences between engines, but before MySQL version 5.6 full-text search only words for MyISAM, but thereafter with InnoDB too.
It might sound silly but Im just curious.
I have a table named posts:
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| title | varchar(50) | YES | | NULL | |
| body | text | YES | | NULL | |
| created | datetime | YES | | NULL | |
| modified | datetime | YES | | NULL | |
+----------+------------------+------+-----+---------+----------------+
The values:
+----+-----------------------+----------------------------------------+---------------------+---------------------+
| id | title | body | created | modified |
+----+-----------------------+----------------------------------------+---------------------+---------------------+
| 2 | A title once again!!! | And the post body follows. Tralalalala | 2013-06-03 13:13:44 | 2013-06-05 09:36:51 |
| 3 | Title strikes back | This is really exciting! Not. | 2013-06-03 13:13:46 | NULL |
| 11 | Tomcat | Tommy boy!!! FFF | 2013-06-04 16:33:22 | 2013-06-04 16:48:40 |
| 12 | FFD | dsfdsf | 2013-06-04 16:48:56 | 2013-06-04 16:55:50 |
| 13 | fdf | dfdsf | 2013-06-04 16:57:47 | 2013-06-05 09:36:54 |
| 14 | GGD | dsfdsf | 2013-06-04 17:02:33 | 2013-06-04 17:02:33 |
| 15 | GG# | dsfdsfff322 | 2013-06-05 09:36:20 | 2013-06-05 09:36:28 |
+----+-----------------------+----------------------------------------+---------------------+---------------------+
Let's say I want to search for row that has the value Th (not case sensitive) regardless of the FIELD. This is like making a quick search function.
Normally I would do something like : SELECT * FROM posts WHERE title LIKE '%Th%' OR body LIKE '%Th%'
I did not include the other fields because obviously they are not gonna accept those values.
I wanna know if there's a shortcut to this? Like SELECT * FROM posts LIKE '%Th%'.
Please advise. Thanks.
Using plain old SQL you need to specify all the column names you wish to include.
If you want more search-box-like behavior, I'd suggest looking at MySQL's fulltext functions; see:
http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
The SQL language is based on the presumption of the schema being known. Thus, there is no "search any column" type of functionality. How would it work against non-text columns? What about columns of different collations? Aside from the language not having a feature, specifying the columns expresses your intent to the next developer and that as much as anything should be an overriding consideration.
Other answers have covered that you need to specify all the columns. Here is an alternative formulation that is a bit shorter:
SELECT *
FROM posts
WHERE concat(title, ' ', body) LIKE '%Th%'
If you are looking for an exact match, then you can do:
select *
from posts
where 'Th' in (title, body)
No there is no shortcut for using a where clause. and specifying the columns. Otherwise the query engine can never know what to filter and what column to filter unless you specify them in the where clause.
If you want a custom shortcut - you can write a function which takes a single parameter (the search string) and returns the required fields.
I'm afraid there isn't.
Not sure what your use case is... does this alternative approach work for your use case?
mysql -u{user} -p{password} -h{hostname} {database_name} -B -e "{query}" | grep "{search_string}"
It connects to the database and runs the specified query, returns query results in new lines, fields separated by tab stop. Then use Unix utility grep to filter returned rows.