I'm about ready to rip my hair out and take up poop flinging as a living!
I have a MySQL query which runs fine in MySQL
SELECT
p.ID AS DataID,
p.timestamp AS Timestamp,
sum(p.Value * v.Factor) AS Value,
v.VirtualProfiles_id AS VProfileID
FROM
profiledata p
JOIN
profilevirtualjoin v
ON
p.Profile_ID=v.Profile_ID
WHERE
v.VirtualProfiles_id = 5
GROUP BY
v.Profile_ID,
p.timestamp
But when I try to run this as a query in a SQLDataSet in Delphi
SQLDataSet2.Active := False;
SQLDataSet2.CommandText := 'SELECT p.ID AS DataID, p.timestamp AS Timestamp, sum(p.Value * v.Factor) AS Value,' +
'v.VirtualProfiles_id AS VProfileID FROM profiledata p JOIN profilevirtualjoin v ON ' +
'p.Profile_ID=v.Profile_ID WHERE v.VirtualProfiles_id = ' + InttoStr(5)
+' GROUP BY v.Profile_ID, p.timestamp';
SQLDataSet2.Active := True;
I get an error
First chance exception at $765BC41F. Exception class TDBXError with message 'Unknown column 'v.VirtualProfiles_id' in 'where clause''. Process EMVS.exe (7556)
If anyone can offer any insight, I would be most appreciative.
EDIT:
I am using the MySQL server 5.5 and Delphi XE
What I am trying to do is this:
I have a tables as follows:
Profile:
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| Designation | varchar(255) | YES | | NULL | |
| Description | text | YES | | NULL | |
| UnitID | int(11) | NO | PRI | NULL | |
+-------------+--------------+------+-----+---------+----------------+
profiledata
+------------+----------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+----------+------+-----+---------+----------------+
| ID | int(11) | NO | PRI | NULL | auto_increment |
| TimeStamp | datetime | YES | | NULL | |
| Value | double | YES | | NULL | |
| Profile_ID | int(11) | NO | PRI | NULL | |
+------------+----------+------+-----+---------+----------------+
Virtualprofiles
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| Designation | varchar(45) | NO | | NULL | |
| Description | varchar(255) | YES | | NULL | |
| Unit_ID | int(11) | NO | PRI | 0 | |
+-------------+--------------+------+-----+---------+----------------+
profilevirtualjoin
+--------------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| VirtualProfiles_id | int(11) | NO | PRI | NULL | |
| Profile_ID | int(11) | NO | PRI | NULL | |
| Factor | double | NO | | 1 | |
+--------------------+---------+------+-----+---------+----------------+
What I need to do is to "produce" a new profile which is the sum of a set of existing profiles. so, the data from the profiledata table must be summed where the ProfileID is included in the virtualprofile and the timestamp values are equal.
The Problem
So, the problem is this. The DBExpress driver provided with Delphi XE can only process Dynamic SQL queries, not MySQL Queries. Although Dynamic SQL is compatible with MySQL, it is not compatible the other way around.
Quoting from the MySQL Manual (sec 12.16.3):
In standard SQL, a query that includes a GROUP BY clause cannot refer to nonaggregated columns in the select list that are not named in the GROUP BY clause.
MySQL extends the use of GROUP BY so that the select list can refer to nonaggregated columns not named in the GROUP BY clause.
The updated DBExpress driver included with Delphi XE3 includes specific support for MySQL code, and so this limitation is not applicable.
The Workaround
The solution to this problem is to create a view in MySQL server and to call it from Delphi using only Dynamic SQL compatible code. In the end the following workaround did the trick:
In MySQL:
CREATE VIEW `VirtualProfileData` AS
SELECT
p.ID AS DataID,
p.timestamp AS Timestamp,
sum(p.Value * v.Factor) AS Value,
v.VirtualProfiles_id AS VProfileID
FROM
profiledata p
JOIN
profilevirtualjoin v
ON
p.Profile_ID=v.Profile_ID
GROUP BY
v.Profile_ID,
p.timestamp
Then in Delphi
SQLDataSet2.Active := False;
SQLDataSet2.CommandText := 'SELECT * FROM VirtualProfileData WHERE VProfileID = ' + InttoStr(5);
SQLDataSet2.Active := True;
You changed the name of the column here:
v.VirtualProfiles_id AS VProfileID
After that point, in most cases (the exception being those involving grouping or aggregation), you need to refer to the column by the new name. I think that's the case here.
Try changing your WHERE clause to use the alias instead:
WHERE v.VirtualProfiles_id = ' + InttoStr(5)
The problem is the compatibility between Mysql Types and Delphi types try to use Basics Types of delphi
In my case changing from
SELECT * FROM table_name WHERE column_name = 'VALUE';
to
SELECT * FROM table_name WHERE column_name LIKE 'VALUE';
solved the problem and return the same result.
I didn't dig dipper to figure it out what was happening, but it is a weird bug because it works fine with every other column except with a specific one.
Related
I have a nested query that deletes a row in table terms only if exactly one row in definitions.term_id is found. It works but it takes like 9 seconds on my system. Im looking to optimize the query.
DELETE FROM terms
WHERE id
IN(
SELECT term_id
FROM definitions
WHERE term_id = 1234
GROUP BY term_id
HAVING COUNT(term_id) = 1
)
The database is only about 4000 rows. If I separate the query into 2 independent queries, it takes about 0.1 each
terms
+-------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| term | varchar(50) | YES | | NULL | |
+-------+------------------+------+-----+---------+----------------+
definitions
+----------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| term_id | int(11) | YES | | NULL | |
| definition | varchar(500) | YES | | NULL | |
| example | varchar(500) | YES | | NULL | |
| submitter_name | varchar(50) | YES | | NULL | |
| approved | int(1) | YES | MUL | 0 | |
| created_at | timestamp | YES | | NULL | |
| updated_at | timestamp | YES | | NULL | |
| votos | int(3) | NO | | NULL | |
+----------------+------------------+------+-----+---------+----------------+
To speed up the process, please consider creating an index on the relevant field:
CREATE INDEX term_id ON terms (term_id)
How about using correlated sub query using exists and try,
DELETE FROM terms t
WHERE id = 1234
AND EXISTS (SELECT 1
FROM definitions d
WHERE d.term_id = t.term_id
GROUP BY term_id
HAVING COUNT(term_id) = 1)
It's often quicker to create a new table retaining only the rows you wish to keep. That said, I'd probably write this as follows, and provide indexes as appropriate.
DELETE
FROM terms t
JOIN
( SELECT term_id
FROM definitions
WHERE term_id = 1234
GROUP
BY term_id
HAVING COUNT(*) = 1
) x
ON x.term_id = t.id
Hehe; this may be a kludgy way to do it:
DELETE ... WHERE id = ( SELECT ... )
but without any LIMIT or other constraints.
I'm depending on getting an error something like "subquery returned more than one row" in order to prevent the DELETE being performed if multiple rows match.
I am using two MYSQL tables on has big log strings for example: "this is a sample log entry with 123.456.789 IP address". Also, there is second table that contains list of Ip addresses in each row. I want to check for all the matching Ip addresses in the log entries and get the result as all the entries in log tables with matching IPs.
I have installed Mysql community version 5.7.22 on RHEL server.
Table 1 : log table
+-------------+---------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+----------------+
| log_id | int(11) | NO | PRI | NULL | auto_increment |
| Id | varchar(30) | NO | | NULL | |
| host | varchar(50) | YES | | NULL | |
| external_id | varchar(40) | NO | | NULL | |
| message | varchar(8000) | YES | | NULL | |
| timestamp | varchar(30) | NO | | NULL | |
+-------------+---------------+------+-----+---------+----------------+
Table 2 : IP table
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| ip | varchar(30) | NO | | NULL | |
| id | int(11) | NO | PRI | NULL | auto_increment |
+-----------+-------------+------+-----+---------+----------------+
I am using below query :
select * from logs where message like '%'ip_table.ip'%';
which is giving a syntax error.
Any other ideas to work on this?
You can fix the syntax error by using concat():
select *
from logs l join
ip_table it
on l.message like concat('%', it.ip, '%');
However, this would match, say, '1.1.1.1' and '1.1.1.10'.
To fix this, you need to take delimiters into account. Assuming this is always a space:
select *
from logs l join
ip_table it
on concat(' ', l.message, ' ') like concat('% ', it.ip, ' %');
Im trying to write a query that utilizes count in sql. The query I am trying to write is.
Find users that reviewed at least 2 restaurants.
Here are the tables that I am using:
explain is_a_restaurant;
+--------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+-------+
| business_id | int(11) | NO | PRI | NULL | |
| cuisine_type | varchar(20) | YES | | NULL | |
| total_seats | int(11) | YES | | 1 | |
+--------------+-------------+------+-----+---------+-------+
explain reviews;
+-------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------+------+-----+---------+-------+
| business_id | int(11) | NO | PRI | NULL | |
| user_id | int(11) | NO | PRI | NULL | |
| review_id | int(11) | NO | PRI | NULL | |
| review_date | date | YES | | NULL | |
| star_rating | int(1) | YES | | 1 | |
+-------------+---------+------+-----+---------+-------+
explain users;
+------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+-------------+------+-----+---------+-------+
| user_id | int(11) | NO | PRI | NULL | |
| name | varchar(50) | YES | | NULL | |
| user_since | date | YES | | NULL | |
Here is what ive tried (ive tried a lot more than this but heres one):
SELECT reviews.user_id FROM reviews JOIN is_a_restaurant ON
(reviews.business_id = is_a_restaurant) WHERE (count(*).is_a_restaurant > 1)
GROUP BY reviews.user_id ASC;
Heres the error that I get
You have an error in your SQL syntax; check the manual that corresponds to
your MySQL server version for the right syntax to use
near '.is_a_restaurant > 1) GROUP BY reviews.user_id ASC' at line 1
I'm not a MySQL guy so my syntax might be a little out, but you probably want to using a HAVING clause.
SELECT reviews.user_id
FROM reviews
JOIN is_a_restaurant ON reviews.business_id = is_a_restaurant.business_id
GROUP BY reviews.user_id ASC
HAVING COUNT(*) > 1;
The HAVING clause is like a WHERE clause but is used for aggregated values (the COUNT in this case).
You were also missing the column name from is_a_restaurant in the JOIN expression.
You need the HAVING clause.
SELECT reviews.user_id
FROM reviews
JOIN is_a_restaurant ON (reviews.business_id = is_a_restaurant.business_id)
GROUP BY reviews.user_id ASC
HAVING count(*) > 1
count() is an aggregate function. SQL aggregate functions are those which return a single value, calculated from values in a column.You can not use where clause with aggregate functions.Instead,you have to use having clause.
Below are the most commonly used aggregate functions.
AVG() - Returns the average value
COUNT() - Returns the number of rows
FIRST() - Returns the first value
LAST() - Returns the last value
MAX() - Returns the largest value
MIN() - Returns the smallest value
SUM() - Returns the sum
So,use having instead of where.
Take care and good luck....!!
I have the following (simplified) Mysql Tables:
Requests:
+----------------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+--------------+------+-----+---------+-------+
| ID | bigint(20) | NO | PRI | NULL | |
| UniqueIdentifier | varchar(255) | YES | MUL | NULL | |
| UniversalServiceId | bigint(20) | YES | MUL | NULL | |
+----------------------+--------------+------+-----+---------+-------+
Observations:
+---------------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+--------------+------+-----+---------+-------+
| ID | bigint(20) | NO | PRI | NULL | |
| Value | varchar(255) | NO | | NULL | |
| RequestId | bigint(20) | NO | MUL | NULL | |
+---------------------+--------------+------+-----+---------+-------+
I have indexed UniqueIdentifier, UniversalServiceId and RequestId.
The tables are queried on UniqueIdentifier and UniversalServiceId with a JOIN on RequestId.
The Observation table has many millions of records. The queries are painfully slow to return and I am wondering if there is anything that I can do to improve performance. I have just started reading about memcache but it seems that it may be useful only after the first query (which is often the only one) for a particular dataset.
This is they type of query that is being used:
select * from Observations where RequestId in (select ID from Requests where UniqueIdentifier = '123456' and UniversalServiceId = '1234'
Any advice / guidance appreciated!
I recommend you use a query using a JOIN operation, rather than an IN (subquery) predicate.
For example:
SELECT o.ID
, o.Value
, o.RequestId
FROM Observations o
JOIN Requests r
ON r.ID = o.RequestId
WHERE r.UniqueIdentifier = '123456'
AND r.UniversalServiceId = '1234'
For optimum performance, suitable indexes would be:
... ON Requests (UniversalServiceId, UniqueIdentifier, ID)
... ON Observations (RequestId, Value, ID)
(The choice of the leading column in the index on the Requests table would depend on the expected cardinality.)
I have two tabels;
mysql> describe ipinfo.ip_group_country;
+--------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+-------------+------+-----+---------+-------+
| ip_start | bigint(20) | NO | PRI | NULL | |
| ip_cidr | varchar(20) | NO | | NULL | |
| country_code | varchar(2) | NO | MUL | NULL | |
| country_name | varchar(64) | NO | | NULL | |
+--------------+-------------+------+-----+---------+-------+
mysql> describe logs.logs;
+----------------------+------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------------+------------+------+-----+---------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| ts | timestamp | NO | | CURRENT_TIMESTAMP | |
| REMOTE_ADDR | tinytext | NO | | NULL | |
| COUNTRY_CODE | char(2) | NO | | NULL | |
+----------------------+------------+------+-----+---------------------+----------------+
I can select country code using ip address from first table:
mysql> SELECT country_code FROM ipinfo.`ip_group_country` where `ip_start` <= INET_ATON('74.125.45.100') order by ip_start desc limit 1;
+--------------+
| country_code |
+--------------+
| US |
+--------------+
In logs.logs, I have all the REMOTE_ADDR (ip address) set, but all COUNTRY_CODE entries are empty. Now, I want to populate COUNTRY_CODE appropriately using the ipinfo table. How can I do this?
thanks!
Try
UPDATE logs.logs
SET COUNTRY_CODE = (
SELECT country_code
FROM ipinfo.ip_group_country
WHERE ipinfo.ip_start <= INET_ATON(logs.REMOTE_ADDR)
LIMIT 1
)
WHERE COUNTRY_CODE IS NULL
If it fails saying the column types must match, you'll have to alter your logs.logs table so that the REMOTE_ADDR column is the same type (varchar(20)) as the ip_cidr table.
In a single-table update you use update t1 set c1=x where y.
In a multi-table update you use update t1, t2 set t1.c1=t2.c2 where t1.c3=t2.c4
Here's the relevant documentation http://dev.mysql.com/doc/refman/5.0/en/update.html
What you're looking for is something along the lines of (editted) update logs.logs as l, ipinfo.ip_group_country as c set l.COUNTRY_CODE=c.country_code where c.ip_start <= INET_ATON(l.REMOTE_ADDR) order by c.ip_start asc
Edit: you're right, the max() in the original answer I provided could not work. The query above should, although it will likely be less efficient than something like the approach in the answer provided below.