I don't understand why this doesn't work, I have a column where I store values comma separated in my "Mysql" database then I want to join two tables to give me results. eg:
SELECT *
FROM users u INNER JOIN
groups g
ON u.id IN ( g.ownerId )
WHERE u.active='1' AND g.gid='15';
And the value of g.ownerId in this senerio is '175,178'.
For some reason this only returns the results from the join with ownerId 175. BUT if I manually enter the values ( 175, 178 ) in the IN clause BOTH rows show up. Why isn't it using both values in the ownerId column?
I have tried this to "separate" the values or force a "list" but it didn't work...
SELECT * FROM users u INNER JOIN groups g ON u.id IN ( SELECT SUBSTRING_INDEX(SUBSTRING_INDEX(g.ownerId, ',', 1), ' ', -1) as x,
SUBSTRING_INDEX(SUBSTRING_INDEX(g.ownerId, ',', 2), ' ', -1) as y ) where g.groupId='15'
Has anyone experienced this before or know what to do?
It doesn't work because the in value consists of a list with a single element that happens to have a comma in it. It is equivalent to:
on uid = '175,178'
You can replace the logic with find_in_set():
on find_in_set(uid, g.id) > 0
However, you really should learn about junctions tables and why your data structure is bad, bad, bad:
You are storing numbers as strings.
You have foreign key relationships with no way to declare them.
You are using string operations inappropriately.
Your query cannot make use of an index.
Fix the data structure.
Related
I've studied and tried days worth of SQL queries to find "something" that will work. I have a table, apj32_facileforms_subrecords, that uses 7 columns. All the data I want to display is in 1 column - "value". The "record" displays the number of the entry. The "title" is what I would like to appear in the header row, but that's not as important as "value" to display in 1 row based upon "record" number.
I've tried a lot of CONCAT and various Pivot queries, but nothing seems to do more than "get close" to what I'd like as the end result.
Here's a screen shot of the table:
The output "should" be linear, so that 1 row contains 9 columns:
Project; Zipcode; First Name; Last Name; Address; City; Phone; E-mail; Trade (in that order). And the values in the 9 columns come from "value" as they relate to the "record" number.
I know there are LOT of examples that are similar, but nothing I've found covers taking all the values from "value" and CONCAT to 1 row.
This works to get all the data I want - SELECT record,value FROM apj32_facileforms_subrecords WHERE (record IN (record,value)) ORDER BY record
But the values are still in multiple rows. I can play with that query to get just the values, but I'm still at a loss to get them into 1 row. I'll keep playing with that query to see if I can figure it out before one of the experts here shows me how simple it is to do that.
Any help would be appreciated.
Using SQL to flatten an EAV model representation into a relational representation can be somewhat convoluted, and not very efficient.
Two commonly used approaches are conditional aggregation and correlated subqueries in the SELECT list. Both approaches call out for careful indexing for suitable performance with large sets.
correlated subqueries example
Here's an example of the correlated subquery approach, to get one value of the "zipcode" attribute for some records
SELECT r.id
, ( SELECT v1.value
FROM `apj32_facileforms_subrecords` v1
WHERE v1.record = r.id
AND v1.name = 'zipcode'
ORDER BY v1.value LIMIT 0,1
) AS `Zipcode`
FROM ( SELECT 1 AS id ) r
Extending that, we repeat the correlated subquery, changing the attribute identifier ('firstname' in place of 'zipcode'. looks like we we could also reference it by element, e.g. v2.element = 2
SELECT r.id
, ( SELECT v1.value
FROM `apj32_facileforms_subrecords` v1
WHERE v1.record = r.id
AND v1.name = 'zipcode'
ORDER BY v1.value LIMIT 0,1
) AS `Zipcode`
, ( SELECT v2.value
FROM `apj32_facileforms_subrecords` v2
WHERE v2.record = r.id
AND v2.name = 'firstname'
ORDER BY v2.value LIMIT 0,1
) AS `First Name`
, ( SELECT v3.value
FROM `apj32_facileforms_subrecords` v3
WHERE v3.record = r.id
AND v3.name = 'lastname'
ORDER BY v3.value LIMIT 0,1
) AS `Last Name`
FROM ( SELECT 1 AS id UNION ALL SELECT 2 ) r
returns something like
id Zipcode First Name Last Name
-- ------- ---------- ---------
1 98228 David Bacon
2 98228 David Bacon
conditional aggregation approach example
We can use GROUP BY to collapse multiple rows into one row per entity, and use conditional tests in expressions to "pick out" attribute values with aggregate functions.
SELECT r.id
, MIN(IF(v.name = 'zipcode' ,v.value,NULL)) AS `Zip Code`
, MIN(IF(v.name = 'firstname' ,v.value,NULL)) AS `First Name`
, MIN(IF(v.name = 'lastname' ,v.value,NULL)) AS `Last Name`
FROM ( SELECT 1 AS id UNION ALL SELECT 2 ) r
LEFT
JOIN `apj32_facileforms_subrecords` v
ON v.record = r.id
GROUP
BY r.id
For more portable syntax, we can replace MySQL IF() function with more ANSI standard CASE expression, e.g.
, MIN(CASE v.name WHEN 'zipcode' THEN v.value END) AS `Zip Code`
Note that MySQL does not support SQL Server PIVOT syntax, or Oracle MODEL syntax, or Postgres CROSSTAB or FILTER syntax.
To extend either of these approaches to be dynamic, to return a resultset with a variable number of columns, and variety of column names ... that is not possible in the context of a single SQL statement. We could separately execute SQL statements to retrieve information, that would allow us to dynamically construct a SQL statement of a form show above, with an explicit set of columns to be returned.
The approaches outline above return a more traditional relational model, (individual columns each with a value).
non-relational munge of attributes and values into a single string
If we have some special delimiters, we could munge together a representation of the data using GROUP_CONCAT function
As a rudimentary example:
SELECT r.id
, GROUP_CONCAT(v.title,'=',v.value ORDER BY v.name) AS vals
FROM ( SELECT 1 AS id ) r
LEFT
JOIN `apj32_facileforms_subrecords` v
ON v.record = r.id
AND v.name in ('zipcode','firstname','lastname')
GROUP
BY r.id
To return two columns, something like
id vals
-- ---------------------------------------------------
1 First Name=David,Last Name=Bacon,Zip Code=98228
We need to be aware that the return from GROUP_CONCAT is limited to group_concat_max_len bytes. And here we have just squeezed the balloon, moving the problem to some later processing, to parse the resulting string. If we have any equal signs or commas that appear in the values, it's going to make a mess of parsing the result string. So we will have to properly escape any delimiters that appear in the data, so that GROUP_CONCAT expression is going to get more involved.
I have two tables
books_tbl:
blocks side-bar top-bar
23,45 3,15 11,56
pages_tbl:
id title
1 ff
3
11
15
I want to select the rows from pages_tbl where pages id has included either blocks, side-bar or tob-bar columns in books_table.
How to process this?
You should really consider fixing your table structure. Never store multiple value in a single cell. See Normalization.
As in this case you can't, try using find_in_set function.
select
from pages_tbl p
where exists (
select 1
from books_tbl b
where find_in_set(
p.id,
concat(b.blocks, ',', b.side_bar, ',', b.top_bar)
) > 0
);
Remember though that this will be slow because the server can't use index if any.
It's usually not a good idea to store comma separated values in a single field. If you really cannot change your data structure, you could use a query like this:
select p.id, p.title
from
pages_tbl p inner join books_tbl b
on (
find_in_set(p.id, b.blocks)
or find_in_set(p.id, b.side-bar)
or find_in_set(p.id, b.top-bar)
)
-- add where condition?
group by p.id, p.title
I have a query, where I check the user has ALL the permissions present in a list of perms.
So, it's something like this...
SELECT DISTINCT account_id
FROM pp_acl_user_roles ur, pp_acl_role_permissions rp
JOIN pp_acl_permissions p ON rp.permission_id=p.id
WHERE (
ur.account_id = '1'
#check for permission ids OR keys depending on what has been passed in.
AND ( p.id IN ('read_accounts', 'write_accounts') OR p.key IN ('read_accounts', 'write_accounts') )
AND ur.role_id = rp.role_id
)
#ensure we have ALL the permissions we asked for, not just some -makes IN() an AND not an OR.
GROUP BY ur.account_id
HAVING COUNT(DISTINCT rp.permission_id) = 2
It checks for either a list of ids or a list of keys for the permissions, so it could be called with either, so this line.
p.id IN ('read_accounts', 'write_accounts') OR p.key IN ('read_accounts', 'write_accounts')
could be
p.id IN (1, 2) OR p.key IN (1, 2)
depending on how it's called.
The HAVING at the end ensures we matched all the items we asked for.
This is fine, but I want to move it to a stored procedure, and I'm hitting a issue.
I had to change IN for FIND_IN_SET so I could pass a string list into the procedure, but the problem is, I have no way to dynamically calculate how many items are in the list, so I can't check they are all present.
Iv'e got this so far...
CREATE PROCEDURE has_permission( IN account_id BIGINT, IN permissions TEXT )
BEGIN
SELECT DISTINCT account_id
FROM pp_acl_user_roles ur, pp_acl_role_permissions rp
JOIN pp_acl_permissions p ON rp.permission_id=p.id
WHERE (
ur.account_id = account_id
#check for permission ids OR keys depending on what has been passed in.
AND ( FIND_IN_SET(p.id, permissions) OR FIND_IN_SET(p.key, permissions) )
AND ur.role_id = rp.role_id
)
#ensure we have ALL the permissions we asked for, not just some -makes IN() an AND not an OR.
GROUP BY ur.account_id;
HAVING COUNT(DISTINCT rp.permission_id) = ????????????
END //
DELIMITER ;
But there is no way to calculate the length of the permissions passed in.
I'm sure there is a way to maybe JOIN to the perms table based on the items in the string and ensure we have matches in both tables, but I cant work it out.
Any pointers much appreciated...
This is what I've got so far...
Not ideal, but you can work out the length of the items in the string with this..
(SELECT LENGTH(permissions) - LENGTH( REPLACE(permissions, ',', '') ) + 1)
It basically counts all the commas in the string and uses that as the total number of perms passed in.
CREATE PROCEDURE has_permission( IN account_id BIGINT, IN permissions TEXT)
BEGIN
SELECT DISTINCT account_id
FROM pp_acl_user_roles ur, pp_acl_role_permissions rp
JOIN pp_acl_permissions p ON rp.permission_id=p.id
WHERE (
ur.account_id = account_id
#check for permission ids OR keys depending on what has been passed in.
AND ( FIND_IN_SET(p.id, permissions) OR FIND_IN_SET(p.key, permissions) )
AND ur.role_id = rp.role_id
)
#ensure we have ALL the permissions we asked for, not just some -makes IN() an AND not an OR.
GROUP BY ur.account_id
HAVING COUNT(DISTINCT rp.permission_id) = (SELECT LENGTH(permissions) - LENGTH( REPLACE(permissions, ',', '') ) + 1);
I have query
SELECT FROM_UNIXTIME(1800 * FLOOR(date/1800)) AS period_start,
COUNT(*) AS count
FROM readed_messages
GROUP BY period_start
ORDER BY period_start ASC
How to add clause if/else, if record doesn't exist. I would like to get commands return 0 and commands return count. I didn't know to do this.
Generating missing data is an annoying problem. Usually it's best left to the application level.
Failing that, and if you have to do it in MySQL, the missing data has to come from somewhere. This can be either a dynamic sequence of numbers built from user variables and a union cross join - or if we know our domain values (like we do in this case) we can pre-build some tables to take care of it.
In this case you're operating on dates, so we need a few tables that will help us generate all the values you're interested in. I'd suggest these tables:
calendar_years(y integer)
calendar_months(m integer)
calendar_days(d integer)
calendar_hours(h integer)
calendar_minutes(m integer)
These tables would all be a single column, containing the range of values you're looking at. calendar_minutes can just have 0 and 30 as values because we're just looking at half hour increments.
This is an instance where a cross join is actually what we want.
select unix_timestamp(concat(y, '-', lpad(m, 2, '0'), '-', lpad(d, 2, '0'), ' ', lpad(h, 2, '0'), ':', lpad(mm, 2, '0'), ':', '00')) tt
from calendar_years
cross join calendar_months
cross join calendar_days
cross join calendar_hours
cross join calendar_minutes
Will now give us every possible combination of our table values, or in essence - every period we're interested in looking at.
We build those columns into a timestamp by using concat and lpad to turn them into a valid datetime string, and then convert it with unix_timestamp.
This gives us a set of values to left join against readed_messages, which will give us your final answer:
select from_unixtime(tt), count(id)
from (
select unix_timestamp(concat(y, '-', lpad(m, 2, '0'), '-', lpad(d, 2, '0'), ' ', lpad(h, 2, '0'), ':', lpad(mm, 2, '0'), ':', '00')) tt
from calendar_years
cross join calendar_months
cross join calendar_days
cross join calendar_hours
cross join calendar_minutes
) q
left join readed_messages rm
on (1800 * floor(`date` / 1800)) = q.tt
where tt <> 0 -- this method is brute force, generates some invalid dates
group by tt
Obviously this will give us a lot of values, probably for a period far greater than you're interested in looking at, you can further filter it in the where clause if you wish.
heres a demo
I am running into some trouble with the following circumstances:
I have a query that creates two temp tables, and the following select to join them together--
SELECT * FROM result
INNER JOIN result2 ON result2.packetDetailsId = result.packetDetailsId
I am then trying to create another column from concatenating a few of the resulting fields and then use that to reference/query against another table. Is there a way to accomplish this in one query? Should I get away from the temp tables?
Thank you again in advance.
update: If I try to alias the combination of the two temp tables I get an error message stating [Err] 1060 - Duplicate column name 'packetDetailsId'
select * from (
SELECT * FROM result
INNER JOIN result2 ON result2.packetDetailsId = result.packetDetailsId) as myalias
Another Update: I almost have it working as one query but I get the result "(BLOB)" in the column I concoctenated:
select packet_details.packetDetailsId,products.productId,Credit,AccountNum,OrderStat, CONCAT(products.productId,Credit,'_',OrderStat) as consol from (
select packetDetailsId, GROUP_CONCAT(Credit) AS Credit, GROUP_CONCAT(AccountNum) AS AccountNum, GROUP_CONCAT(OrderStat) AS OrderStat FROM
( SELECT pd_extrafields.packetDetailsId,
CASE WHEN pd_extrafields.ex_title LIKE ('%Credit%')
THEN pd_extrafields.ex_value ELSE NULL END as Credit,
CASE WHEN pd_extrafields.ex_title LIKE ('%Account%')
THEN pd_extrafields.ex_value ELSE NULL END as AccountNum,
CASE WHEN pd_extrafields.ex_title LIKE ('%Existing%')
THEN pd_extrafields.ex_value ELSE NULL END as OrderStat
FROM pd_extrafields )AS TempTab GROUP BY packetDetailsId ) as alias2
INNER JOIN packet_details ON alias2.packetDetailsId = packet_details.packetDetailsId
INNER JOIN sales ON packet_details.packetDetailsId = sales.packetDetailsId
INNER JOIN sold_products ON sales.saleId = sold_products.saleId
INNER JOIN products ON sold_products.productId = products.productId
If I understand correctly, you already have the temporary tables created and you need to "concatenate" the results, using from ... inner join ...
The only possible restriction you may have is that you can only reference your temporary tables once in your from clause; besides that, there are no other restrictions (I frequently use temporary tables as intermediate steps in the creation of my final result).
Tips
Let's say your temp tables are temp_result1 and temp_result2. Both tables have a field packedDetailsId, on which the join will be performed. Remember to create the appropriate indexes on each table; at the very least you need to index packedDetailsId on both tables:
alter table temp_result1
add index PDI(packedDetailsId);
alter table temp_result2
add index PDI(packedDetailsId);
Now, just execute a query with the desired join and concatenation. If concat returns BLOB, then cast the result as char (of course, I'm assuming you need a text string):
select r1.*, r2.*, cast(concat(r1.field1, ',', r2.field2) as char) as data_concat
from temp_result1 as r1
inner join temp_result2 as r2 on r1.packedDetailsId = r2.packedDetailsId;
I see your problem is that GROUP_CONCAT is returning BLOB values... It's normal (MySQL doesn't know a priori how to return the values, so it returns binary data); just use the cast function.
Hope this helps you
so, if the result2 and result are both temp tables, you will have to include the # if local temp table and ## if global temp table
so your statements should be :
SELECT * FROM #result
INNER JOIN #result2 ON #result2.packetDetailsId = #result.packetDetailsId
My Bad. This is only applicable for MS SQL