I have a dictionary table (words) and another table with concatenated 2 words like "helpme", "helloword" "loveme"...
I want to transform this table to "help me", "hello word", "love me"
I run this sequence :
SELECT
table_concatened.twowords,
t1.word as 'word1',
t2.word as 'word2'
FROM
table_concatened
JOIN dictionary_table AS t1 ON SUBSTRING(table_concatened.twowords,1,len(t1.word)) = t1.word
JOIN dictionary_table AS t2 ON SUBSTRING(table_concatened.twowords,len(t1.word)+1,len(table_concatened.twowords)) = t2.word;
It is working, but is took a very long time with my table.
How can I optimise my sql sequence?
---- exemple of table ---
dictionary_table
|hello|
|word |
|love |
|me |
exemple of table_concatened :
|helloword|
|loveyou |
Edit:
1) The use case is for autocorrection. For example on skype, on iPhone, on chrome, when I type "helloword", I have auto correction to "hello word".
2) The database here is not very important. Our issue is about algo logic and performance optimisation.
If you don't mind going dynamic (and if SQL Server)
-- Generate Some Sample Data
Declare #Dictionary_Table table (word varchar(50));Insert Into #Dictionary_Table values ('hello'),('word'),('love'),('me')
Declare #table_concatened table (ID int,twowords varchar(50));Insert Into #table_concatened values (1,'helloword'),(2,'loveyou')
-- Generate SQL and Execute
Declare #SQL varchar(max)=''
Select #SQL = #SQL+concat(',(',ID,',''||',replace(twowords,'''',''''''),'||'')') From #table_concatened --Where ID=2
Select #SQL = Replace(#SQL,MapFrom,MapTo)
From (
Select MapFrom = word
,MapTo = '|'+ltrim(rtrim(word))+'|'
From #Dictionary_Table
Union All
Select '|',' ' -- Remove Any Remaining |
Union All
Select ' ',' ' -- Remove Any Remaining |
) A
Select #SQL = 'Select ID,Value=ltrim(rtrim(Value)) From ('+Stuff(#SQL,1,1,'values')+') N(ID,Value)'
Exec(#SQL)
Returns
ID Value
1 hello word
2 love you
Related
We have a scenario where users answer some questions related to a parent entity that we'll call a widget. Each question has both a numeric and word answer. Multiple users answer each question for a given widget.
We then display a row for each widget with the average numeric answer for each question. We do that using a MySQL pseudo-pivot with dynamic columns as detailed here So we end up with something like:
SELECT widget_id, ...
ROUND(IFNULL(AVG(CASE
WHEN LOWER(REPLACE(RQ.question, ' ', '_')) = 'overall_size' THEN
if(RA.num = '', 0, RA.num) END),0) + .0001, 2) AS `raw_avg_overall_size`,
...
... where overall_size would be one of the question types related to the widget and might have "answers" from 5 users like 1,2,2,3,1 to that question for a given widget_id based on the answer options below:
Answers
answer_id
answer_type
num
word
111
overall_size
1
x-large
112
overall_size
2
large
113
overall_size
3
medium
114
overall_size
4
small
115
overall_size
5
x-small
So we would end up with a row that had something like this:
widget_id
average_overall_size
115
1.80
What we can't figure out is then given if we round 1.80 to zero precision we get 2 in this example which is the word value 'large' from our data above. We like to include that in the query output too so that end up with:
widget_id
raw_average_overall_size
average_overall_size
115
1.80
large
The issue is that we do not know the average for the row until the query runs. So how can we then reference the word value for that average answer in the same row when executing the query?
As mentioned we are pivoting into a variable and then run another query for the full execution. So if we join in the pivot section, that subquery looks something like this:
SET #phase_id = 1;
SET SESSION group_concat_max_len = 100000;
SET #SQL = NULL;
SET #NSQL = NULL;
SELECT GROUP_CONCAT(DISTINCT
CONCAT(
'ROUND(IFNULL(AVG(CASE
WHEN LOWER(REPLACE(RQ.short_question, '' '', ''_'')) = ''',
nsq,
''' THEN
if(RA.answer = '''', 0, RA.answer) END),0) + .0001, 2) AS `',
CONCAT('avg_raw_',nsq), '`,
REF.value, -- <- ******* THIS FAILS **** --
ROUND(IFNULL(STDDEV(CASE
WHEN LOWER(REPLACE(RQ.short_question, '' '', ''_'')) = ''',
nsq,
''' THEN RA.answer END), 0) + .0001, 3) AS `',
CONCAT('std_dev_', nsq), '`
'
)
ORDER BY display_order
) INTO #NSQL
FROM (
SELECT FD.ref_value, FD.element_name, RQ.display_order, LOWER(REPLACE(RQ.short_question, ' ', '_')) as nsq
FROM review_questions RQ
LEFT JOIN form_data FD ON FD.id = RQ.form_data_id
LEFT JOIN ref_values RV on FD.ref_value = RV.type
WHERE RQ.phase_id = #phase_id
AND FD.element_type = 'select'
AND RQ.is_active > 0
GROUP BY FD.element_name
HAVING MAX(RV.key_name) REGEXP '^[0-9]+$'
) nq
/****** suggested in 1st answer ******/
LEFT JOIN ref_values REF ON REF.`type` = nq.ref_value
AND REF.key_name = ROUND(CONCAT('avg_raw_',nsq), 0);
So we need the word answer (from the REF join's REF.value field in the above code) in the pivot output, but it fails with 'Unknown column REF.value. If we put REF.value in it's parent query field list, that also fails with the same error.
You'll need to join the table/view/query again to get the 'large' value.
For example:
select a.*, b.word
from (
-- your query here
) a
join my_table b on b.answer_id = a.answer_id
and b.num = round(a.num);
An index on my_table (answer_id, num) will speed up the extra search.
This fails, leading to the default of "2":
LOWER(REPLACE(RQ.question, ' ', '_')) = 'overall_size'
That is because the question seems to be "average_overall_size", not "overall_size".
String parsing and manipulation is the pits in SQL; suggest using the application to handle such.
Also, be aware that you may need a separate subquery to compute aggregate (eg AVG()), else it might not be computed over the set of values you think.
Query into temp table, then join
First query should produce table as follows:
CREATE temp table, temp_average_size
widget_id
average_overall_size
rounded_average_size
115
1.80
2
LEFT JOIN
select s.*, a.word
from temp_average_size s LEFT JOIN answers a
ON (s.rounded_average_size = a.num AND a.answer_type = 'overall_size)
I need a list of tables being used in a view in mysql.
For example, if I have a view like:
SELECT * FROM table1
JOIN table2
ON table1.id = table2.id
I want to get: table1,table2
Unfortunately, I don't believe that's possible directly. Instead, you need to query and parse the actual view definition:
SELECT VIEW_DEFINITION
FROM INFORMATION_SCHEMA.VIEWS
WHERE
TABLE_NAME = ?;
.
mysql> CREATE VIEW vw_test AS
-> SELECT * FROM table1 JOIN table2 ON table1.id = table2.id;
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT VIEW_DEFINITION FROM INFORMATION_SCHEMA.VIEWS
-> WHERE TABLE_NAME = 'vw_test';
+------------------------------------------------------------------+
| VIEW_DEFINITION |
+------------------------------------------------------------------+
| select * from table1 join table2 on table1.id = table2.id; |
+------------------------------------------------------------------+
1 row in set (0.00 sec)
then you could use any of the following tools to parse the table names:
Terence Parr's ANTLR parser generator (Java, but can generate parsers in any one of a number of target languages) has several SQL grammars available, including a couple for PL/SQL, one for a SQL Server SELECT statement, one for mySQL, and one for ISO SQL - (http://www.antlr.org/grammar/list).
I took this from SO answer here: SQL parser library for Java - Retrieve the list of table names present in a SQL statement
Data Tools Project - SQL Development Tools (http://www.eclipse.org/datatools/project_sqldevtools/).
Here's the documentation for the SQL Query Parser (http://www.eclipse.org/datatools/project_sqldevtools/sqltools_doc/SQL%20Query%20Parser%20User%20documentation.htm).
Here's a blog with descriptions of how to "Get columns and tables in SQL script (Java version)" http://www.dpriver.com/blog/list-of-demos-illustrate-how-to-use-general-sql-parser/get-columns-and-tables-in-sql-script/
Or write yourself a custom mySQL proc based on the following (found here - http://www.sqlparser.com/fetch-table-column-name-example-extact-all-table-field-name.php):
SELECT c_mandant, hist_datum, parkey1, parkey2, funktionscode, ma_parkey, me_parkey
, CASE WHEN EXISTS (SELECT 1
FROM CDS_H_GRUPPE GRP1
WHERE GRP1.c_mandant = c_mandant
AND GRP1.hist_datum = ADD_MONTHS(LAST_DAY(TRUNC(SYSDATE)), -1)
AND GRP1.funktionscode = 'H'
AND GRP1.parkey1 = ma_parkey)
THEN 1
ELSE NULL
END MA_ME
, CASE WHEN EXISTS (SELECT 1
FROM CDS_H_GRUPPE GRP2
WHERE GRP2.c_mandant = c_mandant
AND GRP2.hist_datum = ADD_MONTHS(LAST_DAY(TRUNC(SYSDATE)), -1)
AND GRP2.funktionscode = 'U'
AND GRP2.parkey1 = me_parkey)
THEN 1
ELSE NULL
END ME_MA
, ROW_NUMBER() OVER (PARTITION BY c_mandant, ma_parkey, me_parkey ORDER BY c_mandant, ma_parkey, me_parkey) ANZ_MA
FROM (SELECT c_mandant, hist_datum, parkey1, parkey2, funktionscode
, CASE WHEN funktionscode = 'U'
THEN parkey1
ELSE parkey2
END MA_PARKEY
, CASE WHEN funktionscode = 'U'
THEN NULL
ELSE parkey1
END ME_PARKEY
FROM
CDS_H_GRUPPE
WHERE
funktionscode IN ('U', 'H')
AND hist_datum = ADD_MONTHS(LAST_DAY(TRUNC(SYSDATE)), -1)
)
this is what you want... this can get table used in Views and table which joined together.. but it can get one join... if want more add few more hint..
hope this would solve your question...
select
case
when view_definition regexp '.*from +.*'
then substring_index(substring_index(view_definition, 'from ', -1), ' ', 1)
end as 'primary table',
case
when view_definition regexp '.*join +.*'
then substring_index(substring_index(view_definition, 'join ', -1), ' ', 1)
end as 'joined table'
from information_schema.views where table_name="YOUR VIEW NAME" and table_schema="shotbot_production";
No that is not possible. You have to look for the definition of the view and get that done by yourself manually.
Data :
Tree Depth
URL1||URL2 2
URL2||URL3 2
URL3||URL4||URL5 3
URL1||URL2||URL3 3
In the above data the Tree column consists of string separated by "||". I need to convert the above data such that I have 3 columns (since the max depth is 3 in this example) , the result should look like:
COL1 COL2 COL3 DEPTH
URL1 URL2 2
URL2 URL3 2
URL3 URL4 URL5 3
URL1 URL2 URL3 3
In the above example the max depth is 3 however in real world it could be N number.
Good day,
In first glance it is look like we need to use user defined SPLIT function but since number of values that you have in each string is not more then 4, there is a much simpler and probably much better solution. We just need to use the built-in PARSENAME function.
I did not test the code but the solution should be something like this:
SELECT PARSENAME(REPLACE(Tree,'||','.'), 1) as col1, PARSENAME(REPLACE(Tree,'||','.'), 2) as col2, PARSENAME(REPLACE(Tree,'||','.'), 3) as col3, Depth
from TableName
I replace the || with dot, since PARSENAME parse names that split by dot. this is the trick :-)
I actually mentioned example like this in my lecture at the sqlsaturday #360. You can see the presentation. The lecture was about WHY to use SQLCLR, and not less important WHEN to use it over transact-SQL. but I also talked about when NOT to use it, and this was one of the examples there.
In any case! if you are going to use SPLIT function then you should use SQLCLR and not T-SQL, as you can see here.
Try this, you just need to enter your Input Table, Output Table, Delimeter and Column to split. It can handle depth of more than 3, unlike PARSENAME function.
It is tested with 100,000 records and 30 split columns. It takes 10 sec to create the desired output.
Declare #Delimiter nvarchar(10) = '||'
Declare #InputTable nvarchar(2000) = '<<input table name>>'
Declare #OutputTable nvarchar(2000) = '<<output table name>>'
Declare #ColumnToSplit nvarchar(2000) = '<<column to split>>'
Declare #lsql nvarchar(max)
Declare #treeDepth int
If Object_id('dbo.treeDepth') is not null
Drop table dbo.treeDepth
CREATE TABLE dbo.treeDepth (depth INT)
declare #ltext nvarchar(max)= 'Select max(1+(len('+#ColumnToSplit+')- len(Replace('+#ColumnToSplit+','''+#Delimiter+''','''')))/(len('''+#Delimiter+'''))) from '+#InputTable
insert dbo.treeDepth EXEC(#ltext)
Select #lsql = isnull(#lsql+',','') +
'xmlname.value(''/Node[1]/Node['+cast(number+1 as nvarchar)+']'',''varchar(1000)'') AS Col_'+cast(number+1 as nvarchar)+''
from master..spt_values where type = 'P' and number < (Select * from dbo.treeDepth)
set #lsql = '
WITH ForXML
AS
(
SELECT *,
CONVERT(XML,''<Node><Node>''
+ REPLACE('+#ColumnToSplit+','''+#Delimiter+''', ''</Node><Node>'') + ''</Node></Node>'') AS xmlname
FROM '+#InputTable+'
)
Select *, '+#lsql+' Into '+#OutputTable+' From ForXML
Alter table '+#OutputTable+'
Drop column xmlname
'
EXEC(#lsql)
I'm trying to make my first function, it creates without any error, but, when I try to use it it gives me error.
Here's the function -
CREATE FUNCTION isie_kontakti (condition CHAR(3))
RETURNS CHAR(100)
BEGIN
DECLARE returnthis CHAR(100);
SELECT DISTINCT Person.name, Person.lastName, Contacts.mobile, Contacts.email
FROM Person JOIN Contacts on Contacts.Person_ID = Person.ID
JOIN ParentChild on ParentChild.parentID = Person.ID
JOIN ChildGroup ON ChildGroup.Person_ID = ParentChild.childID
WHERE ChildGroup.Group_ID = 'condition' INTO returnthis;
RETURN returnthis;
END//
Table schema - http://www.imagesup.net/dm-713886347846.png
You create your function to return a single column of type char(100) yet the returnthis item contains quite a few columns.
You need to match up your query and return type.
How you do that depends on what you're trying to achieve. It's possibly as simple as just concatenating the columns from the select into a single variable, something along the lines of (untested since I don't have my DBMS available at the moment):
SELECT Person.name | ' '
| Person.lastName | ' '
| Contacts.mobile | ' '
| Contacts.email
FROM ...
My requirement is to fetch value from mysql table by passing column names dynamically which I get from function. But the problem is, it displays it as it is rather then fetching the value of column.
The select query is:
SET #stqry =
CONCAT('SELECT DISTINCT
FD.FormsDataID AS FormDataID,
FC.ColumnName AS FieldName,
IFNULL(FC.CustomDisplayName,FC.ColumnName) AS DisplayName,
GetColumnValue(',_FormID,',FC.ColumnName) AS FieldValue,
GetColumnValue(FC.FormID,FC.ColumnName) AS ColIndex
FROM FormsData FD
LEFT JOIN FormsColumns FC ON FD.FormID = FD.FormID
LEFT JOIN FormsRelationship FR ON FR.FormID = FC.FormID
WHERE FD.FormID = ',_FormID,' AND FD.ID = ',_recordID
,' AND FC.FormID=',_FormID
,' AND FD.PARENT_RECORD_ID = 0 AND FC.IsDisplayInEmail = 1
AND FC.IsActive = 1');
select #stqry;
PREPARE n_StrSQL FROM #stqry;
EXECUTE n_StrSQL;
GetColumnValue()is the function I've made which returns columnname from table.(I need it because columns names are added by user dynamically).
The result I am getting is:
FormDataID FieldName fieldValue
-------------------------------
2497 date_time [1]
2497 auto_email [2]
2497 more_email [3]
But actual result should fetch values of [1],[2],[3] in third column. What am I missing here?Can any one help?