hope (and actually think :) ) that some one already had similar problem and solved it ..
the basic issue in short terms .. as the use of a 'wordsearch' feature and the related data will grow rapidly in near future we (in that case means me) need to change to using fulltext search ..
so its all setup now .. but i'm unable to make the querysyntax work (simplified version below)
declare #searchTerm varchar(256)
set #searchTerm = 'test'
declare #searchValues table (code varchar(900), searchterm varchar(256))
-- this is just for testing, usually the table is prefilled with different codes and values
if(#searchTerm is not null)
insert into #searchValues
select code, #searchTerm from dbo.base_question where question_type_id = 'text'
select distinct(a.item_id)
from dbo.answer_text as a with (nolock)
join dbo.base_question as bqt with (nolock) on bqt.id = a.base_question_id
join #searchValues as st on bqt.code = st.code
where a.value is not null
and not LTRIM(RTRIM(a.value)) = ''
and CONTAINS(a.value, st.searchterm)
the where clause is part of a much larger one
.. just guessing i would think the 'CONTAINS'-clause needs a kind of 'static' searchterm input and can't handle a row-dependent value ..
couldn't find anything in the msdn nor here ..
one of the major problems is that the 'base_questions' are not fixed and may change and the search-conditions (which question to search for what term) are completely dynamic .. so no switch-case or something like that .. and i think dynamic sql is also not really an option (for different reasons but that would go far beyond the scope of this question :) .. )
however all searches to this database are done via stored procedures with scalar and table-valued parameters (like #searchValues ) .. i would like to keep it that way if somehow possible
so any hints, tips & suggestions are appreciated
and FYI and completeness
short version of the simplified db-structure:
answer (id bigint (PK), item_id uniqueidentifier (FK), base_question_id uniqueidentifier (FK), value nvarchar(max))
base_question (id uniqueidentifier (PK), code varchar(900), question_type_id (FK), .. descriptive content ..)
item (id uniqueidentifier (PK), .. descriptive content ..)
.. even open to changing the db-structure to make the searches somewhat performant and working again ;) ..
(right now we have around 120k item_ids and around 2.5 mio answers for them but that will increase easily by 10 times and more)
Related
I'm working on a MySQL Way of printing an "affiliate tree" and got the thing working with Common Table Expression. I'm using the following code right now:
WITH RECURSIVE recUsers AS
(
SELECT ID, username, sponsorID, 1 AS depth, username AS path
FROM users
WHERE id = 1
UNION ALL
SELECT c.ID, c.username, c.sponsorID, sc.depth + 1, CONCAT(sc.path, ' > ', c.username)
FROM recUsers AS sc
JOIN users AS c ON sc.ID = c.sponsorID
)
SELECT * FROM recUsers;
This selects the tree underneath the user with the id 1.
Now what I'd need to get is a way to pass that id as a parameter, so I don't need to define everything from the beginning every time I want to get the result.. So my idea is to put everything in a stored prodecure and pass the id in as a parameter.. However, so far I didn't get it working and always getting various errors that are very self speaking...
Basically what I've tried was
DELIMITER //
CREATE PROCEDURE getAffiliateTree(IN userid INT())
BEGIN
---my code here, the userid 1 replaced with userid
END//
DELIMITER;
However, this doesn't seem to work.. How can I get this done?
Two things I would suggest:
Use INT, not INT(). The optional length argument to integer types is deprecated in MySQL 8.0 (which I know you're using, because you're using CTE syntax). Even if you did use the length argument, using an empty argument is not legal syntax.
Make sure that the userid input parameter name is distinct from all of the columns in the tables you reference. That is, if the table has a column named userid (any capitalization), then change the name of your input parameter. Otherwise you may make ambiguous expressions like:
... WHERE userid = userid
Even though you intend one of these to be the column and the other to be the parameter, the SQL parser has no way of knowing that. It ends up treating both as the column name, so it's trivially true on all rows of the table.
Actually, a third thing I would suggest: when you ask questions, "it doesn't seem to work" isn't clear enough. Did it produce an error? If so, what was the full error message? Did it produce no error, but didn't give you the result you wanted? If so, show a mocked-up example of what you expected, and what the query produced that didn't match. It helps to be as clear as you can when you post questions, so readers don't have to guess what trouble you need help with.
The SQLite JSON1 extension has some really neat capabilities. However, I have not been able to figure out how I can update or insert individual JSON attribute values.
Here is an example
CREATE TABLE keywords
(
id INTEGER PRIMARY KEY,
lang INTEGER NOT NULL,
kwd TEXT NOT NULL,
locs TEXT NOT NULL DEFAULT '{}'
);
CREATE INDEX kwd ON keywords(lang,kwd);
I am using this table to store keyword searches and recording the locations from which the search was ininitated in the object locs. A sample entry in this database table would be like the one shown below
id:1,lang:1,kwd:'stackoverflow',locs:'{"1":1,"2":1,"5":1}'
The location object attributes here are indices to the actual locations stored elsewhere.
Now imagine the following scenarios
A search for stackoverflow is initiated from location index "2". In this case I simply want to increment the value at that index so that after the operation the corresponding row reads
id:1,lang:1,kwd:'stackoverflow',locs:'{"1":1,"2":2,"5":1}'
A search for stackoverflow is initiated from a previously unknown location index "7" in which case the corresponding row after the update would have to read
id:1,lang:1,kwd:'stackoverflow',locs:'{"1":1,"2":1,"5":1,"7":1}'
It is not clear to me that this can in fact be done. I tried something along the lines of
UPDATE keywords json_set(locs,'$.2','2') WHERE kwd = 'stackoverflow';
which gave the error message error near json_set. I'd be most obliged to anyone who might be able to tell me how/whether this should/can be done.
It is not necessary to create such complicated SQL with subqueries to do this.
The SQL below would solve your needs.
UPDATE keywords
SET locs = json_set(locs,'$.7', IFNULL(json_extract(locs, '$.7'), 0) + 1)
WHERE kwd = 'stackoverflow';
I know this is old, but it's like the first link when searching, it deserves a better solution.
I could have just deleted this question but given that the SQLite JSON1 extension appears to be relatively poorly understood I felt it would be more useful to provide an answer here for the benefit of others. What I have set out to do here is possible but the SQL syntax is rather more convoluted.
UPDATE keywords set locs =
(select json_set(json(keywords.locs),'$.**N**',
ifnull(
(select json_extract(keywords.locs,'$.**N**') from keywords where id = '1'),
0)
+ 1)
from keywords where id = '1')
where id = '1';
will accomplish both of the updates I have described in my original question above. Given how complicated this looks a few explanations are in order
The UPDATE keywords part does the actual updating, but it needs to know what to updatte
The SELECT json_set part is where we establish the value to be updated
If the relevant value does not exsit in the first place we do not want to do a + 1 on a null value so we do an IFNULL TEST
The WHERE id = bits ensure that we target the right row
Having now worked with JSON1 in SQLite for a while I have a tip to share with others going down the same road. It is easy to waste your time writing extremely convoluted and hard to maintain SQL in an effort to perform in-place JSON manipulation. Consider using SQLite in memory tables - CREATE TEMP TABLE... to store intermediate results and write a sequence of SQL statements instead. This makes the code a whole lot eaiser to understand and to maintain.
I've got self dependant entities (a) in my database, which are referenced from another entity (b), and given a specific (b) entity, I need to get all the (a) entities that are needed. These are many to many mappings, so I have a separate mapping table. I think a recursive Select with a CTE is my best bet, but I'm running into an issue:
This Fiddle illustrates my issue. If some user introduces a circular reference, my recursive select grinds to a screeching halt. I've been wracking my brain to try to find some way to fix this. It should be noted that though I have introduced Foreign Keys in the fiddle, foreign keys aren't actually honored by the system I'm using (long standing argument with the DBAs) - I introduced them to make data flow more clear.
The recursive query, for those who don't want to click through to the fiddle:
WITH recur(objID) AS (
SELECT usesObjID
FROM #otherObj
WHERE otherObjID = 1
UNION ALL
SELECT slaveObjID
FROM #objMap
INNER JOIN recur
on #objMap.masterObjID = recur.objID
)SELECT objID from recur
Any Ideas out there? This design isn't in production, so I can change schema somewhat, but I'd like not to rely on discovering circular references upon insertion, unless it can be done by T-SQL.
It's possible to set the MAXRECURSION of the CTE, which will prevent the infinite loop, but you'll still get weird results since the query will continue to run in the loop until the max recursion is hit.
The challenge is that the loop involves multiple steps, so you can't just check the immediate parent of the child in order to determine if you're in a loop.
One way to handle this would be to add an additional column to the CTE... this new column, tree, tracks all the IDs that have been included so far, and stops when an ID repeats.
WITH recur(objID, Tree) AS (
SELECT
usesObjID,
CAST(',' + CAST(usesObjID AS VARCHAR) + ',' AS VARCHAR) AS Tree
FROM otherObj
WHERE otherObjID = 1
UNION ALL
SELECT
slaveObjID,
CAST(recur.Tree + CAST(slaveObjID AS VARCHAR) + ',' AS VARCHAR) AS Tree
FROM objMap
INNER JOIN recur
ON objMap.masterObjID = recur.objID
WHERE recur.Tree NOT LIKE '%,' + CAST(slaveObjID AS VARCHAR) + ',%'
)SELECT objID from recur
Sql Fiddle Link
I have a table of posts with titles that are in "human" alphabetical order but not in computer alphabetical order. These are in two flavors, numerical and alphabetical:
Numerical: Figure 1.9, Figure 1.10, Figure 1.11...
Alphabetical: Figure 1A ... Figure 1Z ... Figure 1AA
If I orderby title, the result is that 1.10-1.19 come between 1.1 and 1.2, and 1AA-1AZ come between 1A and 1B. But this is not what I want; I want "human" alphabetical order, in which 1.10 comes after 1.9 and 1AA comes after 1Z.
I am wondering if there's still a way in SQL to get the order that I want using string manipulation (or something else I haven't thought of).
I am not an expert in SQL, so I don't know if this is possible, but if there were a way to do conditional replacement, then it seems I could impose the order I want by doing this:
delete the period (which can be done with replace, right?)
if the remaining figure number is more than three characters, add a 0 (zero) after the first character.
This would seem to give me the outcome I want: 1.9 would become 109, which comes before 110; 1Z would become 10Z, which comes before 1AA. But can it be done in SQL? If so, what would the syntax be?
Note that I don't want to modify the data itself—just to output the results of the query in the order described.
This is in the context of a Wordpress installation, but I think the question is more suitably an SQL question because various things (such as pagination) depend on the ordering happening at the MySQL query stage, rather than in PHP.
My first thought is to add an additional column that is updated by a trigger or other outside mechanism.
1) Use that column to do the order by
2) Whatever mechanism updates the column will have the logic to create an acceptable order by surrogate (e.g. it would turn 1.1 into AAA or something like that).
Regardless...this is going to be a pain. I do not evny you.
You can create function which have logic to have human sort order like
Alter FUNCTION [dbo].[GetHumanSortOrder] (#ColumnName VARCHAR(50))
RETURNS VARCHAR(20)
AS
BEGIN
DECLARE #HumanSortOrder VARCHAR(20)
SELECT #HumanSortOrder =
CASE
WHEN (LEN(replace(replace(<Column_Name>,'.',''),'Figure ',''))) = 2
THEN
CONCAT (SUBSTRING(replace(replace(<Column_Name>,'.',''),'Figure ',''),1,1),'0',SUBSTRING(replace(replace(<Column_Name>,'.',''),'Figure ',''),2,2))
ELSE
replace(replace(<Column_Name>,'.',''),'Figure ','')
END
FROM <Table_Name> AS a (NOLOCK)
WHERE <Column_Name> = #ColumnName
RETURN #HumanSortOrder
END
this function give you like 104,107,119,10A, 10B etc as desired
And you can use this function as order by
SELECT * FROM <Table_Name> ORDER BY GetHumanSortOrder(<Column_Name>)
Hope this helps
It is possible?
DECLARE #vTableName varchar(50)
SET #vTableName = (SELECT TableName FROM qms_Types WHERE Id = 1)
SELECT * FROM #vTableName
I have this error:
Msg 1087, Level 16, State 1, Line 3 Must declare the table variable
"#vTableName".
Short answer: No.
Long answer: Noooooooooooooooooooooooooooooooooooooo. Use dynamic SQL if you have to, but if you're structuring your tables in a way where you don't know the table name ahead of time, it might benefit you to rethink your schema.
Here is a great resource for learning how to use dynamic SQL: The Curse and Blessings of Dynamic SQL
if you're trying to select from a table of that name, then you can do something like this:
DECLARE #vTableName varchar(50)
SET #vTableName = (SELECT TableName FROM qms_Types WHERE Id = 1)
EXECUTE('SELECT * FROM [' + #vTableName + ']')
my solution for this:
EXECUTE('SELECT * FROM ' + TableName + '')
It seems as though different folks are interpreting the OP differently.
I'm pretty sure the OP is asking for this type of concept / ability / maneuver...
"Put a table name into a variable and then use that variable as though it were a table name."
DECLARE #TableIWantRecordsFrom varchar(50)
-- ^^^^^^^^^^^^^^^^^^^^^^
SET #TableIWantRecordsFrom = (SELECT TableName FROM qms_Types WHERE Id = 1) -- (L1)
-- ^^^^^^^^^^^^^^^^^^^^^^
-- Let's say, at this point, #TableIWantRecordsFrom ... contains the text 'Person'
-- ^^^^^^^^^^^^^^^^^^^^^^
-- assuming that is the case then...
-- these two queries are supposed to return the same results:
SELECT top 3 fname,lname,mi,department,floor FROM Person
-- ^^^^^^
SELECT top 3 fname,lname,mi,department,floor FROM #TableIWantRecordsFrom -- (L2)
-- ^^^^^^^^^^^^^^^^^^^^^^
From reading all the responses and answers, it appears that this kind of maneuver can't be done - unless - you use dynamic SQL which...
can be a bit of a pain to create and maintain and
can be more work to create than the time it "saves" you in the future.
================================================================
There are other languages where this can be done... in literally, two lines of code (see (L1) and (L2) in above code) and not having to do a lot of formatting and editing.)
(I've done it before - there is another language where all you'd need is L1 and L2...)
================================================================
It is unfortunate that SQL Server will not do this without going to a decent amount of effort...
first write your SQL then
test it to make sure it does, in fact, work then
frame each line with tick marks and then escape your ticks that are now inside THOSE tick marks
declare the variable
set the variable to the sql statement you ticked above
(I may be missing some additional steps)
Oh, and then, if you ever need to maintain it
you need to either, be very careful and just edit it right there, as is, and hope you get it all just right -or- you may have saved a copy of it... un-ticked and un-variablized so you can edit the "real" sql and then when you're done you can RE DO these steps... again.
I think you want this:
DECLARE #vTableName table(TableName varchar(50))
insert into #vTableName
SELECT TableName FROM qms_Types WHERE Id = 1
SELECT * FROM #vTableName
The only way you can do this is through Dynamic SQL which refers to the practice of creating a T-SQL text and executing it using the sp_executesql (or simply exec)
Here is a helpful link about dynamic sql The Curse and Blessings of Dynamic SQL.
You should really think whether or not this is a case for dynamic sql or if there is another way for you to perform this operation.