Hi i've hit a problem with my SQL queries, i have a table that contains 3 columns, one for vehicle brands, one for models and one for model versions.
So my data is split like
BRAND || MODEL || MODEL VERSION
RENAULT || R4 || R4 1.1 GTL
I've been asked to replace our current dropdown system with an input to make it easier for users to select their vehicle.
I'm using jQuery Autocomplete and my query looks something like this.
SELECT DISTINCT CONCAT (brand, ' ', model, ' ', version) as data from vehicles WHERE brand LIKE '%Golf%' OR model LIKE '%Golf%' OR version LIKE '%Golf%' LIMIT 5
So far so good, this will output "RENAULT R4 R4 1.1 GTL" if i type in "RENAULT"... the problem here comes when the user inserts something like Renault R4 instead of just "Renault"
As they've included the Model name as well as the Brand then it doesn't really match any of my columns in the Database and my Ajax call returns no results.
I need to query the actual result set from that concat instead so that anything the users type in will match the results, but i have no idea how i can do this.
In desperation i tried to type where data LIKE '%RENAULT R4%' but as expected this also doesn't work... What can i do in this situation? Any help would be appreciated.
Easy and slow way: Split the string by spaces and ask for each word.
SELECT ...
WHERE
(brand LIKE '%Renault%' OR model LIKE '%Renault%' OR version LIKE '%Renault%')
AND (brand LIKE '%R4%' OR model LIKE '%R4%' OR version LIKE '%R4%')
LIMIT 5
Keep in mind, that query like this one does not allow use of any index, so it is very slow.
The more complicated, but much faster implementation is to use fulltext index. You need recent version of MySQL (5.6 or newer); older versions support fulltext only on MyISAM tables which are not really a database.
CREATE FULLTEXT INDEX idx ON vehicles(brand, model, version);
SELECT ... FROM vehicles
WHERE MATCH(brand, model, version) AGAINST('Renault R4')
LIMIT 5;
(Query not tested, but you should get the idea.)
I can only think of this one, but I believe there are better ways to do it.
OR CONCAT (brand, ' ', model, ' ', version) LIKE '%RENAULT R4%'
Related
I need to take an array and use it for a MySQL query.
I tried looking for methods but all of them seem to be related to PHP arrays and not Ruby.
For example, I have terms = ['genetics', 'elderly', 'health'] and I want to use:
con.query "SELECT col1 FROM data1 WHERE MATCH (col2) AGAINST (terms)"
Is this possible?
You can just join your terms in your against clause:
terms = ['genetics' , 'elderly', 'health']
con.query "SELECT col1 FROM data1 WHERE MATCH col2 AGAINST ('#{terms.join(' ')}')"
Note that using match/against will almost certainly be more performative than using a series of like clauses. See this StackOverflow answer for more information: Which SQL query is better, MATCH AGAINST or LIKE?.
Check out the MySQL documentation for more information on full text searching (including possible operators): http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html.
I'd highly recommend looking into an ORM, such as Sequel, which makes it very easy to generate the proper query in a DBM independent way.
It allows us to use arrays and hashes conveniently. Here's an example using an array to generate the "where" clause in SQL:
my_posts = posts.where(:category => ['ruby', 'postgres', 'linux'])
# WHERE category IN ('ruby', 'postgres', 'linux')
That particular example is part of the "Filtering Records" section.
In a comment, the OP said:
col2 is text with each row having a paragraph, not just one word.
Then you want a LIKE or regex clause that allows each word to be tested. See the "String search functions" section of "Dataset Filtering " for how Sequel allows you to search inside strings.
The code would be something like:
data1.select(:col1).where(Sequel.like(:col2, terms.map{ |t| "%#{ t }%" } ))
which would generate something like:
SELECT col1 FROM data1 WHERE ((col2 LIKE '%genetics%') OR (col2 LIKE '%elderly%') OR (col2 LIKE '%health%'))
I have a table of posts with titles that are in "human" alphabetical order but not in computer alphabetical order. These are in two flavors, numerical and alphabetical:
Numerical: Figure 1.9, Figure 1.10, Figure 1.11...
Alphabetical: Figure 1A ... Figure 1Z ... Figure 1AA
If I orderby title, the result is that 1.10-1.19 come between 1.1 and 1.2, and 1AA-1AZ come between 1A and 1B. But this is not what I want; I want "human" alphabetical order, in which 1.10 comes after 1.9 and 1AA comes after 1Z.
I am wondering if there's still a way in SQL to get the order that I want using string manipulation (or something else I haven't thought of).
I am not an expert in SQL, so I don't know if this is possible, but if there were a way to do conditional replacement, then it seems I could impose the order I want by doing this:
delete the period (which can be done with replace, right?)
if the remaining figure number is more than three characters, add a 0 (zero) after the first character.
This would seem to give me the outcome I want: 1.9 would become 109, which comes before 110; 1Z would become 10Z, which comes before 1AA. But can it be done in SQL? If so, what would the syntax be?
Note that I don't want to modify the data itself—just to output the results of the query in the order described.
This is in the context of a Wordpress installation, but I think the question is more suitably an SQL question because various things (such as pagination) depend on the ordering happening at the MySQL query stage, rather than in PHP.
My first thought is to add an additional column that is updated by a trigger or other outside mechanism.
1) Use that column to do the order by
2) Whatever mechanism updates the column will have the logic to create an acceptable order by surrogate (e.g. it would turn 1.1 into AAA or something like that).
Regardless...this is going to be a pain. I do not evny you.
You can create function which have logic to have human sort order like
Alter FUNCTION [dbo].[GetHumanSortOrder] (#ColumnName VARCHAR(50))
RETURNS VARCHAR(20)
AS
BEGIN
DECLARE #HumanSortOrder VARCHAR(20)
SELECT #HumanSortOrder =
CASE
WHEN (LEN(replace(replace(<Column_Name>,'.',''),'Figure ',''))) = 2
THEN
CONCAT (SUBSTRING(replace(replace(<Column_Name>,'.',''),'Figure ',''),1,1),'0',SUBSTRING(replace(replace(<Column_Name>,'.',''),'Figure ',''),2,2))
ELSE
replace(replace(<Column_Name>,'.',''),'Figure ','')
END
FROM <Table_Name> AS a (NOLOCK)
WHERE <Column_Name> = #ColumnName
RETURN #HumanSortOrder
END
this function give you like 104,107,119,10A, 10B etc as desired
And you can use this function as order by
SELECT * FROM <Table_Name> ORDER BY GetHumanSortOrder(<Column_Name>)
Hope this helps
This is a request from my client to tweak an existing Perl script. However, it is the actual database structure on their end that confuses me.
The requirement looks pretty simple:
only pull records where _X begins with 1, 2, or 9.
However, the underlying database is not that simple, here is the guideline from their DBA:
"_X is a custom metadata field. The database stores this data in rows, not columns, within the customData table. In order to query the custom data table in an efficient manner you need to know the Field_ID for the custom field you get that from the fielddef table:
SELECT Field_ID FROM FieldDef WHERE Name = "_X";
This returns:
10012
"Now you can query CustomData. For example:
SELECT Record_ID FROM CustomData where Field_ID="10012" AND StringValue="2012-04";
He also suggests that in my case, probably it would be:
"SELECT Record_ID FROM CustomData where Field_ID="10012" AND (StringValue LIKE '1%' || StringValue LIKE '2%' || StringValue LIKE '9%')
The weird thing is that the existing Perl script doesn't contain anything like "Select Record_ID FROM" but all like "SELECT StringValue FROM".
So that is why I am very confused here: What is "store in rows, not in columns"? Why first query the Field_ID table then CustomData? I would not be able to communicate with any of them during this weekend but really wish to get some idea on the whole thing, hope experts can help me a little on sorting out the whole structure.
More info(Table schema):
http://pastebin.com/ZiDTCCC0
The existing perl script:(focus on lines 72-136)
http://pastebin.com/JHpikTeZ
Thanks in advance.
What they seem to be using is some kind of Entity-Attribute-Value model, with the entities stored as ints and explained in another table (FieldDef).
You explained pretty well how you queried it (although you can do it in one query, with a join or a subquery), and your problem seems to be that you don't know how the Perl script does it. Unfortunately, without us seeing the Perl script, we can't either :]
What's wrong with:
$term = $_POST['search'];
function buildQuery($exploded,$count,$query)
{
if(count($exploded)>$count)
{
$query.= ' AND column LIKE "%'. $exploded[$count] .'%"';
return buildQuery($exploded,$count+1,$query);
}
return $query;
}
$exploded = explode(' ',$term);
$query = buildQuery($exploded,1,
'SELECT * FROM table WHERE column LIKE "%'. $exploded[0] .'%"');
and then query the db to retrieve the results in a certain order, instead of using the myIsam-only sql match...against?
Would it dawdle performance dramatically?
The difference is in the algorithm's that MySQL uses behind the scenes find your data. Fulltext searches also allow you sort based on relevancy. The LIKE search in most conditions is going to do a full table scan, so depending on the amount of data, you could see performance issues with it. The fulltext engine can also have performance issues when dealing with large row sets.
On a different note, one thing I would add to this code is something to escape the exploded values. Perhaps a call to mysql_real_escape_string()
You can check out my recent presentation I did for MySQL University:
http://forge.mysql.com/wiki/Practical_Full-Text_Search_in_MySQL
Slides are also here:
http://www.slideshare.net/billkarwin/practical-full-text-search-with-my-sql
In my test, using LIKE '%pattern%' was more than 300x slower than using a MySQL FULLTEXT index. My test data was 1.5 million posts from the StackOverflow October data dump.
The following doesn't work, but something like this is what I'm looking for.
select *
from Products
where Description like (#SearchedDescription + %)
SSRS uses the # operator in-front of a parameter to simulate an 'in', and I'm not finding a way to match up a string to a list of strings.
There are a few options on how to use a LIKE operator with a parameter.
OPTION 1
If you add the % to the parameter value, then you can customize how the LIKE filter will be processed. For instance, your query could be:
SELECT name
FROM master.dbo.sysobjects
WHERE name LIKE #ReportParameter1
For the data set to use the LIKE statement properly, then you could use a parameter value like sysa%. When I tested a sample report in SSRS 2008 using this code, I returned the following four tables:
sysallocunits
sysaudacts
sysasymkeys
sysaltfiles
OPTION 2
Another way to do this that doesn't require the user to add any '%' symbol is to generate a variable that has the code and exceute the variable.
DECLARE #DynamicSQL NVARCHAR(MAX)
SET #DynamicSQL =
'SELECT name, id, xtype
FROM dbo.sysobjects
WHERE name LIKE ''' + #ReportParameter1 + '%''
'
EXEC (#DynamicSQL)
This will give you finer controller over how the LIKE statement will be used. If you don't want users to inject any additional operators, then you can always add code to strip out non alpha-numeric characters before merging it into the final query.
OPTION 3
You can create a stored procedure that controls this functionality. I generally prefer to use stored procedures as data sources for SSRS and never allow dynamically generated SQL, but that's just a preference of mine. This helps with discoverability when performing dependency analysis checks and also allows you to ensure optimal query performance.
OPTION 4
Create a .NET code assembly that helps dynamically generate the SQL code. I think this is overkill and a poor choice at best, but it could work conceivably.
Have you tried to do:
select * from Products where Description like (#SearchedDescription + '%')
(Putting single quotes around the % sign?)
Dano, which version of SSRS are you using? If it's RS2000, the multi-parameter list is
not officially supported, but there is a workaround....
put like this:
select *
from tsStudent
where studentName like #SName+'%'
I know this is super old, but this came up in my search to solve the same problem, and I wound up using a solution not described here. I'm adding a new potential solution to help whomever else might follow.
As written, this solution only works in SQL Server 2016 and later, but can be adapted for older versions by writing a custom string_split UDF, and by using a subquery instead of a CTE.
First, map your #SearchedDescription into your Dataset as a single string using JOIN:
=JOIN(#SearchedDedscription, ",")
Then use STRING_SPLIT to map your "A,B,C,D" kind of string into a tabular structure.
;with
SearchTerms as (
select distinct
Value
from
string_split(#SearchedDescription, ',')
)
select distinct
*
from
Products
inner join SearchTerms on
Products.Description like SearchTerms.Value + '%'
If someone adds the same search term multiple times, this would duplicate rows in the result set. Similarly, a single product could match multiple search terms. I've added distinct to both the SearchTerms CTE and the main query to try to suppress this inappropriate row duplication.
If your query is more complex (including results from other joins) then this could become an increasingly big problem. Just be aware of it, it's the main drawback of this method.