I have stored some extra form data as json in a mysql column (extraData).
Example data
{"1":{"name":"sadsdaf ","surname":"sdafa","occupation":"sdaas","email":"dsasdaf#asdf.nl","barcode":"052355491"},"2":{"name":"sadafd","surname":"sdafa","occupation":"sddaf","email":"sda#daf.nl","barcode":"052355492"}}
I would like to do a search query which also does a search for the barcode stored in the json encoded field? So not only this
$sql = "SELECT name FROM visitors WHERE barcode = '".$barcode."'";
But also
$sql = "SELECT name FROM visitors WHERE barcode = '".$barcode."' OR extraData = '".$barcode."'";
What do I need to do to extraData so the variable barcode is checked and I get the corresponding name, surname etc?
SELECT TRIM(BOTH '"' FROM SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(V.extraData,'"name":', -1),',',1),'}',1)
FROM visitors V
WHERE V.extraData LIKE '%"barcode"%'
AND :barcode = TRIM(BOTH '"' FROM SUBSTRING_INDEX(SUBSTRING_INDEX(SUBSTRING_INDEX(V.extraData,'"barcode":', -1),',',1),'}',1);
the LIKE '%"barcode"%' filters the rows with the field extraData containing barcode values
the :barcode means to be the value to match, the TRIM is to get rid of the extra double quotes , the nested SUBSTRING_INDEX searchs for:
the text starting at the end of the first match of '"barcode":' up to
the next match of ',' (if there are more key/values) up to the next
match of '}' (if it's the last key/value at that level)
keep in mind that if you find that a certain key/val is always present in every row and is accessed everywhere you should consider adding an extra field to the table/data model and create an index out of it (ie. name, surname).
the values retrieved must be decoded "always".
"Einstürzende Neubauten" is stored as "Einst\u00fcrzende Neubauten"
doing search and compare numbers are "ok", like in the example (the trim double quotes sometimes would not be necesary as if you json_encode an integer var it will be stored as "number":99 ). but fractional numbers would be stored depending on the locale "1,5" / "1.5"
search and compare strings are more tricky: you have to "jsonencode" the value to match
"https://www.google.com" is stored as "https:\/\/www.google.com"
check your data normalization skills.
check your optimization skills
use an online tool to see how your data will be encoded
Related
I have a read only database. where i want to compare two string. But in the database some column value name has included some special character but i want to compare another string with this string without special character.
For example, In DB the column value name is BMW 3-SERIES. And i have another string BMW3SERIES. So, i want to remove - and space then compare two string in a where condition. Is both string equal or not ?
I am stuck how to remove special characters from model_name .
SELECT * FROM VENDOR WHERE model_name = "BMW3SERIES"
What should be the query?
On MySQL 8+, you could use REGEXP_REPLACE to remove what you deem as special characters, for example:
SELECT *
FROM VENDOR
WHERE REGEXP_REPLACE(model_name, '[ -]+', '') = 'BMW3SERIES';
You may add whatever characters you wish to the above character class [ -]+.
I have a table called messages with a column (BLOB) called message_text. I'm passing a value from the web app when a user sends a new message and I want to check if it's an exact duplicate message text.
SELECT count(message_id) FROM messages WHERE message_text = '$msgTxt' AND user_id = $userId
Where $msgTxt will be a formatted string like...
"Hello there. I don\'t know you.
I\'ve just made a new line. "
The problem is that the comparison isn't working and I'm never finding duplicates. Even if I literally copy/paste an existing value from the database and replace it with $msgTxt in my query I never get any results, and so I'm assuming there's something wrong with the way I'm comparing a blob to a string.
BLOB values are treated as binary strings (byte strings). They have the binary character set and collation, and comparison and sorting are based on the numeric values of the bytes in column values. String or Text values are treated as nonbinary strings (character strings). They have a character set other than binary, and values are sorted and compared based on the collation of the character set.
So, you have to convert either BLOB to String or String to BLOB and then compare both.
If you are using java,
Convert Blob to String
byte[] bdata = blob.getBytes(1, (int)blob.length());
String data1 = new String(bdata);
What API are you using to call MySQL? I see some backslashes, but need to verify that \ is not turning into \\, and that other escapings are not unnecessarily happening or not happening.
Which OS are you using? Windows, when reading stuff, likes to convert NL into CRLF, thereby making it so that it won't match.
Say I have the search string 'mens health' and in my database I have the field with value 'men's health', how can I match my search string against this perhaps using regex? Bearing in mind, this is just one example and in my database there are many records that also contain any number of non alpha characters such as "/,$,',-,(,),1,2..."? The search string is obtained from the URL and should not contain any encoding as to be easily read.
I ended up creating a little function which generates a regex string which I used with an RLIKE in my MySQL Query which should match the sequence of alpha numeric characters regardless of any special characters between the letters.
function GenerateCategoryRegex($SrchTerm) {
$arrChar = str_split(str_replace(' ', '', $SrchTerm));
$RegEx = '(^|;)+[[:space:]]*';
foreach($arrChar as $Char) {
$RegEx .= $Char . '{1}[^\w,]*';
}
$RegEx .= '[[:space:]]*(;|$)+';
return $RegEx;
}
Add another column that contains the "searchable text", such as "mens health". Then search that column, but display the original column ("mean's health").
Now you are free to do other tweaks -- such as adding synonyms, common misspellings, etc. to the search column.
And your REGEXP can become a simple LIKE, or probably even a fast FULLTEXT.
I am using MySQL with MATLAB, and I want to get a name from user, and pass it to the table in mySQL, but it is rejecting a variable name in place of string
var_name=input('enter the name:');
mysql('insert into table (name) values (var_name)');
Any suggestions?
FIRST read the comments to this question - you don't want to shoot yourself in the foot with a mysql injection security problem. You have been warned. Now, to solve your current problem, without addressing the security risk of the whole approach when it comes to building SQL queries, read on...
In principle Amro has already posted two solutions for you which work, but since you have not accepted it I'll explain further.
Your problem is that you are not telling MATLAB which parts of your query it should interpret as a literal string, and which parts it should interpret as a variable name. To solve this, you can just end the literal string where appropriate, i.e. after the opening brackets, and then start them again before the closing brackets.
In between those literal strings you want to add the contents of your variables, so you need to tell MATLAB to concat your literal strings with your variables, since the mysql command probably expects the whole query as a single string. So in essence you want to take the string 'insert into table(' and the string saved in the variable name and the string ') values (' and so on and glue them into one big string. Amro and Isaac have shown you two solutions of how to do this without much explanation:
horzcat('insert into table (', name, ') values (', var_name, ')')
uses the function horzcat, while
['insert into table (' name ') values (' var_name ')']
uses the fact that MATLAB treats strings as arrays of characters so that you can just use square brackets to form a large array containing the strings one after the other.
The third solution, offered by Amro, is a bit more sublte:
sprintf('insert into table (%s) values (%s)',name,var_name)
It tells the function sprintf (which is made for that purpose) "take the string which I supply as first parameter and replace occurences of %s with the strings I supply as the following parameters. This last technique is in particular useful if you also need to insert numbers into your string, because sprintf can also convert numbers to string and allows fine control over how they are formatted. You should have a close look at the help page for sprintf to know more :-).
Try this instead:
mysql(['insert into table (' name ') values (' var_name ')']);
or even:
mysql(sprintf('insert into table (%s) values (%s)',name,var_name));
I believe the problem you are having is the same as the one in this other question. It sounds like you want to create a command string that itself contains a ' delimited string, which would require you to escape each ' with another ' when you create your command string (note the first example in this string handling documentation). Note also you may want to use the 's' option for the INPUT function:
var_name = input('Enter the name: ','s'); %# Treats input like a string
commandString = sprintf('insert into table (name) values (''%s'')', var_name);
%# Note the two apostrophes --^
mysql(commandString);
If I were to enter Ken for the input, the string commandString would contain the following:
insert into table (name) values ('Ken')
And of course, as others have already mentioned, beware injection vulnerabilities.
Is ir possible to sort a column alphabetically but ignoring certain words like e.g 'The'
e.g.
A normal query would return
string 1
string 3
string 4
the string 2
I would like to return
string 1
the string 2
string 3
string 4
Is this possible?
EDIT
Please note I am looking to replace multiple words like The, A, etc... Can this be done?
You can try
SELECT id, text FROM table ORDER BY TRIM(REPLACE(LOWER(text), 'the ', ''))
but note that it will be very slow for large datasets as it has to recompute the new string for every row.
IMO you're better off with a separate column with an index on it.
For multiple stopwords just keep nesting REPLACE calls. :)
This will replace all leading "The " as an example
SELECT *
FROM YourTable
ORDER BY REPLACE(Val,'The ', '')
Yes, it should be possible to use expressions with the ORDER-part:
SELECT * FROM yourTable ORDER BY REPLACE(yourField, "the ", "")
I have a music listing that is well over 75,000 records and I had encountered a similar situation. I wrote a PHP script that checked for all string that began with 'A ', 'An ' or 'The ' and truncated that part off the string. I also converted all uppercase letters to lowercase and stored that string in a new column. After setting an index on that column, I was done.
Obviously you display the initial column but sort by the newly-created indexed column. I get results in a second or so now.