Not sure if this is possible in MySQL, but I have a column that has business names like:
AT&T Store
O'Reilly's Auto Parts
Burger King
which I import into Sphinx Search with a MySQL query. I have MariaDB, so there is a REGEXP_REPLACE(col, regexp, replace) function, but I'm having trouble figuring out the rest.
What I need is to repeat words with non-alphanumeric characters replaced with and without a space. So the above examples would become:
ATT AT T Store
OReillys O Reilly s Auto Parts
Burger King
Is this possible in a MySQL query? Thanks!
This can be done all at once, but maybe not by SQL primitive regex.
I don't know REGEXP_REPLACE, nor modern day SQL.
Typically its done by three regex.
Pseudo code:
$column_val = "O'Reilly's Auto Parts";
$new_column_val = Replace_Globally(
$column_val,
'\b\w+[[:punct:]](?:[[:punct:]]*\w)+\b',
function( $match ) {
$val = $match.value;
$text1 = Replace_Globally( $val, '[[:punct:]]+', "" );
$text2 = Replace_Globally( $val, '[[:punct:]]+', " " );
return $text1 + " " + $text2;
}
);
So, this might not look like something sql can do, so you might have to get creative.
REGEXP_REPLACE is in MariaDB only, MySQL doesn't have it.
select regexp_replace(regexp_replace(
"AT&T Store
O'Reilly's Auto Parts
Burger King",
'([[:alnum:]]+)[[:punct:]]+([[:alnum:]]+)[[:punct:]]+([[:alnum:]]+)',
'\\1\\2\\3 \\1 \\2 \\3'),
'([[:alnum:]]+)[[:punct:]]+([[:alnum:]]+)',
'\\1\\2 \\1 \\2')
Related
I have 4 types of last_name:
"Camp Bell"
"CAMPBELL"
"CampBellJr."
"camp bell jr."
Now, in rails when an user is searched by it's last name like camp bell, I want to show all the 4 records. So, I tried:
RAILS
stripped_name = params[last_name].gsub(/\W/, '')
#=> "campbell"
User.where("LOWER(REPLACE(last_name, '/\W/', '')) LIKE ?", "#{stripped_name}%")
Give me only 2 records with following last_name:
"CAMPBELL"
"CampBellJr."
I guess, this is because, the mysql REPLACE is not working correctly with regex.
Any ideas?
EDIT
Guys, sorry for the confusion. My idea is to strip off all special characters including space. So I'm trying to use \W regex.
For example, the input can be: camp~bell... But, it should still fetch result.
You can check for both stripped_name without space and ones that include both names seperated with space like this.
stripped_name = params[last_name].gsub(/\W/, '')
split_names = params[last_name].split(" ")
User.where('name LIKE ? OR (name LIKE ? AND name LIKE ?)', "%#{stripped_name}%", "%#{split_names[0]}%", "%#{split_names[1]}%")
Next step would to search for complete array of split names not just first two.
Here my solution:
User.where("REPLACE(last_name, ' ', '') ILIKE CONCAT ('%', REPLACE('?', ' ', ''),'%')", stripped_name)
ILIKE is like LIKE but the I is for insensitive case.
To understand easily step by step:
lastname ILIKE '%campbell% you need % because you want lastname
contain this string, not necessary at the begin or the end of you
string.
'campbell%' => search string who begin by campbell
'%campbell' => search string who finish by campbell
We need generate '%campbell%, so we use CONCAT for that
I just use a simply REPLACE, but maybe you should use a regex.
I need an sql select statement to retrieve 04:30 and test.zip from this string:
{"TIME":"04:30","DATE":"11\/25\/2013","FILENAME":["test.zip"]}
use this \[(.*?)\]
it return value between [ and ]
and for 04:30 use TIME":(.*?),
it return value after "TIME":
Can't you just decode it and use PHP? (assuming you can't change the way it's stored in the db)
<?php
$str = '{"TIME":"04:30","DATE":"11/25/2013","FILENAME":["test.zip"]}';
$o = json_decode($str);
$time = $o->TIME;
$file = $o->FILENAME[0];
var_dump($time); //"04:30"
var_dump($file); //"test.zip"
Regex replaces etc in MySQL require a UDF (user-defined function) mysql-udf-regexp
If none of the above are viable solutions (change DB structure, do it with PHP, use a MySQL UDF), you'll need to get creative. It would require a known, static format of that string, but you could replace some parts and substring others. For example:
SELECT SUBSTRING(REPLACE(`column_name`,'{"TIME":"',''),1,5) AS `time` FROM `table_name`
File is more complex, this example assuming only one filename in the array
SELECT REPLACE(SUBSTRING(`column_name`,LOCATE('"FILENAME":["',`column_name`)+13),'"]}','') AS `file` FROM `table_name`
Those two field selections get 04:30 and test.zip respectively (you can of course use those functions in the same statement, rather than separately like I have, by comma separating them)
I am working with PHP and MySQL. I'll provide a simple table below:
------------------------------------------------------
| id | text |
------------------------------------------------------
| 1 | The quick brown fox jumped over the lazy dog |
------------------------------------------------------
I'd like users to be able to search using keywords separated by spaces. I've got a simple SQL query for the above which is SELECT * FROM table WHERE text LIKE %[$keyword]% which means if I search for "The quick fox", I won't get any results. Is there a way to separate the keywords by spaces so if I search for "The quick fox", it will run 3 searches. One for "The", one for "quick", and one for "fox" - removing all white spaces. Though it should only display one result since it all belongs to the same row instead of 3 since all 3 keywords matched the data in the row.
What's the best way to do this? All suggestions to do this better are always welcome. Thanks!
[EDIT]
Just thought of this now, would separating the keywords by comma (,) be a better option?
You might consider a regular expression via REGEXP to separate the words into an or group.
SELECT *
FROM tbl
WHERE
LOWER(`text`) REGEXP '\b(the|quick|fox)\b'
Matches:
The quick brown fox jumps over the lazy dog
Quick, get the fox!
I ate the cake
Doesn't match
Brown dogs
Inside PHP, you can construct this expression by splitting your search string on the spaces and imploding it back on | after escaping each component.
$str = "the quick brown fox";
$kwds = explode(" ", $str);
$kwds = array_map("mysql_real_escape_string", $kwds);
$regexp = "\b(" . implode("|", $kwds) . ")\b";
Then use REGEXP '$regexp' in your statement.
Addendum:
Since you didn't mention it in the OP, I want to be sure you aware of MySQL's full text searching capabilities on MyISAM tables, in case it can meet your need. From your description, full text doesn't sound like exactly your requirement, but you should review it as a possibility: http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
I did it by a different approach of concatenating the query string.
//this line contains four words which are being used for a search
$str = "the quick brown fox";
//the query string is written in little pieces, stored in variable "$uquery"
$uquery = "select * from tablename where";
//a new variable "$keywords" is going to hold the values as an array
$keywords = explode(" ", $str);
$keywords = array_map("mysql_real_escape_string", $keywords);
//"$k" will be storing the total no. of elements in the array, should be holding "4" for this example
$k = count($keywords);
//defining variable "$i" equals to zero
$i = 0;
//initiating a while loop which will be continued 4 times as we have only 4 keywords in this example
while($i <= $k){
//here comes another new variable "$uquery_", following part will be concatenated with the previous piece of "$uquery" later
$uquery_ .= " topic like convert(_utf8 '%$keywords[$i]%' USING latin1)";
//for more explanation let me tell you that how "$keywords[$i]" will help in above piece of code, for every value of "$i" it will be fetching a keyword from the array, see below:
//$keywords[0]= "the" for $i =0
//$keywords[1]= "quick" for $i =1
//$keywords[2]= "brown" for $i =2
//$keywords[3]= "fox" for $i =3
//now concatenating logical && operator after every round of the loop in "$uquery_" but not for the last round as the query needs to be ended after last keyword
if($i != $k){ $uquery_ .= " &&"; }
//adding 1 to $i after every round
$i++;
}
//now our main variable "$uquery" is being concatenated with "$uquery_"
$uquery .= "$uquery_";
The above piece of code will be generating following query:
select * from tablename where topic like convert(_utf8 '%the%' USING latin1) && topic like convert(_utf8 '%quick%' USING latin1) && topic like convert(_utf8 '%brown%' USING latin1) && topic like convert(_utf8 '%fox%' USING latin1)
Note: "topic" is supposed to be the column name in mysql table, you can replace it with the column name as defined in your table.
I hope it will help a few of the people. If you have any questions feel free to ask me via Live Chat feature on my website http://www.79xperts.com
Regards
Adnan Saeed
So I use the PDO for a DB connection like this:
$this->dsn[$key] = array('mysql:host=' . $creds['SRVR'] . ';dbname=' . $db, $creds['USER'], $creds['PWD']);
$this->db[$key] = new PDO($this->dsn[$key]);
Using PDO I can then execute a MySQL SELECT using something like this:
$sql = "SELECT * FROM table WHERE id = ?";
$st = $db->prepare($sql);
$st->execute($id);
$result = $st->fetchAll();
The $result variable will then return an array of arrays where each row is given a incremental key - the first row having the array key 0. And then that data will have an array the DB data like this:
$result (array(2)
[0]=>[0=>1, "id"=>1, 1=>"stuff", "field1"=>"stuff", 2=>"more stuff", "field2"=>"more stuff" ...],
[1]=>[0=>2, "id"=>2, 1=>"yet more stuff", "field1"=>"yet more stuff", 2=>"even more stuff", "field2"=>"even more stuff"]);
In this example the DB table's field names would be id, field1 and field2. And the result allows you to spin through the array of data rows and then access the data using either a index (0, 1, 2) or the field name ("id", "field1", "field2"). Most of the time I prefer to access the data via the field names but access via both means is useful.
So I'm learning the ruby-mysql gem right now and I can retrieve the data from the DB. However, I cannot get the field names. I could probably extract it from the SQL statement given but that requires a fair bit of coding for error trapping and only works so long as I'm not using SELECT * FROM ... as my SELECT statement.
So I'm using a table full of State names and their abbreviations for my testing. When I use "SELECT State, Abbr FROM states" with the following code
st = #db.prepare(sql)
if empty(where)
st.execute()
else
st.execute(where)
end
rows = []
while row = st.fetch do
rows << row
end
st.close
return rows
I get a result like this:
[["Alabama", "AL"], ["Alaska", "AK"], ...]
And I'm wanting a result like this:
[[0=>"Alabama", "State"=>"Alabama", 1=>"AL", "Abbr"=>"AL"], ...]
I'm guessing I don't have the way inspect would display it quite right but I'm hoping you get the idea by now.
Anyway to do this? I've seen some reference to doing this type of thing but it appears to require the DBI module. I guess that isn't the end of the world but is that the only way? Or can I do it with ruby-mysql alone?
I've been digging into all the methods I can find without success. Hopefully you guys can help.
Thanks
Gabe
You can do this yourself without too much effort:
expanded_rows = rows.map do |r|
{ 0 => r[0], 'State' => r[0], 1 => r[1], 'Abbr' => r[1] }
end
Or a more general approach that you could wrap up in a method:
columns = ['State', 'Abbr']
expanded_rows = rows.map do |r|
0.upto(names.length - 1).each_with_object({}) do |i, h|
h[names[i]] = h[i] = r[i]
end
end
So you could collect up the rows as you are now and then pump that array of arrays through something like what's above and you should get the sort of data structure you're looking for out the other side.
There are other methods on the row you get from st.fetch as well:
http://rubydoc.info/gems/mysql/2.8.1/Mysql/Result
But you'll have to experiment a little to see what exactly they return as the documentation is, um, a little thin.
You should be able to get the column names out of row or st:
http://rubydoc.info/gems/mysql/2.8.1/Mysql/Stmt
but again, you'll have to experiment to figure out the API. Sorry, I don't have anything set up to play around with the MySQL API that you're using so I can't be more specific.
I realize that php programmers are all cowboys who think using a db layer is cheating, but you should really consider activerecord.
How do I insert a value in MySQL that consist of single or double quotes. i.e
This is Ashok's Pen.
The single quote will create problems. There might be other escape characters.
How do you insert the data properly?
Put quite simply:
SELECT 'This is Ashok''s Pen.';
So inside the string, replace each single quote with two of them.
Or:
SELECT 'This is Ashok\'s Pen.'
Escape it =)
' is the escape character. So your string should be:
This is Ashok''s Pen
If you are using some front-end code, you need to do a string replace before sending the data to the stored procedure.
For example, in C# you can do
value = value.Replace("'", "''");
and then pass value to the stored procedure.
See my answer to "How to escape characters in MySQL"
Whatever library you are using to talk to MySQL will have an escaping function built in, e.g. in PHP you could use mysqli_real_escape_string or PDO::quote
Use this code:
<?php
$var = "This is Ashok's Pen.";
mysql_real_escape_string($var);
?>
This will solve your problem, because the database can't detect the special characters of a string.
If you use prepared statements, the driver will handle any escaping. For example (Java):
Connection conn = DriverManager.getConnection(driverUrl);
conn.setAutoCommit(false);
PreparedStatement prepped = conn.prepareStatement("INSERT INTO tbl(fileinfo) VALUES(?)");
String line = null;
while ((line = br.readLine()) != null) {
prepped.setString(1, line);
prepped.executeQuery();
}
conn.commit();
conn.close();
There is another way to do this which may or may not be safer, depending upon your perspective. It requires MySQL 5.6 or later because of the use of a specific string function: FROM_BASE64.
Let's say you have this message you'd like to insert:
"Ah," Nearly Headless Nick waved an elegant hand, "a matter of no importance. . . . It's not as though I really wanted to join. . . . Thought I'd apply, but apparently I 'don't fulfill requirements' -"
That quote has a bunch of single- and double-quotes and would be a real pain to insert into MySQL. If you are inserting that from a program, it's easy to escape the quotes, etc. But, if you have to put that into a SQL script, you'll have to edit the text (to escape the quotes) which could be error prone or sensitive to word-wrapping, etc.
Instead, you can Base64-encode the text, so you have a "clean" string:
SWtGb0xDSWdUbVZoY214NUlFaGxZV1JzWlhOeklFNXBZMnNnZD
JGMlpXUWdZVzRnWld4bFoyRnVkQ0JvWVc1a0xDQWlZU0J0WVhS
MFpYCklnYjJZZ2JtOGdhVzF3YjNKMFlXNWpaUzRnTGlBdUlDNG
dTWFFuY3lCdWIzUWdZWE1nZEdodmRXZG9JRWtnY21WaGJHeDVJ
SGRoYm5SbApaQ0IwYnlCcWIybHVMaUF1SUM0Z0xpQlVhRzkxWj
JoMElFa25aQ0JoY0hCc2VTd2dZblYwSUdGd2NHRnlaVzUwYkhr
Z1NTQW5aRzl1SjMKUWdablZzWm1sc2JDQnlaWEYxYVhKbGJXVn
VkSE1uSUMwaUlBPT0K
Some notes about Base64-encoding:
Base64-encoding is a binary encoding, so you'd better make sure that you get your character set correct when you do the encoding, because MySQL is going to decode the Base64-encoded string into bytes and then interpret those. Be sure base64 and MySQL agree on what the character encoding is (I recommend UTF-8).
I've wrapped the string to 50 columns for readability on Stack Overflow. You can wrap it to any number of columns you want (or not wrap at all) and it will still work.
Now, to load this into MySQL:
INSERT INTO my_table (text) VALUES (FROM_BASE64('
SWtGb0xDSWdUbVZoY214NUlFaGxZV1JzWlhOeklFNXBZMnNnZD
JGMlpXUWdZVzRnWld4bFoyRnVkQ0JvWVc1a0xDQWlZU0J0WVhS
MFpYCklnYjJZZ2JtOGdhVzF3YjNKMFlXNWpaUzRnTGlBdUlDNG
dTWFFuY3lCdWIzUWdZWE1nZEdodmRXZG9JRWtnY21WaGJHeDVJ
SGRoYm5SbApaQ0IwYnlCcWIybHVMaUF1SUM0Z0xpQlVhRzkxWj
JoMElFa25aQ0JoY0hCc2VTd2dZblYwSUdGd2NHRnlaVzUwYkhr
Z1NTQW5aRzl1SjMKUWdablZzWm1sc2JDQnlaWEYxYVhKbGJXVn
VkSE1uSUMwaUlBPT0K
'));
This will insert without any complaints, and you didn't have to manually-escape any text inside the string.
You should escape the special characters using the \ character.
This is Ashok's Pen.
Becomes:
This is Ashok\'s Pen.
If you want to keep (') apostrophe in the database use this below code:
$new_value = str_replace("'","\'", $value);
$new_value can store in database.
You can use this code,
<?php
$var = "This is Ashok's Pen.";
addslashes($var);
?>
if mysqli_real_escape_string() does not work.
In PHP, use mysqli_real_escape_string.
Example from the PHP Manual:
<?php
$link = mysqli_connect("localhost", "my_user", "my_password", "world");
/* check connection */
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
mysqli_query($link, "CREATE TEMPORARY TABLE myCity LIKE City");
$city = "'s Hertogenbosch";
/* this query will fail, cause we didn't escape $city */
if (!mysqli_query($link, "INSERT into myCity (Name) VALUES ('$city')")) {
printf("Error: %s\n", mysqli_sqlstate($link));
}
$city = mysqli_real_escape_string($link, $city);
/* this query with escaped $city will work */
if (mysqli_query($link, "INSERT into myCity (Name) VALUES ('$city')")) {
printf("%d Row inserted.\n", mysqli_affected_rows($link));
}
mysqli_close($link);
?>
$var = mysqli_real_escape_string($conn, $_POST['varfield']);
If you are using PHP, just use the addslashes() function.
PHP Manual addslashes
For programmatic access, you can use placeholders to automatically escape unsafe characters for you.
In Perl DBI, for example, you can use:
my $string = "This is Ashok's pen";
$dbh->do("insert into my_table(my_string) values(?)",undef,($string));
Use either addslahes() or mysql_real_escape_string().
The way I do, by using Delphi:
TheString to "escape":
TheString=" bla bla bla 'em some more apo:S 'em and so on ";
Solution:
StringReplace(TheString, #39,'\'+#39, [rfReplaceAll, rfIgnoreCase]);
Result:
TheString=" bla bla bla \'em some more apo:S \'em and so on ";
This function will replace all Char(39) with "\'" allowing you to insert or update text fields in MySQL without any problem.
Similar functions are found in all programming languages!
Maybe you could take a look at function QUOTE in the MySQL manual.
This is how my data as API response looks like, which I want to store in the MYSQL database. It contains Quotes, HTML Code , etc.
Example:-
{
rewardName: "Cabela's eGiftCard $25.00",
shortDescription: '<p>adidas gift cards can be redeemed in over 150 adidas Sport Performance, adidas Originals, or adidas Outlet stores in the US, as well as online at adidas.com.</p>
terms: '<p>adidas Gift Cards may be redeemed for merchandise on adidas.com and in adidas Sport Performance, adidas Originals, and adidas Outlet stores in the United States.'
}
SOLUTION
CREATE TABLE `brand` (
`reward_name` varchar(2048),
`short_description` varchar(2048),
`terms` varchar(2048),
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=latin1;
While inserting , In followed JSON.stringify()
let brandDetails= {
rewardName: JSON.stringify(obj.rewardName),
shortDescription: JSON.stringify(obj.shortDescription),
term: JSON.stringify(obj.term),
}
Above is the JSON object and below is the SQL Query that insert data into MySQL.
let query = `INSERT INTO brand (reward_name, short_description, terms)
VALUES (${brandDetails.rewardName},
(${brandDetails.shortDescription}, ${brandDetails.terms})`;
Its worked....
If nothing works try this :
var res = str.replace(/'/g, "\\'");
var res = res.replace(/"/g, "\\\"");
It adds the \ escape character to all(every) occurrences of ' and "
Not sure if its the correct/professional way to fix the issue
I'm guessing it will work but in actual content, every single and double quotes will be replaced with \ character
As a Python user I replace the quote with a raw "'":
don_not_work_str = "This is Ashok's Pen."
work_str = don_not_work_str.replace("'", r"\'")
For Python，I tried many ways to escape '"', not work, cuz I have various input.
Finally, I just use these code, the data in database was same like the input.
tmp = val.replace('\\', r'\\')
ret = tmp.replace('"', r'\"')