Replace & with & - mysql

When running the following code, I'm encountering an error message saying that the semicolon that I used on this line:
$select_stock->addExpression("REPLACE(b.corporateName, '&', '&')");
for the ampersand is incorrectly placed
InvalidArgumentException: ; is not supported in SQL strings. Use only one statement at a time.
Is there another way to solve this?
public function c_form_db_2($cName) {
$select_stock = $this->connection->select('stock', 'a');
$select_stock->fields('a', ['high', 'low', 'stockname']);
$select_stock->innerJoin('stockdetails', 'b', 'b.high = a.high');
$select_stock->condition('a.isCurrentPrice', 'Yes');
$select_stock->condition('a.isActive', 'Yes');
$select_stock->condition('b.status', 'Closing');
$select_stock->addExpression("REPLACE(b.corporateName, '&', '&')");
$select_stock->escapeLike($cName);
$select_stock->orderBy('a.tickerId', 'DESC');
$select_stock->orderBy('a.volId', 'DESC');
$select_stock_rows = $select_stock->execute()
->fetchAll(\PDO::FETCH_ASSOC);
return $select_stock_rows;
}

I do not know Drupal but I assume REPLACE is the standard MySql function and that Drupal supports all of them. In that case, if by chance you are running MySql 8, then instead of using REPLACE, use REGEXP_REPLACE and match against the regular expression '&amp.' using the wildcard '.' for the ';' character on the assumption that ';' is the only character that will ever be matched by the wildcard.

Related

How to remove extra apostrophe

I wrote a SQL query to find the desired output for my project. I was working fine with the correct output. But suddenly it started to give error and in the SQL query, there is some additional apoatrophe in. How to resolve it?
I tried to add the query to $this->db->query(); but still no use.
public function getStudentConut($id) {
$this->db->select('students.id')
->from('students')
->join('bp','students.pbp = bp.id','left')
->where(condition 1)
->where(condition 2);
$query1 = $this->db->get_compiled_select();
$this->db->select('students.id')
->from('students')
->join('bp','students.dbp = bp.id','left')
->where(condition 1)
->where(condition 2);
$query2 = $this->db->get_compiled_select();
$this->db->select('COUNT(id) as stud_count')
->from('('.$query1." UNION ALL ".$query2.') X')
->group_by('X.id');
$results = $this->db->get();
return $results->num_rows();
}
It was giving correct count earlier. But without any new changes, it started to give the error.
Now I get error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '.id`` WHERE ``bp.some_value`` IS NULL AND ``students.`schoo' at line 2
SELECT COUNT(id) as stud_count FROM (SELECT students.id`` FROM ``students`` LEFT JOIN ``bp`` ON ``students.pbp`` = ``bp.id`` WHERE ``bp..Some other condition.. UNION ALL SELECT students.idFROMstudentsLEFT JOINbpONstudents.dbp=bp.id..some other condition....) X GROUP BYX.id`
I think the issue (at least with the double `) is that CodeIgniter isn't very good with subqueries and such. Basically every time you get the compiled select statement it already has the escape identifiers and then you are putting it in the from statement at the end which will add additional escape identifiers on top of that.
`->from('('.$query1." UNION ALL ".$query2.') X')`
Unfortunately, unlike other methods like set, from doesn't have a 2nd parameter that allows you to set escaping to false (which is what I think you need).
I suggest trying this:
$this->db->_protect_identifiers = FALSE;
$this->db->select('COUNT(id) as stud_count')
->from('('.$query1." UNION ALL ".$query2.') X')
->group_by('X.id');
$results = $this->db->get();
$this->db->_protect_identifiers = TRUE;
and also look in to this: ->where(condition 2); which I'm pretty sure shouldn't compile due to lack of quotes. You probably don't want this escaped so you can do ->where('condition 2', '', false); as per: https://www.codeigniter.com/user_guide/database/query_builder.html#CI_DB_query_builder::where
When all else fails, just know that CodeIgniter has some limitations with "advanced" queries and that maybe you should write it out manually as a string utilizing $this->db->escape_str(...) for escaping user input vars, and $this->db->query(...) to run the SQL.

Escape string for use in MySQL fulltext search

I am using Laravel 4 and have set up the following query:
if(Input::get('keyword')) {
$keyword = Input::get('keyword');
$search = DB::connection()->getPdo()->quote($keyword);
$query->whereRaw("MATCH(resources.name, resources.description, resources.website, resources.additional_info) AGAINST(? IN BOOLEAN MODE)",
array($search)
);
}
This query runs fine under normal use, however, if the user enters a string such as ++, an error is thrown. Looking at the MySQl docs, there are some keywords, such as + and - which have specific purposes. Is there a function which will escape these types of special characters from a string so it can be used in a fulltext search like above without throwing any errors?
Here is an example of an error which is thrown:
{"error":{"type":"Illuminate\\Database\\QueryException","message":"SQLSTATE[42000]: Syntax error or access violation: 1064 syntax error, unexpected '+' (SQL: select * from `resources` where `duplicate` = 0 and MATCH(resources.name, resources.description, resources.website, resources.additional_info) AGAINST('c++' IN BOOLEAN MODE))","file":"\/var\/www\/html\/[...]\/vendor\/laravel\/framework\/src\/Illuminate\/Database\/Connection.php","line":555}}
Solutions I've tried:
$search = str_ireplace(['+', '-'], ' ', $keyword);
$search = filter_var($keyword, FILTER_SANITIZE_STRING);
$search = DB::connection()->getPdo()->quote($keyword);
I'm assuming I will need to use regex. What's the best approach here?
Only the words and operators have meaning in Boolean search mode. Operators are: +, -, > <, ( ), ~, *, ", #distance. After some research I found what word characters are: Upper case, Lower case letters, Numeral (digit) and _. I think you can use one of two approaches:
Replace all non word characters with spaces (I prefer this approach). This can be accomplished with regex:
$search = preg_replace('/[^\p{L}\p{N}_]+/u', ' ', $keyword);
Replace characters-operators with spaces:
$search = preg_replace('/[+\-><\(\)~*\"#]+/', ' ', $keyword);
Only words are indexed by full text search engine and can be searched. Non word characters isn't indexed, so it does not make sense to leave them in the search string.
References:
Boolean Full-Text Searches
Fine-Tuning MySQL Full-Text Search (see: "Character Set Modifications")
PHP: preg_replace
PHP: Unicode character properties
PHP: Possible modifiers in regex patterns
While the answer from Rimas is technically correct, it will suit you only if you do not want users to use the MATCH operators, because it will strip them all completely. For example, I do want to allow use of all of them except #distance in search forms on my site, thus I've come up with this:
#Trim first
$newValue = preg_replace('/^\p{Z}+|\p{Z}+$/u', '', string);
#Remove all symbols except allowed operators and space. #distance is not included, since it's unlikely a human will be using it through UI form
$newValue = preg_replace('/[^\p{L}\p{N}_+\-<>~()"* ]/u', '', $newValue);
#Remove all operators, that can only precede a text and that are not preceded by either beginning of string or space
$newValue = preg_replace('/(?<!^| )[+\-<>~]/u', '', $newValue);
#Remove all double quotes and asterisks, that are not preceded by either beginning of string, letter, number or space
$newValue = preg_replace('/(?<![\p{L}\p{N}_ ]|^)[*"]/u', '', $newValue);
#Remove all double quotes and asterisks, that are inside text
$newValue = preg_replace('/([\p{L}\p{N}_])([*"])([\p{L}\p{N}_])/u', '', $newValue);
#Remove all opening parenthesis which are not preceded by beginning of string or space
$newValue = preg_replace('/(?<!^| )\(/u', '', $newValue);
#Remove all closing parenthesis which are not preceded by beginning of string or space or are not followed by end of string or space
$newValue = preg_replace('/(?<![\p{L}\p{N}_])\)|\)(?! |$)/u', '', $newValue);
#Remove all double quotes if the count is not even
if (substr_count($newValue, '"') % 2 !== 0) {
$newValue = preg_replace('/"/u', '', $newValue);
}
#Remove all parenthesis if count of closing does not match count of opening ones
if (substr_count($newValue, '(') !== substr_count($newValue, ')')) {
$newValue = preg_replace('/[()]/u', '', $newValue);
}
Unfortunately I was not able to figure out a way to do this in 1 regex, thus doing multiple runs. It's also possible, that I am missing some edge cases, as well. Any additions or corrections are appreciated: either here or create an issue for https://github.com/Simbiat/database where I implement this.

MYSQL?PHP using variables to replace filenames in insert statements

I have a page that inserts records into a database file called ports that holds two fields, called id and port.
The data is checked by an include, checkform.php, that strips out any bad data and blank entries.
It works fine, and as I have more data files of a similar construction it seems logical to use the same page for inserting records by passing the file and field names to the page as parameters.
The SQL that is used for the stand alone page is:
$sql='INSERT IGNORE INTO ports(port) VALUES(?)';
I want to do some thing like:
$sql='INSERT IGNORE INTO $filename ($fieldname) VALUES(?)';
I have looked on the forum and found many solutions that do not appear to work
Like :
$sql='INSERT IGNORE INTO '$filename' ('$fieldname') VALUES(?)';
$sql='INSERT IGNORE INTO "'$filename'" ("'$fieldname'") VALUES(?)';
$sql='INSERT IGNORE INTO `$filename` (`$fieldname`) VALUES(?)';
as well as :
$sql="INSERT IGNORE INTO `$filename` (`$fieldname`) VALUES (`$fieldname`);";
and many others. The combination seems endless, and so far I would have been better just copying the pages and changing the variables by hand. The code for the insert is below:
// check if form submitted and has a value
If (isset($_POST['insert']))
{ require('../includes/checkform.inc.php');
// continue if the field is OK
if (empty($missing)) // ** missing is empty if the data is clean and exists
{ // process the input.
require_once('../includes/connection.inc.php');
// initialize a flag
$OK = false;
//create database connection
$conn = mysqli_connect( $DatabaseServer,$DatabaseUser, $DatabasePassword, $DatabaseName);
// Initialize prepared statement
$stmt = $conn->stmt_init();
//create SQL
$sql='INSERT IGNORE INTO ports(port) VALUES(?)'; //#
//bind parameters and execute statement
if($stmt->prepare($sql)) {
$stmt->bind_param('s',$_POST['port']);//#
$stmt->execute();
if ($stmt->affected_rows > 0)
$OK = true;
}//if $tmt
}// if empty
// redirect if successful or display an error - on page below
if ($OK) {
header('Location:insertok.php');
exit;
} else {
$error = htmlspecialchars($stmt->error);
The lines with //# against them are the ones that I need help with.
Most of the code is modified from a book by David Powers.
Howard Walker
To interpolate variables in a string, you have to use double quotes "$var". Note that you shouldn't surround $var with single quotes. And your table and column names might be one of the reserved words. It complains when that happens. You use backticks to escape the reserved words.
$sql="INSERT IGNORE INTO `$filename` (`$fieldname`) VALUES (?);";
This should work just fine.
EDIT
Your file/field might also include the characters that mySQL doesn't like. In that case, escape the query string before executing it. Refer: http://us3.php.net/manual/en/mysqli.real-escape-string.php
$sql = $stmt->real_escape_string($sql);

MySQL to JSON not formed properly

I am trying to return JSON formatted results from a MySQL query but cannot get the correct format - it needs to be e.g.
{comCom:'test 3', comUid:'63',... etc
But what I'm getting is without apostrophes
{comCom:test 3, comUid:63,... etc
I am running the query in PHP as follows (shortened for ease of reading)
$result = mysql_query("select...
...GROUP_CONCAT(CONCAT('{comCom:',ww.comment, ', comUid:',h.user_id,', comName:',h.name,', comPic:',h.live_prof_pic,',comUrl:',h.url,',comWhen:',time_ago(ww.dateadded),'}')) comment,...
How can I get the punctuation?
I know mysql_query is deprecated btw, just in process of moving things to MySQLi
Can you not just escape the ' character with \'?
...GROUP_CONCAT(CONCAT('{comCom:\'',ww.comment, '\', comUid:\'',h.user_id,'\', comName:\'',h.name,'\', comPic:\'',h.live_prof_pic,'\',comUrl:\'',h.url,'\',comWhen:\'',time_ago(ww.dateadded),'\'}'))
or use a mixture of " with '
...GROUP_CONCAT(CONCAT("{comCom:'",ww.comment, "', comUid:'",h.user_id,"', comName:'",h.name,"', comPic:'",h.live_prof_pic,"',comUrl:'",h.url,"',comWhen:'",time_ago(ww.dateadded),"'}"))

How to escape single quotes in MySQL

How do I insert a value in MySQL that consist of single or double quotes. i.e
This is Ashok's Pen.
The single quote will create problems. There might be other escape characters.
How do you insert the data properly?
Put quite simply:
SELECT 'This is Ashok''s Pen.';
So inside the string, replace each single quote with two of them.
Or:
SELECT 'This is Ashok\'s Pen.'
Escape it =)
' is the escape character. So your string should be:
This is Ashok''s Pen
If you are using some front-end code, you need to do a string replace before sending the data to the stored procedure.
For example, in C# you can do
value = value.Replace("'", "''");
and then pass value to the stored procedure.
See my answer to "How to escape characters in MySQL"
Whatever library you are using to talk to MySQL will have an escaping function built in, e.g. in PHP you could use mysqli_real_escape_string or PDO::quote
Use this code:
<?php
$var = "This is Ashok's Pen.";
mysql_real_escape_string($var);
?>
This will solve your problem, because the database can't detect the special characters of a string.
If you use prepared statements, the driver will handle any escaping. For example (Java):
Connection conn = DriverManager.getConnection(driverUrl);
conn.setAutoCommit(false);
PreparedStatement prepped = conn.prepareStatement("INSERT INTO tbl(fileinfo) VALUES(?)");
String line = null;
while ((line = br.readLine()) != null) {
prepped.setString(1, line);
prepped.executeQuery();
}
conn.commit();
conn.close();
There is another way to do this which may or may not be safer, depending upon your perspective. It requires MySQL 5.6 or later because of the use of a specific string function: FROM_BASE64.
Let's say you have this message you'd like to insert:
"Ah," Nearly Headless Nick waved an elegant hand, "a matter of no importance. . . . It's not as though I really wanted to join. . . . Thought I'd apply, but apparently I 'don't fulfill requirements' -"
That quote has a bunch of single- and double-quotes and would be a real pain to insert into MySQL. If you are inserting that from a program, it's easy to escape the quotes, etc. But, if you have to put that into a SQL script, you'll have to edit the text (to escape the quotes) which could be error prone or sensitive to word-wrapping, etc.
Instead, you can Base64-encode the text, so you have a "clean" string:
SWtGb0xDSWdUbVZoY214NUlFaGxZV1JzWlhOeklFNXBZMnNnZD
JGMlpXUWdZVzRnWld4bFoyRnVkQ0JvWVc1a0xDQWlZU0J0WVhS
MFpYCklnYjJZZ2JtOGdhVzF3YjNKMFlXNWpaUzRnTGlBdUlDNG
dTWFFuY3lCdWIzUWdZWE1nZEdodmRXZG9JRWtnY21WaGJHeDVJ
SGRoYm5SbApaQ0IwYnlCcWIybHVMaUF1SUM0Z0xpQlVhRzkxWj
JoMElFa25aQ0JoY0hCc2VTd2dZblYwSUdGd2NHRnlaVzUwYkhr
Z1NTQW5aRzl1SjMKUWdablZzWm1sc2JDQnlaWEYxYVhKbGJXVn
VkSE1uSUMwaUlBPT0K
Some notes about Base64-encoding:
Base64-encoding is a binary encoding, so you'd better make sure that you get your character set correct when you do the encoding, because MySQL is going to decode the Base64-encoded string into bytes and then interpret those. Be sure base64 and MySQL agree on what the character encoding is (I recommend UTF-8).
I've wrapped the string to 50 columns for readability on Stack Overflow. You can wrap it to any number of columns you want (or not wrap at all) and it will still work.
Now, to load this into MySQL:
INSERT INTO my_table (text) VALUES (FROM_BASE64('
SWtGb0xDSWdUbVZoY214NUlFaGxZV1JzWlhOeklFNXBZMnNnZD
JGMlpXUWdZVzRnWld4bFoyRnVkQ0JvWVc1a0xDQWlZU0J0WVhS
MFpYCklnYjJZZ2JtOGdhVzF3YjNKMFlXNWpaUzRnTGlBdUlDNG
dTWFFuY3lCdWIzUWdZWE1nZEdodmRXZG9JRWtnY21WaGJHeDVJ
SGRoYm5SbApaQ0IwYnlCcWIybHVMaUF1SUM0Z0xpQlVhRzkxWj
JoMElFa25aQ0JoY0hCc2VTd2dZblYwSUdGd2NHRnlaVzUwYkhr
Z1NTQW5aRzl1SjMKUWdablZzWm1sc2JDQnlaWEYxYVhKbGJXVn
VkSE1uSUMwaUlBPT0K
'));
This will insert without any complaints, and you didn't have to manually-escape any text inside the string.
You should escape the special characters using the \ character.
This is Ashok's Pen.
Becomes:
This is Ashok\'s Pen.
If you want to keep (') apostrophe in the database use this below code:
$new_value = str_replace("'","\'", $value);
$new_value can store in database.
You can use this code,
<?php
$var = "This is Ashok's Pen.";
addslashes($var);
?>
if mysqli_real_escape_string() does not work.
In PHP, use mysqli_real_escape_string.
Example from the PHP Manual:
<?php
$link = mysqli_connect("localhost", "my_user", "my_password", "world");
/* check connection */
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
mysqli_query($link, "CREATE TEMPORARY TABLE myCity LIKE City");
$city = "'s Hertogenbosch";
/* this query will fail, cause we didn't escape $city */
if (!mysqli_query($link, "INSERT into myCity (Name) VALUES ('$city')")) {
printf("Error: %s\n", mysqli_sqlstate($link));
}
$city = mysqli_real_escape_string($link, $city);
/* this query with escaped $city will work */
if (mysqli_query($link, "INSERT into myCity (Name) VALUES ('$city')")) {
printf("%d Row inserted.\n", mysqli_affected_rows($link));
}
mysqli_close($link);
?>
$var = mysqli_real_escape_string($conn, $_POST['varfield']);
If you are using PHP, just use the addslashes() function.
PHP Manual addslashes
For programmatic access, you can use placeholders to automatically escape unsafe characters for you.
In Perl DBI, for example, you can use:
my $string = "This is Ashok's pen";
$dbh->do("insert into my_table(my_string) values(?)",undef,($string));
Use either addslahes() or mysql_real_escape_string().
The way I do, by using Delphi:
TheString to "escape":
TheString=" bla bla bla 'em some more apo:S 'em and so on ";
Solution:
StringReplace(TheString, #39,'\'+#39, [rfReplaceAll, rfIgnoreCase]);
Result:
TheString=" bla bla bla \'em some more apo:S \'em and so on ";
This function will replace all Char(39) with "\'" allowing you to insert or update text fields in MySQL without any problem.
Similar functions are found in all programming languages!
Maybe you could take a look at function QUOTE in the MySQL manual.
This is how my data as API response looks like, which I want to store in the MYSQL database. It contains Quotes, HTML Code , etc.
Example:-
{
rewardName: "Cabela's eGiftCard $25.00",
shortDescription: '<p>adidas gift cards can be redeemed in over 150 adidas Sport Performance, adidas Originals, or adidas Outlet stores in the US, as well as online at adidas.com.</p>
terms: '<p>adidas Gift Cards may be redeemed for merchandise on adidas.com and in adidas Sport Performance, adidas Originals, and adidas Outlet stores in the United States.'
}
SOLUTION
CREATE TABLE `brand` (
`reward_name` varchar(2048),
`short_description` varchar(2048),
`terms` varchar(2048),
) ENGINE=InnoDB AUTO_INCREMENT=6 DEFAULT CHARSET=latin1;
While inserting , In followed JSON.stringify()
let brandDetails= {
rewardName: JSON.stringify(obj.rewardName),
shortDescription: JSON.stringify(obj.shortDescription),
term: JSON.stringify(obj.term),
}
Above is the JSON object and below is the SQL Query that insert data into MySQL.
let query = `INSERT INTO brand (reward_name, short_description, terms)
VALUES (${brandDetails.rewardName},
(${brandDetails.shortDescription}, ${brandDetails.terms})`;
Its worked....
If nothing works try this :
var res = str.replace(/'/g, "\\'");
var res = res.replace(/"/g, "\\\"");
It adds the \ escape character to all(every) occurrences of ' and "
Not sure if its the correct/professional way to fix the issue
I'm guessing it will work but in actual content, every single and double quotes will be replaced with \ character
As a Python user I replace the quote with a raw "'":
don_not_work_str = "This is Ashok's Pen."
work_str = don_not_work_str.replace("'", r"\'")
For Python,I tried many ways to escape '"', not work, cuz I have various input.
Finally, I just use these code, the data in database was same like the input.
tmp = val.replace('\\', r'\\')
ret = tmp.replace('"', r'\"')