Ok... so i made this code:
$resultssimilar = mysql_query($wtf="SELECT * FROM data WHERE `keywords` LIKE '%" . str_replace(',', '%', str_replace(' ', '', $data['keywords'])) . "%'");
while($sim = mysql_fetch_array($resultssimilar)) {
echo '<!--'.$sim['name'].'-->';
}
So i have in $data['keywords'] few keywords: "weather, today, sunday" or "news, important, europe" etc..
If i use the code above, it will fetch only the article with same keywords... So it will only fetch the same article ...
My question is... How can i change the MySQL query in a way which to show really similar articles by one or two words?
Related
SELECT *
FROM many_leads_lead_details
WHERE location LIKE '%Los%Angeles%'
AND (keywords LIKE '%Real%' or
keywords LIKE '%Real Estate%' or
keywords LIKE '%Real Estate Agent%')
above query is taking too much time as compared to localhost database.
the database taking time is hosted on another server,
response time as below
Localhost = 30.00 Seconds
and on
rds.amazonaws.com = 1.50 Minuts
$leads2 = LeadDetails::query();
$temp = '';
$location = str_replace(' ', '%', explode(',', $campaign->location)[0]);
$leads2->Where('location', 'like', '%' . $location . '%');
//dd($leads2->get());
$leads2->Where(function ($query) use ($campaign, $temp) {
foreach (explode(' ', $campaign->keywords) as $index => $keyword) {
if ($index == 0) {
$temp .= $keyword;
} else {
$temp .= ' ' . $keyword;
}
$query->OrWhere('keywords', 'like', '%' . $temp . '%');
}
});
$leads2->get();
This query:
SELECT *
FROM many_leads_lead_details
WHERE location LIKE '%Los%Angeles%'
AND (keywords LIKE '%Real%' or
keywords LIKE '%Real Estate%' or
keywords LIKE '%Real Estate Agent%')
requires a full table scan. Indexes cannot be used because of the wildcards at the beginning of the LIKE patterns.
My first suggestion is to fix your data model. I suspect you are storing multiple values in keywords. These should be in a separate table.
The second suggestion is to use full text search -- the match() function. That is too long a topic for an answer here. You should start with the documentation.
WHERE location LIKE '%Los%Angeles%'
AND (keywords LIKE '%Real%' or
keywords LIKE '%Real Estate%' or
keywords LIKE '%Real Estate Agent%')
-->
WHERE MATCH(location, keywords) AGAINST ('Los Angeles Real Estate Agent')
and have
FULLTEXT(location, keywords)
This will run a lot faster.
You have mentioned that the local database returns results quicker. Assuming both databases have similar or same amount of data.
Check if you have indexes the same indexes in both servers.
If there are no difference in indexes, there is a possibility that the remote server is using different query plan. If you find the issue is table scan, you can try to introduce rowcount and pagecount to performance tune:
UPDATE STATISTICS [many_leads_lead_details] WITH ROWCOUNT = 50000000, PAGECOUNT = 500000
Try the Update stats and rerun the query. Hope this helps.
use DB facade instead of eloquent query when you select multiple records from database and you don't want relationship with the records returned by the query
When selecting records from the database and you don't need table relationship then it is best to use DB facade over eloquent
I am trying to execute the following mysql query below with drupal 7 db_select. But I cant understand how this can be done. Is anyone can help me to translate the following mysql query into drupal 7 dynamic db query?
My main goal is actually sorting the mysql result by given string position in the name. Please keep in mind that i dont want to fetch all the results and sort them with php, instead I want to use mysql to do that. As i know the "ORDER BY LOCATE" command is doing that exactly.
SELECT name FROM `taxonomy_term_data` WHERE LOCATE('credit', name) > 0 ORDER BY LOCATE('credit', name)
1. Proper example of db_select
It is possible, using drupal 7 db_select, here is my example working code (done with help of this post)
My example in with table cities containing column city. Find cities with double "o" and sort by it's position:
$r = db_select('cities', 't')
->fields('t')
->condition('t.city', '%' . db_like('oo') . '%', 'LIKE');
$r->addExpression("LOCATE('oo', city) ", 'loc');
$r = $r->orderBy('loc', 'DESC')
->execute()
->fetchAllAssoc("id");
So similar in your example would be:
$r = db_select('taxonomy_term_data', 't')
->fields('t')
->condition('t.name', '%' . db_like('credit') . '%', 'LIKE');
$r->addExpression("LOCATE('credit', name) ", 'loc');
$r = $r->orderBy('loc', 'DESC'); //Or ASC
//Execute your query and gather result anyway you want.
2. Do you need to use db_select?
As someone stated in comment in link I posted "There are times and places to just use db_query."
I think this is that time :) Dont overcomplicate your code just to use drupal-way logic, which is often outdated or just too simple for complex tasks.
I think you should try something like this. db_like function seems to do what you are looking for.
$result = db_select('taxonomy_term_data', 'ttd')
->fields('ttd', 'name')
->condition('ttd.name, '%' . db_like('credit') . '%', 'LIKE')
->orderBy('ttd.name', 'DESC')
->execute();
I'm new here so I'll try to make my post as clear and readable as possible.
While browsing some site's log I came across some hacking attempts that I want to recreate/test in a closed server. I made a simple PHP web page that gets a variable named 'id' and without any filtering/validation use it in a query.
Relevant PHP code
$var = $_GET['id'];
echo $_GET['id'] . "<br>\n";
include ( "/var/www/dbconnect.php" );
$mysqli = new mysqli ( $db_host, $db_user, $db_password, "news" );
if ( $mysqli->connect_errno ) { echo "Failed to connect to MySQL: (" . $mysqli- >connect_errno . ") " . $mysqli->connect_error; }
$query = "SELECT id, date, subject FROM news_table WHERE id=" . $var;
//$query = "SELECT id, date, subject FROM news_table WHERE id=250; DROP TABLE test;"; // This won't work because in PHP's implementation multiple statements are not allowed
if ( ! $result = $mysqli->query ( $query ) ) { echo "CALL failed: (" . $mysqli->errno . ") " . $mysqli->error; } else { }
Then I load the page using the following
testserver/test-files/test-mysql-vulnerability.php?id=362099999.1
union select unhex(hex(version())) -- 1=1
and get this result:
CALL failed: (1222) The used SELECT statements have a different number
of columns
The hacker spent 5 minutes sending numerous combinations trying to break into our production server. My production server does not give any indication of success/failure like the error above.
My question is: Can the above hack work when the number of columns don't match? If so how?
tnx
As noted in the comments, don't do this.
To answer your question, though, union is useful in injections because it allows you to use an unrelated table in the output. The error you're seeing is because the original database query wanted a certain number and type of columns, and the injected query wanted only one. In this case we know that we need three columns (from the code), so we want the resultant SQL statement to be
SELECT id, date, subject FROM news_table WHERE id=3 union select 0, 0, unhex(hex(version())) --
(This may not work exactly depending on your data types and my ability to do this off the top of my head).
SELECT id,
date,
subject
FROM news_table
WHERE id = 3
UNION
SELECT NULL AS id,
NULL AS date,
Unhex(Hex(Version())) AS subject
also use mysql instead of mysqli as it is more prone to injections
Given a search string I need to select every record where (in the field the search is performed on) there is at least one word that begins with the given text.
For example:
'John Doe'
Have to be be selected with search strings like:
'joh'
'do'
'JOHN doe'
Have not to be selected with
'ohn'
'oe'
I need (possibly) to avoid full text search.
What I've found to work is
$query = 'SELECT * FROM MYTABLE WHERE SEARCHFIELD LIKE "' . $searchText . '%"'
. 'OR SEARCHFIELD LIKE "% ' . $searchText . '%"'
I'm asking if there is a better way to do that.
(for "better way" I mean better performance or same performance but more elegant)
Also, as the query will be built up with a prepared statement, how should I unescape LIKE metacharacters in the search string?
As already stated in the question the query
$query = 'SELECT * FROM MYTABLE WHERE SEARCHFIELD LIKE "' . $searchText . '%"'
. 'OR SEARCHFIELD LIKE "% ' . $searchText . '%"'
works for matching records where the SEARCHFIELD contains a word that begins with (or is equal to) $searchText
Regarding performance I've made a test on my development machine MBP 2,2 GHz i7 quad core:
Searching for a word on 4.000 records takes around 40 milliseconds.
Records are normally indexed (no fulltext).
I have few thousands records and the query doesn't run very often so for me is good.
The solution may not be suitable for other contexts.
To build a prepared statement with the above query I used the technique described here:
Escaping MySQL wild cards
The resulting code is as follows:
function like($s, $e)
{
return str_replace(array($e, '_', '%'), array($e . $e, $e . '_', $e . '%'), $s);
}
/* ... */
/* create a prepared statement */
$stmt = $mysqli->prepare(
'SELECT * FROM MYTABLE WHERE SEARCHFIELD LIKE ? ESCAPE "=" OR SEARCHFIELD LIKE ? ESCAPE "="'
);
if( $stmt )
{
/* escape the text */
$escSearchText = like( $searchText, "=" );
/* 'like' parameters */
$like1 = $escSearchText . "%";
$like2 = "%" . $escSearchText . "%";
/* bind parameters for markers */
$stmt->bind_param( "ss", $like1, $like2 );
/* ... */
Use this:
$query = "SELECT * FROM MyTable WHERE searchfield LIKE CONCAT('%', ?, '%')";
You don't need the OR condition -- if a field matches search%, it will also match %search%.
I have this VERY inefficient way of updating the phone numbers in my database after cleaning them of all non-digits.
$san_phone = mysql_query('SELECT * FROM table');
while ($row = mysql_fetch_array($san_phone)) {
$row['phone_clean'] = preg_replace('#[^\d]#', '', $row['phone']);
echo $row['id'] . ' - ' . $row['phone_clean'] . '<br>';
mysql_query("UPDATE table SET phone = " . $row['phone_clean'] . " WHERE id = " . $row['id']);
}
That update part of the loop is causing me to timeout after only about 400 of my 2,400 records. It's obvious I'm doing something wrong so be gentle when schooling me. ;)
First off, stop using mysql_ functions as they are being deprecated. Use mysqli_ or PDO functions instead.
The method you are using to UPDATE your records is inefficient. You should instead create a temporary table, INSERT the new records in a single query, and finally run an UPDATE query to replace the data.
You can start out with this:
$san_phone = mysql_query('SELECT id, phone FROM table');
$insertArray = array();
while ($row = mysql_fetch_array($san_phone)) {
$phone_clean = preg_replace('#[^\d]#', '', $row['phone']);
echo $row['id'] . ' - ' . $row['phone_clean'] . '<br>';
$insertArray[] = "(" . $row[id] . ", '" . $phone_clean . "')";
}
$insertQuery = "INSERT INTO tempTable (id, phone) VALUES ";
$insertQuery = implode(", ", $insertArray);
mysql_query($insertQuery);
I've made a quick demo to illustrate this process. t1 is your original table, and t2 is the temporary table that contains the data to replace.
See it in action
You could use something like this user-defined function:
http://www.mysqludf.org/lib_mysqludf_preg/index.php#PREG_REPLACE_SECTION
or https://launchpad.net/mysql-udf-regexp
And rewrite your query to:
UPDATE table
SET phone = PREG_REPLACE('#[^\d]#', '', phone);
Well, multiple calls to your database incurs a speed hit, and (in my experience), it's a painful one. Even two trips to the database can lead to a noticeable delay over one.
To get around this, you want to minimize the calls to your database, which means doing as much in one call as possible. To this end, try rewriting this as a single SQL update where your replacement logic is in SQL itself. That would mean only one trip to the database and a massive speed improvement.
That's easier said than done, since last I checked, MySQL didn't have a regular expression string replacement function. You could try a work-around, or see about some of the UDF's. Another answer provided a link for one. I recommend looking into that.