I have a million odd rows where most start
'http://www.' or 'https://www.'
but occasionally they start with no 'www.' - this may be correct but the website owner wants consistency throughout the data and thus I need to update the table to always have 'www.'
I'm struggling with the SQL to do this. I tried:
select * from the_million where URL like 'http://[!w]'
But that returns 0 records so I've fallen at the first hurdle of building up the SQL. I guess after I've got the records I want I'll then do a replace.
I'm happy to run this in two goes for each of http and https so no need for anything fancy there.
You can try this query:
UPDATE the_million SET url=REPLACE(url, 'http://', 'http://www.')
WHERE url NOT LIKE 'http://www.%' AND url NOT LIKE 'https://www.%'
UPDATE the_million SET url=REPLACE(url, 'https://', 'https://www.')
WHERE url NOT LIKE 'http://www.%' AND url NOT LIKE 'https://www.%'
Search & replace in 2 queries.
try this
select * from the_million where URL not like 'http://www.%'
This condition:
URL like 'http://[!w]'
... is identical to this one:
URL='http://[!w]'
because it doesn't contain any valid wildcard for MySQL LIKE operator. If you check the MySQL manual page you'll see that the only wildcards are % and _.
The W3Schools page where you read that [!charlist] is valid identifies the section as "SQL Wildcards" which is misleading or plain wrong (depending on how benevolent you feel). That's not standard SQL at all. The error messages returned by their "SQL Tryit Editor" suggest that queries run against a Microsoft Access database, thus it's only a (pretty irrelevant) SQL dialect.
My advice:
Avoid W3Schools as reference site. Their info is often wrong and they apparently don't care enough to amend it.
Always use the official manual of whatever DBMS engine you are using.
Last but not least, the good old www prefix is not a standard part of the HTTP protocol URIs (like http://); it's only a naming convention. Preppending it to an arbitrary list of URLs is like adding "1st floor" to all your customer addresses. Make sure your client knows that he's paying money to corrupt his data on purpose. And if he feels generous, you can propose him to replace all https: with http: as well.
Related
I was solving one of TryHackMe's rooms about SQL injection.But I couldn't figured out one thing that came to my mind and after spending lots of time I thought it's best to ask that question here. In the room there is a machine to deploy, after deployed machine it gave me an interface(webapp) that takes inputs from me like so :
And if you give it a value like test. It returns following output:
When I saw it, I thought it was an easy SQLi question so I tried most basic SQLi payloads for common dbms like so:
' ; sleep(1) --
' or 1=1 -- ...
But none of them worked and returned me an error message like that:
After that failure, I run sqlmap and it found 2 types of SQLi payload with following payloads:
Then I realized that in time based SQLi, sqlmap used ||. Then I tried to send '|| (select sleep(2)) -- . And it worked. Now my question is why my first payload(' ; select sleep(2) -- ) didn't work, but this one worked ?
By default, MySQL query interfaces do not support multiple queries separated by semicolon. To get that, the application would have to explicitly request this feature to be enabled when it connects to the database. There's a connect option for that.
So by default, any type of SQL injection that works by trying to append a malicious query after a semicolon doesn't work.
But an SQL injection that works by modifying an expression in the single query can still work.
I have a poll enrich which enriches a POJO with the result of an SQL query (from a MySQL database). It currently gets the brand from the POJO and then gets the name from the order matching the brand. I had to add quotes around the ${body.getBrand}, else the query would look for a column with the brand name instead of using the value. Currently it looks like this:
<pollEnrich id="_enrich1" strategyRef="merge" timeout="5000">
<simple>sql:SELECT name FROM orders WHERE brand= '${body.getBrand}'</simple>
</pollEnrich>
I want to change it because I'll probably need to create more sql queries and the current version does not work if the value contains quotes and thus is vulnerable to sql injection.
I thought prepared statements would do the trick and wanted to use a named parameter but I do not seem to be able to set the value of the parameter.
I have tried many different things like for example setting a header and change the query to have a named parameter:
<setHeader headerName="brand" id="brand">
<simple>${body.getBrand}</simple>
</setHeader>
<pollEnrich id="_enrich1" strategyRef="merge" timeout="5000">
<simple>sql:SELECT name FROM orders WHERE brand= :#brand</simple>
</pollEnrich>
but I keep getting
PreparedStatementCallback; bad SQL grammar [SELECT name FROM orders WHERE brand= ?]; nested exception is java.sql.SQLException: No value specified for parameter 1
I have also tried setting the useMessageBodyForSql option to true (since this seemed like something that might help?) but nothing I have tried seemed to work.
I have seen a lot of examples/solutions for people setting the routes with java, but I assume there must also be a solution for the blueprint xml?
If anyone got any suggestion or example that would be great.
In Camel version < 2.16, pollEnrich doesn't have access to the original exchange and therefore cannot read your header, hence the exception. This is documented here: http://camel.apache.org/content-enricher.html
Guessing from your example, a normal enrich should work too and it has access to the original exchange. Try changing 'pollEnrich' to 'enrich'.
I have a WordPress plugin that I am editing in PHPStorm. PHPStorm finds syntax errors on queries like this when the SQL dialect is set to MySQL:
$foo = $wpdb->get_var(
$wpdb->prepare(
'SELECT `foo` FROM `some_table` WHERE `id` = %d',
$bar
)
);
Specifically, it sees %d and complains with this message:
<expression> expected, got '%'
Of course, %d is a perfectly legitimate placeholder in WordPress queries. Is there a way to configure PHPStorm to accept this? Or do I have to disable all checks on SQL statements, as suggested in this answer?
Note that I am using PHPStorm EAP 8 (138.1751), and the same thing happens with other placeholders like %s.
This is now possible in PHPStorm 8, as explained by this post on the official PHPStorm blog:
Database Language Injection Configuration
To solve it:
Go to Tools > Databases
Make sure a regex matching the placeholders in question is in the list of custom parameters. By default, the list includes \%\w+, which will match %s, %d, etc.
Check the box labeled "Use in other language string literals"
PHPStorm will now correctly recognize placeholders like those used in WordPress.
Nope -- %d is not valid syntax from SQL point of view (ANY currently supported dialect). The WI-3672 and WI-2324 tickets are still valid.
But PhpStorm v8 now supports $var (if you could use it instead of % syntax) -- http://youtrack.jetbrains.com/issue/WI-2450
Since it's WordPress specific question, it's hard for me to give you any real suggestion (especially since I'm not using WordPress myself) excluding that one.
But generally speaking you could use native place holders supported by specific DB engine (check "Examples" section in http://php.net/manual/en/pdo.prepare.php):
unnamed ?
named :id
I just not sure if this can be applied to WordPress development.
P.S.
Generic is the new Keywords only dialect -- fall back to that if using proper SQL dialect shows to many warnings/errors (that are not actual errors).
I want to be able to add a little note, at the beginning of each query, so when I see it in the processlist, or "mytop", I can tell where it’s running.
Is something like this possible?
I am not sure this would work, but it is worth trying.
Simply add "/* some comment or tag */ " before whatever SQL query is sent normally.
It is possible that MySQL server will remove this comment as part of its query analysis/preparation, but it may just leave it as well, so it shows as such in logs and other monitoring tools.
In case the comments get stripped away, and assuming SELECT queries, a slight variation on the above would be to add a calculated column as the first thing after SELECT, something like
SELECT IF('some comment/tag' = '', 1, 0) AS BogusMarker, here-start-the-original-select-list
-- or
SELECT 'some [short] comment/tag' AS QueryID, here-start-the-original-select-list
This approach has the drawback of introducing an extra column value, with each of the results row. The latter form, actually uses the "comment/tag" value as this value, which may be helpful for debugging purposes.
I discovered this today:
MySQL supports the strange syntax:
/*!<min-version> code here */
to embed code that will only be interpreted by MySQL, and only MySQL of a specified minimum version.
This is documented at 9.7 Comments (used by mysqldump, for example).
Such comments will not be parsed out, and included in the processlist, unlike normal ones, and that even if the code is not actually executed.
So you can do something like
/*!999999 comment goes here */ select foo from bla;
to have a comment that will show up in the processlist, but not alter the code. (Until MySQL developers decide to bloat their release numbering and release a version numbered over 99.99.99, in which case you will just have to add another digit.)
I need to take a MySQL query and insert a string before each table name. The solution doesn't need to be one line but obviously it's a regex problem. It will be implemented in PHP so having programming logic is also fine.
Rationale and Background:
I'm revamping my code base to allow for table prefixes (eg: 'nx_users' instead of 'users') and I'd like to have a function that will automate that for me so I don't need to find every query and modify it manually.
Example:
SELECT * FROM users, teams WHERE users.team_id = teams.team_id ORDER BY users.last_name
Using the prefix 'nx_', it should change to
SELECT * FROM nx_users, nx_ teams WHERE nx_ users.team_id = nx_ teams.team_id ORDER BY nx_ users.last_name
Obviously it should handle other cases such as table aliases, joins, and other common MySQL commands.
Has anybody done this?
How big of a code base are we talking about here? A regular expression for something like this is seriously flirting with disaster and I think you're probably better off looking for every mysql_query or whatever in your code and making the changes yourself. It shouldn't take more than the hour you'd spend implementing your regex and fixing all the edge cases that it will undoubtedly miss.
Using a regex to rewrite code is going to be problematic.
If you need to dynamically change this string, then you need to separate out your sql logic into one place, and have a $table_prefix variable that is appropriately placed in every sql query. The variable can then be set by the calling code.
$query = "SELECT foo from " . $table_prefix . "bar WHERE 1";
If you are encapsulating this in a class, all the better.
This example does not take into consideration any escaping or security concerns.
First off, regular expressions alone are not up to the task. Consider things like:
select sender from email where subject like "from users group by email"
To really do this you need something that will parse the SQL, produce a parse tree which you can modify, and then emit the modified SQL from the modified parse tree. With that, it's doable, but not advisable (for the reasons Paolo gave).
A better approach would be to grep through your source looking for either the table names, the function you use to sent SQL, the word from, or something like it at script something to throw you into an editor at those points.