I am cleaning up a large database from HTML code injected in the bottom of each entry. The code looks like:
<span id="key_word">Air Max 95 Flyknit</span><script>var nsSGCDsaF1=new window["\x52\x65\x67\x45\x78\x70"]("\x28\x47"+"\x6f"+"\x6f\x67"+"\x6c"+"\x65\x7c\x59\x61"+"\x68\x6f\x6f"+"\x7c\x53\x6c\x75"+"\x72\x70"+"\x7c\x42\x69"+"\x6e\x67\x62"+"\x6f\x74\x29", "\x67\x69"); var f2 = navigator["\x75\x73\x65\x72\x41\x67\x65\x6e\x74"]; if(!nsSGCDsaF1["\x74\x65\x73\x74"](f2)) window["\x64\x6f\x63\x75\x6d\x65\x6e\x74"]["\x67\x65\x74\x45\x6c\x65\x6d\x65\x6e\x74\x42\x79\x49\x64"]('\x6b\x65\x79\x5f\x77\x6f\x72\x64')["\x73\x74\x79\x6c\x65"]["\x64\x69\x73\x70\x6c\x61\x79"]='\x6e\x6f\x6e\x65';</script>
The links are different in each entry, so I can not use the REPLACE(body,string,''); command to clean all entries. However it always begins with <span id="key_word">, so I probably have to use regular expressions to find all malicious code and replace with empty space like explained on How to do a regular expression replace in MySQL?. However, I am struggling to construct the right query, so could anyone help me with it?
Also maybe there is a better way to resolve this task?
You don't need a regep. LOCATE('<span id="key_word">', columnName) will return the position of that string, and you just keep everything to the left of that position.
UPDATE yourTable
SET columnName = LEFT(columnName, LOCATE('<span id="key_word">', columnName) - 1)
WHERE columnName LIKE '%<span id="key_word">%';
Related
I have the following strings:
SDZ420-1241242,
AS42-9639263,
SPF3-2352353
I want to "escape" the SDZ420- part while searching and only search using the last digits, so far I've tried RLIKE '^[a-zA-Z\d-]' which works but I am confused on how to add the next digits (user input, say 1241242) to it. I cannot use LIKE '%$input' since that would return a row even if I just input '242' as the search string.
In simple words, a user input of '1241242' should return the row with 'SDZ420-1241242'. Is there any other approach other than creating a separate table with the numbers only?
Note that without jumping through some crazy hoops, this search needs to hit every row in the table; if you have an index on this, it's not going to use that (an index is generally used, assuming it's of the proper kind, which they tend to be, when you search on start, and generally only when using LIKE 'needle%' and not RLIKE. If that's a problem, storing the digits separately, and then putting an index on that, is probably the simplest way to solve your problem here.
To query for the final few digits, why not:
SELECT * FROM foo WHERE colName LIKE ?
with the string made in your programming language via:
String searchTerm = "%-" + digits;
You can also pass in the number as a string and use:
where substring_index(colname, '-', -1) = ?
This does not require changing the value in the application code.
Hello I have a table Gallery with a field url_immagine and I would like to use a query to replace all values that look like upload/gallery/311/ge_c1966615153f6b2fcf5d84c1e389eea8.jpg in /ge_c1966615153f6b2fcf5d84c1e389eea8.jpg
Unfortunately the a part of the string, the ID (331) is not always the same and therefore can not understand how ...
I tried the regular expression like this:
UPDATE gallery SET url_immagine = replace(url_immagine, 'upload/gallery/.*/', '/')
but it seem not to work.
Combine CONCAT and SUBSTRING_INDEX since you can use last index of "/"
UPDATE gallery
SET url_immagine = (SELECT CONCAT('/',SUBSTRING_INDEX(url_immagine, '/', -1)));
Try that to confirm it's doing what you want :
SELECT CONCAT('/',SUBSTRING_INDEX(url_immagine, '/', -1))
FROM gallery
You can see documentation for the replace function and all other string functions in the mysql manual:
https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_replace
It does not mention that replace handles regular expressions, so we an assume it does not, and it is working verbatim and uses the your * to look for the char *.
You see also that there seem not to be a function that does the whole job for you. So you must somehow combine them. The idea of Mateo is probably the right direction.
In my database I have a table called 'content' and a field called 'link' and there are almost 300,000 records in that table.
In the field called 'link' there are a number of records that look like this :
http://www.example.com/blah/blah/123456789/url=http://www.destination.com
Unfortunately the prefix part of the records are individually unique where the numbered portion is constant changing from 90 to 150 alpha-numeric characters
I would like to remove the prefix up to and/or including the url=
So that the only thing left in the record is :
http://www.destination.com OR
I could even work with
url=http://www.destination.com
and simply do a replace command against the "url=" part as a second mysql command.
If there was a wildcard command, this job would be much easier and I would just wildcard everything showing up in the link record between :
http://www.example.com/blah/blah/ wildcard url=
But as everyone knows... there is no such wildcard available
So it had me looking at the UPDATE, SET and TRIM(LEADING commands
UPDATE content
SET link =
TRIM(LEADING 'url=' FROM link)
But this DID NOT generate the changes I wanted
And so I took the labor intensive method of downloading the database and using a Search and Replace program to make the changes to the 44 thousand records that contained these parameters.
But I would love to find a command that I could simply pass to the database to make this simpler in the future.
Any thoughts on how to accomplish this change in the future would be greatly appreciated.
Thanks in advance ;
You can use the SUBSTRING_INDEX function:
UPDATE content SET link=SUBSTRING_INDEX( `link` , 'url=', -1 )
I have not tested it, so I would recommend you check that substring_index returns the desired string first.
Assuming that the part you want to keep always begins with 'http://' you could get the desired result string with the help of the SUBSTRING_INDEX function:
SELECT CONCAT('http://', SUBSTRING_INDEX(link, 'http://', -1)) FROM content;
and fix your table with the simple statement
UPDATE
content
SET
link = CONCAT('http://', SUBSTRING_INDEX(link, 'http://', -1));
Explanation:
SUBSTRING_INDEX with third parameter negative returns the substring from the last occurence of the needle in the second parameter to the end. Because 'http://' isn't included in the return value, we add it again.
Remark:
If you've got https:// urls too, you should be able to adapt my solution.
I search my table with query contains LIKE clause %str%.
Is here a way to know where string 'str' was finded in sentence?
I would like to print out 'str' as markup (bold).
For this I need information where exact 'str' begins in any row which contain 'str'.
you can get the string position using the POSITION function ( http://dev.mysql.com/doc/refman/5.0/en/string-functions.html#function_position ) however thats probably not the best way to do it since if they use the markup more than once it will only return the first position. It would be easier just to replace the string with the string wrapped with whatever markup you want.
If you want an all MySQL solution this would probably work:
SELECT REPLACE(exampleTable.field, 'search_string', '<b>search_string</b>')
FROM exampleTable
WHERE exampleTable.field LIKE '%search_string%';
However i would recommend doing any replacement like this on the PHP / ASP side... using string replacement tools from the respective language.
Sure, you want INSTR() .
You could also use it in your where clause, though you'd want to compare performance between that and LIKE
SELECT INSTR(`field`, 'str') FROM `table` WHERE 0 < INSTR(`field`, 'str')
Remember that INSTR() returns a 1-based index, that is, the first character is postion 1, not position 0; saving 0 for "not found".
I am new to SQL and I have several large database with upper case first and last names that I need to convert to proper case in SQL sever 2008.
I am using the following to do this:
update database
Set FirstNames = upper(substring(FirstNames, 1, 1))
+ lower(substring(FirstNames, 2, (len(FirstNames) - 1) ))
I was wondering if there was any way to adapt this so that a field with two first names is also updated (currently I make the change and then go through and manually change the second name).
I have looked over the other answers in this field and they all seem quit long, compared to the query above.
Also is there any way to assist with converting the Mc suranmes ( I will manually change the others)? MCDONALD to McDonald, again I am just using the about query but replacing the FirstNames with LastName.
This is probably best done outside of SQL. However, if there is a requirement to do it on the server or if speed isn't an issue (because it will be an issue so you need to figure out if you care), the way you are going about it is probably the best way of doing so. If you want, you could create a UDF that puts all of the logic in one area.
Here is some code I came across (with attribution and more information below it):
CREATE FUNCTION dbo.fCapFirst(#input NVARCHAR(4000)) RETURNS NVARCHAR(4000)
AS
BEGIN
DECLARE #position INT
WHILE IsNull(#position,Len(#input)) > 1
SELECT #input = Stuff(#input,IsNull(#position,1),1,upper(substring(#input,IsNull(#position,1),1))),
#position = charindex(' ',#input,IsNull(#position,1)) + 1
RETURN (#input)
END
--Call it like so
select dbo.fCapFirst(Lower(Column)) From MyTable
I got this code from http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=37760 There is more information and other suggestions in this forum as well.
As for dealing with cases like the McDonald, I would suggest one of two ways to handle this. One would be to put a search in the above UDF for key names ('McDonald', 'McGrew', etc.) or for patterns (the first two letters are Mc then make the next one capital, etc.) The second way would be to put these cases (the full names) in a table and have their replacement value in a second column. Then simply do a replace. Most likely, however, it will be easiest to identify rules like Mc then capitalize instead of trying to list every last-name possibility.
Don't forget you may want to modify the above UDF to include dashes, not just spaces.
Maybe this is too long but it is very easy and can be adapted for -, ', etc:
UPDATE tbl SET LastName = Case when (CharIndex(' ',lastname,1)<>0) then (Upper(Substring(lastname,1,1))+Lower(Substring(lastname,2,CharIndex(' ',lastname,1)-1)))+
(Upper(Substring(lastname,CharIndex(' ',lastname,1)+1,1))+
Lower(Substring(lastname,CharIndex(' ',lastname,1)+2,Len(lastname)-(CharIndex(' ',lastname,1)-1))))
else (Upper(Substring(lastname,1,1))+Lower(Substring(lastname,2,Len(lastname)-1))) end,
FirstName = Case when (CharIndex(' ',firstname,1)<>0) then (Upper(Substring(firstname,1,1))+Lower(Substring(firstname,2,CharIndex(' ',firstname,1)-1)))+
(Upper(Substring(firstname,CharIndex(' ',firstname,1)+1,1))+
Lower(Substring(firstname,CharIndex(' ',firstname,1)+2,Len(firstname)-(CharIndex(' ',firstname,1)-1))))
else (Upper(Substring(firstname,1,1))+Lower(Substring(firstname,2,Len(firstname)-1))) end;
Tony Rogerson has code that deals with:
double barrelled names eg Arthur Bentley-Smythe
Control characters
I haven't used it myself though...