Search & replace 'http' to 'https' in database - mysql

Using phpmyadmin, I want to run a query that will search my entire database for:
http://example.com
And replace with:
https://example.com
My SQL knowledge is limited, maybe something like:
UPDATE ?? = REPLACE(??, 'http://example.com', 'https://example.com');
The database is over 1gb, so what can I run that will not crash the server.
Update: Note that while there are other answers posted here on SO that deals with search and replace, they don't seem to cover the entire database.

use REPLACE. and if there is a index on the field then the UPDATE can use them
UPDATE t
set url = REPLACE(url, 'http:', 'https:')
WHERE url LIKE '%http:%';
only change example.com
this will only find row with 'http://example.com'
UPDATE t
set url = REPLACE(url, 'http:', 'https:')
WHERE url LIKE '%http://example.com%';
or this will find all rows with http:// but only change only this http://example.com to https://example.com
UPDATE t
set url = REPLACE(url, 'http://example.com', 'https://example.com')
WHERE url LIKE '%http:%';

Warning, the answers given so far will mess up serialized data!
For example, say your site stores serialized data in a row with the URL in it, like this:
a:1:{i:0;s:19:”http://example.com”;}
Notice that the value of this item has 19 characters, and is denoted by s:19 in the array.
If you replace content using a SQL query, the same row on your new environment would end up like this:
a:1:{i:0;s:19:”https://example.com”;}
But after this change, the value is now 20 characters long meaning s:19 is incorrect. This invalidates the array and the entire row.
So either you make sure your SQL statements deal with serialized data, or if you happen to be using WordPress then there are a few options to search using PHP so as to not break the serialized rows:
The Better Search Replace plugin automatically handles serialized
data
The Search and Replace plugin offers an option which handles
serialized data
Taken and adapted from: https://wpengine.com/support/wordpress-serialized-data/

I would use insert:
update t
set url = insert(url, 5, 0, 's')
where url like 'http:%';

Related

MySQLAdmin replace text in a field with percent in text

Using MySQLAdmin. Moved data from Windows server and trying to replace case in urls but not finding the matches. Need slashes as I don't want to replace text in anything but the urls (in post table). I think the %20 are the problem somwhow?
UPDATE table_name SET field = replace(field, '/user%20name/', '/User%20Name/')
The actual string is more like:
https://www.example.com/forum/uploads/user%20name/GFCI%20Stds%20Rev%202006%20.pdf
In a case you are using MariaDB you have REGEXP_REPLACE() function.
But best approach is to dump the table into the file. Open it in a Notepad ++
and run regex replace like specified on a pic:
Pattern is: (https:[\/\w\s\.]+uploads/)(\w+)\%20(\w+)((\/.*)+)
Replace with: $1\u$2\%20\u$3$4
Then import the table again
Hope this help
If its MariaDB, you can do the following:
UPDATE table_name SET field = REGEXP_REPLACE(field, '\/user%20name\/', '\/User%20Name\/');
First, please check, what is actually stored in the database: %20 is a html-entity which represents a whitespace. Usually, when you are storing this inside the database, it will be represented as an actual whitespace (converted before you store it) -> Hence your replace doesn't match the actual data.
The second option that might be possible - depending on what you want to do: You are seeing the URL containing %20, therefore you created your database records (which you would like to fetch) with that additional %20 - And when you now try to query your results based on the actual url, the %20 is replaced with an "actual" whitespace (before your query) and hence it doesn't match your stored data.

Add a line break using MySQL search & replace

I just migrated a SMF forum to WordPress BBPress. The problem is video urls in posts are not on their own line so WordPress shows the url instead of displaying the video.
I'd like to do a search and replace in the database and put in a return before any YouTube url.
Example:
What do you think???https://www.youtube.com/watch?v=SbbM_v2_5wA
Would become:
What do you think???
https://www.youtube.com/watch?v=SbbM_v2_5wA
How would I do this?
Try this on a backup instance of your data first as it may not do exactly what you expect; in short the below will replace every FIELD containing the string 'https://www.youtube.com/' with the same string cut by a newline preceeding the url. If you run the code twice on your data it WILL insert a second newline, which is probably not what you want.
update TABLENAME set FIELD = concat(substring(FIELD, 1, locate('https://www.youtube.com/', FIELD)-1),'\n',substring(FIELD, locate('https://www.youtube.com/', FIELD))) where locate('https://www.youtube.com/', FIELD) > 0;
You will need to change identifiers TABLENAME and FIELD to reflect your schema.

mysql to update a database using UPDATE SET and TRIM(LEADING wildcard prefix in record

In my database I have a table called 'content' and a field called 'link' and there are almost 300,000 records in that table.
In the field called 'link' there are a number of records that look like this :
http://www.example.com/blah/blah/123456789/url=http://www.destination.com
Unfortunately the prefix part of the records are individually unique where the numbered portion is constant changing from 90 to 150 alpha-numeric characters
I would like to remove the prefix up to and/or including the url=
So that the only thing left in the record is :
http://www.destination.com OR
I could even work with
url=http://www.destination.com
and simply do a replace command against the "url=" part as a second mysql command.
If there was a wildcard command, this job would be much easier and I would just wildcard everything showing up in the link record between :
http://www.example.com/blah/blah/ wildcard url=
But as everyone knows... there is no such wildcard available
So it had me looking at the UPDATE, SET and TRIM(LEADING commands
UPDATE content
SET link =
TRIM(LEADING 'url=' FROM link)
But this DID NOT generate the changes I wanted
And so I took the labor intensive method of downloading the database and using a Search and Replace program to make the changes to the 44 thousand records that contained these parameters.
But I would love to find a command that I could simply pass to the database to make this simpler in the future.
Any thoughts on how to accomplish this change in the future would be greatly appreciated.
Thanks in advance ;
You can use the SUBSTRING_INDEX function:
UPDATE content SET link=SUBSTRING_INDEX( `link` , 'url=', -1 )
I have not tested it, so I would recommend you check that substring_index returns the desired string first.
Assuming that the part you want to keep always begins with 'http://' you could get the desired result string with the help of the SUBSTRING_INDEX function:
SELECT CONCAT('http://', SUBSTRING_INDEX(link, 'http://', -1)) FROM content;
and fix your table with the simple statement
UPDATE
content
SET
link = CONCAT('http://', SUBSTRING_INDEX(link, 'http://', -1));
Explanation:
SUBSTRING_INDEX with third parameter negative returns the substring from the last occurence of the needle in the second parameter to the end. Because 'http://' isn't included in the return value, we add it again.
Remark:
If you've got https:// urls too, you should be able to adapt my solution.

Change part of record with regular expression in mysql

I work on my site in localhost, I have a field in my database which I store absolute url of some pictures, so all records are like http://localhost/my_project/images/picture.jpg and now in my website this records not work, I should change all records to like this 'http://www.mysite.com/images/picture.jpg', so I found replace command for mysql but this command will replace all part of field:
UPDATE tablename SET tablefield = replace(tablefield, "findstring", "replacestring");
How can I change just http://localhost/ in my records with phpmyadmin ?
First, you should remove the domain name from all of your url's. Simple using /images/picture.jpg will allow it to work on all hosts.
To replace it globally, I would recommend doing a mysqldump and opening the file in a text editor, replacing the strings, and importing it back into the database. That's assuming you don't have any serialized strings (wordpress) in your database.
To simply strip the 'http://localhost' prefix, in order to use host-less absolute paths as recommended in #Rob's answer:
UPDATE tablename
SET tablefield = SUBSTR(tablefield FROM 17)
WHERE tablefield LIKE 'http://localhost/%'
See it on sqlfiddle.
To replace with 'http://www.mysite.com' as asked in the original question:
UPDATE tablename
SET tablefield = CONCAT('http://www.mysite.com', SUBSTR(tablefield FROM 17))
WHERE tablefield LIKE 'http://localhost/%'
See it on sqlfiddle.
Of course, neither of these answers use regular expressions as requested in your question title (although they do use simple pattern matching); to use regular expression pattern replacement, you will need to install and use a UDF such as provided by lib_mysqludf_preg.

Format list of urls in mysql

I have a list of a million or urls in an mysql table.
I need to cleanse the data (extract domains) so I can be confident about DISTINCT type queries.
Data is in several different types: -
www.domain.tld
domain.tld
http://domain.tld
https://vhost.domain.tld
domain.tld/
There are invalid domains and empty data.
Ideally I'd like to do something along the lines of : -
UPDATE table1 SET domain = website REGEXP '^(https?://)?[a-zA-Z0-9\\\\.\\\\-]+(/|$|\\\\?)'
domain being a new empty field, website being the original url.
You can't use regex like that in MySQL as is, but apparently you can some some UDFs that implement it. See:
How to do a regular expression replace in MySQL?
https://launchpad.net/mysql-udf-regexp
http://www.mysqludf.org/lib_mysqludf_preg/