I'm working on a WordPress-based project and I need to batch edit the posts from the database.
At the moment I have cells, where the content is Text <pre> Text\r\nText\r\nText </pre>
What I need to do is find all the \r\n strings and replace them with \r\n\r\n. So far I have this:
UPDATE `table_name`
SET `field_name` = replace(field_name, '\r\n', '\r\n\r\n\')
The issue is, there is also \r\n\ text outside the <pre> tags, which I don't want to affect. That's why I need to do something like find (start: '<pre>' end: '</pre>') before calling the replace. I just have no idea how to do this in MySQL.
When web searching, I found some stuff about regex but I have no idea how that works.
MySQL doesn't have the regexp replace functionality. I'd say you have two options - either create an user defined function that will do this kind of replacement, or, if you can't / don't know how to, you have to use a script/program written in a different language. It may be e.g. a PHP script that will connect to the db and do the desired replacement. Another way would be to dump the table containing your posts to a file, use sed or something similar and import the data once again.
Using sed would probably take less time than PHP and the syntax is fairly easy to learn.
Whatever you choose remember it's always a good idea to make a backup before.
Related
I have many pages (posts) in a WP site with legacy short code that needs to be removed and cannot accomplish what I need using WordPress find/replace plug-ins - so I'm turning to MySQL queries.
The shortcodes to be removed all follow the same pattern: "[nivo... (variable content) .../nivo]
I need to remove the entire nivo shortcode and ONLY the nivo shortcode because other shortcodes exist on some pages.
I found something very close to what I think I need, and modified the obvious parameters for this particular application as follows...
UPDATE `post_content`
REPLACE(txt, SUBSTRING(txt, LOCATE('[nivo', txt), LENGTH(txt) - LOCATE('nivo]', REVERSE(txt)) - LOCATE('nivo]', txt) + 10), '')
WHERE txt LIKE '%(%)%'
That ^^^ is accepted in the SQL query window (no stop signs) but returns a #1064 error when executed.
Ideally I would like to TEST this first on a specific post ID just to be sure it's really catching everything but I couldn't figure out how to write that into the query.
I know VERY little about MySQL (I'm a designer) but I have DB backups ready for rollback just in case.
Help would be greatly appreciated.
The key is that you want to use a plugin or tool that allows you to use regex (regular expressions). I have used this Php tool, you just put it in your public_html and then access the path via a web browser: https://interconnectit.com/search-and-replace-for-wordpress-databases/
Just use a little regex in the Php tool or find a find and replace plugin that allows you to use regex. It would look something like this:
\[nivo.*?/nivo\]
\ means escaping a character, since the bracket has a special meaning in regex.
. means any character
* means any number of characters
? means 0 or more
.*? together means anything or nothing
For reference (and probably a better answer):
PHP - Remove Shortcodes and Content in between with Regex Pattern
I have a MySQL 5.7.29 database on which a website is built. I need to construct a query to find all table rows containing lines such as
https://example.com/index.php?topic=7989.0
or similar and replace them with
https://example.com/my-redirect-page/?7989.0
The wildcard here is the ?topic=7989.0 as this can be something like ?topic=1234 or even ?topic=3456.0#anchor
I can display the rows they appear in (in PHPMyAdmin) using this (thanks to 'sticky bit' below) :
SELECT * FROM `abc_posts` WHERE `post_content` LIKE '%example.com\index.php?topic%'
My problem is that I then need to change just that URL when there is also text content around it.
Thanks in advance.
The question mark doesn't need to be escaped.
But 'https://example.com/index.php?topic=7989.0' isn't LIKE '%example.com\?topic%' as the question mark doesn't immediately follow the host name.
Try:
...
post_content LIKE '%example.com/index.php?topic%'
...
You could do something like thin to find them
SELECT 'https://example.com/index.php?topic=7989.0'
WHERE 'https://example.com/index.php?topic=7989.0' REGEXP 'example.com/index.php?topic=';
Which would find all rows. But for replacing it, you must also tell which database version youh have mysql 5.x have not many regex fucntions mariadb and mysql 8 have much more
I need something like this:
Attivato;Nome;Categorie;Prezzo tasse escluse;Descrizione;Immagini
1;"Bracciale rock";11,12,13;130;"This is a long description.";http://s20.postimg.org/r08w8i4i5/perle.jpg,http://s20.postimg.org/tmjtbp6bx/bracciale.jpg
But if I open it with neooffice calc (or anyway in some spreadsheet program) it then export like this, at the best:
Attivato;Nome;Categorie;Prezzo tasse escluse;Descrizione;Immagini
1;"Bracciale rock";"11,12,13";130;"This is a long description.";"http://s20.postimg.org/r08w8i4i5/perle.jpg,http://s20.postimg.org/tmjtbp6bx/bracciale.jpg"
It won't retain things like 11,12,13 without converting them to strings
How can I fix this?
I tried really everything but no way... Tried any kind of import/export options, different programs, etc... I cannot do it.
I finally found a couple of ways.
(1) In neooffice/openoffice:
left empty the text separator field
check the 'detect special numbers' option
(2) As another alternative, I used google docs. That's good also to use for clients who usually use exel. Google docs/drive saves csv files in utf-8 encoding by default.
Also, for the problem of "string conversion" of multiple values like 5,7,9,6, I found that if you use semicolons (;) instead of commas, that works (I mean, it doesn't add "" when you save as csv AND doesn't read them as dates or other wrong data types). And in prestashop you can set the field and test separators accordingly.
Hope it helps other people.
Somewhere along the way, between all the imports and exports I have done, a lot of the text on a blog I run is full of weird accented A characters.
When I export the data using mysqldump and load it into a text editor with the intention of using search-and-replace to clear out the bad characters, searching just matches every "a" character.
Does anyone know any way I can successfully hunt down these characters and get rid of them, either directly in MySQL or by using mysqldump and then reimporting the content?
This is an encoding problem; the  is a non-breaking space (HTML entity ) in Unicode being displayed in Latin1.
You might try something like this... first we check to make sure the matching is working:
SELECT * FROM some_table WHERE some_field LIKE BINARY '%Â%'
This should return any rows in some_table where some_field has a bad character. Assuming that works properly and you find the rows you're looking for, try this:
UPDATE some_table SET some_field = REPLACE( some_field, BINARY 'Â', '' )
And that should remove those characters (based on the page you linked, you don't really want an nbsp there as you would end up with three spaces in a row between sentences etc, you should only have one).
If it doesn't work then you'll need to look at the encoding and collation being used.
EDIT: Just added BINARY to the strings; this should hopefully make it work regardless of encoding.
The accepted answer did not work for me.
From here http://nicj.net/mysql-converting-an-incorrect-latin1-column-to-utf8/ I have found that the binary code for  character is c2a0 (by converting the column to VARBINARY and looking what it turns to).
Then here http://www.oneminuteinfo.com/2013/11/mysql-replace-non-ascii-characters.html found the actual solution to remove (replace) it:
update entry set english_translation = unhex(replace(hex(english_translation),'C2A0','20')) where entry_id = 4008;
The query above replaces it to a space, then a normal trim can be applied or simply replace to '' instead.
I have had this problem and it is annoying, but solvable. As well as  you may find you have a whole load of characters showing up in your data like these:
“
This is connected to encoding changes in the database, but so long as you do not have any of these characters in your database that you want to keep (e.g. if you are actually using a Euro symbol) then you can strip them out with a few MySQL commands as previously suggested.
In my case I had this problem with a Wordpress database that I had inherited, and I found a useful set of pre-formed queries that work for Wordpress here http://digwp.com/2011/07/clean-up-weird-characters-in-database/
It's also worth noting that one of the causes of the problem in the first place is opening a database in a text editor which might change the encoding in some way. So if you can possibly manipulate the database using MySQL only and not a text editor this will reduce the risk of causing further trouble.
Every week, I have to completely replace the data in several very large MySQL tables. So what I normally do is delete the existing data, import the new data, and then run my usual queries to modify the new data as needed.
Unfortunately, these days I have noticed that the new data contains unwanted characters, such as quotes and extra spaces. With well over 100,000 records in some of these tables (AFAIK), I cannot easily open the data in notepad to strip out unwanted characters, prior to importing.
I realize I could write a separate find and replace query for every single column in every table, like this:
UPDATE mytablename SET mycolumn = REPLACE(mycolumn, '"', '');
But having to name every column is a bother. Anyway, I would like to find a more elegant solution. Today, I found a snippet on the internet that looks like a start:
SELECT * FROM INFORMATION_SCHEMA.COLUMNS WHERE
table_name = 'myTable' and ordinal_position = 1
I think the next step might be to loop through the ordinal positions, and then replace and update each column, but I don't know how to do this in MySQL. I also don't know how to stop the loop after the last column is reached, to avoid error messages.
Is there an easy way to do this? Or am I hoping for too much?
I am a beginner, so a clear, simple explanation would be much appreciated.
Thanks in advance.
MORE INFORMATION:
Since my first post, I have discovered that stored procedures are not allowed on my server. Too bad.
Anyway, I have tried this new code, just to get started:
set #mytablestring='mytable';
set #mycolumnnumber=1;
set #mycolumnname=(SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE table_name = #mytablestring and ordinal_position = #mycolumnnumber);
SELECT #mycolumnname FROM mytable;
Unfortunately, in the final SELECT query, #mycolumnname is interpreted as a string, not as a column name. So the query does not work. If I could get past this, I believe I could write some code to loop through the columns by incrementing #mycolumnnumber.
If anyone knows how to solve this, I would really appreciate some help.
Many thanks.
I suggest that you take a look at vim, sed, awk and many of the other text editors and text processing utilities that you can find on Linux (and sometimes on Windows too). 100,000 records may be a pain in Notepad, but it's a piece of cake for real text processing utilities.
For example, to strip all # characters from foobar.txt:
sed 's/#//g' foobar.txt > foobar-clean.txt
Or, the same thing with the file opened in (g)vim:
:%s/#//g