I am trying to transfer a list of domain names from an old system to a newer one.
The problem is the the data in the old database was used as a reference and contains additional information but the new system will integrate with cpanel and thus the domain has to be correct.
I am trying to automate the import of the old data that does conform to my requirements and leave aside for manual import the ~4% that does not.
I have used a regular expression to achieve this but for some reason it is not working as I expect it.
This is the condition I use:
`domain` REGEXP '^[\.A-Za-z0-9\-]+\\.[a-zA-Z]{2,4}$' = 1
It correctly identifies the following as not being valid:
https://test-1.example.com:8443/login_up.php3
118.18.187.15
But fails for the wollowing:
the-example.com mchannel
example.com NEW
I know regex decently well but I can't figure out why in this case it does not work.
Fiddle URL: http://sqlfiddle.com/#!2/a9d70/5
Example what should validate: http://www.regexr.com/39f4v
This regex should match everything from the first regex and the ips
[[:<:]][\.A-Za-z0-9\-]+\.[a-zA-Z]{2,4}[[:>:]]|[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+\.[[:digit:]]+
Related
I am using latest version of Server version: 8.0.11 MySQL Community Server - GPL. Download link
Table structure is web_url < id, url >
Sample Data in the url column.
www.google.com
www.yahoo.com
how to extract domain name only like google and yahoo using the function
REGEXP_REPLACE. so it's meaning is the replace the www. and .com part be replace by space or with ''. if you have any other solution then you are welcome.
I am either missing some. So help.
I have written the regular expression but don't getting how use it properly in query. Please give query base answer.
my query
select REGEXP_REPLACE(url,'^(www.)[a-zA-z0-9]*(.[a-zA-Z]{3})(\/[a-zA-Z09-]*)*','') from web_url;
So far i done this query.
So far i done this may be someone get help
done this query to test for comparison is happening or not.
kindly ignore last two rows as they are just for testing purpose of regexp_like function
select regexp_like(url,'^(www.)(.*)(.com)$') from web_url;
If url is misleading and is in fact an FQDN, what the comments suggest,
reverse(regexp_replace(reverse(regexp_replace(url, '\\.[^\\.]*\\.?$', '')), '\\..*', ''))
should give you the second label from the right (or the third if there is the optional empty label at the end), what seems to be what you want.
The reverse() are necessary since we don't know how many dots are in the string. Because of the reverse() after we've removed the the last (or the last two) labels, we can simply remove from the first dot on.
If there were capture groups with regexp_replace() it would be easier, but I didn't find anything about that in the documentation. Please comment and point me to that, if I just missed that.
not sure how far I'm going to get with this, but I'm going through a database removing certain bits and pieces in preparation for a conversion to different software.
I'm struggling with the image tags as on the site they currently look like
[img:<string>]<image url>[/img:<string>]
those strings are in another field called bbcode_uid
The query I'm running to make the changes so far is
UPDATE phpbb_posts SET post_text = REPLACE(post_text, '[img:]', '');
So my actual question, is there any way of pulling in each string from bbcode_uid inside of that SQL query so that I don't have to run the same command 10,000+ times, changing the unique string every time.
Alternatively could I include something inside [img:] to also include the next 8 characters, whatever they may be, as that is the length of the string that is used.
Hoping to save time with this, otherwise I might have to think of another way of doing it.
As requested.
The text I wish to replace would be
[img:1nynnywx]http://i.imgur.com/Tgfrd3x.jpg[/img:1nynnywx]
I want to end up with just
http://i.imgur.com/Tgfrd3x.jpg
Just removing the code around the URL, however each post_text has a different string which is contained inside bbcode_uid.
Method 1
LIB_MYSQLUDF_PREG
If you want more regular expression power in your database, you can consider using LIB_MYSQLUDF_PREG. This is an open source library of MySQL user functions that imports the PCRE library. LIB_MYSQLUDF_PREG is delivered in source code form only. To use it, you'll need to be able to compile it and install it into your MySQL server. Installing this library does not change MySQL's built-in regex support in any way. It merely makes the following additional functions available:
PREG_CAPTURE extracts a regex match from a string. PREG_POSITION returns the position at which a regular expression matches a string. PREG_REPLACE performs a search-and-replace on a string. PREG_RLIKE tests whether a regex matches a string.
All these functions take a regular expression as their first parameter. This regular expression must be formatted like a Perl regular expression operator. E.g. to test if regex matches the subject case insensitively, you'd use the MySQL code PREG_RLIKE('/regex/i', subject). This is similar to PHP's preg functions, which also require the extra // delimiters for regular expressions inside the PHP string
you can refer this link :github.com/hholzgra/mysql-udf-regexp
Method 2
Use php program, fetch records one by one , use php preg_replace
refer : www.php.net/preg_replace
reference:http://www.online-ebooks.info/article/MySql_Regular_Expression_Replace.html
You might be able to do this with substring_index().
The following will work on your example:
select substring_index(substring_index(post_text, '[/img:', 1), ']', -1)
I'm writing a PHP program. Each content has some options which stores users preferences. For example:
Where the content should be shown in the template?
In which categories or pages, the content should be loaded?
Now, I store these data in a single field named options as JSON. the final result is something like:
{
"locales":["en"],
"themes":{
"default":{
"width":"0",
"height":"0",
"top":"0",
"right":"0",
"bottom":"0",
"left":"0"
}
},
"pages":["aboutus\/"],
"categories":["all"],
"homepage":"false"
}
I have no problem to select data and filter rows using mysql regexp statement. But I have 3 other questions:
How to update a JSON field value in a single update statement?
Does it make sense to do such a trick at all (I mean using JSON)?
What other solutions do you recommend to store options, while there a lots of property to be stored?
AFAIK, there is no absolute way to do that in none-NoSQL database like MySQL. However, not exactly for JSON, you can see how FriendFeed uses MySQL to store schema-less data and it may give you some ideas.
UPDATE:
MySQL 5.7.8 and newer versions are now supporting JSON data type. There are some tutorials to help.
I have a list of a million or urls in an mysql table.
I need to cleanse the data (extract domains) so I can be confident about DISTINCT type queries.
Data is in several different types: -
www.domain.tld
domain.tld
http://domain.tld
https://vhost.domain.tld
domain.tld/
There are invalid domains and empty data.
Ideally I'd like to do something along the lines of : -
UPDATE table1 SET domain = website REGEXP '^(https?://)?[a-zA-Z0-9\\\\.\\\\-]+(/|$|\\\\?)'
domain being a new empty field, website being the original url.
You can't use regex like that in MySQL as is, but apparently you can some some UDFs that implement it. See:
How to do a regular expression replace in MySQL?
https://launchpad.net/mysql-udf-regexp
http://www.mysqludf.org/lib_mysqludf_preg/
In field post_content I have a string like this in nearly 800 rows:
http://somesite.com/">This is some site</a>
I need to remove everything from "> onwards so that it leaves just the URL. I can't do a straight find and replace because the text is unique.
Any clues? This is really my first foray into MySQL database modifications but I did do an extensive search before posting here.
Thanks,
~Kyle~
From this site: http://www.regular-expressions.info/mysql.html
LIB_MYSQLUDF_PREG
If you want more regular expression power in your database, you can consider using LIB_MYSQLUDF_PREG. This is an open source library of MySQL user functions that imports the PCRE library. LIB_MYSQLUDF_PREG is delivered in source code form only. To use it, you'll need to be able to compile it and install it into your MySQL server. Installing this library does not change MySQL's built-in regex support in any way. It merely makes the following additional functions available:
Here it comes...
PREG_CAPTURE extracts a regex match from a string. PREG_POSITION returns the position at which a regular expression matches a string. PREG_REPLACE performs a search-and-replace on a string. PREG_RLIKE tests whether a regex matches a string.
Sounds exactly what you're looking for.
All these functions take a regular expression as their first parameter. This regular expression must be formatted like a Perl regular expression operator. E.g. to test if regex matches the subject case insensitively, you'd use the MySQL code PREG_RLIKE('/regex/i', subject). This is similar to PHP's preg functions, which also require the extra // delimiters for regular expressions inside the PHP string.
See this post: How to do a regular expression replace in MySQL?
Either that or you could just write a script in any lanugage which goes through each record, does a regex replacement and then updates the field. For more info on regex, see here: http://www.regular-expressions.info/reference.html
There's a number of options. One might be to use SUBSTRING_INDEX():
UPDATE
table
SET field = SUBSTRING_INDEX( field, '">', 1 )
It's possible - there is a syntax for User Defined Functions which would let you pass in a regular expression pattern that matches the link and strips everything else.
However, this is quite complicated for somebody new to MySQL, and from your question, this sounds like a one-off. In which case - why not just use Excel and then reimport the data?
Great stuff!
All seems doable with a little bit of time and self education.
In the end, I exported that table as a CSV in Sequel Pro and did some nifty find and replace work in Coda. Not as sophisticated as your suggestions, but it worked.
Thanks again,
~Kyle~