Related
I have a table of UK postcodes. All of them are in different format, some are capitalized with white space some are not. What I want to do is format them so they can follow the UK postcode standard. For instance AB1 2BB.
I used this query for the purpose which does work, but some postcodes have a longer or shorter first part so it does not succeed for all.
SELECT UPPER(INSERT((REPLACE(postcode , ' ', '')) , 4, 0, ' ')) AS postcode
However if I try to do it the other way around
SELECT UPPER(INSERT((REPLACE(postcode , ' ', '')) , -4, 0, ' ')) AS postcode
It does not work and returns all the postcodes glued together e.g AB12BB
What I want is to put a space before the last 3 characters.
I think you want:
select concat_ws(' ',
left(replace(postcode, ' ', ''), 3),
right(replace(postcode, ' ', ''), 3
) as standardized_postcode
Insert a space at the 3th char from the end of the string, after you remove all the spaces:
SELECT UPPER(INSERT(REPLACE(postcode , ' ', ''), LENGTH(REPLACE(postcode , ' ', '')) - 2, 0, ' ')) AS postcode
It sounds like you're going to be dealing with postcodes like this:
LS10 1DH
LS101DH
LS63DR
etc. We should start by removing all of the spaces:
REPLACE(postcode,' ','') -- LS10 1DH becomes LS101DH
taking the last 3 characters:
RIGHT(REPLACE(postcode,' ',''), 3) -- 1DH
and all of the characters up to the 3rd from last:
LEFT(REPLACE(postcode,' ',''), LEN(REPLACE(postcode,' ','')) - 3))
Then use CONCAT to bring it all together:
SET #postcode = 'LS101DH';
SELECT CONCAT(LEFT(REPLACE(#postcode,' ',''), LENGTH(REPLACE(#postcode,' ','')) - 3),
' ', -- add a space in
RIGHT(REPLACE(#postcode,' ',''), 3));
for pc in postcodes:
print('{} {}'.format(pc[:-3], pc[-3:]))
Seems to work for a given list of UK postcodes for me.
I have around 200,000 records of data with phone numbers, but the numbers are inconsistent.
for example, some may be 10 digits (missing a 0 at the beginning), some have spaces in there, some have a '-' in the middle and some begin with '+44' instead of 0.
Is there a way in mySQL to condition all these and cleanse the data in one query?
Without sample data and without an example output this is purely speculative and assuming you want the output in the format of 01234567891.
Use a combination of LENGTH, REPLACE' ANDLEFT` functions to resolve the 4 issues you highlighted:
Missing 0 at beggining.
Spaces in the string.
-'s in the string.
+44 rather than 0.
SELECT CASE WHEN LENGTH(REPLACE(REPLACE(numberfield, '-', ''), ' ', '')) = 10
THEN CONCAT('0', REPLACE(REPLACE(numberfield, '-', ''), ' ', ''))
WHEN LEFT(REPLACE(REPLACE(numberfield, '-', ''), ' ', ''), 3) = '+44'
THEN REPLACE(REPLACE(REPLACE(numberfield, '-', ''), ' ', ''), '+44', '0'
END AS Cleannumber
FROM yourtable
Assuming the phone number field is a string - the following should deal with the conditions you specified :
RIGHT( LPAD( REPLACE( REPLACE( REPLACE('phonenumber', '-', ''), '+44', ''), ' ', ''), 11, '0' ), 11 )
first any '-' are removed, then '+44' is removed, then spaces are removed, then 11 '0's are added to the start of the number, finally the rightmost 11 characters are taken.
So you would do an UPDATE query replacing the phonenumber column.
Currently I'm working on a database redesign project. A large bulk of this project is pulling data from the old database and importing it into the new one.
One of the columns in a table from the old database is called 'name'. It contains a forename and a surname all in one field (ugh). The new table has two columns; forenames and surname. I need to come up with a clean, efficient way to split this single column into two.
For now I'd like to do everything in the same table and then I can easily transfer it across.
3 columns:
Name (the forename and surname)
Forename (currently empty, first half of name should go here)
Surname (currently empty, second half of name should go here)
What I need to do: Split name in half and place into forename and surname
If anyone could shed some light on how to do this kind of thing I would really appreciate it as I haven't done anything like this in SQL before.
Database engine: MySQL
Storage engine: InnoDB
A quick solution is to use SUBSTRING_INDEX to get everything at the left of the first space, and everything past the first space:
UPDATE tablename
SET
Forename = SUBSTRING_INDEX(Name, ' ', 1),
Surname = SUBSTRING_INDEX(Name, ' ', -1)
Please see fiddle here. It is not perfect, as a name could have multiple spaces, but it can be a good query to start with and it works for most names.
Try this:
insert into new_table (forename, lastName, ...)
select
substring_index(name, ' ', 1),
substring(name from instr(name, ' ') + 1),
...
from old_table
This assumes the first word is the forename, and the rest the is lastname, which correctly handles multi-word last names like "John De Lacey"
For the people who wants to handle fullname: John -> firstname: John, lastname: null
SELECT
if( INSTR(`name`, ' ')=0,
TRIM(SUBSTRING(`name`, INSTR(`name`, ' ')+1)),
TRIM(SUBSTRING(`name`, 1, INSTR(`name`, ' ')-1)) ) first_name,
if( INSTR(`name`, ' ')=0,
null,
TRIM(SUBSTRING(`name`, INSTR(`name`, ' ')+1)) ) last_name
It works fine with John Doe. However if user just fill in John with no last name, SUBSTRING(name, INSTR(name, ' ')+1)) as lastname will return John instead of null and firstname will be null with SUBSTRING(name, 1, INSTR(name, ' ')-1).
In my case I added if condition check to correctly determine lastname and trim to prevent multiple spaces between them.
This improves upon the answer given, consider entry like this "Jack Smith Smithson", if you need just first and last name, and you want first name to be "Jack Smith" and last name "Smithson", then you need query like this:
-- MySQL
SELECT
SUBSTR(full_name, 1, length(full_name) - length(SUBSTRING_INDEX(full_name, ' ', -1)) - 1) as first_name,
SUBSTRING_INDEX(full_name, ' ', -1) as last_name
FROM yourtable
Just wanted to share my solution. It also works with middle names. The middle name will be added to the first name.
SELECT
TRIM(SUBSTRING(name,1, LENGTH(name)- LENGTH(SUBSTRING_INDEX(name, ' ', -1)))) AS firstname,
SUBSTRING_INDEX(name, ' ', -1) AS lastname
I had a similar problem but with Names containing multiple names, eg. "FirstName MiddleNames LastName" and it should be "MiddleNames" and not "MiddleName".
So I used a combo of substring() and reverse() to solve my problem:
select
SystemUser.Email,
SystemUser.Name,
Substring(SystemUser.Name, 1, instr(SystemUser.Name, ' ')) as 'First Name',
reverse(Substring(reverse(SystemUser.Name), 1, instr(reverse(SystemUser.Name), ' '))) as 'Last Name',
I do not need the "MiddleNames" part and maybe this is not the most efficient way to solve it, but it works for me.
Got here from google, and came up with a slightly different solution that does handle names with more than two parts (up to 5 name parts, as would be created by space character). This sets the last_name column to everything to the right of the 'first name' (first space), it also sets full_name to the first name part. Perhaps backup your DB before running this :-) but here it is it worked for me:
UPDATE users SET
name_last =
CASE
WHEN LENGTH(SUBSTRING_INDEX(full_name, ' ', 1)) = LENGTH(full_name) THEN ''
WHEN LENGTH(SUBSTRING_INDEX(full_name, ' ', 2)) = LENGTH(full_name) THEN SUBSTRING_INDEX(del_name, ' ', -1)
WHEN LENGTH(SUBSTRING_INDEX(full_name, ' ', 3)) = LENGTH(full_name) THEN SUBSTRING_INDEX(del_name, ' ', -2)
WHEN LENGTH(SUBSTRING_INDEX(full_name, ' ', 4)) = LENGTH(full_name) THEN SUBSTRING_INDEX(del_name, ' ', -3)
WHEN LENGTH(SUBSTRING_INDEX(full_name, ' ', 5)) = LENGTH(full_name) THEN SUBSTRING_INDEX(del_name, ' ', -4)
WHEN LENGTH(SUBSTRING_INDEX(full_name, ' ', 6)) = LENGTH(full_name) THEN SUBSTRING_INDEX(del_name, ' ', -5)
ELSE ''
END,
full_name = SUBSTRING_INDEX(full_name, ' ', 1)
WHERE LENGTH(name_last) = 0 or LENGTH(name_last) is null or name_last = ''
SUBSTRING_INDEX didn't work for me in SQL 2018, so I used this:
declare #fullName varchar(50) = 'First Last1 Last2'
declare #first varchar(50)
declare #last varchar(50)
select #last = right(#fullName, len(#fullName)-charindex(' ',#fullName, 1)), #first = left(#fullName, (charindex(' ', #fullName, 1))-1);
Yields #first = 'First', #last = 'Last1 Last2'
I use Oracle and MySql, so if you have any answer in both codes please let me know,
Issue:
I have 2 columns in one table called: USers
Column#1= Names
Column#2= UCNames
This list contain names that are from different sources but partially match like:
Names
Alex Jones Marfex
UCNames
Alex Jonnes Mike Marfex
I want to compare both of the columns and find match based on the following attributes:
Search on the first 4 for letters and 4 last words and to store in new column called: verifiyed
Thanks
This gives you the first word in a string
Substring(Col,1,(Locate (' ',Col + ' ')-1)) First
This gives you the last word in a string
Reverse(Substring(Reverse(Col), 1, Locate(' ',Reverse(Col)) - 1)) Last
So your compare could be
Where
Substring(Col1,1,( Locate(' ',Col1 + ' ')-1))
= Substring(Col2,1,( Locate(' ',Col2 + ' ')-1))
And
Reverse(Substring(Reverse(Col1), 1, Locate(' ',Reverse(Col1)) - 1))
= Reverse(Substring(Reverse(Col2), 1, Locate(' ',Reverse(Col2)) - 1))
I went with words which seemed safer for what you are trying to do with just a bit more effort. If you do want to keep the 4 characters, just replace the Locate es with 4.
Based on the answer by #asantaballa , but using MySQL SUBSTRING_INDEX function :-
WHERE SUBSTRING_INDEX(Col1, ' ', 1) = SUBSTRING_INDEX(Col2, ' ', 1)
AND SUBSTRING_INDEX(Col1, ' ', -1) = SUBSTRING_INDEX(Col2, ' ', -1)
or to crudely check the first 4 characters and the last 4 words:-
WHERE SUBSTRING(Col1, 1, 4) = SUBSTRING(Col2, 1, 4)
AND SUBSTRING_INDEX(Col1, ' ', -4) = SUBSTRING_INDEX(Col2, ' ', -4)
I am trying to extract a certain part of a column that is between delimiters.
e.g. find foo in the following
test 'esf :foo: bar
So in the above I'd want to return foo, but all the regexp functions only return true|false,
is there a way to do this in MySQL
Here ya go, bud:
SELECT
SUBSTR(column,
LOCATE(':',column)+1,
(CHAR_LENGTH(column) - LOCATE(':',REVERSE(column)) - LOCATE(':',column)))
FROM table
Yea, no clue why you're doing this, but this will do the trick.
By performing a LOCATE, we can find the first ':'. To find the last ':', there's no reverse LOCATE, so we have to do it manually by performing a LOCATE(':', REVERSE(column)).
With the index of the first ':', the number of chars from the last ':' to the end of the string, and the CHAR_LENGTH (don't use LENGTH() for this), we can use a little math to discover the length of the string between the two instances of ':'.
This way we can peform a SUBSTR and dynamically pluck out the characters between the two ':'.
Again, it's gross, but to each his own.
This should work if the two delimiters only appear twice in your column. I am doing something similar...
substring_index(substring_index(column,':',-2),':',1)
A combination of LOCATE and MID would probably do the trick.
If the value "test 'esf :foo: bar" was in the field fooField:
MID( fooField, LOCATE('foo', fooField), 3);
I don't know if you have this kind of authority, but if you have to do queries like this it might be time to renormalize your tables, and have these values in a lookup table.
With only one set of delimeters, the following should work:
SUBSTR(
SUBSTR(fooField,LOCATE(':',fooField)+1),
1,
LOCATE(':',SUBSTR(fooField,LOCATE(':',fooField)+1))-1
)
mid(col,
locate('?m=',col) + char_length('?m='),
locate('&o=',col) - locate('?m=',col) - char_length('?m=')
)
A bit compact form by replacing char_length(.) with the number 3
mid(col, locate('?m=',col) + 3, locate('&o=',col) - locate('?m=',col) - 3)
the patterns I have used are '?m=' and '&o'.
select mid(col from locate(':',col) + 1 for
locate(':',col,locate(':',col)+1)-locate(':',col) - 1 )
from table where col rlike ':.*:';
If you know the position you want to extract from as opposed to what the data itself is:
$colNumber = 2; //2nd position
$sql = "REPLACE(SUBSTRING(SUBSTRING_INDEX(fooField, ':', $colNumber),
LENGTH(SUBSTRING_INDEX(fooField,
':',
$colNumber - 1)) + 1)";
This is what I am extracting from (mainly colon ':' as delimiter but some exceptions), as column theline255 in table loaddata255:
23856.409:0023:trace:message:SPY_EnterMessage (0x2003a) L"{#32769}" [0081] WM_NCCREATE sent from self wp=00000000 lp=0023f0b0
This is the MySql code (It quickly did what I want, and is straight forward):
select
time('2000-01-01 00:00:00' + interval substring_index(theline255, '.', 1) second) as hhmmss
, substring_index(substring_index(theline255, ':', 1), '.', -1) as logMilli
, substring_index(substring_index(theline255, ':', 2), ':', -1) as logTid
, substring_index(substring_index(theline255, ':', 3), ':', -1) as logType
, substring_index(substring_index(theline255, ':', 4), ':', -1) as logArea
, substring_index(substring_index(theline255, ' ', 1), ':', -1) as logFunction
, substring(theline255, length(substring_index(theline255, ' ', 1)) + 2) as logText
from loaddata255
and this is the result:
# LogTime, LogTimeMilli, LogTid, LogType, LogArea, LogFunction, LogText
'06:37:36', '409', '0023', 'trace', 'message', 'SPY_EnterMessage', '(0x2003a) L\"{#32769}\" [0081] WM_NCCREATE sent from self wp=00000000 lp=0023f0b0'
This one looks elegant to me. Strip all after n-th separator, rotate string, strip everything after 1. separator, rotate back.
select
reverse(
substring_index(
reverse(substring_index(str,separator,substrindex)),
separator,
1)
);
For example:
select
reverse(
substring_index(
reverse(substring_index('www.mysql.com','.',2)),
'.',
1
)
);
you can use the substring / locate function in 1 command
here is a mice tutorial:
http://infofreund.de/mysql-select-substring-2-different-delimiters/
The command as describes their should look for u:
**SELECT substr(text,Locate(' :', text )+2,Locate(': ', text )-(Locate(' :', text )+2)) FROM testtable**
where text is the textfield which contains "test 'esf :foo: bar"
So foo can be fooooo or fo - the length doesnt matter :).