I have a MySQL column that contains phone numbers, the problem is that they're in different formats, such as:
2125551212
212-555-1212
(212)5551212
I'd like to know if it's possible to take the existing 10 digits, remove the formatting, and change them all to this format: (212) 555-1212
Not a duplicate, as I'm looking to update several thousand entries instead of masking new entries.
Unfortunately, no REGEXP_MATCHES() or TRANSLATE() function comes with standard MySQL installation (they do with Postgres), so you could do this a way which I find really dirty, but it works.
First you cleanse your column by removing characters that aren't numbers using replace()
Then you take several parts of the string to separate them out using substr()
Finally, you concatenate them adding symbols between your substrings with concat()
If you have any more characters that you need truncate, just add another replace() on top of 3 already existing.
Sample data
create table nums ( num text );
insert into nums values
('2125551212'),
('212-555-1212'),
('(212)5551212');
Query formatting your data
select
num,
concat('(',substr(num_cleansed,1,3),') ',substr(num_cleansed,4,3),'-',substr(num_cleansed,7)) AS num_formatted
from (
select
num,
replace(replace(replace(num,'(',''),')',''),'-','') as num_cleansed
from nums
) foo
Result
num num_formatted
2125551212 (212) 555-1212
212-555-1212 (212) 555-1212
(212)5551212 (212) 555-1212
Click here SQLFiddle to preview output.
I'm leaving UPDATE statement as a homework for the reader.
Related
Please look at my screenshots and help me to understand what I am missing.
What datatype should I choose for these columns in MYSQL? I keep getting mistakes in decimal datatype columns. I chose decimаl12,3 because no columns(revenue, product&purchase price) with currency have more than 12 digits in total, 9 before and 3 after the decimal point. Could someone help me to understand what data type to choose with examples?
if we have an integer number e.g. 85192 we choose int?
for currency we choose the decimal, right? then what have I done wrong that I keep getting errors? 0 records imported.
if we have a combination of numbers and letters or just letters then we choose varchar? and varchаr1 equals 1 character, eg. apple32 = 7 characters, therefore vаrchar7?
turning to decimal, 12,464.87 in total 7 digits, 5 before and 2 after the decimal point, hence mysql decimаl7,2 should be enough, right? or would it be better to put decimаl10,3 with a margin so to say.
excel
mysql
data
$1,000.00 contains two characters that cannot be part of a numeric literal: the dollar sign and the comma that is used as a thousands separator.
Find a way to change '$1,000.00' to '1000.00' in the input file. Then, the load will succeed.
Alternatively, create an intermediate table where product_price is a VARCHAR(32), load into that, and then:
INSERT INTO target_table
SELECT
other_col1
,other_col2
, ....
,CAST(REPLACE(REPLACE(product_price,',',''),'$','') AS DECIMAL(15,2)
,other_col_n
,...
FROM staging_table;
You don't need an intermediate table. When doing LOAD DATA, put and columns into #variables; then use a SET to convert as needed:
LOAD DATA
...
col1, col2, #price, ...,
SET price = CAST(REPLACE(REPLACE(product_price,',',''),'$','') AS DECIMAL(15,2))
Dates need to be like this: "2022-07-25 22:02:22". Either change what Excel is delivering, or use STR_TO_DATE(...) in the SET.
Firstly I want this to be purely done with MySQL query.
I have a series of Invoice numbers
invoice_number
INV001
INV002
INV003
INV004
INV005
001
002
003
006
007
009
010
INVOICE333
INVOICE334
INVOICE335
INVOICE337
INVOICE338
INVOICE339
001INV
002INV
005INV
009INV
I want to output something like this
from_invoice_no to_invoice_no total_invoices
INV001 INV005 5
001 010 7
INVOICE333 INVOICE339 6
001INV 009INV 4
The invoice number pattern cannot be fixed. They can change in future
Please help me to achieve this.
Thanks in advance.
I will first show a general idea how to solve this problem and provide some code which will be ugly, but easily understandable. Then I'll explain what the issues are and how to remedy them.
STEP 1: Deriving the grouping criterion
For the first step, I assume you have the right (privilege) to create an additional column in your table. Let us name it invoice_text. Now, the general idea is to remove all digits from the invoice number so that only the "text pattern" remains. Then we can group by the text pattern.
Assuming that you have already created the column mentioned above, you could do the following:
UPDATE Invoices SET invoice_text = REPLACE(invoice_number, '0', '');
UPDATE Invoices SET invoice_text = REPLACE(invoice_text, '1', '');
UPDATE Invoices SET invoice_text = REPLACE(invoice_text, '2', '');
...
UPDATE Invoices SET invoice_text = REPLACE(invoice_text, '9', '');
After having done that, you will have the pure text pattern without digits in invoice_text and can use that for grouping:
SELECT COUNT(invoice_number) AS total_invoices FROM Invoices
GROUP BY invoice_text
This is nice, but it is not yet what you wanted. It does not show the first and last invoice number for each group.
STEP 2: Deriving the first and last invoice for each group
For this step, create one more column in your table. Let us name it invoice_digits. As the name implies, it is meant to take only the pure invoice number without the "pattern text".
Assuming you have that column, you could do the following:
UPDATE Invoices SET invoice_digits = REPLACE(invoice_number, 'A', '');
UPDATE Invoices SET invoice_digits = REPLACE(invoice_digits, 'B', '');
UPDATE Invoices SET invoice_digits = REPLACE(invoice_digits, 'C', '');
...
UPDATE Invoices SET invoice_digits = REPLACE(invoice_digits, 'Z', '');
Now, you can use that column to get the minimum and maximum invoice number (without "pattern text"):
SELECT
MIN(invoice_digits) AS from_invoice_no,
MAX(invoice_digits) AS to_invoice_no,
COUNT(invoice_number) AS total_invoices
FROM Invoices
GROUP BY invoice_text
Problems and how to solve them
1) According to your question, you want to get the minimum and maximum full invoice number text. The solution above will show only the minimum and maximum invoice number text without the text parts, i.e. only the digits.
We could remedy this by doing a further JOIN, but since I can very well imagine that you won't insist on this :-), and since it won't make the general idea more clear, I am leaving this to you. If you are interested, let us know.
2) It might be difficult to decide what a digit (i.e. what the actual invoice number) is. For example, if you have invoice numbers like INV001, INV002, this will be no problem, but what if you have INV001/001, INV001/002, INV002/003 and so on? In this example, my code would would yield 001001, 001002, 002003 as actual invoice numbers and use that to decide what the minimum and maximum numbers are.
This might not be what you want to do in that case. The only way around this is that you thoroughly think about what you should consider a digit and what not, and to adapt my code accordingly.
3) My code currently uses string comparisons to get the minimum and maximum invoice numbers. This may yield other results than comparing the values as numbers. If you are wondering what that means: Compare '19' to '9' as string, and compare 19 to 9 as number.
If this is a problem, then use MySQL's CAST to convert the text to a number before feeding it to MAX or MIN. But please be aware that this has its own caveats:
If you have very long invoice numbers with so many digits that they don't fit into MySQL's numeric data types, this method will fail. It will also fail if you have defined a character like / to be digits (due to the issues described in 2)) since MySQL can't convert this into a number.
Instead of converting to numbers, you can also pad the values in invoice_digits with leading zeroes, for example using MySQL's LPAD function. This will avoid the problems described above and sort the numbers as expected, even if they include non-digits like /, but you will have to know the maximum length of the digit string in advance.
4) The code is ugly! Do you really have to remove all possible characters from A to Z one by one by doing UPDATE statements to get the digit string?
Actually, it is even worse. I just have assumed that you only have the "text characters" A to Z in your invoices. But there could be any character Unicode defines: Russian or Chinese ones, special characters, in other words: thousands of different characters.
Unfortunately, AFAIK, MySQL still does not provide a REGEX-REPLACE function. I don't see any chance to get this problem solved unless you extend MySQL with an appropriate UDF (user defined function). There are some cool guys out there who have recognized the problem and have added such functions to MySQL. Since recommending libraries seems to be discouraged on SO, just google for "mysql regex replace".
When having extended MySQL that way, you can replace the ugly bunch of UPDATE statements which remove the digits / the text from the invoice number by a single one (using a REGEX, you can replace all digits or all non-digits at once).
For the sake of completeness, you could avoid the many UPDATE statements by doing UPDATE ... SET ... = REPLACE(REPLACE(REPLACE(...))) and thus applying all updates with one statement. But this is even more ugly and error prone, so if you are serious about your problem, you'll really have to extend MySQL by a REGEX-REPLACE.
5) The solution will only work if you have the privilege to create new columns in the table.
This is true for the solution as-is. But I have chosen to go that way solely because it makes the general idea clear and understandable. Instead of adding columns to your original table, you could also create a new table where you store the pure text / digits (this table might be a temporary one).
Furthermore, since MySQL supports grouping by computed values, you don't need additional columns / tables at all. You should decide by yourself what is the best way to go.
I have downloaded a MySQL table as CSV, which has over thousand entries of the following type:
id,gender,garment-color
1,male,white
2,"male,female",black
3,female,"red,pink"
Now, when I am trying to create a chart out of this data, it is taking "male" as one value, and "male,female" as a separate value.
So, for the above example, rather than counting 2 "male", and 3 "female", the chart is showing 3 separate categories ("male", "female", "male,female"), with one count each.
I want the output as follows, for chart to have the correct count:
id,gender,garment-color
1,male,white
2,male,black
2,female,black
3,female,red
3,female,pink
The only way I know is to copy the row in MS Excel and adjust the values manually, which is too tedious for 1000+ entries. Is there a better way?
From MySQL command line or whatever tool you are using to send queries to MySQL:
select * from the_table
into outfile '/tmp/out.txt' fields terminated by ',' enclosed by '"'
Then download /tmp/out.txt' from the server and it should be good to go assuming your data is good. If it is not, you might need to massage it with some SQL function use in theselect`.
The csv likely came from a poorly designed/normalized database that had both those values in the same row. You could try using selects and updates, along some built in string functions, on such rows to spawn additional rows containing the additional values and update their original rows to remove those values; but you will have to repeat until all commas are removed (if there is more than one in some field), and will have to determine if a row containing multiple fields with such comma-separated lists need multiplied out (i.e. should 2 gender and 4 color mean 8 rows total).
More likely, you'll probably want to create additional tables for X_garmentcolors, and X_genders; where X is whatever the original table is supposed to be describing. These tables would have an X_id field referencing the original row and a [garmentcolor|gender] value field holding one of the values in the original rows lists. Ideally, they should actually reference [gender|garmentcolor] lookup tables instead of holding actual values; but you'd have to do the grunt work of picking out all the unique colors and genders from your data first. Once that is done, you can do something like:
INSERT INTO X_[garmentcolor|gender] (X_id, Y_id)
SELECT X.X_id, Y.Y_id
FROM originalTable AS X
INNER JOIN valueTable AS Y
ON X.Y_valuelist LIKE CONCAT('%,' Y.value) -- Value at end of list
OR X.Y_valuelist LIKE CONCAT('%,' Y.value, ',%') -- Value in middle of list
OR X.Y_valuelist LIKE CONCAT(Y.value, ',%') -- Value at start of list
OR X.Y_valuelist = Y.value -- Value is entire list
;
There is a table with fields name, age, city and state. Now I need to select rows based on city name. The value of the column city is surrounded with ", for example "LA".
How can I write a SELECT statement for getting data based on city.
\" is the escape combination for double quotes:
SELECT * FROM mytable WHERE city = '\"LA\"';
See MySQL documentation "String Literals".
Just suggestion, if you are collecting and storing the information in the table to be queried later (ie you are in control of the input), try to clean the data up before storing it to make it easier to query?
If the input has quotes and white space, clean that before inserting the values into the table. Use programming to do this, or mySQL: TRIM() and REPLACE() to remove the characters that might make a query hard to build and then store the resulting value into the table.
Of course, if you do not have control of the input data, that is where the answers above and the challenge to a programmer begins, trying to figure out the different input possibilities and dealing with that.
SELECT *
FROM table1
WHERE city LIKE '%LA%'
or
SELECT *
FROM table1
WHERE city REGEXP '^[[:space:]]*LA[[:space:]]*$'
I have 2000 products with row that is using serialized data and I need to update specific string
this is the row name data
a:35:{s:11:"expire_days";s:3:"30d";s:12:"trial1_price";s:0:"";s:11:"trial1_days";s:0:"";s:12:"is_recurring";s:0:"";s:10:"start_date";s:0:"";s:5:"terms";s:24:"$150 for 1 Per license";s:12:"rebill_times";s:0:"";s:15:"paypal_currency";s:0:"";s:4:"##11";N;s:3:"url";s:0:"";s:8:"add_urls";s:0:"";s:4:"##12";N;s:5:"scope";s:0:"";s:5:"order";s:4:"1010";s:11:"price_group";s:1:"7";s:13:"renewal_group";s:2:"28";s:14:"need_agreement";s:0:"";s:13:"require_other";a:1:{i:0;s:0:"";}s:16:"prevent_if_other";N;s:4:"##13";N;s:19:"autoresponder_renew";s:0:"";s:16:"dont_mail_expire";s:0:"";s:13:"joomla_access";s:2:"36";s:10:"files_path";s:108:"products/Boxes8.zip|Box 8
products/Boxes9.zip|Box 9";s:14:"download_count";s:0:"";s:18:"download_unlimited";}
and only thing I need changed is
s:24:"$150 for 1 Per license";
any help is appreciated.
You should probably SELECT the row, make your changes, then UPDATE with the new value. The answer to this question may be helpful if you need to do this database side.
How to do a regular expression replace in MySQL?
If you want to replace the value of that single field with something else, you can use the following query:
UPDATE table SET col = CONCAT(
LEFT(col, LOCATE('s:24:"', col) + 5), -- up to and including the opening quote
'Now for free', -- new replacement text
SUBSTR(col, LOCATE('"', col, LOCATE('s:24:"', col)+6)) -- closing quote and everything after that
) WHERE col LIKE '%s:24:"$150 for 1 Per license"%'
Note that there is potential for trouble: if the value of one of your fields should end in 's:24:', then that combined with the closing quote would get misinterpreded as the location you're looking at. I consider this risk unlikely, but if you want to play it safe, you might want to check for that with an elaborate regular expression that can deal with quoted strings and escaped quotes.