I have pretty large ASCII file (1.7mil rows) that I need to insert commas into at specific column positions. I am doing this because I am trying to convert the file to csv so I can import it into mysql. Unless there is a better approach (no doubt), what I am trying to do is insert comma at the specific column positions where I know fields end. This is not a job for column mode as dragging through 1.7mil rows would be insane.
I have tried this solution - How do I add a character at a specific postion in a string?
but it did not work. Does anyone have a suggestion?
Thanks!
To insert after the 4th character on each line:
Find: ^(.{4})
Replace: \1,
(Ticking Regular Expression in the find/replace dialog)
Another way to do it is to import the txt file you have into mysql as a table with a single column. Then split the string using SUBSTRING()
SELECT
SUBSTRING(col, 1, 8) AS Column1
, SUBSTRING(col, 9, 8) AS Column2
, SUBSTRING(col, 17, 16) AS Column3
FROM table
You can modify this query to do SELECT INTO or INSERT INTO. Depends on how you want to get it to the final table.
I've had to do it this way before because it was a recurring process and needed to be automated.
Related
I have a MySQL column that contains phone numbers, the problem is that they're in different formats, such as:
2125551212
212-555-1212
(212)5551212
I'd like to know if it's possible to take the existing 10 digits, remove the formatting, and change them all to this format: (212) 555-1212
Not a duplicate, as I'm looking to update several thousand entries instead of masking new entries.
Unfortunately, no REGEXP_MATCHES() or TRANSLATE() function comes with standard MySQL installation (they do with Postgres), so you could do this a way which I find really dirty, but it works.
First you cleanse your column by removing characters that aren't numbers using replace()
Then you take several parts of the string to separate them out using substr()
Finally, you concatenate them adding symbols between your substrings with concat()
If you have any more characters that you need truncate, just add another replace() on top of 3 already existing.
Sample data
create table nums ( num text );
insert into nums values
('2125551212'),
('212-555-1212'),
('(212)5551212');
Query formatting your data
select
num,
concat('(',substr(num_cleansed,1,3),') ',substr(num_cleansed,4,3),'-',substr(num_cleansed,7)) AS num_formatted
from (
select
num,
replace(replace(replace(num,'(',''),')',''),'-','') as num_cleansed
from nums
) foo
Result
num num_formatted
2125551212 (212) 555-1212
212-555-1212 (212) 555-1212
(212)5551212 (212) 555-1212
Click here SQLFiddle to preview output.
I'm leaving UPDATE statement as a homework for the reader.
Can someone give me an example of how to remove anything after a dash "-" in a zip code field using SQL commands?
For example, change any of this:
XXXXX-X
XXXXX-
XXXXX-XX
XXXXX-XXXX
to this:
XXXXX
Thanks for the examples. I also need to remove any instances of "-", "-X', "-XXXX", etc in the databases so the zip codes just contain five digits. Can someone include an example of this?
Given that your field is named zip , then just do this:
SELECT SUBSTRING_INDEX(zip, '-', 1) as zip
that would return what you want.
To update the data on the table you can do:
update table set zip = SUBSTRING_INDEX(zip, '-', 1) where condition = foo
that would update only records matching some condition, if you want to update them all remove the where part
Instead of using a regex, you could use MySQL's SUBSTRING_INDEX() method:
SELECT SUBSTRING_INDEX(zip_code, '-', 1) FROM your_table
EDIT (to support updates)
UPDATE your_table SET zip_code = SUBSTRING_INDEX(zip_code, '-', 1);
I'd recommend creating a second column, maybe zip_code_short and running SET zip_code_short = instead of overwriting the main data - just to make sure it doesn't cause any errors first (if feasible).
I have 2000 products with row that is using serialized data and I need to update specific string
this is the row name data
a:35:{s:11:"expire_days";s:3:"30d";s:12:"trial1_price";s:0:"";s:11:"trial1_days";s:0:"";s:12:"is_recurring";s:0:"";s:10:"start_date";s:0:"";s:5:"terms";s:24:"$150 for 1 Per license";s:12:"rebill_times";s:0:"";s:15:"paypal_currency";s:0:"";s:4:"##11";N;s:3:"url";s:0:"";s:8:"add_urls";s:0:"";s:4:"##12";N;s:5:"scope";s:0:"";s:5:"order";s:4:"1010";s:11:"price_group";s:1:"7";s:13:"renewal_group";s:2:"28";s:14:"need_agreement";s:0:"";s:13:"require_other";a:1:{i:0;s:0:"";}s:16:"prevent_if_other";N;s:4:"##13";N;s:19:"autoresponder_renew";s:0:"";s:16:"dont_mail_expire";s:0:"";s:13:"joomla_access";s:2:"36";s:10:"files_path";s:108:"products/Boxes8.zip|Box 8
products/Boxes9.zip|Box 9";s:14:"download_count";s:0:"";s:18:"download_unlimited";}
and only thing I need changed is
s:24:"$150 for 1 Per license";
any help is appreciated.
You should probably SELECT the row, make your changes, then UPDATE with the new value. The answer to this question may be helpful if you need to do this database side.
How to do a regular expression replace in MySQL?
If you want to replace the value of that single field with something else, you can use the following query:
UPDATE table SET col = CONCAT(
LEFT(col, LOCATE('s:24:"', col) + 5), -- up to and including the opening quote
'Now for free', -- new replacement text
SUBSTR(col, LOCATE('"', col, LOCATE('s:24:"', col)+6)) -- closing quote and everything after that
) WHERE col LIKE '%s:24:"$150 for 1 Per license"%'
Note that there is potential for trouble: if the value of one of your fields should end in 's:24:', then that combined with the closing quote would get misinterpreded as the location you're looking at. I consider this risk unlikely, but if you want to play it safe, you might want to check for that with an elaborate regular expression that can deal with quoted strings and escaped quotes.
I have a string such as,
"This is a sting and I dont know how long I am"
I want to turn every word in string into a row for my sql table so that I get:
ThisIs a string and I dont know etc...
I need to be able to do this with the MySql command line. (I also need an adjacent column to all be filled with ones on every row, incase that helps/changes your answer) I was thinking I could somehow use INSERT String (Words, num) Values (#words, 1) but I dont know how to get it to add every word. Is there any easy way to do this? If not, how would it be done?
MySQL does not have a function to split a delimited string. This problem is heavily discussed on MySQL manual page (search "split"), although there is no direct solution to handle variable number of elements.
Instead of that, I would help myself to generate such a query:
SELECT CONCAT('INSERT INTO t1 VALUES ("', REPLACE(REPLACE(TRIM(string_column), '"', '\\"'), ' ', '", "'), '")') FROM t2_with_string
I have a table "locales" with a column named "name". The records in name always begin with a number of characters folowed by an underscore (ie "foo_", "bar_"...). The record can have more then one underscore and the pattern before the underscore may be repeated (ie "foo_bar_", "foo_foo_").
How, with a simple query, can I get rid of everything before the first underscore including the first underscore itself?
I know how to do this in PHP, but I cannot understand how to do it in MySQL.
SELECT LOCATE('_', 'foo_bar_') ... will give you the location of the first underscore and SUBSTR('foo_bar_', LOCATE('_', 'foo_bar_')) will give you the substring starting from the first underscore. If you want to get rid of that one, too, increment the locate-value by one.
If you now want to replace the values in the tables itself, you can do this with an update-statement like UPDATE table SET column = SUBSTR(column, LOCATE('_', column)).
select substring('foo_bar_text' from locate('_','foo_bar_text'))
MySQL REGEXs can only match data, they can't do replacements. You'd need to do the replacing client-side in your PHP script, or use standard string operations in MySQL to do the changes.
UPDATE sometable SET somefield=RIGHT(LENGTH(somefield) - LOCATE('_', somefield));
Probably got some off-by-one errors in there, but that's the basic way of going about it.