Here's my code.
SELECT *
FROM `accounts`
WHERE NOT name REGEXP '^[[.NUL.]-[.DEL.]]*$'
I want all non keyboard characters across all tables to be replaced with a space.
Hoping that someone can actually do this.
You aren't going to be able to do this easily in SQL.
The most straightforward approach would be to take a logical backup of your database, use sed or perl (or some similar tool to do the string replacement), and then re-import the data.
You should test this by importing the data into a test database (or at least a test schema) to make sure it doesn't harm your data.
Assuming by "non keyboard" characters you are referring to "non-printable" characters, and that you are on linux, you can do this with a combination of mysqldump and sed like so:
# dump your schema and data
mysqldump --single-transaction your_schema > /tmp/your_schema.sql
# copy the dump file and replace all non-printable characters with a space
sed -e 's/[^[:print:]]/ /g' /tmp/your_schema.sql > /tmp/your_schema_test.sql
# create an empty test schema to test the import
mysqladmin create your_schema_test
# import the data into the test schema
mysql -f your_schema_test < /tmp/your_schema_test.sql
Related
Is it possible to query Postgresql in order to get correct CSV line? For instance select concat (a,',',b) from t but with correctly escaped commas and quotes.
A couple of options.
Using psql
select * from some_table \g (format=csv) output.csv
This will create a CSV file named output.csv.
\copy cell_per to 'output.csv' WITH(format csv, header, delimiter '|');
The above allows you to use the options as explained here COPY to do things like change the delimiter, quoting, etc.
You can also use COPY directly as a query. Though in that case it is important to note that COPY runs as the server user and can only write files to directories the server user has permissions on. The work around is to make the output go to STDOUT and and capture it. For instance using the Python driver psycopg2 there are copy methods copy.
I have hundreds of SQL file which I want to restore all of the databases in different database name for each file.
I look around for a solution, but what I got is something like concat all the files into one SQL file using cat.* and then restore using the concatenated file.
But, what I want is to restore it to a different database so, I think concat is not suitable for my case.
Here's one solution: alternate USE commands with your sql files, so you change the default database before the respective database's content. Gather the whole collection together and then pipe that to the input of the mysql client.
Example using bash syntax:
(
echo "USE database1;"
cat file1.sql
echo "USE database2;"
cat file2.sql
...
) | mysql
Another solution is to run the mysql client once for each file, and specify the database name as the argument:
mysql database1 < file1.sql
mysql database2 < file2.sql
...
Re your comment:
You can write a loop in bash too.
for file in *.sql
do
db=...
mysql $db < $file
done
The tricky part above is the "..." — deciding which db goes with each input SQL file. You haven't described any way to match them, so I don't know what you'd have to do to figure that out. But if you can make that inference somehow from the filename, then you can do this without having to type every file.
grep -n "Table Structure" dumpfile.sql
returns
XXXXXX:-- Table structure for table `table_name_1`
XXXXXX:-- Table structure for table `table_name_2`
XXXXXX:-- Table structure for table `table_name_3`
But after this point, it breaks. Not sure why ?
AND also
For retrieving a single table from huge dump file (Around 489GB), I used:
sed -n -e '/Table Structure 'table_name'/p' dump_file_name.sql > extracted_file.sql
But it is not able to locate the table_name.
So my question here is. How can all the tables be accessed ? Or why is it after certain table, it is not able to find the table.
Please If anyone can help me with this. It will be a greatest deed !
You have two problems with your sed command.
First, you're using single quotes inside the string that's delimited by single quotes. That won't work, because the inside quotes will just end the shell string, not be included literally.
Second, the quotes in the dump file are backticks, not single quotes.
Also, you're missing for table in your pattern, and the s in structure should be lowercase.
sed -n -e '/Table structure for table `table_name`/p' dump_file_name.sql > extracted_file.sql
But you can just use grep for this, you don't need sed:
grep 'Table structure for table `table_name`' dump_file_name.sql > extracted_file.sql
I recently migrated domains and in my database, I had stored full paths containing the old domain, which now broke :)
What I need to do is change values in the database table from
http://www.olddomain.com/img/some/path
to
http://www.newdomain/same/dir/structure/as/old/domain
The only caveat is that the photo names at the end of the url must be preserved. So essentially, I have to just change the host name.
Is that possible to do? If so, how? :)
Try this:
UPDATE table SET column = REPLACE(column,"www.olddomain.com","www.newdomain.com");
ALWAYS make sure you do a backup of your database before running a query that updates many records (as this will).
With a MySQL database, do this on the command line:
1 - Put full DB in a file:
mysqldump -uYOURUSERNAME -pYOURPASSWORD YOURDBNAME > YOURDBNAME.sql
2 - Replace olddomain.com with newdomain.com in the previous DB file:
sed -i 's/olddomain.com/newdomain.com/g' YOURDBNAME.sql
3 - Delete all tables in original database (make sure you have a backup), and update database with replaced domain in all rows of all tables, where applicable:
mysql -uYOURUSERNAME -pYOURPASSWORD YOURDBNAME < YOURDBNAME.sql
This is guaranteed to work. I've used this to update domains on Magento databases (300+ tables) several times.
FYI, sed is a linux/unix command line tool for "filtering and transforming text", I don't know if there is a windows version.
PS - If you really need to put slashes (/) in the domain (like if you're replacing www.example.com/sitedir with www.example.com), you should escape the slashes inside the sed string, i.e. instead of using /, use \/. For this example you would do:
sed -i 's/www.example.com\/sitedir/www.example.com/g' YOURDBNAME.sql
What's the easiest way to get the data for a single table, delete a single table or break up the whole dump file into files each containing individual tables? I usually end up doing a lot of vi regex munging, but I bet there are easier ways to do these things with awk/perl, etc. The first page of Google results brings back a bunch of non-working perl scripts.
When I need to pull a single table from an sql dump, I use a combination of grep, head and tail.
Eg:
grep -n "CREATE TABLE" dump.sql
This then gives you the line numbers for each one, so if your table is on line 200 and the one after is on line 269, I do:
head -n 268 dump.sql > tophalf.sql
tail -n 69 tophalf.sql > yourtable.sql
I would imagine you could extend upon those principles to knock up a script that would split the whole thing down into one file per table.
Anyone want a go doing it here?
Another bit that might help start a bash loop going:
grep -n "CREATE TABLE " dump.sql | tr ':`(' ' ' | awk '{print $1, $4}'
That gives you a nice list of line numbers and table names like:
200 FooTable
269 BarTable
Save yourself a lot of hassle and use mysqldump -T if you can.
From the documentation:
--tab=path, -T path
Produce tab-separated data files. For each dumped table, mysqldump
creates a tbl_name.sql file that contains the CREATE TABLE statement
that creates the table, and a tbl_name.txt file that contains its
data. The option value is the directory in which to write the files.
By default, the .txt data files are formatted using tab characters
between column values and a newline at the end of each line. The
format can be specified explicitly using the --fields-xxx and
--lines-terminated-by options.
Note This option should be used only when mysqldump is run on the
same machine as the mysqld server. You must have the FILE privilege,
and the server must have permission to write files in the directory
that you specify.
This shell script will grab the tables you want and pass them to splitted.sql.
It’s capable of understanding regular expressions as I’ve added a sed -r option.
Also MyDumpSplitter can split the dump into individual table dumps.
Maatkit seems quite appropriate for this with mk-parallel-dump and mk-parallel-restore.
I am a bit late on that one, but if it can help anyone, I had to split a huge SQL dump file in order to import the data to another Mysql server.
what I ended up doing was splitting the dump file using the system command.
split -l 1000 import.sql splited_file
The above will split the sql file every 1000 lines.
Hope this helps someone