I'm trying to update multiple rows in a DB using a small script.
I need to update the rows based on some specific user_ids which I have in a list on Linux machine.
#! /bin/bash
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in ()";
As you see above, the user_ids are in a file, let's say /opt/test/user_ids_txt.
How can I import them into this command?
This really depends on the format of user_ids_txt. If we assume it just happens to be in the correct syntax for your SQL in statement, the following will work:
#! /bin/bash
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in ($(< /opt/test/user_ids_txt))";
The bash interpreter will substitute in the contents of the file. This can be dangerous for SQL queries, so I would echo out the command on the terminal to make sure it is correct before implementing it. You should be able to preview your SQL query by simply running the following on the command line:
echo "update device set in_use=0 where user_id in ($(< /opt/test/user_ids_txt))"
If your file is not in the SQL in syntax you will need to edit it (or a copy of it) before running your query. I would recommend something like sed for this.
Example
Let's say your file /opt/test/user_ids_txt is just a list of user_ids in the format:
aaa
bbb
ccc
You can use sed to edit this into the correct SQL syntax:
sed 's/^/\'/g; s/$/\'/g; 2,$s/^/,/g' /opt/test/user_ids_txt
The output of this command will be:
'aaa'
,'bbb'
,'ccc'
If you look at this sed command, you will see 3 separate commands separated by semicolons. The individual commands translate to:
1: Add ' to the beginning of every line
2: Add ' to the end of every line
3: Add , to the beginning of every line but the first
Note: If your ID's are strictly numeric, you only need the third command.
This would make your SQL query translate to:
update device set in_use=0 where user_id in ('aaa'
,'bbb'
,'ccc')
Rather than make a temporary file to store this, I would use a bash variable, and simply plug that into the query like this:
#! /bin/bash
in_statement="$(sed 's/^/\'/g; s/$/\'/g; 2,$s/^/,/g' /opt/test/user_ids_txt)"
mysql -u user-ppassword db -e "update device set in_use=0 where user_id in (${in_statement})";
All of the other variables that make this work are tested and working correctly so I'm obviously doing this wrong.
I have a bash script that first selects some mysql data and stores into a new variable.
Then it goes on to connect again and update the database.
title=$(mysql -u $user -p$pass -h $host dbname | SELECT post_title FROM wp_posts WHERE ID=$8);
mysql --host=$host --user=$user --password=$pass dbname <<EOF
UPDATE wp_my_music_lib SET title = "$title" WHERE track_id=${4}${6};
EOF
The title entry is always blank which says to me that the initial SELECT isn't working properly. It should also be noted that the data expected from the select result has white space and special chars in it ie :
Some Artist (10/10/13)
I thought quoting the var "$title" would fix any potential problems with gobbling but that isn't the issue here as I've tried selecting a single numerical object from a different column and that doesn't work either.
If I hard code the title var it works as expected.
1) Can you see what I'm doing wrong?
2) Is it possible to perform all of the above with one db connection instead as that would make more sense?
mysql | SELECT pipes the output of mysql to a command called SELECT, which is сertainly not what you want.
To execute a query via mysql and capture the output you can use this syntax:
title=$(mysql -B dbname <<< "SELECT post_title FROM wp_posts WHERE ID=$8")
You could also execute the SELECT in a subquery to avoid multiple calls to mysql:
mysql --host=$host --user=$user --password=$pass dbname <<EOF
UPDATE wp_my_music_lib SET title = (
SELECT post_title FROM wp_posts WHERE ID=$8)
WHERE track_id=${4}${6}
EOF
I am storing output of MySQL query in a varible using shell scripting. The output of SQL query is in multiple rows. When I checked the count of the variable (which I think is an array), it is giving 1. My code snippet is as follows:
sessionLogin=`mysql -ugtsdbadmin -pgtsdbadmin -h$MYSQL_HOST -P$MYSQLPORT CMDB -e " select distinct SessionID div 100000 as 'MemberID' from SessionLogin where ClientIPAddr like '10.104%' and LoginTimestamp > 1426291200000000000 order by 1;"`
echo "${#sessionLogin[#]}"
How can I store the MySQL query output in an array in shell scripting?
You can loop over the output from mysql and append to an existing array. For example, in Bash 3.1+, a while loop with process substitution is one way to do it (please replace the mysql parameters with your actual command)
output=()
while read -r output_line; do
output+=("$output_line")
done < <(mysql -u user -ppass -hhost DB -e "query")
echo "There are ${#output[#]} lines returned"
Also take a look at the always excellent BashFaq
Say I have a number of similar mysql databases, where one table consists of users, which I want to query, while the name of the specific table is also variable.
I would say I need something similar, but it gives me a hard time to get the quotes rigth:
user_id = $(mysql --user=$dba --password=$dbp $dbn -e "SELECT id FROM ${user_tbl} WHERE username = \"guest\"")
Addition:
Ok, now knowing what the initial query should have been, comes the other question, how to update a field again using a variable for the tabelname.
mysql --user=$dba --password=$dbp $dbn -e "UPDATE ${user_tbl} SET password=${pass} WHERE username = 'guest'"
gives the following error:
Unknown column 'some_encrypted_pass' in 'field list'
user_id=$(mysql --user=$dba --password=$dbp $dbn -e "SELECT id FROM ${user_tbl} WHERE username = 'guest'")
you should use single quote to search guest string.
btw, why are you using that method, simply run mysqli_query and get value in array [with fetch methods] ? that will be better and more secure and reliable.
Usually I use manual find to replace text in a MySQL database using phpMyAdmin. I'm tired of it now, how can I run a query to find and replace a text with new text in the entire table in phpMyAdmin?
Example: find keyword domain.example, replace with www.domain.example.
For a single table update
UPDATE `table_name`
SET `field_name` = replace(same_field_name, 'unwanted_text', 'wanted_text')
From multiple tables-
If you want to edit from all tables, best way is to take the dump and then find/replace and upload it back.
The easiest way I have found is to dump the database to a text file, run a sed command to do the replace, and reload the database back into MySQL.
All commands below are bash on Linux.
Dump database to text file
mysqldump -u user -p databasename > ./db.sql
Run sed command to find/replace target string
sed -i 's/oldString/newString/g' ./db.sql
Reload the database into MySQL
mysql -u user -p databasename < ./db.sql
Easy peasy.
Running an SQL query in phpMyAdmin to find and replace text in all WordPress blog posts, such as finding mysite.example/wordpress and replacing that with mysite.example/news
Table in this example is tj_posts
UPDATE `tj_posts`
SET `post_content` = replace(post_content, 'mysite.example/wordpress', 'mysite.example/news')
Put this in a php file and run it and it should do what you want it to do.
// Connect to your MySQL database.
$hostname = "localhost";
$username = "db_username";
$password = "db_password";
$database = "db_name";
mysql_connect($hostname, $username, $password);
// The find and replace strings.
$find = "find_this_text";
$replace = "replace_with_this_text";
$loop = mysql_query("
SELECT
concat('UPDATE ',table_schema,'.',table_name, ' SET ',column_name, '=replace(',column_name,', ''{$find}'', ''{$replace}'');') AS s
FROM
information_schema.columns
WHERE
table_schema = '{$database}'")
or die ('Cant loop through dbfields: ' . mysql_error());
while ($query = mysql_fetch_assoc($loop))
{
mysql_query($query['s']);
}
phpMyAdmin includes a neat find-and-replace tool.
Select the table, then hit Search > Find and replace
This query took about a minute and successfully replaced several thousand instances of oldurl.ext with the newurl.ext within Column post_content
Best thing about this method : You get to check every match before committing.
N.B. I am using phpMyAdmin 4.9.0.1
Another option is to generate the statements for each column in the database:
SELECT CONCAT(
'update ', table_name ,
' set ', column_name, ' = replace(', column_name,', ''www.oldDomain.example'', ''www.newDomain.example'');'
) AS statement
FROM information_schema.columns
WHERE table_schema = 'mySchema' AND table_name LIKE 'yourPrefix_%';
This should generate a list of update statements that you can then execute.
UPDATE table SET field = replace(field, text_needs_to_be_replaced, text_required);
Like for example, if I want to replace all occurrences of John by Mark I will use below,
UPDATE student SET student_name = replace(student_name, 'John', 'Mark');
If you are positive that none of the fields to be updated are serialized, the solutions above will work well.
However, if any of the fields that need updating contain serialized data, an SQL Query or a simple search/replace on a dump file, will break serialization (unless the replaced string has exactly the same number of characters as the searched string).
To be sure, a "serialized" field looks like this:
a:1:{s:13:"administrator";b:1;}
The number of characters in the relevant data is encoded as part of the data.
Serialization is a way to convert "objects" into a format easily stored in a database, or to easily transport object data between different languages.
Here is an explanation of different methods used to serialize object data, and why you might want to do so, and here is a WordPress-centric post: Serialized Data, What Does That Mean And Why is it so Important? in plain language.
It would be amazing if MySQL had some built in tool to handle serialized data automatically, but it does not, and since there are different serialization formats, it would not even make sense for it to do so.
wp-cli
Some of the answers above seemed specific to WordPress databases, which serializes much of its data. WordPress offers a command line tool, wp search-replace, that does handle serialization.
A basic command would be:
wp search-replace 'an-old-string' 'a-new-string' --dry-run
However, WordPress emphasizes that the guid should never be changed, so it recommends skipping that column.
It also suggests that often times you'll want to skip the wp_users table.
Here's what that would look like:
wp search-replace 'https://old-domain.example' 'https://shiney-new-domain.com' --skip-columns=guid --skip-tables=wp_users --dry-run
Note: I added the --dry-run flag so a copy-paste won't automatically ruin anyone's database. After you're sure the script does what you want, run it again without that flag.
Plugins
If you are using WordPress, there are also many free and commercial plugins available that offer a gui interface to do the same, packaged with many additional features.
Interconnect/it PHP script
Interconnect/it offers a PHP script to handle serialized data: Safe Search and Replace tool. It was created for use on WordPress sites, but it looks like it can be used on any database serialized by PHP.
Many companies, including WordPress itself, recommends this tool. Instructions here, about 3/4 down the page.
UPDATE `MySQL_Table`
SET `MySQL_Table_Column` = REPLACE(`MySQL_Table_Column`, 'oldString', 'newString')
WHERE `MySQL_Table_Column` LIKE 'oldString%';
I believe "swapnesh" answer to be the best ! Unfortunately I couldn't execute it in phpMyAdmin (4.5.0.2) who although illogical (and tried several things) it kept saying that a new statement was found and that no delimiter was found…
Thus I came with the following solution that might be usefull if you exeprience the same issue and have no other access to the database than PMA…
UPDATE `wp_posts` AS `toUpdate`,
(SELECT `ID`,REPLACE(`guid`,'http://old.tld','http://new.tld') AS `guid`
FROM `wp_posts` WHERE `guid` LIKE 'http://old.tld%') AS `updated`
SET `toUpdate`.`guid`=`updated`.`guid`
WHERE `toUpdate`.`ID`=`updated`.`ID`;
To test the expected result you may want to use :
SELECT `toUpdate`.`guid` AS `old guid`,`updated`.`guid` AS `new guid`
FROM `wp_posts` AS `toUpdate`,
(SELECT `ID`,REPLACE(`guid`,'http://old.tld','http://new.tld') AS `guid`
FROM `wp_posts` WHERE `guid` LIKE 'http://old.tld%') AS `updated`
WHERE `toUpdate`.`ID`=`updated`.`ID`;
the best you export it as sql file and open it with editor such as visual studio code and find and repalace your words.
i replace in 1 gig file sql in 1 minutes for 16 word that total is 14600 word.
its the best way.
and after replace it save and import it again.
do not forget compress it with zip for import.
In the case of sentences with uppercase - lowercase letters,
We can use BINARY REPACE
UPDATE `table_1` SET `field_1` = BINARY REPLACE(`field_1`, 'find_string', 'replace_string')
Here's an example of how to find and replace in Database
UPDATE TABLE_NAME
SET COLUMN = replace(COLUMN,'domain.example', 'www.domain.example')
TABLE_NAME => Change it with your table name
COLUMN => Change it to your column make sure it exists
I have good luck with this query when doing a search and replace in phpmyadmin:
UPDATE tableName SET fieldName1 = 'foo' WHERE fieldName1 = 'bar';
Of course this only applies to one table at a time.
Generate change SQL queries (FAST)
mysql -e "SELECT CONCAT( 'update ', table_name , ' set ', column_name, ' = replace(', column_name,', ''www.oldsite.example'', ''www.newsite.example'');' ) AS statement FROM information_schema.columns WHERE table_name LIKE 'wp_%'" -u root -p your_db_name_here > upgrade_script.sql
Remove any garbage at the start of the file. I had some.
nano upgrade_script.sql
Run generated script with --force options to skip errors. (SLOW - grab a coffee if big DB)
mysql -u root -p your_db_name_here --force < upgrade_script.sql