I need to insert the contant of a .txt file into a mysql table. The command is this, but I'm having problems with the LOAD_FILE:
(already tried with absolut and relative path for the file)
FILE="LOAD_FILE('/home/vector5/scripts/log_Backup_HG3DiesSetmana.txt')"
mysql --host=localhost --user=MYUSER --password=MYPASSWORD vector5_apLogis2 << EOF
insert into tableNameBackups(data,tipus,titol,text)
values('2015-06-02 00:47:17','111','asdadad',$FILE);
EOF
And this is the error I have, looks like is not reading the file. I've tried almost everything with no luck:
ERROR 1048 (23000) at line 1: Column 'text' cannot be null
Looks like $taulaAUsar is undefined. Check if it contains the table name.
UPDATE
$file is null make sure that the file exists. Think you have to add the complete path to it.
Related
I'm running a python script which returns several lines of text, which I use the following truncate command tr and want to add to my database.
I start here: this removes the unformatted line delimiters.
tr -d '\15\32' < long_text > unixfile.txt
I'm then left with an output which looks like this:
Happy Birthday Stackoverflow
Happy Birthday to you
Happy Birthday Stackoverflow
I use the following command to place this into a variable:
lyrics=$(cat unixfile.txt)
mysql --user=USER --password=PASSWORD --database='DB' --execute='INSERT INTO `song_lyrics` (`id`, `song_id`, `lyrics`, `info`) VALUES ('"'$i'"', '"'$i'"', '"$lyrics"', '0');'
ERROR 1064 (42000) at line 1:
It seems that MySQL is seeing my entire 3 line string (shown above) in three different commands, because the output of the unixfile.txt appears in the error output.
This problem can be solved using the following:
Ensure the long text string is being captured in a file, instead of a variable - then re-capture the file/variable.
lyrics=$(cat unixfile.txt)
mysql --user="$user" --password="$password" --database="$my_db" <<END
use my_db;
INSERT INTO song_lyrics (id, song_id, lyrics, info) VALUES ("$i", "$i", "$lyrics", '0');
END
this method ensures the variable is passed properly.
the only remaining issues would be with various types of punctuation
although this post does not describe this.
I have a table that holds relative paths to real files on HDD. for example:
SELECT * FROM images -->
id | path
1 | /files/1.jpg
2 | /files/2.jpg
Can I create a query to select all records pointing to non-existent files? I need to check it by MySql server exactly, without using an iteration in PHP-client.
I would go with a query like this:
SELECT id, path, ISNULL(LOAD_FILE(path)) as not_exists
FROM images
HAVING not_exists = 1
The function LOAD_FILE tries to load the file as a string, and returns NULL when it fails.
Please notice that a failure in this case might be due to the fact that mysql simply cannot read that specific location, even if the file actually exists.
EDIT:
As #ostrokach pointed out in comments, this isn't standard SQL, even though MySQL allows it, to follow the standard it could be:
SELECT *
FROM images
WHERE LOAD_FILE(PATH) IS NULL
The MySQL LOAD_FILE command has very stringent requirements on the files that it can open. From the MySQL docs:
[LOAD_FILE] Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full path name to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes. If the secure_file_priv system variable is set to a non-empty directory name, the file to be loaded must be located in that directory.
So if the file can't be reached by the mysql user or any of the other requirements are not satisfied, LOAD_FILE will return Null.
You can get a list of IDs that correspond to missing files using awk:
mysql db_name --batch -s -e "SELECT id, path FROM images" \
| awk '{if(system("[ -e " $2 " ]") == 1) {print $1}}' \
>> missing_ids.txt
or simply using bash:
mysql db_name --batch -s -e "SELECT id, path FROM images" \
| while read id path ; if [[ -e "$path" ]] ; then echo $id ; done
>> missing_ids.txt
This also has the advantage of being much faster than LOAD_FILE.
MYSQL only handles the Database so there is no way for you to fire an SQL Statement to check on the HDD if the file exists. You need to iterate over the rows and check it with PHP.
It's not possible using stock MySQL. However you can write UDF (user-defined function), probably in C, load it using CREATE FUNCTION statement and use it from MySQL as you would use any built-in function.
I want to store images and .docx/.doc, .pptx/.ppt, .pdf files using the front end of my software. I don't understand how to implement this and how to insert the BLOB and CLOB files into the table. Please help.
I am using Kubuntu 11.04, MySQL5, Qt 4.7.3.
Two ways:
1 - Use a LOAD_FILE function -
INSERT INTO table1 VALUES(1, LOAD_FILE('data.png'));
2 - Insert file as hex string, e.g. -
INSERT INTO table1 VALUES
(1, x'89504E470D0A1A0A0000000D494844520000001000000010080200000090916836000000017352474200AECE1CE90000000467414D410000B18F0BFC6105000000097048597300000EC300000EC301C76FA8640000001E49444154384F6350DAE843126220493550F1A80662426C349406472801006AC91F1040F796BD0000000049454E44AE426082');
INSERT INTO MY_TABLE(id, blob_col) VALUES(1, LOAD_FILE('/full/path/to/file/myfile.png')
LOAD_FILE has many conditions attached to it. From the MySQL documentation:
LOAD_FILE(file_name)
Reads the file and returns the file contents as a string. To use this
function, the file must be located on the server host, you must
specify the full path name to the file, and you must have the FILE
privilege. The file must be readable by all and its size less than
max_allowed_packet bytes. If the secure_file_priv system variable is
set to a nonempty directory name, the file to be loaded must be
located in that directory.
If the file does not exist or cannot be read because one of the
preceding conditions is not satisfied, the function returns NULL.
Also, there there are bugs with LOAD_FILE in Linux. See http://bugs.mysql.com/bug.php?id=38403 for the bug, and MySQL LOAD_FILE returning NULL for workarounds. On Ubuntu 12.04, MySQL 5.5.32, this works for me:
Copy file to /tmp
Change ownership to mysql user chown mysql:mysql /tmp/yourfile
Log into mysql as mysql root user so you are sure you have FILE privilege
Run your insert statement
Or you could merely use the MySQL Workbench, select the rows, last rows, insert a row without the blob, then just right click and select "Load Value From File".
INSERT INTO table1 VALUES(1, LOAD_FILE(data.png));
won't work but
INSERT INTO table1 VALUES(1, LOAD_FILE('data.png'));
should (assuming data.png exists in the local directory)
for those People who are getting "Column 'image' cannot be null" error while saving Blob through query :-
Open your MySql Command Line Client and login with root user and type
mysql> SHOW VARIABLES LIKE "secure_file_priv";
this will show you the secure path used by MySql to access the files. something like
+------------------+-----------------------+
| Variable_name | Value |
+------------------+-----------------------+
| secure_file_priv | /var/lib/mysql-files/ |
+------------------+-----------------------+
you can either paste files inside this folder or change the "secure_file_priv" variable value to "empty string" so that it can read file from anywhere.
If you are using mysql workbench, just right click on the field (cell) and select 'load value from file' option and then browse to the file and click open and then click on apply. It will automatically generate query like this
UPDATE `dbname`.`tablename` SET `columnname` = ? WHERE (`row` = '1');
I have a database named "crea" with a table named "assets" and in this table i have 11 column named (name,description,assetType,local,temporary,data,id,create_time,access_time,asset_flags,CreatorID)
I also have a directory folder with many picture in .jp2 format (xxxx.jp2)
What im trying to do is to bulk insert these picture in the table "assets" of my database, so i decided to do it with 2 shell scripts, the both are in the directory with the pictures.
When i launch ./assetadd.sh from the terminal, i get this error from MySQL :
ERROR 1048 (23000) at line 1: Column 'data' cannot be null
I checked many time and im sure that the column 'data' isnt NULL (Type: LONGBLOB Binaries Null: NO), so i really dont understand why i get this error.
Help will be apreciated. Thank you
- Script 1 : assetsadd.sh
#!/bin/bash
path=$(pwd)
find $path/ -type f \( -iname *.jp2 \) -exec ./insertjp2.sh {} \;
echo "finished!!"
- Script 2 : insertjp2.sh
#!/bin/bash
user="crea"
password="crea"
database="crea"
dbhost="localhost"
creator="crea"
param=$#
basenam=${param##*/}
filenam=${basenam%.*}
MYSQL=`/usr/bin/mysql -u$user -p$password -D$database -e"INSERT INTO assets (name,description,assetType,local,temporary,data,id,create_time,access_time,asset_flags,CreatorID) VALUES ('$filenam','$filenam',0,0,0,LOAD_FILE('$param'),'$filenam',UNIX_TIMESTAMP(),1325304546,0,'$creator' );"`
echo $param >> assetadd.log
echo $MYSQL
From the documentation for LOAD_FILE:
Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full path name to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes. If the secure_file_priv system variable is set to a nonempty directory name, the file to be loaded must be located in that directory.
If the file does not exist or cannot be read because one of the preceding conditions is not satisfied, the function returns NULL.
It looks like at the very least, you are not specifying the full path to the file name.
I've been trying to get a shell(bash) script to insert a row into a REMOTE database, but I've been having some trouble :(
The script is meant to upload a file to a server, get a URL, HASH, and a file size, connect to a remote mysql database, and insert the data into an existing table. I've gotten it working until the remote MYSQL database bit.
It looks like this:
#!/bin/bash
zxw=randomtext
description=randomtext2
for file in "$#"
do
echo -n *****
ident= *****
data= ****
size=` ****
hash=`****
mysql --host=randomhost --user=randomuser --password=randompass randomdb
insert into table (field1,field2,field3) values('http://www.example.com/$hash','$file','$size');
echo "done"
done
I'm a total noob at programming so yeah :P
Anyway, I added the \ to escape the brackets as I was getting errors. As it is right now, the script is works fine until connects to the mysql database. It just connects to the mysql database and doesn't do the insert command (and I don't even know if the insert command would work in bash).
PS: I've tried both the mysql commands from the command line one by one, and they worked, though I defined the hash/file/size and didn't have the escaping "".
Anyway, what do you guys think? Is what I'm trying to do even possible? If so how?
Any help would be appreciated :)
The insert statement has to be sent to mysql, not another line in the shell script, so you need to make it a "here document".
mysql --host=randomhost --user=randomuser --password=randompass randomdb << EOF
insert into table (field1,field2,field3) values('http://www.site.com/$hash','$file','$size');
EOF
The << EOF means take everything before the next line that contains nothing but EOF (no whitespace at the beginning) as standard input to the program.
This might not be exactly what you are looking for but it is an option.
If you want to bypass the annoyance of actually including your query in the sh script, you can save the query as .sql file (useful sometimes when the query is REALLY big and complicated). This can be done with simple file IO in whatever language you are using.
Then you can simply include in your sh scrip something like:
mysql -u youruser -p yourpass -h remoteHost < query.sql &
This is called batch mode execution. Optionally, you can include the ampersand at the end to ensure that that line of the sh script does not block.
Also if you are concerned about the same data getting entered multiple times and your rdbms getting inconsistent, you should explore MySql transactions (commit, rollback, etc).
Don't use raw SQL from bash; bash has no sane facility for sanitizing the data beforehand. Generate a CSV file and upload that instead.