I have a database named "crea" with a table named "assets" and in this table i have 11 column named (name,description,assetType,local,temporary,data,id,create_time,access_time,asset_flags,CreatorID)
I also have a directory folder with many picture in .jp2 format (xxxx.jp2)
What im trying to do is to bulk insert these picture in the table "assets" of my database, so i decided to do it with 2 shell scripts, the both are in the directory with the pictures.
When i launch ./assetadd.sh from the terminal, i get this error from MySQL :
ERROR 1048 (23000) at line 1: Column 'data' cannot be null
I checked many time and im sure that the column 'data' isnt NULL (Type: LONGBLOB Binaries Null: NO), so i really dont understand why i get this error.
Help will be apreciated. Thank you
- Script 1 : assetsadd.sh
#!/bin/bash
path=$(pwd)
find $path/ -type f \( -iname *.jp2 \) -exec ./insertjp2.sh {} \;
echo "finished!!"
- Script 2 : insertjp2.sh
#!/bin/bash
user="crea"
password="crea"
database="crea"
dbhost="localhost"
creator="crea"
param=$#
basenam=${param##*/}
filenam=${basenam%.*}
MYSQL=`/usr/bin/mysql -u$user -p$password -D$database -e"INSERT INTO assets (name,description,assetType,local,temporary,data,id,create_time,access_time,asset_flags,CreatorID) VALUES ('$filenam','$filenam',0,0,0,LOAD_FILE('$param'),'$filenam',UNIX_TIMESTAMP(),1325304546,0,'$creator' );"`
echo $param >> assetadd.log
echo $MYSQL
From the documentation for LOAD_FILE:
Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full path name to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes. If the secure_file_priv system variable is set to a nonempty directory name, the file to be loaded must be located in that directory.
If the file does not exist or cannot be read because one of the preceding conditions is not satisfied, the function returns NULL.
It looks like at the very least, you are not specifying the full path to the file name.
Related
I am trying to create an sql script using bash but I keep getting this line after each iteration of my loop
: command not found
This is the case on Ubuntu as well as OSX.
At this stage I am not executing the sql script, I simply trying to create it. What am I missing so that it will not try to "execute" the query?
The queries are fine when tested in phpmyadmin
I don't understand why would need to set the $PATH variable if I am not executing the actual query, I am just creating the text file.
The code I use is:
SQL="";
cat people.txt | while read line
do
PW="my"$line"db";
DB="test_"$line;
$SQL=$SQL"CREATE DATABASE \`$DB\`;CREATE USER \`$line\`#\`localhost\`;SET PASSWORD FOR \`$line\`#\`localhost\` = PASSWORD(\"$PW\") ;GRANT ALL PRIVILEGES ON $DB.* TO \`$line\`#\`localhost\` IDENTIFIED BY \"$PW\";";
done
echo $SQL > t1.sql;
The list I am using for my imports:
bob123
john123
jane123
The output I am getting is:
./02_setup_mysqldb.sh: line 14: =CREATE DATABASE `test_bob123`;CREATEUSER `bob123`#`localhost`;SET PASSWORD FOR `bob123`#`localhost` = PASSWORD("mybob123db") ;GRANT ALL PRIVILEGES ON test_bob123.* TO `bob123`#`localhost` IDENTIFIED BY "mybob123db";: command not found
./02_setup_mysqldb.sh: line 14: =CREATE DATABASE `test_john123`;CREATE USER `john123`#`localhost`;SET PASSWORD FOR `john123`#`localhost` = PASSWORD("myjohn123db") ;GRANT ALL PRIVILEGES ON test_john123.* TO `john123`#`localhost` IDENTIFIED BY "myjohn123db";: command not found
There's an error in your variable assignment, it should be:
SQL=$SQL"CREATE DATABASE...
Note the missing $ in front of the variable name - variables don't need $ in front of the name when values are assigned.
Notes:
1.The assignment to SQL variable is incorrect, just change to SQL="$SQL...", just remove the $ character.
2.When you do cat people.txt | while read LINE, you are ignoring the last line, being necessary to have a blank line after the last line.
3.Your script has a large string concatenation in one line, just create variables to make it more readable.
Finally, code:
#!/bin/bash
while read line
do
PW="my${line}db"
DB="test_$line"
create_database="CREATE DATABASE \`$DB\`;\n"
create_user="CREATE USER \`$line\`#\`localhost\`;\n"
change_password="SET PASSWORD FOR \`$line\`#\`localhost\` = PASSWORD(\"$PW\");\n"
grant_privileges="GRANT ALL PRIVILEGES ON $DB.* TO \`$line\`#\`localhost\` IDENTIFIED BY \"$PW\";\n"
SQL="${SQL}${create_database}${create_user}${change_password}${grant_privileges}"
done < people.txt
echo -e ${SQL} > t1.sql
What's wrong with this code, I want to insert an image to table, but when I was executed this code the result of image field is NULL.
I try with MySQL Workbench executing:
CREATE TABLE image(keyh int, img blob);
INSERT INTO image VALUES(1, load_file('d:\Picture\cppLogo.png'));
To use this function, the file must be located on the server host, you
must specify the full path name to the file, and you must have the
FILE privilege. The file must be readable by all and its size less
than max_allowed_packet bytes. If the secure_file_priv system variable
is set to a nonempty directory name, the file to be loaded must be
located in that directory.
If the file does not exist or cannot be read because one of the
preceding conditions is not satisfied, the function returns NULL.
http://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_load-file
What can you do?
Check which user mysql is running with, and make sure the file is readable by that user. Make sure the security settings allow the file to be read and it is not of greater size than max_allowed_packet.
See SHOW VARIABLES LIKE 'max_allowed_packet'.
For me, it looks like the file is on your localhost and you try to upload it. This is not possible using LOAD_FILE(). The file must be already on the server.
The issue can also be caused by your windows directory seperator character \ (like RiggsFolly said), which is used for escaping instead, switch to unix style / then:
LOAD_FILE('D:/Picture/cppLogo.png')
Or your Image is of greater filesize than a BLOB field can hold, like Balazs Vago said.
i was found the correct syntax is following this:
C:/wamp/binsql5.5.20/data/56VRLRFE.jpg'
not this
C:\wamp\binsql5.5.20\data\56VRLRFE.jpg'
thanks guys for all your Answer :D
Open your MySql Command Line Client and login with root user and type
mysql> SHOW VARIABLES LIKE "secure_file_priv";
this will show you the secure path used by MySql to access the files. something like
+------------------+-----------------------+
| Variable_name | Value |
+------------------+-----------------------+
| secure_file_priv | /var/lib/mysql-files/ |
+------------------+-----------------------+
you can either paste files inside this folder or change the "secure_file_priv" variable value to "empty string" so that it can read file from anywhere.
On Windows the fundamental problem is that MySql, by default, runs as a Windows service under the Network account which means that there are only a few file locations the server can access. Thus for load_file to work, the file must be placed in a folder on the server which can be read by the service. There seems to be no documentation on this. In my investigation the only folder that works with load_file is C:\ProgramData\MySQL\MySQL Server 8.0\Uploads
Run a query to test the load...
select load_file('C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\1.txt') ;
Note on windows you have to use either double \ or / to separate the path elements. This will return NULL on failure, otherwise the contents of the file.
Assume now a table named db.image with columns source and image. Source is character and image is blob. The command to load a.jpg into the table would be
insert into db.image (source,image) values ('a.jpg',load_file('c:/programdata/mysql/mysql server 8.0/uploads/a.jpg'));
Store Directly Without folder name for example-
create table myimg(id int, image mediumblob);
insert into myimg values(101, load_file("E://xyz.png"));
I need to insert the contant of a .txt file into a mysql table. The command is this, but I'm having problems with the LOAD_FILE:
(already tried with absolut and relative path for the file)
FILE="LOAD_FILE('/home/vector5/scripts/log_Backup_HG3DiesSetmana.txt')"
mysql --host=localhost --user=MYUSER --password=MYPASSWORD vector5_apLogis2 << EOF
insert into tableNameBackups(data,tipus,titol,text)
values('2015-06-02 00:47:17','111','asdadad',$FILE);
EOF
And this is the error I have, looks like is not reading the file. I've tried almost everything with no luck:
ERROR 1048 (23000) at line 1: Column 'text' cannot be null
Looks like $taulaAUsar is undefined. Check if it contains the table name.
UPDATE
$file is null make sure that the file exists. Think you have to add the complete path to it.
I have a table that holds relative paths to real files on HDD. for example:
SELECT * FROM images -->
id | path
1 | /files/1.jpg
2 | /files/2.jpg
Can I create a query to select all records pointing to non-existent files? I need to check it by MySql server exactly, without using an iteration in PHP-client.
I would go with a query like this:
SELECT id, path, ISNULL(LOAD_FILE(path)) as not_exists
FROM images
HAVING not_exists = 1
The function LOAD_FILE tries to load the file as a string, and returns NULL when it fails.
Please notice that a failure in this case might be due to the fact that mysql simply cannot read that specific location, even if the file actually exists.
EDIT:
As #ostrokach pointed out in comments, this isn't standard SQL, even though MySQL allows it, to follow the standard it could be:
SELECT *
FROM images
WHERE LOAD_FILE(PATH) IS NULL
The MySQL LOAD_FILE command has very stringent requirements on the files that it can open. From the MySQL docs:
[LOAD_FILE] Reads the file and returns the file contents as a string. To use this function, the file must be located on the server host, you must specify the full path name to the file, and you must have the FILE privilege. The file must be readable by all and its size less than max_allowed_packet bytes. If the secure_file_priv system variable is set to a non-empty directory name, the file to be loaded must be located in that directory.
So if the file can't be reached by the mysql user or any of the other requirements are not satisfied, LOAD_FILE will return Null.
You can get a list of IDs that correspond to missing files using awk:
mysql db_name --batch -s -e "SELECT id, path FROM images" \
| awk '{if(system("[ -e " $2 " ]") == 1) {print $1}}' \
>> missing_ids.txt
or simply using bash:
mysql db_name --batch -s -e "SELECT id, path FROM images" \
| while read id path ; if [[ -e "$path" ]] ; then echo $id ; done
>> missing_ids.txt
This also has the advantage of being much faster than LOAD_FILE.
MYSQL only handles the Database so there is no way for you to fire an SQL Statement to check on the HDD if the file exists. You need to iterate over the rows and check it with PHP.
It's not possible using stock MySQL. However you can write UDF (user-defined function), probably in C, load it using CREATE FUNCTION statement and use it from MySQL as you would use any built-in function.
I want to store images and .docx/.doc, .pptx/.ppt, .pdf files using the front end of my software. I don't understand how to implement this and how to insert the BLOB and CLOB files into the table. Please help.
I am using Kubuntu 11.04, MySQL5, Qt 4.7.3.
Two ways:
1 - Use a LOAD_FILE function -
INSERT INTO table1 VALUES(1, LOAD_FILE('data.png'));
2 - Insert file as hex string, e.g. -
INSERT INTO table1 VALUES
(1, x'89504E470D0A1A0A0000000D494844520000001000000010080200000090916836000000017352474200AECE1CE90000000467414D410000B18F0BFC6105000000097048597300000EC300000EC301C76FA8640000001E49444154384F6350DAE843126220493550F1A80662426C349406472801006AC91F1040F796BD0000000049454E44AE426082');
INSERT INTO MY_TABLE(id, blob_col) VALUES(1, LOAD_FILE('/full/path/to/file/myfile.png')
LOAD_FILE has many conditions attached to it. From the MySQL documentation:
LOAD_FILE(file_name)
Reads the file and returns the file contents as a string. To use this
function, the file must be located on the server host, you must
specify the full path name to the file, and you must have the FILE
privilege. The file must be readable by all and its size less than
max_allowed_packet bytes. If the secure_file_priv system variable is
set to a nonempty directory name, the file to be loaded must be
located in that directory.
If the file does not exist or cannot be read because one of the
preceding conditions is not satisfied, the function returns NULL.
Also, there there are bugs with LOAD_FILE in Linux. See http://bugs.mysql.com/bug.php?id=38403 for the bug, and MySQL LOAD_FILE returning NULL for workarounds. On Ubuntu 12.04, MySQL 5.5.32, this works for me:
Copy file to /tmp
Change ownership to mysql user chown mysql:mysql /tmp/yourfile
Log into mysql as mysql root user so you are sure you have FILE privilege
Run your insert statement
Or you could merely use the MySQL Workbench, select the rows, last rows, insert a row without the blob, then just right click and select "Load Value From File".
INSERT INTO table1 VALUES(1, LOAD_FILE(data.png));
won't work but
INSERT INTO table1 VALUES(1, LOAD_FILE('data.png'));
should (assuming data.png exists in the local directory)
for those People who are getting "Column 'image' cannot be null" error while saving Blob through query :-
Open your MySql Command Line Client and login with root user and type
mysql> SHOW VARIABLES LIKE "secure_file_priv";
this will show you the secure path used by MySql to access the files. something like
+------------------+-----------------------+
| Variable_name | Value |
+------------------+-----------------------+
| secure_file_priv | /var/lib/mysql-files/ |
+------------------+-----------------------+
you can either paste files inside this folder or change the "secure_file_priv" variable value to "empty string" so that it can read file from anywhere.
If you are using mysql workbench, just right click on the field (cell) and select 'load value from file' option and then browse to the file and click open and then click on apply. It will automatically generate query like this
UPDATE `dbname`.`tablename` SET `columnname` = ? WHERE (`row` = '1');