I have a couple files that I want to store together in a blob type column of a mysql table. So I just put them into a folder and then zipped it. I've never had any trouble storing images, text and pdf files using the load_file() function, but when I try with the .zip folder I get back a NULL value.
What am I missing? Thanks!
I have noted the same phenomenon.
It does seem a bit strange indeed and OS related. Here is the result of my investigation (using MARIA DB 10.4, Windows 10 Pro20H2):
In a given folder, C:\zipfolder for ex., I've created a textfile zipdoc.txt with some text content and a zip file containing the textfile.
This gives the folowing load_file output:
select load_file('C:\\zipfolder\\zipdoc.txt');
+----------------------------------------+
| load_file('C:\\zipfolder\\zipdoc.txt') |
+----------------------------------------+
| zipcontent text |
+----------------------------------------+
select load_file('C:\\zipfolder\\zipdoc.zip');
+----------------------------------------+
| load_file('C:\\zipfolder\\zipdoc.zip') |
+----------------------------------------+
| NULL |
+----------------------------------------+
Changing the file extension from .zip to .zip_ for ex. fixes the issue:
select load_file('C:\\zipfolder\\zipdoc.zip_');
+---------------------------------------------------------------------------------------------------------------------------------------+
| load_file('C:\\zipfolder\\zipdoc.zip_') |
+---------------------------------------------------------------------------------------------------------------------------------------+
| PK♥♦¶ FÄLR├SAÏ☼ ☼
zipdoc.txtzipcontent textPK☺☻¶ ¶ FÄLR├SAÏ☼ ☼
☺ zipdoc.txtPK♣♠ ☺ ☺ 8 7 |
+---------------------------------------------------------------------------------------------------------------------------------------+
So, it looks like Windows 10 is blocking the access to .zip files in a more restrictive way than other files.
Giving EVERYONE access to the zip-file allows the load_file function accessing of the original zip-file. After granting the access with the following Powerhell script (adopted from here):
$acl = Get-Acl C:\zipfolder\zipdoc.zip
$AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule("Jeder","Read","Allow")
$acl.SetAccessRule($AccessRule)
$acl | Set-Acl C:\zipfolder\zipdoc.zip
load_file is able to access the zipfile:
select load_file('C:\\zipfolder\\zipdoc.zip');
+---------------------------------------------------------------------------------------------------------------------------------------+
| load_file('C:\\zipfolder\\zipdoc.zip') |
+---------------------------------------------------------------------------------------------------------------------------------------+
| PK♥♦¶ FÄLR├SAÏ☼ ☼
zipdoc.txtzipcontent textPK☺☻¶ ¶ FÄLR├SAÏ☼ ☼
☺ zipdoc.txtPK♣♠ ☺ ☺ 8 7 |
+---------------------------------------------------------------------------------------------------------------------------------------+
So, the solution is to grant EVERYONE access to the zip-files or just changing the extension of the files (It remains a task for admins to find a more restrictive working access level).
Complement: As mentioned by #Álvaro González, the use of an archiving program that sets the appropriate rights is also a solution.
I cannot reproduce the problem. See console output:
mysql> CREATE TABLE test (val BLOB);
Query OK, 0 rows affected (0.29 sec)
mysql> INSERT INTO test SELECT LOAD_FILE('C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\test.sql');
Query OK, 1 row affected (0.05 sec)
Records: 1 Duplicates: 0 Warnings: 0
mysql> INSERT INTO test SELECT LOAD_FILE('C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\test.zip');
Query OK, 1 row affected (0.04 sec)
Records: 1 Duplicates: 0 Warnings: 0
mysql> SELECT LENGTH(val) FROM test;
+-------------+
| LENGTH(val) |
+-------------+
| 5603 |
| 17725 |
+-------------+
2 rows in set (0.00 sec)
Related
I have a MySql server locally (on Windows 10) and want to run a python script from a trigger. For taht, i want to use sys_exec().
I got the lib_mysqludf_sys_64.dll from here: https://github.com/rapid7/metasploit-framework/tree/master/data/exploits/mysql
and set the UDF up as described here: https://osandamalith.com/2018/02/11/mysql-udf-exploitation/
sys_exec is working:
mysql> SELECT sys_exec('python --version')//
+------------------------------+
| sys_exec('python --version') |
+------------------------------+
| 0 |
+------------------------------+
1 row in set (0.52 sec)
but when i want to run the script:
mysql> SELECT sys_exec('python C:\dbtest.py test')//
+--------------------------------------+
| sys_exec('python C:\dbtest.py test') |
+--------------------------------------+
| 2 |
+--------------------------------------+
1 row in set (0.07 sec)
As far as i understand, the 2 indicates, that the file was not found.
How can i fix this?
When i use '\\', the exit code changes to 1 (Invalid_Function ?):
mysql> SELECT sys_exec('python C:\\dbtest.py test')//
+---------------------------------------+
| sys_exec('python C:\\dbtest.py test') |
+---------------------------------------+
| 1 |
+---------------------------------------+
1 row in set (0.57 sec)
the MySQL UDF SYS is not installed.
Here you can find the Answer to do that : Mysql sys_exec Can't open shared library 'lib_mysqludf_sys.so' (errno: 11, wrong ELF class: ELFCLASS32)
here also the git for download: https://github.com/mysqludf/lib_mysqludf_sys
I found the problem was with the access permissions of the mysql user. I wasn't able to find out wich (windows) user actually runs the python script and needs access to the used files, but the way i solved the problem was to grant "everyone" access to the files involved.
I am trying to do IPv6 address range matching using LineString and Point. But I believe this question is more related to how MySQL handles large number precision in general. (Note the following works without issues with smaller numbers, i.e. IPv4 decimal representation).
For IPv4, the ranges are defined as LineString(Point(-1, ip_start), Point(1, ip_end)).
I seem to be losing precision when creating a Point() with a large number, therefore my range matching is flawed.
MariaDB [mydb]> set #point=Point(0, 42541828702485584113142439188939931648);
Query OK, 0 rows affected (0.00 sec)
MariaDB [mydb]> select astext(#point);
+-------------------------------+
| astext(#point) |
+-------------------------------+
| POINT(0 4.254182870248558e37) |
+-------------------------------+
1 row in set (0.00 sec)
MariaDB [mydb]> select st_y(#point);
+----------------------+
| st_y(#point) |
+----------------------+
| 4.254182870248558e37 |
+----------------------+
1 row in set (0.00 sec)
MariaDB [mydb]> select cast(st_y(#point) as Decimal(39));
+----------------------------------------+
| cast(st_y(#point) as Decimal(39)) |
+----------------------------------------+
| 42541828702485580000000000000000000000 |
+----------------------------------------+
1 row in set (0.00 sec)
MariaDB [mydb]>
How can I use a Point() with large numbers accurately?
SPATIAL functions (st_x, point, etc) work with DOUBLEs, not DECIMAL.
DECIMAL(39) should work. But why not just keep it as a string or hex string (VARCHAR(39) CHARACTER SET ascii) or BINARY(16)?
Please explain your ultimate goal that led you to want to use Spatial tools.
Here is code to handle ranges of IP addresses (separate code for IPv4 or IPv6): http://mysql.rjweb.org/doc.php/ipranges
I create a table in mysql on macos commandline using the 'utf-8' charset,
mysql> CREATE TABLE tb_stu (id VARCHAR(20), name VARCHAR(20), sex CHAR(1), birthday DATE) default charset=utf8;
Query OK, 0 rows affected (0.02 sec)
mysql> SHOW TABLES;
+----------------+
| Tables_in_test |
+----------------+
| pet |
| tb_stu |
+----------------+
2 rows in set (0.00 sec)
mysql> show create table tb_stu \G
*************************** 1. row ***************************
Table: tb_stu
Create Table: CREATE TABLE `tb_stu` (
`id` varchar(20) DEFAULT NULL,
`name` varchar(20) DEFAULT NULL,
`sex` char(1) DEFAULT NULL,
`birthday` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.00 sec)
I want to add some values to the 'tb_stu' table, I have a txt file containing Chinese string :
1 小明 男 2015-11-02
2 小红 女 2015-09-01
3 张三 男 2010-02-12
4 李四 女 2009-09-10
and the txt file is 'utf-8' charset too!
➜ ~ file /Users/lee/Desktop/JAVA/Java从入门到精通/第18章--使用JDBC操作数据库/Example_18_02/tb_stu.txt
/Users/lee/Desktop/JAVA/Java从入门到精通/第18章--使用JDBC操作数据库/Example_18_02/tb_stu.txt: UTF-8 Unicode text
so I execute the mysql command line:
mysql> LOAD DATA LOCAL INFILE '/Users/lee/Desktop/JAVA/Java从入门到精通/第18章--使用JDBC操作数据库/Example_18_02/tb_stu.txt' INTO TABLE tb_stu;
Query OK, 4 rows affected, 4 warnings (0.01 sec)
Records: 4 Deleted: 0 Skipped: 0 Warnings: 4
but I get the messy code in mysql :
mysql> select * from tb_stu;
+------+----------------+------+------------+
| id | name | sex | birthday |
+------+----------------+------+------------+
| 1 | å°æ˜Ž | ç | 2015-11-02 |
| 2 | å°çº¢ | å | 2015-09-01 |
| 3 | å¼ ä¸‰ | ç | 2010-02-12 |
| 4 | æŽå›› | å | 2009-09-10 |
+------+----------------+------+------------+
4 rows in set (0.00 sec)
it makes me confused, the tabel in mysql and the txt are both 'utf-8' charset, why I get the messy code? thanks a lot!
You will need to investigate some more to understand your problem. One of the options for example is that your data was written into DB correctly but in your command line it is just displayed incorrectly due to some wrong setting of encoding in your operating system environment. Or the problem might be that the data was garbled (corrupted) when it was written and that means that it is wrongly stored in the DB. So I would suggest to take your original file with properly displayed Chinese characters and convert them to unicode sequence, and then take the data in DB and also convert them into unicode sequence and compare to see if your DB data is just displayed incorrectly or the data itself is corrupted. This will help you to understand your problem and then to find a way to fix it. Here is tool that can help you:
There is an Open Source java library MgntUtils (written by me) that has a Utility that converts Strings to unicode sequence and vise versa:
result = "Hello World";
result = StringUnicodeEncoderDecoder.encodeStringToUnicodeSequence(result);
System.out.println(result);
result = StringUnicodeEncoderDecoder.decodeUnicodeSequenceToString(result);
System.out.println(result);
The output of this code is:
\u0048\u0065\u006c\u006c\u006f\u0020\u0057\u006f\u0072\u006c\u0064
Hello World
The library can be found at Maven Central or at Github It comes as maven artifact and with sources and javadoc
Here is javadoc for the class StringUnicodeEncoderDecoder
I have a MYSQL database with a column named img of BLOB type.
When I insert a value into that column like this :
LOAD_FILE('C:/Documents and Settings/All Users/Documents/My Pictures/Sample Pictures/Sunset.jpg')
it works !
But like this :
LOAD_FILE('C:/Documents and Settings/Administrator/My Documents/My Pictures/picture.jpg')
it doesn't work and it tells me that the column img cannot be null !
And in both cases the file exists, and I'm connecting to the database as the root user (all privileges), so I don't understand why I'm getting this error.
Thanks in advance
Maybe problem with max_allowed_packet
1.jpg is a small picture and 2.jpg is a big picture
mysql> DESCRIBE blob_files;
+-------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------+---------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| file | blob | YES | | NULL | |
+-------+---------+------+-----+---------+----------------+
2 rows in set (0.01 sec)
mysql> INSERT INTO blob_files(file) VALUE(LOAD_FILE('D:/2.jpg'));
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> SHOW WARNINGS;
+---------+------+--------------------------------------------------------------
------------------+
| Level | Code | Message
|
+---------+------+--------------------------------------------------------------
------------------+
| Warning | 1301 | Result of load_file() was larger than max_allowed_packet (104
8576) - truncated |
+---------+------+--------------------------------------------------------------
------------------+
1 row in set (0.00 sec)
mysql> INSERT INTO blob_files(file) VALUE(LOAD_FILE('D:/1.jpg'));
Query OK, 1 row affected (0.05 sec)
I'm going to post this as an answer, and I'll modify it as needed. In the comments I mentioned mysqld needs to run as administrator. Upon consideration, I realized this is actually not a good idea, since windows UAC is in place for a reason. A better option is to add the necessary permission to the folder.
Go to your My Documents folder under administrator, right click on My Pictures, go to security, and add "LOCAL SERVICE" to your permissions with the read attribute. Then your MySQL server should be able to read from that folder.
If you want to verify LOCAL SERVICE is the proper account, go to start -> run and type services.msc and press enter. Find MySQL, right click and hit properties, go to the Log On tab and see which account it runs as. This is the one you should add to the security tab of the folder with the read permissions.
This may be a little late but how I insert blob files when I just need them as sample data is like this:
Using MysqlWorkbench I enter a record into my blob table without applying the insert.
I right-click the blob field, open value editor then upload an image there.
Now before applying the insert I copy the row and paste it repeatedly (a procedure loop can be used as well)
I do this same thing when updating blob fields as well except in the WHERE clause I would use WHERE primaryKey >= this value this would update all the samples
Using a decimal (I have tried variations of it) 0.5 is always converted to 5.
I can store 1.5, etc, no problem... Just curious on how I need to set up my data type to correctly store a '0.5' number.
Thanks
EDIT:
The current data type is Decimal (3,1). I have tried float as well.
I simply retreive the number through $_POST from a Form text box which is restricted to numbers only through Java. I never considered that maybe the browser is simply sending it as a 5, I don't know.
EDIT 2: I have confirmed through echo that the 0.5 is sent fine, it's once it gets into the database that it becomes 5
You know, I might get voted down for this, but I've always had success (as a sort of hack) in storing values like this as a VARCHAR and then using format() function in PHP when I need to make calculations. MySQL still seems to parse it well enough for SQL functions as well. Just my 2 cents :)
I used a decimal value in a database with various inputs:
I have tried to recreate you issue, but I get the following:
mysql> create table test(myDec decimal(2,2));
Query OK, 0 rows affected (0.02 sec)
mysql> insert into test values(0.5);
Query OK, 1 row affected (0.01 sec)
mysql> insert into test values(.5);
Query OK, 1 row affected (0.00 sec)
mysql> select * from test;
+-------+
| myDec |
+-------+
| 0.50 |
| 0.50 |
+-------+
2 rows in set (0.00 sec)
Given your edit, I think that you might be getting values from international visitors, where the decimal point might be formatted using a comma instead of a dot?
mysql> insert into test values('0,5');
Query OK, 1 row affected, 1 warning (0.00 sec)
mysql> select * from test;
+-------+
| myDec |
+-------+
| 0.50 |
| 0.50 |
| 0.00 |
+-------+
3 rows in set (0.00 sec)