I'm receiving IEEE754 32-bit floating point data from a remote system and storing it in a MySQL (MariaDB) database. I'm trying to ensure that I can retrieve the data from the database exactly as it was received, so ideally I would be able to insert the binary data directly (to avoid any rounding errors).
Since MySQL uses IEEE754 for floats, I'm wondering if there is any way to work with the binary data directly, or do I have to store data as binary data and convert it outside of MySQL (or store two copies of the data, one in a binary column and one as float column)?
Related
I'm using Visual Basic 2010 to build a Windows application, using an Access *.mdb for it's memory. Most of the data reads and writes work correctly, except for where I'm entering data into a datagridview, and using a table adapter update to save the data to the memory via a dataset data source. Again, that works fine until I get to 9 significant figures, and I start to loose precision, for example, 1,033,241.78 saves as 1,033,241.75.
Where I use an oledb connection, I get no such precision loss; 1,033,241.78 saves as 1,033,241.78.
The numeric fields of the database are double, the numeric fields of the dataset are double, so according to all of the best search results, I should be no-where near my precision limit.
Please, any clues?
First of all, i understand that storing images into DB is a bad practice, however i need to do it.
I have a table in MySQL (..., img BINARY, ..) and have some images on my machine.
How should I transform my input data, to make it avaliable store in BINARY field? And how to decode it into original format again?
I'd like to know if you can query a blob (large/medium any kind) column and retrieve the bytes from N to M so you can query a huge blob file and only get small chunks of it in your resultset. If this is possible in MySQL, how can you do it (an example please!)?
I found this question for plain text but what about doing the same for bytes?
You can find the answer right here: MySQL blob: how to get just a subset of the stored data
MySQL treats blobs the same as strings (more or less):
BLOB values are treated as binary strings (byte strings). They have no character set, and sorting and comparison are based on the numeric values of the bytes in column values.
So all the usual string functions work on blobs. In particular, you can use substring to grab just part of of a blob.
That said, storing a multi-gigabyte data file in a relational database as a BLOB isn't the best thing to do. You'd be better off storing the file's metadata in the database and leaving the file itself in the file system; file systems are pretty good at managing files, relational databases are good at handling structured data.
I have converted from a MySQL database to Postgres. During the conversion, the picture column in Postgres was created as bytea.
This Xojo code works in MySQL but not Postgres.
Dim mImage as Picture
mImage = rs.Field("Picture").PictureValue
Any ideas?
I don't know about this particular issue, but here's what you can do to find out yourself, perhaps:
Pictures are stored as BLOBs in the database. Now, this means that the column must also be declared as BLOB (or a similar binary type). If it was accidentally marked as TEXT, this would work as long as the database does not get exported by other means. I.e, as long as only your Xojo code reads and writes to the record, using the PictureValue functions, that takes care of keeping the data in BLOB form. But if you'd then convert to another database, the BLOB data would be read as text, and in that process it might get mangled.
So, it may be relevant to let us know how you converted the DB. Did you perform a export as SQL commands and then imported it into Postgres by running these commands again? Do you still have the export file? If so, find a record with picture data in it and see if that data is starting with: x' and then contains hex byte code, e.g. x'45FE1200... and so on. If it doesn't, that's another indicator for my suspicion.
So, check the type of the Picture column in your old DB first. If that specifies a binary data type, then the above probably does not apply.
Next, you can look at the actualy binary data that Xojo reads. To do that, get the BlobValue instead of the PictureValue, and store that in a MemoryBlock. Do the same for a single picture, both with the old and the new database. The memoryblock should contain the same bytes. If not, that would suggest that the data was not transferred correctly. Why? Well, that depends on how you converted it.
I would like to use MYSQL as a storage system for a huge number of files.
I would like to read/write just a portion of the data stored in a column (data is stored as bytes) so I don't have to load the entire file into the application (because it can be > than a GB).
So, in brief, I would like to have random read/write access in a blob column without loading the entire data into memory.
Are there functions available to perform these operations? Thank you.
MySQL treats blobs the same as strings (more or less):
BLOB values are treated as binary strings (byte strings). They have no character set, and sorting and comparison are based on the numeric values of the bytes in column values.
So all the usual string functions work on blobs. In particular, you can use substring to grab just part of of a blob.
That said, storing a multi-gigabyte data file in a relational database as a BLOB isn't the best thing to do. You'd be better off storing the file's metadata in the database and leaving the file itself in the file system; file systems are pretty good at managing files, relational databases are good at handling structured data.
You can try this approach. Store the meta data of your files (like path, name, etc.) in the database and store the files under a directory.
From the database you can fetch the filepath and the read the file in random access mode. Using the file-offset you can get the required subset of the stored data.
You could use e.g. MID() [1] to cut portions of the BLOB; though I would prefer to store files in the file system, not in a database. MySQL performs rather poor on BLOBs.
[1]
http://dev.mysql.com/doc/refman/5.1/en/string-functions.html#function_mid