I have uploaded a csv file into database as a byte[] data, using hibernate and spring, I have 3 columns in my database (id, file name and byte[] data). I would like to display the 3rd column which is a csv file, into a front page. Any help?
From the database perspective, this is nothing more than querying the table by ID and reading the BLOB from the database.
The tricky thing is that unformatted BLOBs/CLOBs are usually read using streams of some sort, and rarely are available as straight columns. Depends on your database implementation, hibernate can sometimes hide the complexity, but something to watch out for.
In terms of displaying it, it's a matter of using the returned BLOB and throwing it into something to force it to display. A quick search found this, but this is really the more difficult part of the problem, the database side can be fairly boring here.
Related
I have JSON file that I want to save as a blob to Microsoft SQL Server.
The pros for zipping is saving space, the cons is the readability that getting lost.
I want to know if T-SQL has any optimization in which it zips the blobs on its own. I know that columnar databases work this way, like Vertica or Postgres for example.
I personally would not compress them if I wanted to be able to search by them. I do not believe it compresses a blob on it's own. I know for a fact even just very large VARCHAR columns do not compress on their own, so I would not expect a blob to. However there is built in compression you can turn on:
https://blogs.msdn.microsoft.com/sqlserverstorageengine/2015/12/08/built-in-functions-for-compressiondecompression-in-sql-server-2016/
https://learn.microsoft.com/en-us/sql/relational-databases/data-compression/enable-compression-on-a-table-or-index?view=sql-server-2017
There are some advantages to it but usually at the cost of CPU. So if I were you, I'd probably not zip up the files to put in SQL, but I might compress the tables I store. It would depend on exactly what the data was, json probably gets a lot of space back on compression, but a .jpeg would not.
An option I have done in the past is to simply store my files on a content server somewhere, and store in SQL the meta data about the file (name, tags, patch to where I stored it, file extension, etc.) That way my data is easy to get at/put there and I simply use SQL to look it up. Additionally it has allowed me when it was large text files to also use Lucene indexes from solr to make a full text searchable solution since the data wasn't stuffed into a SQL table. Just an idea! :)
One more thought, if I were to store big json files into SQL I would probably choose VARCHAR(MAX) or NVARCHAR(MAX) as my datatype. Anytime I have tried to use TEXT, IMAGE, etc. I would later run into some kind of SQL error if I tried to do a tricky query. I believe Microsoft is trying to use VARCHAR(MAX) to replace the blob type of data types and is slowly deprecating them.
I am designing a system with 30,000 objects or so and can't decide between the two: either have a JSON file pre computed for each one and get data by pointing to URL of the file (I think Twitter does something similar) or have a PHP/Perl/whatever else script that will produce JSON object on the fly when requested, from let's say database, and send it back. Is one more suited for than another? I guess if it takes a long time to generate the JSON data it is better to have already done JSON files. What if generating is as quick as accessing a database? Although I suppose one has a dedicated table in the database specifically for that. Data doesn't change very often so updating is not a constant thing. In that respect the data is static for all intense and purposes.
Anyways, any thought would be much appreciated!
Alex
You might want to try MongoDB which retrieves the objects as JSON and is highly scalable and easy to setup.
I want to store file data of a directory in a file. i.e., file name, file size etc so that I can reduce search time. My problem now is to find an efficient way to do it. I have considered json and xml but can't decide between these two. Also if there is a better way let me know.
I'd say that it's up to what kind of data you prefer to work with and to what structure of data you have (very simple as a list of word, less simple as a list of word and the number of time each word was searched,...)
For a list of word you can use a simple text file with one word per line or coma separated (csv), for a less simple structure, json or xml will work fine.
I like to work with json as it's more light than xml and less verbose. If you didn't plan to share this data and/or it isn't complex, you don't need the validation (xsd,...) offered by xml.
And even if you plan to share this data, you can work with json.
You'll need some server side code to write the data to a file, like php, java, python, ruby,...
I would recommend Json file if you use alomst like a properties file.
If you plan to store the data in the file into database then you can go for XML where u have to the option to use JAXB/JPA in java environment
I'm sitting on a CSV import into a database trying to think of ways to add more data without having to change the API (add new fields).
Since working with JSON quite a bit on the client, I'm thinking of storing data into MYSQL as JSON string. So if I have a field
image_filenames
Where I'm currently storing data like this:
img123.jpg
WOuld it make sense to store multiple images in a JSON array like so:
{"img_base":"img123.jpg", "img_alt_1":"img123-1.jpg", "img_alt_2":"img123-2" }
I can deserialize server side, so it woudn't be much of a problem to grab the image I need from the JSON array, while it does not bloat up the API.
Question:
I can't find anything at all on importing CSV with JSON strings. So, what's good and bad in doing so? Are there security concerns (SQL-injections)?
Thanks!
Transferred from comments to here:
If you have a data model scheme that changes or is inconsistent, then the relational database storage isn't the best choice. Sure, you can serialize it and store as binary string, but why? IMO, and I'm not a fan of NoSQLs, but MongoDB looks like something you might make use of. Its document scheme is JSON-based, it'd be familiar to you if you work with JSON-based code on a daily basis. I'd use that to store the data rather than relational db.
Non-relational ones do less work, so they work faster in some scenarios. They also don't have a scheme, so there's no alter table statement as such, therefore you can add "columns" as much as you like. If you don't have relations and need something to store data in JSON format, but that it can be searchable - MongoDB would be great.
I'm trying to migrate from a custom designed blog software system to a WordPress.com site. I can access my MySQL database of posts and comments without too much difficulty, thanks in part to this post: How do I connect to a MySQL database from Clojure?.
I think my next step is to generate the pseudo-xml file known as a "WordPress eXtended RSS file", and that's where I'm having difficulty. I can navigate my way through the recordsets returned from my database connection (as of now, I'm doing two SQL searches, one for posts, and another for comments, giving me two recordsets), but I don't know what sort of data structure I should put those results into in order to eventually export them into the xml file.
I've been using a lot of file spit/slurps while working my way through this, but I have a feeling that's neither the most efficient way to do it, nor the most "lispy" way.
I think what I need is help defining a clojure-based data structure that I can append both new posts and new comments to, iteratively, for eventual export into the xml file format I need it to finally get to.
Look at clojure.xml/emit and clojure.xml/emit-element; they expect data that look as described in (doc clojure.xml/parse) and walk this data to output XML.