I have an application that modifies a table dynamically, think spreadsheet), then upon saving the form (which the table is part of) ,I store that changed table (with user modifications) in a database column named html_Spreadhseet,along with the rest of the form data. right now I'm just storing the html in a plain text format with basic escaping of characters...
I'm aware that this could be stored as a separate file, the source table (html_workseeet) already is. But from a data handling perspective its easier to save the changed html table to and from a column so as to avoid having to come up with a file management strategy (which folder will this live in, now must include folder in backups, security issues now need to apply to files, how to sync db security with file system etc.), so to minimize these issues I'm only storing the ... part in the database column.
My question is should I gzip the HTML , maybe use JSON, or some other format to easily store and retrieve the HTML from the database column, what is the best practice to store HTML content in a datbase? Or just store it as I currently am as an escaped text column?
If what you are trying to do is save the HTML for redisplay, what's wrong with saving it as is, then just retrieving it via a stored proc, and re-displaying it for them when needed?
Say you have an HTML page, which can select some kind of ID from a list, either on a ThickBox page, or from a select option.
Normally for this kind of situation, you would probably query the DB via $Ajax possibly JSon, or not.
Then the result sent back to the $Ajax call will be your resultant data.
Then you replace the Div which holds your SpreadSheet with the DB SpreadSheet.
So, in answer to your original question, you could store the SpreadSheet with some sort of ID, storing it as the HTML of the Div.
When retrieved, you merely replace the Div HTML, with what you have stored.
It depends on the size of the HTML. You could store it as a binary BLOB after zipping it. Most of the time it is just best to store the data directly after escaping the SQL characters that may cause problems.
As asked many times why are you storing the view instead of the model ?
You probably should bit the bullet and parse the table (using an HTML parser perhaps), as otherwise you risk the user storing damaging JavaScript in the table data. (This means the cell contents should be parsed as well). The data can still be stored as a blob (maybe compressed CSV or JSON), but you need to make certain it is not damaging.
Related
Theorically i have a group (2000 - 3000) of data structure it's a local obituary project, with 5 item/value for each (name, birth, death, function, photo url) and i want :
to load this data in html page,
format each data into styled div,
allow user to filter (name, function etc) these divs (hide/show) via js or jquery and inputs
i know the best way to do this is mysql + php, but since i have no knowledge yet in this field My question is :
Is it correct to use a json file to store these data and populate divs ?
can we filter json data with jquery ?
Thanks
Define database. Can it be used to store data? Yes. But it has no search abilities, it isn't ACID compliant, and accessing it would run in the current process. Any non-trivial filtering would require an O(N) process, unless you're going to build index tables in memory. It would also be nearly impossible to keep in sync across processes/machines if the user is allowed to edit any of the data at all. If its read only and N is small, it can work. If N is not small or you need to be able to update it, a JSON file is a bad idea.
I have a table in My Sql qhere i have fields lie name, location, description and picture.
What I want to do is store multiple picture links in the picture column.
Is there a way of doing that without creating a separate table for picture?
Thank you
Well you need to perform some sort of serialization in order to do that. I used to do that before I moved to document-oriented databases. Quite possibly your best option is to store everything in a json format as it is pretty universal and I can't think of any language that cannot handle it and convert it back to an object, array, dictionary or whatever the language requires. Assuming you need to save the name of the file as in somefile.png, what you could do is store ["image1.png","image2.png","image3.png"] and so on. If you want to store a blob however it's a bit more complicated. You either have to create a second table or read the contents of each image, convert it to base64, load all base64 strings into an object and then serialize it into a json. I wouldn't recommend that as each operation would cost a lot of system resources.
My basic task is to import parts of data from one single file, into several different tables as fast as possible.
I currently have a file per table, and i manage to import each file into the relevant table by using LOAD DATA syntax.
Our product received new requirements from a client, he is no more interested to send us multiple files but instead he wants to send us single file which contains all the original records instead of maintaining multiple such files.
I thought of several suggestions:
I may require the client to write a single raw before each batch of lines in file describing the table to which he want it to be loaded and the number of preceding lines that need to be imported.
e.g.
Table2,500
...
Table3,400
Then i could try to apply LOAD DATA for each such block of lines discarding the Table and line number description. IS IT FEASIBLE?
I may require each record to contain the table name as additional attribute, then i need to iterate each records and inserting it , although i am sure it is much slower vs LOAD DATA.
I may also pre-process this file using for example Java and execute the LOAD DATA as statement in a for loop.
I may require almost any format changes i desire, but it have to be one single file and the import must be fast.
(I have to say that what i mean by saying table description, it is actually a different name of a feature, and i have decided that all relevant files to this feature should be saved in different table name - it is transparent to the client)
What sounds as the best solution? is their any other suggestion?
It depends on your data file. We're doing something similar and made a small perl script to read the data file line by line. If the line has the content we need (for example starts with table1,) we know that it should be in table 1 so we print that line.
Then you can either save that output to a file or to a named pipe and use that with LOAD DATA.
This will probably have a much better performance that loading it in temporary tables and from there into new tables.
The perl script (but you can do it in any language) can be very simple.
You may have another option which is to define a single table and load all your data into that table, then use select-insert-delete to transfer data from this table to your target tables. Depending on the total number of columns this may or may not be possible. However, if possible, you don't need to write an external java program and can entirely rely on the database for loading your data which can also offer you a cleaner and more optimized way of doing the job. You will much probably need to have an additional marker column which can be the name of the target tables. If so, this can be considered as a variant of option 2 above.
This is a complex problem, so I'm going to try to simplify it.
I have a mysql instance on my server hosting a number of schemas for different purposes. The schemas are structured generally (not perfectly) in a EAV fashion. I need to transition information into and out of that structure on a regular basis.
Example1: in order to present the information on a webpage, I get the information, stick it into a complex object, which I then pass via json to the webpage, where I convert the json into a complex javascript object, which I then present with knockoutjs and similar things.
Conclusion: This resulted in a lot of logic being put into multiple places so that I could associate the values on the page with the values in the database.
Example2: in order to allow users to import information from a pdf, I have a lot of information stored in pdf form fields. In this case, I didn't write the pdf though, so the form fields aren't named in such a way that all of this logic is easy enough to write 3 or more times for CRUD.
Conclusion: This resulted in my copying a list of the pdf form fields to a table in the database, so that I could then somehow associate them with where their data should be placed. The problem that arose is that the fields on the pdf would need to associate with schema.table.column and the only way I found to store that information was via a VARCHAR
Neither of the examples are referring to a small amount of data (something like 6 tables in example 1 and somewhere around 1400 pdf form fields in example 2). Given Example1 and the resulting logic being stored multiple places, it seemed logical to build Example2, where I could store the relationships between data in the database where they could be accessed and changed consistently and for all involved methods.
Now, it's quite possible I'm just being stupid and all of my googling hasn't come across that there's an easy way to associate this data with the correct schema.table.column If this is the case, then telling me the right way to do that is the simple answer here.
However, and this is where I get confused. I have always been told that you never want to store information about a database in the database, especially not as strings (varchar). This seems wrong on so many levels and I just can't figure out if I'm being stupid, and it's better to follow Example1 or if there's some trick here that I've missed about database structure.
Not sure where you got "... never ... store information about a database in the database". With an EAV model it is normal to store the metamodel (the entity types and their allowable attributes) in the database itself so that it is self-describing. If you had to change the metamodel, would you rather change code or a few rows in a table?
The main drawback to EAV databases is that you lose the ability to do simple joins. Join-type operations become much more complex. Like everything else in life, you make tradeoffs depending on your requirements. I have seen self-describing EAV architectures used very successfully.
I'm building a very simple MTurk-ish app in Rails. The idea is that people will upload csvs containing whatever columns they want (e.g., some id, name of a user, some piece of text, a link, whatever -- these columns will change from task to task), and these csvs will contain all the information for the MTurk task.
My question is: how would I store these csvs in a database? One way is to store each csv row as a blob of unstructured data in MySQL (i.e., I basically leave each row as a string and stick this into a MySQL column). A maybe better way is to use a NoSQL database like MongoDB, where I don't need a predefined schema.
Suggestions? Which way is better, or is there another option? I am using Rails for this, so options that work well with Rails would be great.
Well you pretty much answered your own question.
Either use a NoSQL Document based database (like MongoDB) or split up the cvs and save it in a 1:n correlation within your database as key value pairs attached to a row and column each. your idea to store blobs isn't quite ideal however as it would restrict you from searching within the columns.