tl;dr
In my node.js application I create pdf documents. What is the best/right way to save them? Right now I use node.js fileserver and shell.js to do it.
I am working on a node.js web application to manage apartments and tenants for learning purpose and on some point I create PDF Documents that I want to save under a path
/documents/building_name/apartment_name/tenant_name/year/example.pfd
Now if the user wants to change the building, apartment or tenant name via an http PUT request I change the database but also the want to change the path.
Well both works but I can't write good tests for these functions.
Now a friend told me that it's a bad practice to save documents on a file server and I better should use BLOB.
On the other side google doesn't really agree on using blobs
So what is the right way to save documents?
Thanks
Amit
You should first define a source of truth. Unless you're legally obliged to keep copies of those files and they are not being accessed very often, I wouldn't even bother storing those at all and just generate them upon request.
If not, keep the DB clean, blobs will make it huge. Put them into cold storage (again assuming they are not being accessed too frequently) without those paths. If the paths are reliant on often changing information, that can't be performant for neither the file server nor your system.
Instead store a revision number in your DB that the file can be found under and limit the path structure to information that rarely change.
Like {building}/{apartment}/{tenant}_{revision}.pfd
That - depending on your backup structure - will allow you to time-travel if necessary and doesn't force a re-index all the time.
Note: I don't know too much about your use case.
Related
I was wondering whether is that possible to have xlsm file as backend while having html as frontend? How can I achieve this if yes?
Thanks in advance.
Since the question lacks the understanding of an application structure in the programming realm, I will put this as an answer hoping to clarify a few things.
First of all I don't think you understand what the term "back-end" means.
Please read https://en.wikipedia.org/wiki/Front_and_back_ends AND http://blog.teamtreehouse.com/i-dont-speak-your-language-frontend-vs-backend
hopefully these will clarify a few thing for you.
Just to explain these concepts shortly:
In an application Front-end and back-end refer to two interfaces that communicate with each other and exchange data in some form. Such separation is made when the program and the user are separate (such as when you have a server and a client such as in distributed programming). This however, is only one of many programming patterns today. Although rare in today's world, there are programs that do not separate functionality in such way and thus delegate all this functionality to the core program that is statically installed on the clients computer. But in other cases here is what the terms front-end vs. Back-end means:
Reason why such separation is necessary:
In today's world many applications (such as web applications and mobile applications) are deployed on common servers to provide wider and faster access, better support and to reduce the cost of access for the client (not requiring any space, no download time etc.). However in such cases, since the client doesn't have access to the program locally, they need to access it over internet protocols such as TCP (which is used by today's http). The problem is that the frontend files are served everytime the application is loaded and can not keep track of states of data (they are stateless) [excluding the edge cases of cookies and caches]
Front End:
The sole reason that the front end exists for the user to interact with the application and to collect data from the user such as login information etc. (User Interface)
Back end:
Now back-end is a little bit more complicated. There are 2 major components to a good back-end design:
Logic
Data
The backend is responsible for processing the data from the user (front-end) in a correct and meaningful way. For example in a really simple program which adds two numbers the front end would be responsible of asking the user for two numbers and the back-end would carry out the actual addition and send the result back to front end to be displayed.
If the data has states. The backend would also need to save the last state of the data somewhere on the server. This is where the second component comes in. The most common practice is to have a ".db" file(s) which represents a database. However there is no obligation for you to do so. When necessary, if you wanted your backend could read data any where from a plain text file to STDIN.
Why do we use databases? ==> The queries. Query languages that come with data bases make it so much easier for us to extract and isolate the relevant data
After processing and modifying the data, the backend sends it back to the front end to be displayed to the user. The common data transferring ways are JSON, XML and SExpressions.
So following this short lecture, back to your question:
Can I have an xlsm file in the backend?
Yes. You can preserve the data in the backend(server) in anyway that you want. The only thing you need to make sure is that the endpoint the front end communicates to reads data from this file and writes back onto this file. (Sometimes CSV files are used in such a way that is similar to xlsm files)
Is it a good idea?
No. Databases exists for a reason. Use them.
Hope this sheds light on a few things. I highly advise you understand the application stack before writing any code
I am building a website with ROR 3. I need to provide a page to my clients wherein he could edit his pricing info regarding the application. I am quite confused on how to do this. The pricing page needs to be displayed as an html table with different columns which has got the pricing info.
I am thinking of different ways to do this.
1) Allow the client to create and upload an html page and then save it as a file in my public directory and render as an when the client clicks on the pricing link.
2) The clients may not have bare technical knowledge, hence make the client upload some other formats like Word, Excel etc and then parse it and store it as an HTML file in the public directory.
3) Provide the client with some real time editing tools where in the client could edit in a fixed format, and after wards save the file and render it later.
Also, I wouldn't like to store these infos in my database. There would be quite a few number of clients and hence managing all these data in my database would become cumbersome. Storing all these as plain html files and rendering it later would be the most ideal thing for me.
There might be other better steps in doing this as well. Could you please suggest which might the better, or any other option that could suit my needs? Basically I would want my clients to have a mechanism where they could provide there pricing details, edit it later and display it back as an html table, all this without using an Database backend. Any suggestions would be mostly appreciated.
Good way is Excel(csv format).
You can do PHP with Excel. I thing this is the best solutions for your requirement.
Try this.
http://php.net/manual/en/function.fgetcsv.php
If you are give authority to user to change edit contain and you have to used " CSV or Excel" please see these links:
Importing CSV and Excel
Exporting CSV and Excel
If you really don't want to use database then you can use YAML as a structured storage.
e.g. ( you, most probably, could come up with a better structure )
SMS_Pack:
Sl_No:
1: 10000
2: 25000
3: 50000
You can read those .yml files and parse them as hashes. Should be fairly easy to represent that hash as a HTML table.
For the creation, I'm sure you can come up with some dynamic form input. Or to just let the client send this kind of file ( which might not be the best solution ).
But it just might be easier to manage all of this information within a database.
I was wondering what the best pattern was to handle the management of images these days when using spring/java and mysql.
I have several options. Some of the
images are just small avatars for
the users. Is it fine to put these
directly into mysql? Or use the file
system?
For the larger images, is file
system pretty much the only option,
and then use mysql to store the
location on the file system?
Where is a good spot to put them on
a linux server? /var/files/images?
Since the files are hidden from the
war deployment directory, what is
the best way to stream them? Use
some kind of a file output stream as
the response body for an http
request?
Also, do I have to develop all of
the file management stuff myself,
like cleaning up unused files and
the like?
What about image security? Some images should not be accessed by everyone. I think I'd need to use a separate url with Spring security checking the current user for this.
I'd appreciate advice on all of these questions. Thanks.
You could use MySQL, and that would have the advantage of centralization and easy cleanup, but IMHO it's a waste of the database's resources if you plan to scale.
For data like images where everything is public, consider something like Amazon S3 which allows you to serve images directly from S3's web servers. If you plan to host everything yourself, just serve from a directory. Just remember to turn directory listings off :)
What is the best way to create an Archive of image documents in the database ?
Given we have about 2-10 million records and each record includes 2-4 images and about 20 text fields , what is the best way for create this archive so that we have good speed and high security for data?
Also, what database is good for this project?
Definitely use the file system as Minor suggested.
One option is SQL Server FILESTREAM. See http://msdn.microsoft.com/en-us/library/cc949109.aspx.
Use file system storage for archive image. You must save link in DB for the image file. And if you use a HTTP content you can use the cache proxy server such as Squid, Nginx, etc.
More questions for you:
How dynamic is the data? Do you store it once and never change it or it gets frequently changed?
Do you need versioning for the documents or the latest version overwrites the previous and that's it.
Are the documents always edited using one application or they can be changed outside (ex: using Word)
Are the documents related to other "non-document" data (database rows) or is it the only thing that you need to store?
File system won't offer any real security, so I would discount that straight off.
In Oracle there is built-in image support through the ORDImage type.
Check out Marcel's blog as he, and the Piction company, do a lot of work in this area and he has lots of useful material to download.
You can use control downloads. Look at http://kovyrin.net/2006/11/01/nginx-x-accel-redirect-php-rails/lang/en/
When I say "configure" I mean where to save those values that could change very often (constants values like taxes rates or something similar) and then when you need to change them you don't want to re-compile your application.
Where to save those values? Database? XML File? Flat File?
It depends on how often these change and who or what changes them. For some application specific settings, it's best to use an XML or config file, where the developers are the ones responsible for updating it. For other "businessy" values (like exchange rates, tax rates, etc), it's best to keep them in the database and provide a UI for users (not developers) to update.
It also depends on how many apps depend on this value, for example, if several applications depend on some setting (such as email server addres), it's best to put it in a database since it'll be easily accessible from any machine where the app is running.
I use INI files for potentially user-configurable files, and BIN files for data that save session state between runs.
But, it is very dependent upon what type of application you are developing.
it depends on how your app is architecture. you could design your app in such way that you could change the location of you configuration. by just injecting the provider.
Normally I use Ini files or XML if the data is structured.
For applications that already use a database and you don't want to have the user to change the data easily, you can use the database.
I almost never use binary data unless you want to obfuscate the data for the user.
Regardless of app, you're probably going to have at least 3 sources of configuration data:
Command line flags, usually for bootstrapping your run-time environment, e.g, finding config files, setting debug flags, include paths, class paths, etc
Config files, potentially more than one that may override each other. These usually boot strap your application: connection strings, cache settings, build-specific settings, etc
Control data in a database. Things like timezones, conversion rates, stable display values, etc. This data should also be versioned in the database (as in, a "Data Version" field, not living in a version control system). Versioning it will save a lot of headaches when you find you need to change a setting for a new release, but the old release will break if you change it.
Generally, anything that changes at run-time should go in the database. Anything that is sensitive and rarely changing should go into the config files, and any hacks should go on the command line (--[no]enable-bug-287438-hack can be very handy when you need it).
I prefer the simplicity of a flat ini file. Here's an example Setting class that you might find useful.