Nowadays, we always think like "send your data to a server, it computes it for you, then send you back the response".
But imagine something else : i want my client to compute the data itself.
The question is : is there something like a universal protocol to send actions rather than data through http ? So that the server can send the action to the client, whatever system it uses. If it does not exist, what are the technical difficulties you can face creating this kind of system ?
I'm talking about "static" actions, like mathematical functions for example.
You're unfortunately going to run into a problem pretty quick because, technically speaking, a universal language is impossible. Systems are going to have different architecture, different languages available, and different storage means. I believe what you intend (correct me if I'm wrong) is a "widespread" protocol. One way or another, you're going to have to drill down based on your personal use-case.
For a widespread example, you could keep a set of JavaScript files with functions server-side, and refer a web client to the one they need to run it by loading a javascript file during some event. Pass the location of the file and the function name, load it using the link above, then call the JavaScript function by name to run it. I could see this being an admitedly somewhat roundabout solution. This also may work in Java due to its built in JavaScript engine, although I haven't tested it.
Beyond that, I am unaware of anything particularly widespread. Most applications limit what they accept as instructions quite strictly to prevent security breaches (Imagine a SQL Injection that can run free on a client's machine). In fact, JavaScript limits itself quite severely, perhaps most notably in regards to local file reading.
Hopefully this helps with your ideas. Let me know in a comment if you have any questions/issues about what I've said.
Related
12-Factor Apps suggest that you configure your application using environment variables. So far, so good. I can easily imagine that this is a good way to do it if you need to set a connection string, e.g.
But what if you have more complex configuration with lots and lots of values? I for sure do not want to have 50+ environment variables, do I?
How could I solve this, and still be compliant to the idea of 12-Factor Apps?
From a quick read of the configure link you provided, I agree with the author's claim that there is a widespread problem, but I am not convinced that their proposed solution is going to always be best. Like you, I don't relish the idea of having to define dozens of environment variables to configure an application. So here are some alternative ideas.
First, read Chapter 2 of the Config4* Getting Started Guide (disclaimer: I am the main author of that software). In particular, notice that its support for what I call adaptive configuration can go a long way towards addressing the concern that you ask about. Is Config4* the ultimate solution? Possibly not, but I think it is a good step in the right direction.
Second, the chances are that whatever application you are developing/maintaining has already settled on a particular configuration technology, such as XML files or Java property files, and it won't be feasible to migrate to using Config4*. This raises the question: is there anything you can do to avoid having a proliferation of, say, XML-based configuration files when you have multiple environments (such as dev, UAT, staging and production) in which the application will be deployed? I have outlined an approach for dealing with this issue in another StackOverflow article.
So I've gotten very familiar with Javascript MVC's over the past few weeks. Now I'm looking to learn the how to program a backend(specifically using asp.net MVC's implementation). I'm learning about the razor's view engine etc.
One learning block I'm running into when reading up on examples and tutorials is that I am thinking to myself "well..can't I just do that in the front-end with javascriptMVC" for most of the logic, and If I need to talk to a database I can just use a JSON call. There must be some value in back-end coding but right now I don't see it(hoping to get that solved).
The client is always exposed to attackers, hence you can never trust the code.
In other words: Any security-related things, verification and validation logic belongs to the server, all authentication and authorization stuff, … and: when you need to make sure that there is one reliable instance to decide some things, e.g. on prices, discounts, and so on.
There is a saying in web programming, and that is: All input is evil.
So whatever comes from your frontend (which basically is your JavaScript application) should be handled with care. Always black- or whitelist input, encode it, transform it, check it, and so on … and the only place where you can do this reliably, as it's the only place that is under YOUR control is the server.
Moreover: Never put secrets into the client, such as credentials (for your database, e.g.).
Hope this helps.
Summary:
Are there good HTML5/javascript options for selectively reading chunks of data (let's say to be eventually converted to JSON) from a large local file?
Problem I am trying to solve:
Some existing program locally and outputs a ton of data. I want to provide a browser-based interactive viewer that will allow folks to browse through these results. I have control over how the data is written out. I can write it all out in one big file, but since it's quite large, I can't just read the whole thing in memory. Hence, I am looking for some kind of indexed or db-like access to this from my webapp.
Thoughts on solutions:
1. Brute-force: HTML5 FileReader API has a nice slice() method for random access. So I could write out some kind of an index in the beginning of the file, use it to look up positions of other stored objects, and read them whenever they're needed. I figured I'd ask if there are already javascript libraries that do something like this (or better) before trying to implement this ugly thing.
2. HTML5 local database. Essentially, I am looking for an analog of HTML5 openDatabase() call that would open (a read-only) connection to a database based on a user-specified local file. From what I understand, there's no way to specify a file with a pre-loaded database. Furthermore, even if there was such a hack, it's not clear whether the local file format would be the same across browsers. I've seen the phonegap solution that populates the browser local database from SQL statements. I can do that too, but the data I am talking about is quite large (5-10GB): it will take a while to load, and such duplication seems rather pointless.
HTML5 does not sound like the appropriate answer for your needs. HTML5's focus is on the client side, and based on your description you're asking a lot out of the browsers, most likely more than they can handle.
I would instead recommend you look at a server-based solution to deliver the desired goal/results to the client view, something like Splunk would be a good product to consider.
Say there is some functionality needed for an application under development which could be achieved by making a system call to either a command line program or utilizing a library. Assuming efficiency is not an issue, is it bad practice to simply make a system call to a program instead of utilizing a library? What are the disadvantages of doing this?
To make things more concrete, an example of this scenario would be an application which needs to download a file from a web server, either the cURL program or the libcURL library could be used for this.
Unless you are writing code for only one OS, there is no way of knowing if your system call will even work. What happens when there is a system update or an OS upgrade?
Never use a system call if there is a library to do the same function.
I prefer libraries because of the dependency issue, namely the executable might not be there when you call it, but the library will be (assuming external library references get taken care of when the process starts on your platform). In other words, using libraries would seem to guarantee a more stable, predictable outcome in more environments than system calls would.
There are several factors to take into account. One key one is the reliability of whether the external program will be present on all systems where your software is installed. If there is a possibility that it will be missing, then maybe it is better to do it inside your program.
Weighing against that, you might consider that the extra code loaded into your program is prohibitive - you don't need the code bloat for such a seldom-used part of your application.
The system() function is convenient, but dangerous, not least because it invokes a shell, usually. You may be better off calling the program more directly - on Unix, via the fork() and exec() system calls. [Note that a system call is very different from calling the system() function, incidentally!] OTOH, you may need to worry about ensuring all open file descriptors in your program are closed - especially if your program is some sort of daemon running on behalf of other users; that is less of a problem if your are not using special privileges, but it is still a good idea not to give the invoked program access to anything you did not intend. You may need to look at the fcntl() system call and the FD_CLOEXEC flag.
Generally, it is easier to keep control of things if you build the functionality into your program, but it is not a trivial decision.
Security is one concern. A malicious cURL could cause havoc in your program. It depends if this is a personal program where coding speed is your main focus, or a commercial application where things like security play a factor.
System calls are much harder to make safely.
All sorts of funny characters need to be correctly encoded to pass arguments in, and the types of encoding may vary by platform or even version of the command. So making a system call that contains any user data at all requires a lot of sanity-checking and it's easy to make a mistake.
Yeah, as mentioned above, keep in mind the difference between system calls (like fcntl() and open()) and system() calls. :)
In the early stages of prototyping a c program, I often make external calls to programs like grep and sed for manipulation of files using popen(). It's not safe, it's not secure, and it's certainly not portable. But it can allow you to get going quickly. That's valuable to me. It lets me focus on the really important core of the program, usually the reason I used c in the first place.
In high level languages, you'd better have a pretty good reason. :)
Instead of doing either, I'd Unix it up and build a script framework around your app, using the command line arguments and stdin.
Other's have mentioned good points (reliability, security, safety, portability, etc) - but I'll throw out another. Performance. Generally it is many times faster to call a library function or even spawn a new thread then it is to start an entire new process (and then you still have to correctly check/verify it's execution and parse it's output!)
I'm looking for best practices for using the same data in different places without repeating yourself - this could include configuration or business rules.
Example 1. Data validation rules where you want to validate on the client using javascript, but you want to make sure by validating on the server.
Example 2. Database access where your web server and your cronjobs use the same password, username.
Ease of processing and a human-readable solution would be a plus.
Encode your data in JSON. There's a JSON library for pretty much any language you'd care to think of, or if not, it's pretty easy to code one up. If JSON is not enough, perhaps look at YAML.
XML is pretty globally used. Easy to read, easy to write, and human readable. If you're concerned about the space overhead (which you actually aren't if you want human readable) then just compress it before you send it out, XML compresses quite well.
See answers to this question. I think they are applicable here, especially the one with a DSL.
As much hate as they get, for sharing data validation rules, I'm going to have to say Regular Expressions.
I know, I know, everyone hates them, but they are (generally) language-agnostic.
Use O/S Environment Variables (envvars) to store application configuration info (such as db passwords)
Validation rules often require logic. You could write your rules in JavaScript, and then run them in the browser, server (using Nashorn), and database (PLV8 with Postgres).