I'm working on a web application (Java EE backend) which contains a fairly complex input modal. This input modal allows the user to capture data, but it has a bunch of (JavaScript) restrictions, such as mandatory fields, fields only being available if a specific value is entered, etc.
I have to expose this functionality to external systems and allow them to submit this data to my server. These external systems can be both web or client based (but I can assume that the clients will have internet access). My first thought is to provide some kind of definition of the fields and stuff like mandatory to these systems through services, and have them render the input modal however they want. This has been met with resistance though, because the types of fields and restrictions will likely change quite a bit during the next few months of development. These external systems have different deployment timelines, and for this to work we'll have to firstly duplicate all the logic handling these restrictions across all systems, and secondly synchronize our deployments.
An alternative which has been proposed is to have the external systems call my modal through standard HTTP and render it either in an iframe or in an embedded rendered. This solves all of the previous complaints, but it leaves me feeling a little uneasy.
Are there any alternatives we are not thinking of? Maybe some kind of UI schema with existing render libraries for the different platforms? What are your thoughts on the second proposal, any major concerns or is this the "best" solution?
Edit: To clarify, I'll of course still perform backend validation regardless of the frontend decision, as I can't just trust the incoming data.
The constraints that you mention (mandatory fields etc.) really have nothing to do with the user interface. You are also right that it is not a good idea to have your backend render web content.
Your first proposal sounds like a good idea, here's how I would solve the issues you mentioned:
Do all the validation on the backend and send a model object to the client, representing the current state of the UI (field name, type, enabled/disabled, error message etc.).
Keep the client as dumb as possible. It should only be responsible for rendering the model on a window / webpage. Whenever a field is changed and it requires validation, submit the model to the backend for validation and get back a new model to be displayed. (You could optimize this by only returning the fields that changed.)
Doing it this way will keep your validation logic in one place (the backend) and the clients rarely need to be modified.
I have been faced with same issues in several previous projects. Based on this experience I can honestly say that server-side validation is the thing you will likely have to implement to avoid rubbish being committed from client side regardless if it comes from GUI or other third party system via API. You can choose one of available validation frameworks, I used Apache Commons Validator and think it is well, or you can implement your own one. On the other hand client side pre-validation, auto-completion and data look up are the solutions you should have to make human users happy. Do not consider about code duplication, just make your system right way from the business point of view.
Related
I'm using Wix as an e-commerce solution and the way I understand it, I can only add code (not edit current code) to make specific changes to the site. The one change I want to make is to have the ability to authorize/capture PayPal payments at a later date for all the products I am selling.
I've read through the PayPal authorization/capture documentation here but am still confused for my specific use case considering the only button I have is a "Check Out with Paypal" once customers have added products to their cart as opposed to "Buy Now" or some of the other button options available.
Is there a way to easily integrate authorize/capture in this case and if so, can someone help me out with how? Hoping I can make one change no matter how many different products a customer is purchasing that allows me to either capture all or part of the entire purchase amount and void the rest.
I've scoured the internet, but don't feel like anything I've come across is directly applicable. See here and here. The latter link makes it look insanely easy, but again I think the problem lies in the fact that I'm using Wix and can't directly edit existing code.
If anyone can provide directions/code necessary to implement this I'd be extremely grateful. Thanks so much in advance!
Note: It appears Wix integrates with PayPal Standard and all I need is the "Basic Authorization" capability, NOT "Order Authorizations."
The latter link makes it look insanely easy, but again I think the problem lies in the fact that I'm using Wix and can't directly edit existing code.
You've nailed it. It would be that easy, just add the parameter paymentaction=authorization when redirecting to PayPal, but Wix needs to provide a way for this to happen.
Many shopping carts -- especially those that integrate via an API rather than Payments Standard -- have a checkbox in their settings to toggle on authorization mode vs. immediate capture (sale) mode. The reason API-based integrations are much more likely to implement such a setting, is that an API is needed for the shopping cart to be able to capture the authorization at a later point in time. API-less integrations (like payments standard) cannot do the capture themselves (because this requires an API call), and so with standard you'll always need to log into www.paypal.com for the capture later on.
Anyway, there's probably no way forward unless Wix provides you with one.
Wix could well be using an API integration rather than the HTML-only Payments Standard, but the problem is the same: the API needs to specify 'authorization' (instead of 'sale'/'capture'), and from their lack of documenting the feature it does not appear Wix has implemented this.
Most shopping cart/platform providers do support authorization and capture, so you could make the feature request to Wix, and/or consider switching providers if it's a must-have.
On a general note, using authorization and capture adds complexity to payment processing, and the capture is not guaranteed to be successful. You can get a successful authorization and then have the payment fail when you try to capture it (happens in a certain % of cases, when funds are no longer available or the card decides to decline). So in general, you use immediate sale/capture at checkout time (not authorization) unless you have very specific and well-defined business needs to not be capturing up front (and refunding in the case of exceptions).
Maybe I'm confusing things or over-complicating them, but I'm struggling to develop a RESTful API that supports both HTML and JSON content types. Take for example a simple user management feature. I would expect to have an API that looks like this:
GET /users: lists all users
GET /users/{id}: views a single user
POST /users: creates a new user
A programmatic client posting to /users with a JSON payload would expect a 201 Created response with a Location header specifying the URL to the newly created user, e.g. /users/1. However, a person creating a user through his web browser would post to the same URL with a form-encoded payload and would expect to be redirected to the user list page, requiring the API to return a 302/303 redirect with a Location header of /users.
From a purely conceptual point of view, I find it surprising that an API would react differently based on the submitted content type, and wonder if this is bad design. Then again, maybe it's a mistake to consider the programmatic API and the web-centric API to be the same API and one shouldn't worry about such concerns and worry more about providing a good experience to the different clients.
What do you think?
You've stumbled upon two separate issues.
One, the typical web browser is a pretty lousy REST client.
Two, web application APIs are not necessarily REST APIs (see #1).
And thus, your conundrum of trying to serve two masters.
Arguably representation has little to do with application semantics when it comes to details such as workflow, particularly if you have equally rich media types (vs a constrained media type such as an image, or something else).
So, in those terms, it's really not appropriate to have the application behave differently given similar media types.
On the other hand, media type IS Yet Another aspect of the request which can influence operation on the back end. You could, for example be requesting an elided "lite" data type that may well not offer links to other parts of the api that a richer media type would, or your authorization level is a factor on what data you can view, as well as what other relations are available, or even what media types are supported at all.
So it's fair that every aspect of the request payload can have impact on the particular semantics and effect of any particular request to the server. In that case, you're scenario is not really off the mark.
In the end, it's down to documentation to clarify your intent as an API designer.
(Note: these two questions are similar, but more specific to ASP.Net)
Consider a typical web app with a rich client (it's Flex in my case), where you have a form, an underlying client logic that maps the form's input to a data model, some way of remoting these objects to a server logic, which usually puts it in a database.
Where should I - generally speaking - put the validation logic, i. e. ensuring correct format of email adresses, numbers etc.?
As early as possible. Rich client frameworks like Flex provide built-in validator logic that lets you validate right upon form submission, even before it reaches your data model. This is nice and responsive, but if you develop something extensible and you want the validation to protect from programming mistakes of later contributors, this doesn't catch it.
At the data model on the client side. Since this is the 'official' representation of your data and you have data types and getters / setters already there, this validation captures user errors and programming errors from people extending your system.
Upon receiving the data on the server. This adds protection from broken or malicious clients that may join the system later. Also in a multi-client scenario, this gives you one authorative source of validation.
Just before you store the data in the backend. This includes protection from all mistakes made anywhere in the chain (except the storing logic itself), but may require bubbling up the error all the way back.
I'm sort of leaning towards using both 2 and 4, as I'm building an application that has various points of potential extension by third parties. Using 2 in addition to 4 might seem superfluous, but I think it makes the client app behave more user friendly because it doesn't require a roundtrip to the server to see if the data is OK. What's your approach?
Without getting too specific, I think there should validations for the following reasons:
Let the user know that the input is incorrect in some way.
Protect the system from attacks.
Letting the user know that some data is incorrect early would be friendly -- for example, an e-mail entry field may have a red background until the # sign and a domain name is entered. Only when an e-mail address follows the format in RFC 5321/5322, the e-mail field should turn green, and perhaps put a little nice check mark to let the user know that the e-mail address looks good.
Also, letting the user know that the information provided is probably incorrect in some way would be helpful as well. For example, ask the user whether or not he or she really means to have the same recipient twice for the same e-mail message.
Then, next should be checks on the server side -- and never assume that the data that is coming through is well-formed. Perform checks to be sure that the data is sound, and beware of any attacks.
Assuming that the client will thwart SQL injections, and blindly accepting data from connections to the server can be a serious vulnerability. As mentioned, a malicious client whose sole purpose is to attack the system could easily compromise the system if the server was too trusting.
And finally, perform whatever checks to see if the data is correct, and the logic can deal with the data correctly. If there are any problems, notify the user of any problems.
I guess that being friendly and defensive is what it comes down to, from my perspective.
There's only a rule which is using at least some kind of server validation always (number 3/4 in your list).
Client validation (Number 2/1) makes the user experience snappier and reduces load (because you don't post to the server stuff that doesn't pass client validation).
An important thing to point out is that if you go with client validation only you're at great risk (just imagine if your client validation relies on javascript and users disable javascript on their browser).
There shoudl definitely be validation on the server end. I am thinking taht the validation should be done as early as possible on the server end, so there's less chance of malicious (or incorrect) data entering the system.
Input validation on the client end is helpful, since it makes the interface snappier, but there's no guarantee that data coming in to the server has been through the client-side validation, so there MUST be validation on the server end.
Because of security an convenience: server side and as early as possible
But what is also important is to have some global model/business logic validation so when you have for example multiple forms with common data (for example name of the product) the validation rule should remain consistent unless the requirements says otherwise.
Some of my recent web projects that I worked on, use a flow engine as the central abstraction in the presentation and/or (more or less the) business layer. Reflecting on my experiences, I can honestly say that I am not a fan of the flow-centric approach. On the contrary even. I see the same symptoms pop up in projects that use flows as central abstraction.
Everything is a flow. You don't just start an application, no, you "enter the main flow" even if it is just to show a menu with a huge dispatcher behind it. I am not against flows as such. Some use cases keep popping up everywhere and need to be included at various points in other use cases (LookupCustomer, ...), but for flow-centric people everything is a flow, even things that are... not flows.
Fragmentation. Flow-based applications tend to have many pieces of logic (actions, commands, fragments of code to prepare the view...) dispersed throughout the code. Mapping in and out of these actions adds overhead, is tedious and bloats the code. Although it is easy to follow the abstract flow, actually figuring out what is happening inside these little (or big) chunks of code is another thing. While every style of application allows people to write bad and inconsistent code, flow-centric applications make it particularly easy to do so.
Config files. Most applications use some XML format to declare flows and actions that accompany state changes. The language in which the application is written (say Java, C#, Ruby, ...) is probably far more richer and expressive than the XML format ever will be. Why bother?
Flows break encapsulation. If you give me a component that has a certain embedded flow logic, then the flow should be part of the component, and should not be an external abstraction. In other words: the flow is part of the component and the component is self-contained. It is a detail. Sure, it can be parameterized and stuff, but a component should "just work". People writing a Swing, GWT, or whatever fat or rich interface application, don't bother with explicit flow abstractions. Why should my web application have one? Give me the flow diagram of GMail.
(Edit) Flows are procedural. If you look at "rich" patterns like MVC with events and everything, flows really pale in comparison. You are using an modern and expressive language to implement your application, right? So you can do better than the rigid "do this, then that, and that, and ..." way from the time when punchcards and assembler were in fashion.
Examples of frameworks that promote flow-centric development are Struts, BTT, Spring Webflow, and JSF. I've also come across homegrown flow engines built on top of ordinary servlets.
This is also an interesting article: http://chillenious.wordpress.com/2006/07/16/on-page-navigation/
Do you (still) think a flow-centric approach for (the front-end of) a web-application is a good idea?
In general, flows seem to be an unnecessarily enterprisey approach to what should be a relatively simple problem: we would like to ensure that users take one of several particular paths through our application. What's more instructive and insightful is to examine why we need this path to occur. Is it because...
... we don't want them to interact with our application except in rigidly predefined ways? Then we've limited the utility of our application, and we make our application much harder to change and use.
... we're worried about the ability of our application to handle unexpected input or deal with states we haven't anticipated if people stray off the beaten path? Then that says a lot about our technical choices for a validation framework.
... we can't envision a scenario other than the predefined ones under which someone would use the site? Then we are implicitly assuming that only we know how best to use it; we limit the ability of the user to control their interaction.
Notice how each of these underscores an issue intrinsic to the application's development and team members, and one that's not the fault of a user. So I support your general premise that flow-based approaches tend to have a number of issues.
The primary problem is that flows unnecessarily increase brittleness that is already better abstracted by other mechanisms. For example, to achieve a rule like "you need to fill out your order form before you confirm checkout", don't make a workflow; have a better CustomerOrder model that knows when it doesn't have all the information necessary to allow an OrderConfirmation. If you try to skip ahead, your model and controller should take care of failing validation on the next POST.
Essentially, flows extract out disparate fragments of each participating controller and collect them into a new "flow controller" that's specific to each flow. That's not necessarily a bad idea, but it suggests that the original controllers may have been taking on too much responsibility to begin with if that sort of path was so easy to define separately. For example, if you previously had OrderConfirmation, CustomerOrder, and OrderCheckout controllers, and you're thinking about an Order flow to link all three together, what you should probably be thinking about is an Order controller instead.
I think defining flows is useful in a web application. In answer to your main points:
Everything is a Flow.
There is nothing intrinsically wrong with that, it's just a name to give something. A flow can be short or long - I agree it's a bit weird that there is a "main" flow that starts everything but it doesn't really cause any problems in practice.
Fragmentation
You have some valid points here, although I get the feeling that the greatest contributor to this is the design of the DSL. For example, Spring WebFlow v2 is a vast improvement over SWF v1 in terms of readability and understandability.
Config files
I strongly disagree with this point. I feel that xml is the best way to express this code. If you think about it - managing controllers, views, state changes and actions is really just "configuration" rather than "code". And xml (in my opinion) is the best way to express configuration. Just think about the word "Controller". All a controller does is direct and configure things - call services, return views and models etc. There is no need for any richness or expressiveness of Java to define what is basically just configuration of your web application.
Flows break encapsulation
GMail could expressed in a series of flows. Think about the number of steps it takes to compose and send an email. Flows really just define the wiring of how the application works - sure you could have a number of components that interact with each other, but the way that you configure them to work together is essentially the flow you have defined in your application. Making this flow explicit in a separate DSL seems like a good idea to me, as fundamentally it is separate.
The first question that should be answered is whether a flow framework is really the best tool for your specific web application. I'm a fan of Spring Web Flow, myself, but I'll only use it if my web app can easily be broken down into flows, and if navigation should be tightly controlled. If the navigation is very loose, where you can get to almost any page from any other page, then SWF isn't the right tool for the job.
As you mention, there are other drawbacks to flow frameworks. They usually aren't RESTful, and thus not bookmarkable. If that conflicts with what you want for your application, then SWF probably isn't for you.
That said, SWF, and some of the other flow frameworks, offer some features that few other web frameworks deliver. This includes complete solutions for double-submit issues and browser back button and history handling. SWF's implementation of these features lends some additional security. Since the flow execution IDs for each page change as the application is used, you get immunity to forced browsing and some protection against cross-site request forgery.
The concept of flows is quite nice, in my opinion, since flows tend to mirror use cases. Scoping data to a flow or a conversation removes the responsibility for its cleanup from the developer, which I think is a very good thing. It's like the difference between manual memory management and garbage collection. Not only does it make less work for me, but it eliminates the possibility of introducing bugs should I forget to cleanup attributes. One thing I hated about Struts was that I needed to duplicate my cleanup code in several actions to ensure correctness. It's much easier just to scope the data to the use case.
Flows also present a context for related actions and views. If I look at a struts-config or faces-config file, I can see all kinds of navigation rules or action mappings, but there is no immediate context for me to mentally group related items together. I have to manually trace through the configuration, and even then sometimes I get stuck. With Struts, I need to look at the specific web pages in order to figure out which actions can be invoked from a view.
With SWF, I can clearly see all the actions, views, and models related to a flow. With Eclipse plugins, I can see this as a state diagram. Even if you're not using eclipse, it's very easy to translate a flow definition to a state diagram. These diagrams are useful for myself, my project manager, and pretty much anyone who wants to understand the high-level of how a use case is performed. In short, chunking related things together allows for easier understanding, and a shallower learning curve. That's one reason why OOP is so popular. With web apps, the idea of chunking these elements together to form a use case just feels natural.
Everything is a flow
Everything really is a flow. Computer programs had always been a flow and will always be a flow containing of theese processes:
input -> process -> output
The MVC design pattern in fact corresponds with this..
controller -> model -> view
Fragmentation
You're right. But I think this "issue" might be reduced by a good suppport in IDEs.
Config files
There's no doubt xml is the best way to express configuration.
Flows break encapsulation
I would disagree with this. You can make black boxes using flows and then use these black boxes in another flows.
IMHO, web apps are best developed as independent modules rather than modules that are "bound by flow".
Since most web apps today are ajaxy apps, having independent modules on the page helps a lot.
Configurations can be handled by XML or JSON files.
Web 2.0 presents a serious challenge to the notion that "everything is a flow". And when the presentation tier is fully transposed to the client layer, we'll be back on the solid, and familiar (from GUIs of yore) ground of event based processing.
Flows arise because of the inherit mismatch between traditional application interaction and the way web applications actually work. Flows are merely a convenient way to describe what would be more traditionally modeled as a series of GUI dialogs (think wizard) in a way that is compatible with the way in which web pages are delivered and interacted with. Imagine if you will that you were writing a traditional program, but every time the user ran the program you could only display a single dialog box, and when the user clicked "Ok" (or "Cancel", or "Next", or "Previous") your program would terminate. In that situation, how would you go about modeling the expected behavior of the program (to further complicate matters, assume many users are running the program at different times)? I think you would find you would rather quickly arrive at something similar to flows.
I think perhaps what you're really asking is, "Why are most flow frameworks so easy to abuse?", which naturally leads to the followup question "What can be done to fix that?".
This is often situation, but here is latest example:
Companies have various contact data (addresses, phone numbers, e-mails...) when they make job ad, they have checkboxes where they choose how they want to be contacted. It is basically descriptive data. User when reading an ad sees something like "You can apply by mail, in person...", except if it's "through web portal" or "by e-mail" because then appropriate buttons should appear. These options are stored in database, and client (owner of the site, not company making an ad) can change them (e.g. they can add "by telepathy" or whatever), yet if they tamper with "e-mail" and "web-portal" options, they screw their web site.
So how should I handle data where everything behaves same way except "this thing" that behaves this way, and "that thing" that behaves some other way, and data itself is live should be editable by client.
You've tagged your question as "language-agnostic", and not all languages cleanly support polymorphism, but that's the way I would approach this.
Each option has some type, and different types require different properties to be set. However, every type supports some sort of "render" method that can display the contact method as needed. Since the properties (phone number, or web address, etc.) are type-specific, you can validate the administrator's input when creating these "objects", to make sure that the necessary data is provided and valid. Since you implement the render method, rather than spitting out HTML provided by a user, you can ensure that the rendered page is correct. It's less flexible, but safer and more user friendly.
In the database, you can have one sparsely populated table that holds data for all types of contacts, or a "parent" table with common properties and sub-tables with type-specific properties. It depends on how many types you have and how different they are. In either case, you would have some sort of type indicator, so that you know the type of object to which the data should be be bound.
First of all, think twice do you really need it. Reason is simple. You are supposed to serve specific need and input data is a mean to provide that service. If data does not fit with existing service then what is its value and who are consumer of that specific information?
There are two possible answers: You are expanding your client base or you need to change existing service because of change of demand. In both cases you need to star from development of business model. If you describe what service you need and what information it should provide you will avoid much of specific data and come with clear requirements easy to implement in software.
I'd recommend the resolution pattern for this, based on the mention of a database. The link above describes it, but it's actually a lot simpler than it sounds. You write a database query that returns all the possible options (for example, you read the standard options and the customized options together using perhaps a UNION or a JOIN depending on your schema) - the COALESCE SQL keyword is then useful to find the first 'resolution' of the option value that isn't NULL.
Well, if all it is is that you have two options that are special, and then anything else is dealt with in the same way, then store your options as strings, and if either of the two special ones appears in that list, then show the appropriate stuff for that special item.
Just check your list of items for the two special ones. Nothing fancy.
By writing a very simple Rules Engine. You can use an out-of-the box implementation, or you can roll your own. Since your case seems so simple, I tend to roll my own, because it means less dependencies (YMMV).