Related
Standard way of working on new API (library, class, whatever) usually looks like this:
you think about what methods would API user need
you implement API that you suspect user will need
So basically you trying to guess what your API should look like. It very often leads to over engineering stuff, huge APIs that you think user will need and it is very possible that great part of your code won't be used at all.
Some time ago, maybe few years even, I read some article that promoted writing client code first. I don't remember where I found it but author pointed out several advantages like better understanding how API will be used, what it should provide and what is basically obsolete. I think idea was that it goes along with SCRUM methodology and user stories but on implementation level.
Just out of curiosity for my latest private project I started not with actual API (some kind of toolkit library) but with client code that would use this API. Of course my code is all in red because classes, methods and properties does not exist and I can forget about help from intellisense but what I noticed is that after few days of coding my application "has" all basic functionalities and my library API "is" a lot smaller than I imagined when starting a project.
I don't say that if somebody took my library and started using it it wouldn't lack some features but I think it helped me to realize that my idea of this API was somewhat flawed because I usually try to cover all bases and provide methods "just in case". And sometimes it bites me badly because I made some stupid mistake in basic functions being more focused on code that somebody maybe would need.
So what I would like to ask you do you ever tried this approach when needed to create a new API and did it helped you? Is it some recognized technique that has a name?
So basically you're trying to guess what your API should look like.
And that's the biggest problem with designing anything this way: there should be no (well, minimal) guesswork in software design. Designing an API based on assumptions rather than actual information is dangerous, for several reasons:
It's directly counter to the principle of YAGNI: in order to get anything done, you have to assume what the user is going to need, with no information to back up those assumptions.
When you're done, and you finally get around to using your API, you'll invariably find that it sucks to use (poor user experience), because you weren't thinking about how the library is used (UX), you were thinking about what the library must do (features).
An API, by definition, is an interface for users (i.e., developers). Designing as anything else just makes for a bad design, without fail.
Writing sample code is like designing a GUI before writing the backend: a Good Thing. It forces you to think about user experience and practical effects of design decisions without getting bogged down in useless theorising and assumption.
And contrary to Gabriel's answer, this is not bottom-up design: it's top-down. Rather than design the concrete backend of your library and then force an abstract interface on top of it, you first design the interface and then worry about the implementation.
Generally speaking, the idea of designing the concrete first and abstracting from that afterwards is called bottom-up design. Test Driven Development uses similar principle to what you describe to support better design. Firstly you write a test, which is an use of code you are going to write afterwards. It is important to proceed stepwise, because you have to proove the API is implementable. IMportant part of each part is refactoring - this allows you design more concise API and reuse parts of your code.
Obviously, "Hello World" doesn't require a separated, modular front-end and back-end. But any sort of Enterprise-grade project does.
Assuming some sort of spectrum between these points, at which stage should an application be (conceptually, or at a design level) multi-layered? When a database, or some external resource is introduced? When you find that the you're anticipating spaghetti code in your methods/functions?
when a database, or some external resource is introduced.
but also:
always (except for the most trivial of apps) separate AT LEAST presentation tier and application tier
see:
http://en.wikipedia.org/wiki/Multitier_architecture
Layers are a mean to keep a design loosely coupled and highly cohesive.
When you start to have a few classes (either implemented or just sketched with UML), they can be grouped logically, into layers - or more generally packages, or modules. This is called the art of separating the concerns.
The sooner the better: if you do not start layering early enough, then you risk to have never do it as the effort can be too important.
Here are some criteria of when to...
Any time you anticipate the need to
replace one part of it with a
different part.
Any time you find
yourself need to divide work amongst
parallel team.
There is no real answer to this question. It depends largely on your application's needs, and numerous other factors. I'd suggest reading some books on design patterns and enterprise application architecture. These two are invaluable:
Design Patterns: Elements of Reusable Object-Oriented Software
Patterns of Enterprise Application Architecture
Some other books that I highly recommend are:
The Pragmatic Programmer: From Journeyman to Master
Refactoring: Improving the Design of Existing Code
No matter your skill level, reading these will really open your eyes to a world of possibilities.
I'd say in most cases dealing with multiple distinct levels of abstraction in the concepts your code deals with would be a strong signal to mirror this with levels of abstraction in your implementation.
This does not override the scenarios that others have highlighted already though.
I think once you ask yourself "hmm should I layer this" the answer is yes.
I've worked on too many projects that probably started off as proof of concept/prototype that ended up being full projects used in production, which are horribly written and just wreak of "get it done quick, we'll fix it later." Trust me, you wont fix it later.
The Pragmatic Programmer lists this as the Broken Window Theory.
Try and always do it right from the start. Separate your concerns. Build it with modularity in mind.
And of course try and think of the poor maintenance programmer who might take over when you're done!
Thinking of it in terms of layers is a little limiting. It's what you see in whitepapers about a product, but it's not how products really work. They have "boxes" that depend on each other in various ways, and you can make it look like they fit into layers but you can do this in several different configurations, depending on what information you're leaving out of the diagram.
And in a really well-designed application, the boxes get very small. They are down to the level of individual interfaces and classes.
This is important because whenever you change a line of code, you need to have some understanding of the impact your change will have, which means you have to understand exactly what the code currently does, what its responsibilities are, which means it has to be a small chunk that has a single responsibility, implementing an interface that doesn't cause clients to be dependent on things they don't need (the S and the I of SOLID).
You may find that your application can look like it has two or three simple layers, if you narrow your eyes, but it may not. That isn't really a problem. Of course, a disastrously badly designed application can look like it has layers tiers if you squint as hard as you can. So those "high level" diagrams of an "architecture" can hide a multitude of sins.
My generic rule of thumb is to at least to separate the problem into a model and view layer, and throw in a controller if there is a possibility of more than one ways of handling the model or piping data to the view.
(Or as the first answer, at least the presentation tier and the application tier).
Loose coupling is all about minimising dependencies, so I would say 'layer' when a dependency is introduced. i.e. a database, third party application, etc.
Although 'layer' is probably the wrong term these days. Most of the time I use Dependency Injection (DI) through an Inversion of Control container such as Castle Windsor. This means that I can code on one part of my system without worrying about the rest. It has the side effect of ensuring loose coupling.
I would recommend DI as a general programming principle all of the time so that you have the choice on how to 'layer' your application later.
Give it a look.
R
I seem to use bland words such as node, property, children (etc) too often, and I fear that someone else would have difficulty understanding my code simply because the parts' names are vague, common words.
How do you find creative names for classes and components to make them more memorable?
I am particularly having trouble with generic tools which have no real description except their rather generic functional purpose. I would like to know if others have found creative ways to name things rather than simply naming them by their utility, such as AnonymousFunctionWrapperCallerExecutorFactory.
It's hard to answer. I find them just because they seem to 'fit'.
What I do know, however, is that I find it basically impossible to move on writing code unless something is named correctly, and it 'feels' good. If it isn't named right, I find it hard to use, and the code is generally confusing.
I'm not too concerned about something being 'memorable', only 'accurate'.
I have been known to sit around thinking out loud about what to name something. Take your time, and make sure you are really happy with the name. don't be afraid of using common/simple words.
I don't really have an answer, but three things for you to think about.
The late Phil Karlton famously said: "There are only two hard problems in computer science. Cache Invalidation and Naming Things." So, the fact that you are having trouble coming up with good names is entirely normal and even expected.
OTOH, having trouble naming things can also be a sign of bad design. (And yes, I am perfectly aware, that #1 and #2 contradict each other. Or maybe one should think of it more like balancing each other.) E.g., if a thing has too many responsibilities, it is pretty much impossible to come up with a good name. (Witness all the "Service", "Util", "Model" and "Manager" classes in bad OO designs. Here's an example Google Code Search for "ManagerFactoryFactory".)
Also, your names should map to the domain jargon used by subject matter experts. If you can't find a subject matter expert, that's a sign that you are currently worrying about code that you're not supposed to worry about. (Basically, code that implements your core business domain should be implemented and designed well, code in ancillary domains should be implemented and designed so-so, and all other code should not be implemented or designed at all, but bought from a vendor, where what you are buying is their core business domain. [Please interpret "buy" and "vendor" liberally. Community-developed Free Software is just fine.])
Regarding #3 above, you mentioned in another comment that you are currently working on implementing a tree data structure. Unless your company is in the business of selling tree data structures, that is not a part of your core domain. And the reason that you have trouble finding good names could be that you are working outside your core domain. Now, "selling tree data structures" may sound stupid, but there are actually companies that do that. For example, the BCL team inside Microsoft's developer division: they actually sell (well, for certain definitions of "sell", anyway) the .NET framework's Base Class Libraries, which include, among others, tree data structures. But note that for example Microsoft's C++ compiler team actually (literally) buys their STL from a third-party vendor – they figure that their core domain is writing compilers, and they leave the writing of libraries to a company who considers writing STLs their core domain. (And indeed, AFAIK, that company does nothing but write and sell STL implementations. That's their sole product.)
If, however, selling tree data structures is your core domain, then the names you listed are just fine. They are the names that subject matter experts (programmers, in this case) use when talking about the domain of tree data structures.
Using 'metaphors' is a common theme in agile (and pattern) literature.
'Children' (in your question) is an example of a metaphor that is extensively used and for good reasons.
So, I'd encourage the use of metaphors, provided they are applicable and not a stretch of the imagination.
Metaphors are everywhere in computing. From files to bugs to pointers to streams... you can't avoid them.
I believe that for the purpose of standardization and communication, it's good to use a common vocab, like in the same case for design patterns. I have a problem with a programmer who keeps 'inventing' his own terms and I have trouble understanding him. (He kept using the term 'events orchestrating' instead of 'scripting' or 'FCFS process'. Kudos for creativity though!)
Those common vocab describe stuff we are used to. A node is a point, somewhere in a graph, in a tree, or what-not. One way is to be specific to the domain. If we are doing a mapping problem, instead of 'node', we can use 'location'. That helps in a sense, at least for me. So I find there is a need to balance being able to communicate with other programmers, and at the same time keeping the descriptor specific enough to help me remember what it does.
I think node, children, and property are great names. I can already guess the following about your classes, just by their "bland" names:
Node - this class is part of a graph of objects
children - this variable holds a list of nodes belonging to the containing node.
I don't think "node" is either vague or common, and if you're coding a generic data structure, it's probably ok to have generic names! (With that being said, if you are coding up a tree, you could use something like TreeNode to emphasize that the node is part of a tree.) One way you can make the life of developers who will use your API easier is to follow the naming conventions of your platform's built in libraries. If everyone calls a node a node, and an iterator an iterator, it makes life easy.
Names that reflect the purpose of the class, method or property are more memorable than creative ones. Modern IDEs make it easier to use longer names so feel fee to be descriptive. Getting creative won't help as much as getting accurate.
I recommend to pick nouns from a specific application domain. E.g. if you are putting cars in a tree, call the node class Car - the fact that it is also a node should be apparent from the API. Also, don't try to be too generic in your implementation - don't put all attributes of the car into a hashtable named properties, but create separate attributes for make, color, etc.
A lot of languages and coding styles like to use all sorts of descriptive prefixes. In PHP there are no clear types, so this may help greatly. Instead of doing
$isAvailable = true;
try
$bool_isAvailable = true;
It is admittedly a pain, but usually well worth the time.
I also like to use long names to describe things. It may seem strange, but is usually easier to remember, especially when I go back to refactor my code
$leftNode->properties < $leftTreeNode->arrayOfNodeProperties;
And if all else fails. Why not fall back on a solid star wars themed program.
$luke->lightsaber($darth[$ewoks]);
And lastly, in college I named my classes after my professor, and then my class methods all the things I wanted to do to that jerk.
$Kube->canEat($myShorts, $withKetchup);
Some of my recent web projects that I worked on, use a flow engine as the central abstraction in the presentation and/or (more or less the) business layer. Reflecting on my experiences, I can honestly say that I am not a fan of the flow-centric approach. On the contrary even. I see the same symptoms pop up in projects that use flows as central abstraction.
Everything is a flow. You don't just start an application, no, you "enter the main flow" even if it is just to show a menu with a huge dispatcher behind it. I am not against flows as such. Some use cases keep popping up everywhere and need to be included at various points in other use cases (LookupCustomer, ...), but for flow-centric people everything is a flow, even things that are... not flows.
Fragmentation. Flow-based applications tend to have many pieces of logic (actions, commands, fragments of code to prepare the view...) dispersed throughout the code. Mapping in and out of these actions adds overhead, is tedious and bloats the code. Although it is easy to follow the abstract flow, actually figuring out what is happening inside these little (or big) chunks of code is another thing. While every style of application allows people to write bad and inconsistent code, flow-centric applications make it particularly easy to do so.
Config files. Most applications use some XML format to declare flows and actions that accompany state changes. The language in which the application is written (say Java, C#, Ruby, ...) is probably far more richer and expressive than the XML format ever will be. Why bother?
Flows break encapsulation. If you give me a component that has a certain embedded flow logic, then the flow should be part of the component, and should not be an external abstraction. In other words: the flow is part of the component and the component is self-contained. It is a detail. Sure, it can be parameterized and stuff, but a component should "just work". People writing a Swing, GWT, or whatever fat or rich interface application, don't bother with explicit flow abstractions. Why should my web application have one? Give me the flow diagram of GMail.
(Edit) Flows are procedural. If you look at "rich" patterns like MVC with events and everything, flows really pale in comparison. You are using an modern and expressive language to implement your application, right? So you can do better than the rigid "do this, then that, and that, and ..." way from the time when punchcards and assembler were in fashion.
Examples of frameworks that promote flow-centric development are Struts, BTT, Spring Webflow, and JSF. I've also come across homegrown flow engines built on top of ordinary servlets.
This is also an interesting article: http://chillenious.wordpress.com/2006/07/16/on-page-navigation/
Do you (still) think a flow-centric approach for (the front-end of) a web-application is a good idea?
In general, flows seem to be an unnecessarily enterprisey approach to what should be a relatively simple problem: we would like to ensure that users take one of several particular paths through our application. What's more instructive and insightful is to examine why we need this path to occur. Is it because...
... we don't want them to interact with our application except in rigidly predefined ways? Then we've limited the utility of our application, and we make our application much harder to change and use.
... we're worried about the ability of our application to handle unexpected input or deal with states we haven't anticipated if people stray off the beaten path? Then that says a lot about our technical choices for a validation framework.
... we can't envision a scenario other than the predefined ones under which someone would use the site? Then we are implicitly assuming that only we know how best to use it; we limit the ability of the user to control their interaction.
Notice how each of these underscores an issue intrinsic to the application's development and team members, and one that's not the fault of a user. So I support your general premise that flow-based approaches tend to have a number of issues.
The primary problem is that flows unnecessarily increase brittleness that is already better abstracted by other mechanisms. For example, to achieve a rule like "you need to fill out your order form before you confirm checkout", don't make a workflow; have a better CustomerOrder model that knows when it doesn't have all the information necessary to allow an OrderConfirmation. If you try to skip ahead, your model and controller should take care of failing validation on the next POST.
Essentially, flows extract out disparate fragments of each participating controller and collect them into a new "flow controller" that's specific to each flow. That's not necessarily a bad idea, but it suggests that the original controllers may have been taking on too much responsibility to begin with if that sort of path was so easy to define separately. For example, if you previously had OrderConfirmation, CustomerOrder, and OrderCheckout controllers, and you're thinking about an Order flow to link all three together, what you should probably be thinking about is an Order controller instead.
I think defining flows is useful in a web application. In answer to your main points:
Everything is a Flow.
There is nothing intrinsically wrong with that, it's just a name to give something. A flow can be short or long - I agree it's a bit weird that there is a "main" flow that starts everything but it doesn't really cause any problems in practice.
Fragmentation
You have some valid points here, although I get the feeling that the greatest contributor to this is the design of the DSL. For example, Spring WebFlow v2 is a vast improvement over SWF v1 in terms of readability and understandability.
Config files
I strongly disagree with this point. I feel that xml is the best way to express this code. If you think about it - managing controllers, views, state changes and actions is really just "configuration" rather than "code". And xml (in my opinion) is the best way to express configuration. Just think about the word "Controller". All a controller does is direct and configure things - call services, return views and models etc. There is no need for any richness or expressiveness of Java to define what is basically just configuration of your web application.
Flows break encapsulation
GMail could expressed in a series of flows. Think about the number of steps it takes to compose and send an email. Flows really just define the wiring of how the application works - sure you could have a number of components that interact with each other, but the way that you configure them to work together is essentially the flow you have defined in your application. Making this flow explicit in a separate DSL seems like a good idea to me, as fundamentally it is separate.
The first question that should be answered is whether a flow framework is really the best tool for your specific web application. I'm a fan of Spring Web Flow, myself, but I'll only use it if my web app can easily be broken down into flows, and if navigation should be tightly controlled. If the navigation is very loose, where you can get to almost any page from any other page, then SWF isn't the right tool for the job.
As you mention, there are other drawbacks to flow frameworks. They usually aren't RESTful, and thus not bookmarkable. If that conflicts with what you want for your application, then SWF probably isn't for you.
That said, SWF, and some of the other flow frameworks, offer some features that few other web frameworks deliver. This includes complete solutions for double-submit issues and browser back button and history handling. SWF's implementation of these features lends some additional security. Since the flow execution IDs for each page change as the application is used, you get immunity to forced browsing and some protection against cross-site request forgery.
The concept of flows is quite nice, in my opinion, since flows tend to mirror use cases. Scoping data to a flow or a conversation removes the responsibility for its cleanup from the developer, which I think is a very good thing. It's like the difference between manual memory management and garbage collection. Not only does it make less work for me, but it eliminates the possibility of introducing bugs should I forget to cleanup attributes. One thing I hated about Struts was that I needed to duplicate my cleanup code in several actions to ensure correctness. It's much easier just to scope the data to the use case.
Flows also present a context for related actions and views. If I look at a struts-config or faces-config file, I can see all kinds of navigation rules or action mappings, but there is no immediate context for me to mentally group related items together. I have to manually trace through the configuration, and even then sometimes I get stuck. With Struts, I need to look at the specific web pages in order to figure out which actions can be invoked from a view.
With SWF, I can clearly see all the actions, views, and models related to a flow. With Eclipse plugins, I can see this as a state diagram. Even if you're not using eclipse, it's very easy to translate a flow definition to a state diagram. These diagrams are useful for myself, my project manager, and pretty much anyone who wants to understand the high-level of how a use case is performed. In short, chunking related things together allows for easier understanding, and a shallower learning curve. That's one reason why OOP is so popular. With web apps, the idea of chunking these elements together to form a use case just feels natural.
Everything is a flow
Everything really is a flow. Computer programs had always been a flow and will always be a flow containing of theese processes:
input -> process -> output
The MVC design pattern in fact corresponds with this..
controller -> model -> view
Fragmentation
You're right. But I think this "issue" might be reduced by a good suppport in IDEs.
Config files
There's no doubt xml is the best way to express configuration.
Flows break encapsulation
I would disagree with this. You can make black boxes using flows and then use these black boxes in another flows.
IMHO, web apps are best developed as independent modules rather than modules that are "bound by flow".
Since most web apps today are ajaxy apps, having independent modules on the page helps a lot.
Configurations can be handled by XML or JSON files.
Web 2.0 presents a serious challenge to the notion that "everything is a flow". And when the presentation tier is fully transposed to the client layer, we'll be back on the solid, and familiar (from GUIs of yore) ground of event based processing.
Flows arise because of the inherit mismatch between traditional application interaction and the way web applications actually work. Flows are merely a convenient way to describe what would be more traditionally modeled as a series of GUI dialogs (think wizard) in a way that is compatible with the way in which web pages are delivered and interacted with. Imagine if you will that you were writing a traditional program, but every time the user ran the program you could only display a single dialog box, and when the user clicked "Ok" (or "Cancel", or "Next", or "Previous") your program would terminate. In that situation, how would you go about modeling the expected behavior of the program (to further complicate matters, assume many users are running the program at different times)? I think you would find you would rather quickly arrive at something similar to flows.
I think perhaps what you're really asking is, "Why are most flow frameworks so easy to abuse?", which naturally leads to the followup question "What can be done to fix that?".
I create business applications with heavy database use. Most of the programming work is just to connect components to the database and modifying components to adapt to general interface behaviour. I mostly use Delphi with its rich VCL library, and generally buy components needed. I keep most of the business logic in the database. I rarely get the chance to build a nice class hierarchy from the bottom up as there really is no need. Anyone else have this experience?
For me, occasionally a problem is clearer or easier with subclassing, but not often.
This also changes quite a bit in a given design as it's refactored.
My biggest problem is that programming courses and texts give so much weight to inheritance, hierarchies, and polymorphism through base classes (vs. interfaces or dynamic typing). This helps create legions of programmers that subclass everything and their mother.
The answer to this question is not totally language-agnostic;
Some languages like Java have a fairly limited set of language features available, meaning that subclassing is fairly often used because it's a convenient method for re-use, technical inheritance.
Closures and lambdas of C# make inheritance for technical reasons much less relevant. So normally inheritance is used for semantic reasons (like cat extends animal).
The last C# project I worked on, we more or less made all of the class hierarchies within a few weeks. After that it was more or less over.
On my current java project we create new class hierarchies all of the time.
Other languages will have other features that similarly affect this composition (mixins come to mind)
I put on my architecting/class design hat probably once or twice a month. It's probably the best hat I have and is the most fun to wear.
Depends what stage of the lifecycle your project is in though.
When your tackling problem domains you are well familiar with and already have a common code base to work from, you often have no need to create a new class hierarchy. It's when you stumble upon problems you have no ready solutions for, that you start building your own.
It's also very dependant on the type of applications you develop. If your domain already has well accepted conventions and libraries to work from, there probably isn't any need to reinvent the wheel (other than personal / academic interests). Some areas have inherently less available resources to work with, and in those you'll find yourself building everything from scratch most of the time.
A majority of applications, especially business applications, contains at least some kind of business logic in it. I would contend that business should not be in the database, but should rather be in the application. You can put referential integrity in the database as I think this is a good choice, but business logic should be only in the application.
By class hierarchy, I suppose you mean do you always have to end up with some inheritance in your object model, then the answer is no. But chances are you can often find some common code, factor it out and create a base class to contain the common code.
If you agree with me on the point that business logic should not be in the database, but should be in the application, then I recommend you look into the MVC Design Pattern to guide your design. You will find your design contain classes or objects. Your VCLs will represent your View, and you can have your Model classes map directly to the database table, i.e. each member in the class in the model corresponds to a field in a database table (again, this is the norm but there will be exception, where this simplicity fails to apply). Then you'll need a layer to handle the CRUD (Create, Read, Update, Delete) of the Model classes to the database tables. You will end up with an "layered" application that is easier to maintain and enhance.
It depends on what you mean by hierarchy - inheritance or layering?
When object oriented languages first came out, inheritance was overused. Complicated hierarchies were common. Now, interfaces (as in Java and C#) provide a simpler way to get the benefit of polymorphism without the complications of inheritance. I rarely use inheritance anymore.
Layering, however, is vital when creating a large application. Layering prevents general low-level classes (like lists) from directly referencing specific high-level classes (like web browser windows). As far as I know, there isn't a formal way to describe layering, but there are general guidelines (model-view-controller (MVC), separate GUI logic from business logic, separate data from presentation, etc.).
It really depends on the types/phases of the projects you're working on. I happen to do that everyday because I'm working on database internals for a new database, creating related libraries/frameworks. I'd imagine doing that a lot less if I'm working within a mature framework using other people's libraries.
I'm doing Infrastructure for our companys' product, so I'm writing a lot of code that will be used later by guys in other teams. So I end up writing lots of abstract classes, interfaces, hierarchies and so on. Mostly it's just a pattern of "default behaviour in an abstract/virtual class, which other programmers may override".
Very challenging, I must say.
The time that I find class hierarchies most beneficial is when the relationship between objects actually does match a true "is-a" relationship in the domain.
However if I can avoid large hierarchies I will due to the fact that they are often a little more tricky to map to relational databases and can really complicate your database designs. Since you say most of your applications make heavy use of databases this would be something to take into consideration.