Is a process design really declarative programming? - integration

I've heard from someone that they´re using a business process automation tool (like Weblogic Integration) as a programming language (what sounds like something kind of stupid) to make things declarative. Then they put all the logic inside a process, every single if and while.
But, isn´t a process a how to step-by-step entity to reach a target?
For me it makes a process completely imperative. What do you think?

Orchestration languages are in fact imperative scripting languages with conditionals, looping and other traditionally imperative constructs, typically expressed through a flowchart-based user interface. They certainly do not (in my experience) implement tail-recursive functional programming, backward chaining or any other paradigm that might reasonably described as declarative in the generally accepted sense.
MS Workflow Foundation is advertised as having a rules engine, but this is fairly simplistic and doesn't really do forward chaining, except in a somewhat roundabout way. ILOG actually makes an adaptor for their rules engine specifically to drop it into MS workflow foundation.
Other workflow tools have better rule engines and a proper forward chaining system that could be viewed as declarative. However, once you get into the workflows themselves with looping and conditional branches you are most definitely in the territory of imperative programming.
However, some systems also implement a petri-net or state change based markup system for workflow, which might reasonably be described as declarative, but they still have an imperative mode of interaction with the underlying system. They still update variables and have side-effects.
I have seen one or two applications (for example TOAD for data anlaysis) actually using MS Workflow Foundation as a scripting language. As such it allows you to add a scripting facility to the application that (at least for marketing purposes) doesn't require programming skill to use.
In practice, a tool designed for writing, editing and running SQL queries being fitted with a scripting framework for 'non-programmers' makes one wonder what audience it's really aimed at. As a scripting language, workflow modelling tools are fairly clumsy and offer very limited opportunities for abstraction; in practice a .Net based scripting language such as IronPython or Boo, particularly in conjunction with a decent templating mechanism, would be a very powerful addition to such a tool.
One point about graphical languages of this sort is that they do not scale well with complexity. A similar issue applies with ETL tools as well. I have seen a provisioning application (see below) that was done (ironically) with Crossworlds (now known as Websphere Integrator). Within a month of starting on the application it became obvious that the graphical workflow language was not going to scale with the complexity of the application and it was re-built, based on a custom rules engine written in Java and a fairly large body of bespoke java code.
This type of issue is not uncommon with EAI and Orchestration systems and is one of the reasons that SOA is hard to implement in practice. What you are doing is actually pushing business logic into a very clumsy programming environment that is not being officially acknowledged as such. This will work in a simple case but is hard to make work on a complex system - this is sort of a guilty secret in SOA circles.
Coda:
A provisioning application is a system that takes plans for telecommunication services contracts (in this case for a mobile phone network) and pushes configuration information
based on rules out to various switches, billing applications and other applications. They tend to be fairly complex. When you buy a mobile phone plan with so many minutes and so many texts per month, a provisioning application is pushing out configuration information to the rest of the system about your access and billing rules.

It is definitely not what people usually mean when they talk about declarative programming, even if it some sense can be called declarative.

Related

creating a library in a single language with interfaces to everything else

Our business currently revolves around the development of a library, which can be used in a wide spectrum of industries (desktop, mobile, web and embedded). At the moment we only have customers within the desktop and web world and we already see that we are basically having to duplicate our code across multiple languages (c#, java and c++). How is this typically managed in other companies, do we really have to settle for having our middleware written in multiple languages to meet very different industries.
We have tried c# interop along with with java jni but the results where not as good as we had expected and it didn't seem like a good and professional solution.
Anyone have any ideas how we could keep the core in a single language but develop interfaces in different languages allowing for the different industries.
I have not a lot of ideas when it comes to industry, but I guess that the problem is similar to other not-so-strict scenarios. It is unclear also what do you mean by "middleware". Anyway, I would souggest you to not duplicate code and effectively have, if needed, multiple interfaces to the other languages.
In an ideal scenario, it would be very convenient to have one isolated process with your application logic, written in whatever language you feel more comfortable, and have this process to communicate using some inter-process-communication mechanisms (e.g., pipes, sockets, shared memory, files) with thin client-libraries which could be use by the clients in the other languages. That way, you are forced to create a very clean communication/usage protocol and to have some separation of concerns (because all the communication has to be serialized); you gain in robustness (a fatal error in your application process or in your client does not necessarily means a crash in the other one) and you don't have to use ffi mechanisms, but rather things like sockets and pipes. Even more, if you decide to publish your comm. protocol, then third party developers can create wrappers in their exotic predilect language without need of direct linking/importing to your development components.
This kind of paradigm has been in use with success over the years in the unix world and softwares like Mathematica . In a similar move, the developers of java have made the java plugin to work out-of-browser, in a separated process, to reduce interop complexity.
Ah, I forgot, maybe you may use this: ICE (http://www.zeroc.com/overview.html). It is probably the lighter interop layer that you can use to communicate in a clean way java, c++, c#, python and a few others. They even have some embedding solution...
Pick common denominator standards: REST or SOAP/XML over HTTP can be consumed by a lot of languages, including both Java and C#.
You'll be saddled with heavyweight WS-* standards and lots of network latency, but if interoperability is your goal this is one way to achieve it.
Might take a look at the Haxe language. It "compiles" to a dozen other languages.
https://haxe.org/

Are there any tools that make keeping the UML models in-sync with the code completely seamless?

UML Round-Trip Engineering tools with seamless synchronization?
The Rational suite purports to do it. But it's so pricey and clunky at drawing (worse than the Rose days) that it's not in the reach of most departments.
What’s amazing is that the free Bouml seems to do a fantastic job. It’s just feels too clunky to use. It has a great deal of functionality, is free (!), very fast, and reverse-engineers complex C++ very well. It also has some nice diagram support, including a very nice sequence diagram. Although the interface is unpolished (and constantly opens dialogs on the rightmost monitor), it does have the beginnings of a very capable product. It's a shame that the interface is so bare-bones and requires the expenditure of a lot of effort. Maybe it's because the author puts most of his time into the actual functionality. Does anyone have experience using Bouml throughout the product lifecycle?
That leaves the pricey MagicDraw, the very-capable yet reasonably-priced Enterprise Architect, and the slick-looking Visual Paradigm. Of these, only Visual Paradigm had an issue reverse-engineering my project's C++ headers.
MagicDraw has a strange, old feel. It does a good job at reverse-engineering on its own, although it remains to be seen whether round-trip engineering of complex C++ projects is seamless. They want over $1800 for the multi-language version, so it's priced similarly to Rational tools.
Enterprise Architect, although far less expensive than most, seems like it may be the most feature complete. It parses and generates C++ flawlessly. Even the comments and formatting are left intact. There are great training materials. But it doesn't handle Objective-C, so less useful for iOS and Mac OS X mixed code projects. The automatic Sequence Diagram generation sounds awesome, but sounds like it only works on Windows .NET projects.
Visual Architect (>$800 for multi-language 2-way) is bar far the best-looking software modeling tool I've come across. Although it may have some round-trip issues remaining, it is a pleasure to use for building the models by hand. It's even nicer than Rose was in some ways. It has an intuitive way of bringing up the tools you need right at the cursor. Yet as I mentioned, it currently falls short of the goal to keep the model in sync with the source. And it often doesn't even give notification that the import didn't fully work, or that duplicate classes have been created (with the same names). It also makes entry of message parameters difficult, using dialogs, whereas others allow the parameters to be changed right on the diagram. (The free Bouml excels at this, as does MagicDraw and others.)
Has anyone found a multi-language (Java, C++, C#, ObjC++, Python, Ruby, SQL) round-trip engineering tool that will hold up to real world projects, where customizations are handled (like custom parameters on messages), yet are not wiped out by the next source code import?
And where all the formatting and comments are completely preserved on generation. Close is not really good enough. If the tools mess up the source code formatting, no developer is going to want the tool run on his source.
Peter Coad's Together-J used to have diagrams and an editor together in one IDE (hence the name). Change a diagram and the code changes; same for the other way as well.
The UML tool and editor were both a bit slow. I think machines of the day were underpowered and didn't show it off to best advantage.
I believe Peter Coad sold it to Borland. Looks like Borland is out of the IDE business. You can still get it here.
I think IntelliJ is the best Java IDE there is. You can generate some nice UML diagrams using it.
The real question is: Why is UML so important? I'd rather have code. I usually do enough UML to get the idea across, write the code with unit tests, and then reverse engineer it for documentation. You can't debug or unit test UML diagrams. Better to have working code.
Bouml ... constantly opens dialogs on the rightmost monitor
in a multiple monitor configuration the best is to indicate to Bouml which monitor must be used by default, else for Bouml you have just a very large monitor including all your monitors. Of course to indicate a default monitor doesn't means you can't use the other one(s), and it is possible to move the dialogs/main window where you want. The definition of the default monitor to use is done through the environment dialog.
Enterprise Architect seems to do a good job at this. As you point out, it's reasonably-priced. And it will also generate diagrams and documentation, as well as import/export source code.

Scripting Languages vs. Compiled Languages for web development

Though I come from a purely PHP background on the web development side of programming, I have also spent much time with C# and C++ on the desktop.
I don't really want to spark any flame wars, but:
When should you use scripting languages over compiled languages for website development?
(and vice versa)
Just to clarify, for the sake of this question, I define a "scripting language" to mean an interpreted language like PHP, Python, or Ruby, and a "compiled language" to mean a strongly typed, compiled language like C#, C++, Java, or VB.
It depends :-)
On...
...where and how you want to deploy the application
...the skillsets of the engineers in your organization
...what third-party components you want to integrate with or incorporate
Deployment
If you need to be able to deploy the solution on any of dozens of different possible platforms, you may find that you're better off with PHP than Java (for example). There are hundreds of thousands of Java hosting providers out there, but there are probably millions of PHP hosting providers. (And I say this as a Java-head who finds PHP "so so" at best.)
This goes to OS as well. Mono aside, .Net stuff is going to limit you to Windows-based deployment (or lagging behind the cutting edge and having to very, very rigorously test each and every 3rd party component you bring in, to ensure that it doesn't have Mono...issues).
Skillsets
Coming up to speed in an environment or language is non-trivial. For most of us, picking up the basics is pretty quick, but you may not be making the best architectural/design decisions because you're (comparatively) weak on the environment/language. Skillsets count.
Related to this: Skillset hiring counts. Is it easier (and/or cheaper) to hire PHP devs with 3-4 years of experience, or Java devs with 3-4 years of experience, or C# devs, or...?
Buying/finding/integrating vs. building
In your target area of development, which server-side components or packages will you want to integrate with? PHP has a vast array of things available for it, as does Java, as does C# or ASP.Net. But they're different things (by and large), so you'll want to look at what you actually want to use.
Conclusion
So I think it's less a matter of compiled vs. scripted (in today's world), and more a matter of what's the best fit by other criteria for what you're trying to do.
Addendum: Both/And
And of course, there's always "both/and". For instance, I do work in two main, unrelated environments right now, both using a combination of scripted and compiled resources. (One of them is Java + JavaScript via Rhino on Tomcat, the other is compiled COM objects + JScript [again, server-side] on IIS.)
A programmer can write good/bad fast/slow scalable/unscable code in any language. Although, some language and technologies make it harder to do. In my experience, with scripting languages you can produce a small to medium scale application faster than you can with compiled languages like Java. However, as applications grow in size, compiled languages become more suited to the task I think this comes from strongly typing objects, deeper layers of architecture to manage tasks, and more QA frameworks to verify things are running as they should be as changes occur.
I find it to be mostly a matter of opinion. At first I hated the pre-compiled web applications asp.net provides, but I've gotten used to it so I don't hate it anymore. It has advantages and disantages:
Pro
pre-compiled web applications are easy to deploy, often you'll only have to update the bin-directory
pre-compiled web applications perform well
you don't have to upload source code, which is nice imho.
Con
updating a pre-compiled web app generally means the web application is reset, so unless you've changed the session state, it'll end all sessions and log everyone out
rebuilding a large web application can take some time, which is added to the time it took you to write the changes in the first place. I am sometimes impatient.
I've always liked how easy it is to just update one file in a PHP project without having to rebuild a project or something like that, on the other hand, .net has a nice IDE that allows you to debug everying, from back end (C#, VB.net) to front end (Javascript), in one package.
But again; both have advantages and disadvantages.
I wouldn't draw such a sharp distinction between compiled and interpreted languages - this is really just an implementation detail, and tends to change with time (faster than the languages themselves change.) Case in point - thanks to Facebook, PHP is now a "compiled language" too. Another case in point - I enjoy web development with Scheme - and my preferred Scheme implementation now runs a VM and in that sense is at least as compiled as Java is.
So I think the issues to focus on are the expressiveness of the language, its performance, and its ease of deployment - compiled vs. interpreted is only important insofar as it relates to these things.
I'm a big fan of compiled languages everywhere, if for nothing more than the static typing. On the other hand, scripting languages are very convenient -- no binaries to deal with, only text files, which is a big win for web servers.
In the end, it doesn't really matter -- use whatever language you know and feel most comfortable with for the job.
I think that speed is a key concern in a web application, in particular
how fast is it to write my code
how fast is it to fix my code
how fast is it to refactor my code
how fast is it to test my code
That is, I am concerned about the speed of the slowest link: myself. Anything else is fast enough for Twitter-like loads.
Today, the number one on my evaluation list for a new project would be Tornado and Python.
If I had a choice of platforms, of course.
Ah, Python is among the fastest in scripting languages.
For scripting languages, anyone that has a copy of your software could potentially modify your source code because it's open source.
For programming languages, anyone that has a copy the software cannot simply modify your source code because it is compiled.
So I guess, it depends upon your preferences.

How is web programming different from back-end programming?

I have worked on single threaded business logic/back-end programming for most of my career. I now wish to learn web programming but would like to know how web programming is different from non-GUI programming (e.g. writing an API or a file processing application). I am not talking about the GUI design aspects (someone has already asked that question here) but more about programming complexity.
On the few occasions when I have worked on a web application, I felt that web applications are relatively more non-deterministic and unpredictable (for example, due to the event driven, multi-threaded model of web applications, there are several permutations and combinations of events and actions one needs to take care of) .
What would you say are some of the basic features of web programming that makes it different from non-GUI applications? What are the pitfalls/mistakes a back-end developer might commit while working on web applications?
EDIT
My definition of back-end programming means non-GUI applications like an API or a file processing batch application that parses a large data file, reads the records, does a lot of number crunching calculations on the data and spews out the results into another file or database. Anothe example could be a library of date and time utilities.
The biggest challenge with web programming is dealing with state. HTTP is a stateless protocol. This can make maintaining state more challenging than in a desktop application. Web applications tend to have a different life cycle due to this. Each web development platform deals with this somewhat differently, but they all need to deal with it in some way.
Web applications generally feel like single threaded applications, as you - the application developer - rarely create threads of your own. If anything, it's actually a lot easier, because the stateless nature of the web transactions means that you have to load the data for the page each time from the database. Therefore, you don't have to worry about concurrency, since 'whatever is there' is usually good enough.
The biggest problem with Web development is all of the background knowledge that you have to accumulate over time. How do you lay out web pages? How do you style things with CSS? How do you get parameters from the query string? How do you validate a field value in JavaScript? All of those things are actually really easy to learn, but there's just so many of them that it can be a real pain.
The biggest pitfalls I've witnessed Application developers make when moving into Web is not considering the costs of their code. Either they abuse MySQL to much too the point of bogging the RDBMS down, they write code that uses too much memory, or they make front end pages that are to big to fit in dialup/cellphones or low end broadband/dsl pipeline.
Sometimes it can't be avoid in writing a heavy duty page, but considerations can be made to attempt to cache as much as possible or when writing a page that will be hit a lot they will make no effort to profile and optimize queries before they go out the door.
Its not that these people are stupid, just a lack of experience and awareness that they need to play nice and write code that's somewhat lean.
Back-end programming is infinitely easier than web programming. (You have been warned!) Web programming is the easiest to show off to everybody.
Most web sites have a back end component as well. Typical structure will be something like:
UI - html/css/javascript
Controller - if using MVC
Business Logic/Services - this is backend
Database - this is also backend
So building web sites will still mean a lot of back end work. In regards to the UI, the main difference is that you will need to have a good eye for design and layout to do it well. The html/css technology is pretty simple in itself.
HTML was actually developed to deliver physics papers. You can still see it in some of the old meta tags. At any rate the difference is web programming is stateless and thick client development is not.
As you have adeptly indicated, its all driven by events. True javascript has mucked up web development a bit by creating the illusion of a stateful enviornment but in the end everything comes down to simple HTML.
Its never too late to start learning, I would say start making some static HTML pages and move your way up to an MVC Framework, I suggest Microsoft MVC Framework. Its pretty fantastic, there are others you could use like ASP.Net Webforms but you won't learn anything by dragging and dropping things onto a designer ;).
web & GUI applications interface with humans .. back-end applications interface with services and databases .. As such your specifications need to include significant consideration of your user's mental model - making things behave as people expect them to. And doing that - understanding how users think - is not always easy or logical. You may have elegant algorithmic solutions that simply fail to engage, because people don't always think logically. Many times, quite elegant UI's are extremely twisted coding-wise .. which is very contrary to system->system programming
Depending on problem-space, much of this can be more art than science.
One consideration (amongst many) with web programming is that users won't just be stupid (not that they all are, but you always have to factor that in), they will sometimes (assume always) be downright malicious and nasty, and will do everything in their power to destroy your application, your database, your weekends, your sanity...
Be as paranoid as a very small nun at a penguin shoot. Do not trust your users.
Another consideration is that Back End programming as per your definition is easier to test.
Once you begin web programming you're at the mercy of the various browsers' different interpretations of the same code. Plus the user, with inputs of mouse and keyboard, has a variety of ways to break what you produce.
Web programming isn't back-end programing. It shows stuff on the front end, the web.
Are you defining it otherwise?
EDIT
Web programming pulls you into presenting data consistently, visually, to everyone. Back end coding means constructing that data, in the same way for presentation, but not presenting it.
Based on your definition of "back end programming," your question applies not only to web applications, but to any GUI application.
It kind of depends what kind of GUI application we're talking about. For example:
Internal business applications tend to involve lots of business process workflow logic, record keeping, and interoperability between separate systems. No fancy alorithms or number crunching are needed. Your audience is limited, so performance is not a big deal, but cross-platform compatibility is important so these tend to be web applications. Your main concerns are making it easy to tie business sytems together, and keeping the API layered to ensure that the GUI code does not have to deal with any of the business logic code.
Public web sites (such as this one) tend to involve less of a formal architecture, and more of a mentality of "just get this cool feature to work so we can get more visitors." Again, no number crunching or algorithms unless performance is an issue. Performance is more of an issue for massively popular sites like Slashdot or Google, so if you anticipate rapid growth it pays to design for scalability in advance.
Public e-commerce web sites are kind of like both of the above: features and performance are important, but equally important is the structured architecture underneath it that ties all of the commerce business systems together (purchasing, supplier, shopping cart, payment gateways, etc.)
For the actual GUI portion, the complexity of the application kind of determines how complex the GUI code will be. For highly complex, nested GUIs where your requirements change often, it's easy to fall into the trap of putting too much GUI stuff into one page. Soon the page exceeds most people's complexity threshold, making the page very difficult to maintain. It pays to think in advance how you can separate different portions of the GUI into separate components, and then tie them together. If you're new to GUI programming, read some articles on the Model-View-Controller (MVC) pattern.
For simple web sites, where most pages are fairly static, this issue doesn't come up so much because each individual page is easy to maintain.
Most web programming is done in the style popular in the early seventies, before Dijkstra's 'goto considered harmful' was well-known.

Flow Based Programming

I have been doing a little reading on Flow Based Programming over the last few days. There is a wiki which provides further detail. And wikipedia has a good overview on it too. My first thought was, "Great another proponent of lego-land pretend programming" - a concept harking back to the late 80's. But, as I read more, I must admit I have become intrigued.
Have you used FBP for a real project?
What is your opinion of FBP?
Does FBP have a future?
In some senses, it seems like the holy grail of reuse that our industry has pursued since the advent of procedural languages.
1. Have you used FBP for a real project?
We've designed and implemented a DF server for our automation project (dispatcher, component iterface, a bunch of components, DF language, DF compiler, UI). It is written in bare C++, and runs on several Unix-like systems (Linux x86, MIPS, avr32 etc., Mac OSX). It lacks several features, e.g. sophisticated flow control, complex thread control (there is only a not too advanced component for it), so it is just a prototype, even it works. We're now working on a full-featured server. We've learnt lot during implementing and using the prototype.
Also, we'll make a visual editor some day.
2. What is your opinion of FBP?
2.1. First of all, dataflow programming is ultimate fun
When I met dataflow programming, I was feel like 20 years ago, when I met programming first. Altough, DF programming differs from procedural/OOP programming, it's just a kind of programming. There are lot of things to discover, even sooo simple ones! It's very funny, when, as an experienced programmer, you met a DF problem, which is a very-very basic thing, but it was completely unknown for you before. So, if you jump into DF programming, you will feel like a rookie programmer, who first met the "cycle" or "condition".
2.2. It can be used only for specific architectures
It's just a hammer, which are for hammering nails. DF is not suitable for UIs, web server and so on.
2.3. Dataflow architecture is optimal for some problems
A dataflow framework can make magic things. It can paralellize procedures, which are not originally designed for paralellization. Components are single-threaded, but when they're organized into a DF graph, they became multi-threaded.
Example: did you know, that make is a DF system? Try make -j (see man, what -j is used for). If you have multi-core machine, compile your project with and without -j, and compare times.
2.4. Optimal split of the problem
If you're writing a program, you often split up the problem for smaller sub-problems. There are usual split points for well-known sub-problems, which you don't need to implement, just use the existing solutions, like SQL for DB, or OpenGL for graphics/animation, etc.
DF architecture splits your problem a very interesting way:
the dataflow framework, which provides the architecture (just use an existing one),
the components: the programmer creates components; the components are simple, well-separated units - it's easy to make components;
the configuration: a.k.a. dataflow programming: the configurator puts the dataflow graph (program) together using components provided by the programmer.
If your component set is well-designed, the configurator can build such system, which the programmer has never even dreamed about. Configurator can implement new features without disturbing the programmer. Customers are happy, because they have personalised solution. Software manufacturer is also happy, because he/she don't need to maintain several customer-specific branches of the software, just customer-specific configurations.
2.5. Speed
If the system is built on native components, the DF program is fast. The only time loss is the message dispatching between components compared to a simple OOP program, it's also minimal.
3. Does FBP have a future?
Yes, sure.
The main reason is that it can solve massive multiprocessing issues without introducing brand new strange software architectures, weird languages. Dataflow programming is easy, and I mean both: component programming and dataflow configuration building. (Even dataflow framework writing is not a rocket science.)
Also, it's very economic. If you have a good set of components, you need only put the lego bricks together. A DF program is easy to maintain. The DF config building requires no experienced programmer, just a system integrator.
I would be happy, if native systems spread, with doors open for custom component creating. Also there should be a standard DF language, which means that it can be used with platform-independent visual editors and several DF servers.
Interesting discussion! It occurred to me yesterday that part of the confusion may be due to the fact that many different notations use directed arcs, but use them to mean different things. In FBP, the lines represent bounded buffers, across which travel streams of data packets. Since the components are typically long-running processes, streams may comprise huge numbers of packets, and FBP applications can run for very long periods - perhaps even "perpetually" (see a 2007 paper on a project called Eon, mostly by folks at UMass Amherst). Since a send to a bounded buffer suspends when the buffer is (temporarily) full (or temporarily empty), indefinite amounts of data can be processed using finite resources.
By comparison, the E in Grafcet comes from Etapes, meaning "steps", which is a rather different concept. In this kind of model (and there are a number of these out there), the data flowing between steps is either limited to what can be held in high-speed memory at one time, or has to be held on disk. FBP also supports loops in the network, which is hard to do in step-based systems - see for example http://www.jpaulmorrison.com/cgi-bin/wiki.pl?BrokerageApplication - notice that this application used both MQSeries and CORBA in a natural way. Furthermore, FBP is natively parallel, so it lends itself to programming of grid networks, multicore machines, and a number of the directions of modern computing. One last comment: in the literature I have found many related projects, but few of them have all the characteristics of FBP. A list that I have amassed over the years (a number of them closer than Grafcet) can be found in http://www.jpaulmorrison.com/cgi-bin/wiki.pl?FlowLikeProjects .
I do have to disagree with the comment about FBP being just a means of implementing FSMs: I think FSMs are neat, and I believe they have a definite role in building applications, but the core concept of FBP is of multiple component processes running asynchronously, communicating by means of streams of data chunks which run across what are now called bounded buffers. Yes, definitely FSMs are one way of building component processes, and in fact there is a whole chapter in my book on FBP devoted to this idea, and the related one of PDAs (1) - http://www.jpaulmorrison.com/fbp/compil.htm - but in my opinion an FSM implementing a non-trivial FBP network would be impossibly complex. As an example the diagram shown in
is about 1/3 of a single batch job running on a mainframe. Every one of those blocks is running asynchronously with all the others. By the way, I would be very interested to hearing more answers to the questions in the first post!
1: http://en.wikipedia.org/wiki/Pushdown_automaton Push-down automata
Whenever I hear the term flow based programming I think of LabView, conceptually. Ie component processes who's scheduling is driven primarily by a change to its input data. This really IS lego programming in the sense that the labview platform was used for the latest crop of mindstorm products. However I disagree that this makes it a less useful programming model.
For industrial systems which typically involve data collection, control, and automation, it fits very well. What is any control system if not data in transformed to data out? Ie what component in your control scheme would you not prefer to represent as a black box in a bigger picture, if you could do so. To achieve that level of architectural clarity using other methodologies you might have to draw a data domain class diagram, then a problem domain run time class relationship, then on top of that a use case diagram, and flip back and forth between them. With flow driven systems you have the luxury of being able to collapse a lot of this information together accurately enough that you can realistically design a system visually once the components are build and defined.
One question I never had to ask when looking at an application written in labview is "What piece of code set this value?", as it was inherent and easy to trace backwards from the data, and also mistakes like multiple untintended writers were impossible to create by mistake.
If only that was true of code written in a more typically procedural fashion!
1) I build a small FBP framework for an anomaly detection project, and it turns out to have been a great idea.
You can also have a look at some of the KNIME videos, that give a good idea of what a flow based framework feels like when the framework is put together by a great team. Admittedly, it is batch based and not created for continuous operation.
By far the best example of flow based programming, however, is UNIX pipes which is one of the oldest, most overlooked FBP framework. I don't think I have to elaborate on the power of nix pipes...
2) FBP is a very powerful tool for a large set of problems. The intrinsic parallelism is a great advantage, and any FBP framework can be made completely network transparent by using adapter modules. Smart frameworks are also absurdly fault tolerant, and able to dynamically reload crashed modules when necessary. The conceptual simplicity also allows cleaner communication with everybody involved in a project, and much cleaner code.
3) Absolutely! Pipes are here to stay, and are one of the most powerful feature of unix. The power inherent in a FBP framework compared to a static program are many, and trivialise change, to the point where some frameworks can be reconfigured while running with no special measures.
FBP FTW! ;-)
In automotive development, they have a language agnostic messaging protocol which is part of the MOST specification (Media Oriented Systems Transport), this was designed to communicate between components over a network or within the same device. Systems usually have both a real and visualized message bus - therefore you effectively have a form of flow based programming.
That was what made the light bulb go on for me several years ago and brought me here. It really is a fantastic way to work and so much more fun than conventional programming. The message catalog form the central specification and point of reference. It works well for both developers and management. i.e. Management are able to browse the message catalog instead of looking at source.
With integrated logging also referencing the catalog to produce intelligible analysis things can get really productive. I have real world experience of developing commercial products in this way. I am interested in taking things further, particularly with regards to tools and IDEs. Unfortunately I think many people within the automotive sector have missed the point about how great this is and have failed to build on it. They are now distracted by other fads and failed to realize that there was far more to most development than the physical bus.
I've used Spring Web Flow extensively in Java Web applications to model (typically) application processes, which tend to be complex wizard-like affairs with lots of conditional logic as to which pages to display. Its incredibly powerful. A new product was added and I managed to recut the existing pieces into a completely new application process in an hour or two (with adding a couple of new views/states).
I also looked into using OS Workflow to model business processes but that project got canned for various reasons.
In the Microsoft world you have Windows Workflow Foundation ("WWF"), which is becoming more popular, particularly in conjunction with Sharepoint.
FBP is just a means of implementing a finite state machine. It's nothing new.
I realize that it is not exactly the same thing, but this model has been used for years in PLC programming. ISO calls it Sequential Flow Chart, but many people call it Grafcet after a popular implementation. It offers parallel processing and defines transitions between states.
It's being used in the Business Intelligence world these days to mashup and process data. Data processing steps like ETL, querying, joining , and producing reports can be done by the end-user. I'm a developer on an open system - ComposableAnalytics.com In CA, the flow-based apps can be shared and executed via the browser.
This is what MQ Series, MSMQ and JMS are for.
This is cornerstone of Web Services and Enterprise Service Bus implementations.
Products like TIBCO and Sun's JCAPS are basically flow-based without using this particular buzz-word.
Most of the work of the application is done with small modules that pass messages through a processing network.