I'm using joint positions from a Kinect camera as my state space but I think it's going to be too large (25 joints x 30 per second) to just feed into SARSA or Qlearning.
Right now I'm using the Kinect Gesture Builder program which uses Supervised Learning to associate user movement to specific gestures. But that requires supervised training which I'd like to move away from. I figure the algorithm might pick up certain associations between joints that I would when I classify the data myself (hands up, step left, step right, for example).
I think feeding that data into a deep neural network and then pass that into a reinforcement learning algorithm might give me a better result.
There was a paper on this recently. https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
I know Accord.net has both deep neural networks and RL but has anyone combined them together? Any insights?
If I understand correctly from your question + comment, what you want is to have an agent that performs discrete actions using a visual input (raw pixels from a camera). This looks exactly like what DeepMind guys recently did, extending the paper you mentioned. Have a look at this. It is the newer (and better) version of playing Atari games. They also provide an official implementation, which you can download here.
There is even an implementation in Neon which works pretty well.
Finally, if you want to use continuous actions, you might be interested in this very recent paper.
To recap: yes, somebody combined DNN + RL, it works and if you want to use raw camera data to train an agent with RL, this is definitely one way to go :)
I've been looking at the N=36 tutorial which introduces new RIO support in MvvmCross 3.09. Is it safe to combine INC fields and old school properties in the same class? I ask because some of my property setters and getters are complex so it maybe be easier to leave them as-is. However the vast majority of my existing properties are simple and as such seem excellent candidates for fields.
Thanks
Mark
'safe' is an interesting word to use here - I'm not entirely sure what it means in this context.
I personally believe it is safe to mix and match INotifyChanged and INotifyPropertyChanged in the same project and in the same view model - there's nothing that should go bang as a result and the memory and processing speed performance of INotifyChanged should be as good as or better than the performance of INotifyPropertyChanged.
The only potential areas of unsafe risk I can think of are:
team development and later code maintenance - using the two different approaches together might confuse yourself or other coders either now or later in maintenance - it would be fair for them to ask "where do I use one approach or the other?" and "why?"
lack of 'change all' support - INotifyPropertyChanged allows ViewModels to send a everything has changed notification - they can do this using a null or empty property name. INotifyChanged does not currently join in with this notification. In my experience, this 'change all' mechanism is used very infrequently and is not well known by Mvvm developers - so the risk here is small. However, if anyone did try to use it, then they might be surprised that the INotifyChanged bound-fields didn't update.
portability to other Mvvm libraries - Rio is a binding mechanism MvvmCross has introduced - so it isn't yet available in other Mvvm platforms. If you were ever to port back to something like Prism then this might be a risk for you (you might have to rewrite those fields as properties)
confusing to Windows developers - experienced Xaml developers have been used to using INotifyPropertyChanged all the way back to 2005 - so it might confuse them to have to use the MvvmCross Xaml Binding Extensions in order to get the fields bound inside Xaml. (Whether or not this confusion is good or bad for them depends on your world view!)
Which presentation patterns do you think Ext favors or have you successfully used to achieve high testability and also maintainability?
Since Ext component instances usually come tightly coupled with state and some sort of presentation logic (e.g. format validation for text fields), Passive View is not a natural fit. Supervising Presenter seems like it can work (and I've painlessly used it in one occasion). How about the suitability of Presentation Model? Any others?
While this question is specifically for Ext, it can apply to similar frameworks like SmartClient and even RIA technologies like Flex. So, if you have any first-hand pattern experiences with any other web UI technologies, your input would still be appreciated.
When thinking of presentation patterns, this is a great quote:
Separating user interface code from
everything else is a key principle in
well-engineered software. But it’s not
always easy to follow and it leads to
more abstraction in an application
that is hard to understand. Quite a
lot design patterns try to target this
scenario: MVC, MVP, Supervising
Controller, Passive View,
PresentationModel,
Model-View-ViewModel, etc. The reason
for this variety of patterns is that
this problem domain is too big to be
solved by one generic solution.
However, each UI Framework has its own
unique characteristics and so they
work better with some patterns than
with others.
As far as Ext is concerned, in my opinion the closest pattern would be the Model-View-Viewmodel, however this pattern is inherently difficult to code for whilst maintaining the separation of the key tenets (state, view, model).
That said, as per the quote above, each pattern tries to solve/compartmentalise/simplify a problem/situation often too complex for the individual application at hand, or which often fails when you try and take it to its absolute. As such, think about getting a 'best fit' as opposed to an absolute when pattern matching application development.
And remember:
The reason
for this variety of patterns is that
this problem domain is too big to be
solved by one generic solution.
I hope this helps!
2 yeas have passed since this question was aksed and now Ext-JS 4 has a built-in implementation of the MVC pattern. However, instead of an MVP (which I prefer), it favors a straight controller because the views attachment themselves to the models through stores.
Here's the docs on the controller:
http://docs.sencha.com/ext-js/4-1/#!/api/Ext.app.Controller
Nonetheless it can be made to act more like a supervising controller. One nice aspect of Ext-JS is the global application objects ability to act like an event bus for handling controller to controller communication. See this post on how to do that:
http://www.sencha.com/forum/showthread.php?176495-How-to-listen-for-custom-events-fired-in-application
Of course the definitive explanation of all these patterns can be found here:
http://martinfowler.com/eaaDev/uiArchs.html
UML Round-Trip Engineering tools with seamless synchronization?
The Rational suite purports to do it. But it's so pricey and clunky at drawing (worse than the Rose days) that it's not in the reach of most departments.
What’s amazing is that the free Bouml seems to do a fantastic job. It’s just feels too clunky to use. It has a great deal of functionality, is free (!), very fast, and reverse-engineers complex C++ very well. It also has some nice diagram support, including a very nice sequence diagram. Although the interface is unpolished (and constantly opens dialogs on the rightmost monitor), it does have the beginnings of a very capable product. It's a shame that the interface is so bare-bones and requires the expenditure of a lot of effort. Maybe it's because the author puts most of his time into the actual functionality. Does anyone have experience using Bouml throughout the product lifecycle?
That leaves the pricey MagicDraw, the very-capable yet reasonably-priced Enterprise Architect, and the slick-looking Visual Paradigm. Of these, only Visual Paradigm had an issue reverse-engineering my project's C++ headers.
MagicDraw has a strange, old feel. It does a good job at reverse-engineering on its own, although it remains to be seen whether round-trip engineering of complex C++ projects is seamless. They want over $1800 for the multi-language version, so it's priced similarly to Rational tools.
Enterprise Architect, although far less expensive than most, seems like it may be the most feature complete. It parses and generates C++ flawlessly. Even the comments and formatting are left intact. There are great training materials. But it doesn't handle Objective-C, so less useful for iOS and Mac OS X mixed code projects. The automatic Sequence Diagram generation sounds awesome, but sounds like it only works on Windows .NET projects.
Visual Architect (>$800 for multi-language 2-way) is bar far the best-looking software modeling tool I've come across. Although it may have some round-trip issues remaining, it is a pleasure to use for building the models by hand. It's even nicer than Rose was in some ways. It has an intuitive way of bringing up the tools you need right at the cursor. Yet as I mentioned, it currently falls short of the goal to keep the model in sync with the source. And it often doesn't even give notification that the import didn't fully work, or that duplicate classes have been created (with the same names). It also makes entry of message parameters difficult, using dialogs, whereas others allow the parameters to be changed right on the diagram. (The free Bouml excels at this, as does MagicDraw and others.)
Has anyone found a multi-language (Java, C++, C#, ObjC++, Python, Ruby, SQL) round-trip engineering tool that will hold up to real world projects, where customizations are handled (like custom parameters on messages), yet are not wiped out by the next source code import?
And where all the formatting and comments are completely preserved on generation. Close is not really good enough. If the tools mess up the source code formatting, no developer is going to want the tool run on his source.
Peter Coad's Together-J used to have diagrams and an editor together in one IDE (hence the name). Change a diagram and the code changes; same for the other way as well.
The UML tool and editor were both a bit slow. I think machines of the day were underpowered and didn't show it off to best advantage.
I believe Peter Coad sold it to Borland. Looks like Borland is out of the IDE business. You can still get it here.
I think IntelliJ is the best Java IDE there is. You can generate some nice UML diagrams using it.
The real question is: Why is UML so important? I'd rather have code. I usually do enough UML to get the idea across, write the code with unit tests, and then reverse engineer it for documentation. You can't debug or unit test UML diagrams. Better to have working code.
Bouml ... constantly opens dialogs on the rightmost monitor
in a multiple monitor configuration the best is to indicate to Bouml which monitor must be used by default, else for Bouml you have just a very large monitor including all your monitors. Of course to indicate a default monitor doesn't means you can't use the other one(s), and it is possible to move the dialogs/main window where you want. The definition of the default monitor to use is done through the environment dialog.
Enterprise Architect seems to do a good job at this. As you point out, it's reasonably-priced. And it will also generate diagrams and documentation, as well as import/export source code.
I have been doing a little reading on Flow Based Programming over the last few days. There is a wiki which provides further detail. And wikipedia has a good overview on it too. My first thought was, "Great another proponent of lego-land pretend programming" - a concept harking back to the late 80's. But, as I read more, I must admit I have become intrigued.
Have you used FBP for a real project?
What is your opinion of FBP?
Does FBP have a future?
In some senses, it seems like the holy grail of reuse that our industry has pursued since the advent of procedural languages.
1. Have you used FBP for a real project?
We've designed and implemented a DF server for our automation project (dispatcher, component iterface, a bunch of components, DF language, DF compiler, UI). It is written in bare C++, and runs on several Unix-like systems (Linux x86, MIPS, avr32 etc., Mac OSX). It lacks several features, e.g. sophisticated flow control, complex thread control (there is only a not too advanced component for it), so it is just a prototype, even it works. We're now working on a full-featured server. We've learnt lot during implementing and using the prototype.
Also, we'll make a visual editor some day.
2. What is your opinion of FBP?
2.1. First of all, dataflow programming is ultimate fun
When I met dataflow programming, I was feel like 20 years ago, when I met programming first. Altough, DF programming differs from procedural/OOP programming, it's just a kind of programming. There are lot of things to discover, even sooo simple ones! It's very funny, when, as an experienced programmer, you met a DF problem, which is a very-very basic thing, but it was completely unknown for you before. So, if you jump into DF programming, you will feel like a rookie programmer, who first met the "cycle" or "condition".
2.2. It can be used only for specific architectures
It's just a hammer, which are for hammering nails. DF is not suitable for UIs, web server and so on.
2.3. Dataflow architecture is optimal for some problems
A dataflow framework can make magic things. It can paralellize procedures, which are not originally designed for paralellization. Components are single-threaded, but when they're organized into a DF graph, they became multi-threaded.
Example: did you know, that make is a DF system? Try make -j (see man, what -j is used for). If you have multi-core machine, compile your project with and without -j, and compare times.
2.4. Optimal split of the problem
If you're writing a program, you often split up the problem for smaller sub-problems. There are usual split points for well-known sub-problems, which you don't need to implement, just use the existing solutions, like SQL for DB, or OpenGL for graphics/animation, etc.
DF architecture splits your problem a very interesting way:
the dataflow framework, which provides the architecture (just use an existing one),
the components: the programmer creates components; the components are simple, well-separated units - it's easy to make components;
the configuration: a.k.a. dataflow programming: the configurator puts the dataflow graph (program) together using components provided by the programmer.
If your component set is well-designed, the configurator can build such system, which the programmer has never even dreamed about. Configurator can implement new features without disturbing the programmer. Customers are happy, because they have personalised solution. Software manufacturer is also happy, because he/she don't need to maintain several customer-specific branches of the software, just customer-specific configurations.
2.5. Speed
If the system is built on native components, the DF program is fast. The only time loss is the message dispatching between components compared to a simple OOP program, it's also minimal.
3. Does FBP have a future?
Yes, sure.
The main reason is that it can solve massive multiprocessing issues without introducing brand new strange software architectures, weird languages. Dataflow programming is easy, and I mean both: component programming and dataflow configuration building. (Even dataflow framework writing is not a rocket science.)
Also, it's very economic. If you have a good set of components, you need only put the lego bricks together. A DF program is easy to maintain. The DF config building requires no experienced programmer, just a system integrator.
I would be happy, if native systems spread, with doors open for custom component creating. Also there should be a standard DF language, which means that it can be used with platform-independent visual editors and several DF servers.
Interesting discussion! It occurred to me yesterday that part of the confusion may be due to the fact that many different notations use directed arcs, but use them to mean different things. In FBP, the lines represent bounded buffers, across which travel streams of data packets. Since the components are typically long-running processes, streams may comprise huge numbers of packets, and FBP applications can run for very long periods - perhaps even "perpetually" (see a 2007 paper on a project called Eon, mostly by folks at UMass Amherst). Since a send to a bounded buffer suspends when the buffer is (temporarily) full (or temporarily empty), indefinite amounts of data can be processed using finite resources.
By comparison, the E in Grafcet comes from Etapes, meaning "steps", which is a rather different concept. In this kind of model (and there are a number of these out there), the data flowing between steps is either limited to what can be held in high-speed memory at one time, or has to be held on disk. FBP also supports loops in the network, which is hard to do in step-based systems - see for example http://www.jpaulmorrison.com/cgi-bin/wiki.pl?BrokerageApplication - notice that this application used both MQSeries and CORBA in a natural way. Furthermore, FBP is natively parallel, so it lends itself to programming of grid networks, multicore machines, and a number of the directions of modern computing. One last comment: in the literature I have found many related projects, but few of them have all the characteristics of FBP. A list that I have amassed over the years (a number of them closer than Grafcet) can be found in http://www.jpaulmorrison.com/cgi-bin/wiki.pl?FlowLikeProjects .
I do have to disagree with the comment about FBP being just a means of implementing FSMs: I think FSMs are neat, and I believe they have a definite role in building applications, but the core concept of FBP is of multiple component processes running asynchronously, communicating by means of streams of data chunks which run across what are now called bounded buffers. Yes, definitely FSMs are one way of building component processes, and in fact there is a whole chapter in my book on FBP devoted to this idea, and the related one of PDAs (1) - http://www.jpaulmorrison.com/fbp/compil.htm - but in my opinion an FSM implementing a non-trivial FBP network would be impossibly complex. As an example the diagram shown in
is about 1/3 of a single batch job running on a mainframe. Every one of those blocks is running asynchronously with all the others. By the way, I would be very interested to hearing more answers to the questions in the first post!
1: http://en.wikipedia.org/wiki/Pushdown_automaton Push-down automata
Whenever I hear the term flow based programming I think of LabView, conceptually. Ie component processes who's scheduling is driven primarily by a change to its input data. This really IS lego programming in the sense that the labview platform was used for the latest crop of mindstorm products. However I disagree that this makes it a less useful programming model.
For industrial systems which typically involve data collection, control, and automation, it fits very well. What is any control system if not data in transformed to data out? Ie what component in your control scheme would you not prefer to represent as a black box in a bigger picture, if you could do so. To achieve that level of architectural clarity using other methodologies you might have to draw a data domain class diagram, then a problem domain run time class relationship, then on top of that a use case diagram, and flip back and forth between them. With flow driven systems you have the luxury of being able to collapse a lot of this information together accurately enough that you can realistically design a system visually once the components are build and defined.
One question I never had to ask when looking at an application written in labview is "What piece of code set this value?", as it was inherent and easy to trace backwards from the data, and also mistakes like multiple untintended writers were impossible to create by mistake.
If only that was true of code written in a more typically procedural fashion!
1) I build a small FBP framework for an anomaly detection project, and it turns out to have been a great idea.
You can also have a look at some of the KNIME videos, that give a good idea of what a flow based framework feels like when the framework is put together by a great team. Admittedly, it is batch based and not created for continuous operation.
By far the best example of flow based programming, however, is UNIX pipes which is one of the oldest, most overlooked FBP framework. I don't think I have to elaborate on the power of nix pipes...
2) FBP is a very powerful tool for a large set of problems. The intrinsic parallelism is a great advantage, and any FBP framework can be made completely network transparent by using adapter modules. Smart frameworks are also absurdly fault tolerant, and able to dynamically reload crashed modules when necessary. The conceptual simplicity also allows cleaner communication with everybody involved in a project, and much cleaner code.
3) Absolutely! Pipes are here to stay, and are one of the most powerful feature of unix. The power inherent in a FBP framework compared to a static program are many, and trivialise change, to the point where some frameworks can be reconfigured while running with no special measures.
FBP FTW! ;-)
In automotive development, they have a language agnostic messaging protocol which is part of the MOST specification (Media Oriented Systems Transport), this was designed to communicate between components over a network or within the same device. Systems usually have both a real and visualized message bus - therefore you effectively have a form of flow based programming.
That was what made the light bulb go on for me several years ago and brought me here. It really is a fantastic way to work and so much more fun than conventional programming. The message catalog form the central specification and point of reference. It works well for both developers and management. i.e. Management are able to browse the message catalog instead of looking at source.
With integrated logging also referencing the catalog to produce intelligible analysis things can get really productive. I have real world experience of developing commercial products in this way. I am interested in taking things further, particularly with regards to tools and IDEs. Unfortunately I think many people within the automotive sector have missed the point about how great this is and have failed to build on it. They are now distracted by other fads and failed to realize that there was far more to most development than the physical bus.
I've used Spring Web Flow extensively in Java Web applications to model (typically) application processes, which tend to be complex wizard-like affairs with lots of conditional logic as to which pages to display. Its incredibly powerful. A new product was added and I managed to recut the existing pieces into a completely new application process in an hour or two (with adding a couple of new views/states).
I also looked into using OS Workflow to model business processes but that project got canned for various reasons.
In the Microsoft world you have Windows Workflow Foundation ("WWF"), which is becoming more popular, particularly in conjunction with Sharepoint.
FBP is just a means of implementing a finite state machine. It's nothing new.
I realize that it is not exactly the same thing, but this model has been used for years in PLC programming. ISO calls it Sequential Flow Chart, but many people call it Grafcet after a popular implementation. It offers parallel processing and defines transitions between states.
It's being used in the Business Intelligence world these days to mashup and process data. Data processing steps like ETL, querying, joining , and producing reports can be done by the end-user. I'm a developer on an open system - ComposableAnalytics.com In CA, the flow-based apps can be shared and executed via the browser.
This is what MQ Series, MSMQ and JMS are for.
This is cornerstone of Web Services and Enterprise Service Bus implementations.
Products like TIBCO and Sun's JCAPS are basically flow-based without using this particular buzz-word.
Most of the work of the application is done with small modules that pass messages through a processing network.