Demonstration set calculation from learning from demonstration reinforcement learning - reinforcement-learning

From this paper "Robotic Handling of Compliant Food Objects by Robust Learning from Demonstration" I didn't found the method that they use to calculate the demonstration set from state space and action space.
Really appreciate if you can help me
But I can't understand what they mean
I've try to read several paper, the mention the teleoperation use mapping from "A survey of robot learning from demonstration"
But still didn't make sense to me

Related

How could i code hard attention in keras?

How could i code hard attention in keras?
what should i do to make it in my projects?
i tried the soft attention and i can do it but i don't know how can i make the hard one.
i know some reinforcement learning but i didn't implement any algorithm yet.
i want to make one like the one implemented in "ask attend and tell" http://kelvinxu.github.io/projects/capgen.html
https://arxiv.org/pdf/1502.03044.pdf

Self-taught wanting to clean up (Packages, structure, best practices...)

First off I apologize for the slightly vague question.
I'm a self-taught coder and the more I work, the more I find myself wanting to invest in the whole process (and God knows there is room for improvement).
I'm looking for very "basic" material (books recommendation if you can!) about things like how to structure code and comments to make things clear, how to structure a project directory (How far should I divide my stuff into packages? How to properly name classes, subclasses, make it all understandable and clear?), this kind of stuff. Not the actual how make things work, but rather how to make things clean, understandable and well structured.
I've been reading a lot on design patterns, good software design and things like that, but there is this "gap" in my knowledge where I can think of a good solution to my problems, but the execution itself is pretty messy and ugly.
I've always worked solo, and chances are I will soon be working with other coders on my projects. Right now I sometimes have trouble understanding my own code, so there is no way others could dive in and help me.
I'm coding in AS 3.0, should anyone wonder.
Thank you very much :)
Depending on how new you are to coding, I'd recommend looking into why 'encapsulation' makes sense.
When I was new to object oriented programming I couldn't understand the appeal of encapsulation, but now I see how it really can help keep your code 'clean' - and I strongly suspect it's fundamental to a lot of other object oriented concepts; Design patterns, avoiding 'tight coupling', composition, code reuse and so on.
Here's the wiki info on encapsulation...
http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_programming)
..but here's my summary:
I've heard the analogy of a car used: The user doesn't need to know HOW the engine works; they just need to know that, when they turn the key, the engine WORKS. Encapsulation keeps the implementation of a class's methods secret. The methods themselves are public (to be accessible outside the class) but properties are private.
By making properties private and limiting the number of public-facing methods in your classes you can have your classes operate as independently as possible. It also means you can totally change HOW a method is implemented inside a class (so long as it retains the same name and argument signature) without anyone OUTSIDE of that class needing to know anything about it. Even if that 'anyone' is YOU it can still really speed up development because you're not having to track down multiple references to public variables of the class throughout your code.
So AIM to have your classes perform one function; Think about if a method would, in fact, be better 'hidden away' in another class. Even if you have multiple objects instantiated 'inside' another, your code will be much more clean and you can be more sure about WHERE that specific piece of code you're searching for is - if a class mostly has one function, it's easier to remember what it does.
Other than that, I'd also recommend using only shallow inheritance; Having a complex hierarchy of classes and subclasses can lead to objects that need to inherit from more than one class. Look into 'composition' and interfaces too. (Interfaces are kind of AS3's solution to not having multiple inheritance).

Guides for UI designers working in Google Web Toolkit

Can anyone point me to a useful guide for UI designers working in google web toolkit?
As per my comment to bhargava's answer, your designers should be learning UiBinder. The whole chapter about building user interfaces seems appropriate too (to get a better perspective), but UiBinder is what they will be mostly dealing with. Without using UiBinder in your project, you are stuck with Java and that's not something your designers are likely to know (and are probably not keen to :)).
I'd recommend building a simple example (but not too simple - maybe you should "strip" the official mail example) that uses UiBinder and show the designers exactly what you expect them to provide and what should be left as stubs. It all depends on the designers in question - whether all they know is HTML and CSS or maybe they have experience with Java, etc. Tailor the example to your needs - you probably won't "get it right" the first time, but with feedback from the designers (what's hard to understand, what they think should be the responsibility of the programmer, etc.), you should arrive at a good learning tool for future employees and a reference for current ones :)
Well if you're looking on how to use widgets and panels in gwt then i would recommend Roughian Examples
This website provides us the basic usage of the GWT widgets and panels and provides us with enough information just to get things started.

Pure AS 3 project structure?

I am kinda new to AS 3 development.. and the thing is I am always getting into problems how to structure the different views of an application state?
Do you use a MVC framework for this? I have looked at some puremvc, robotlegs but our flash projects are usually relativly small so I wonder if it is not a overkill :)
Design patters are good as long as you don't try to use them for the sake of using them. I always have my views separate to model as it is, but rarely work on something big enough to warrant a flow blown implementation of something like PureMVC. On large projects then yes they can be beneficial. The hard part is deciding how "big" is big enough to use them. Probably comes down to experience to make that judgment.
Anyway just make sure objects are loosely coupled and that your model and view are not one and the same. Quite often instead of extending a MovieClip I prefer to use composition and have a class have a MovieClip. For me I think it helps separate visuals from the data.
To be honest I'm not a fan of applying traditional design patterns to Flash development. If you're writing pure classes then you can do anything you like, but if you're linking classes to visual elements (like MovieClips), I find that the proper way to structure things (i.e. the way that best avoids duplication of effort and preserves separation of concerns) winds up being dictated by how you need to stack and/or nest your display timelines.
In other words, I find it better to work out at the display level what things ultimately will need to be in the same or in different timelines, and what will ultimately need to be nested or not nested, and then work out which classes should extend or compose each other afterwards.
But note that this is all assuming you're making stuff where the logic is closely coupled with the display elements, which is what I consider the area where Flash shines. Naturally if you're making a loan simulator with no animations or skinning or whatever, it's sensible enough to just use an MVC pattern and then slap a static display layer on top, with each button pointed to a C and each text box pointed to a V. But then that's exactly what you'd do if you were making the same thing in Java, or as a JS-powered HTML form, or whatever else. The advice in this answer is what I find works best when you're making the sorts of heavily visual things that would be ill-advised to attempt in anything but Flash.
A framework can be particularly valuable if you intend on handing the code off to someone else. Whatever framework you choose, you'll introduce a common set of expected relationships from which someone can build an understanding of the codebase.
If the view logic is largely sequential, or at least modal, an easy non-framework mode of organization is to group your views into "pages" and have a single controller manage which page is on-stage at a time and feed in model data from wherever.
if you are really new to AS3 and are planning to do "pure" coding (neither using the graphical approach promoted by the Flash authoring tool, nor diving into the Flex world), i suggest you don't waste your time on AS3 and move on to Haxe ...
Haxe is not very well known, so the only opinions you'll get are from people using it ... there's a little bias of course, but you may read their statements here, here and here ...
now when it comes to patterns and framworks, there are many possibilities ... design patterns are solutions to a set of problems ... do not make the mistake of trying to transform all the problems you encounter into problems that fit the patterns you know ...
in my book, there is one fundamental principle for software design: DRY & KISS - I admittedly still have a lot to work on the latter ... :) ... for more concrete and guiding principles, I stick to SOLID and GRASP ... if your read them carefully and reflect upon the (good) patterns you know, you are likely to find out, that they incorporate said principles ... do not overuse patterns ... do not overuse any tools in fact ... use whatever tool is the best for the job ...
IMHO, pure AS3 is for geeks ... and I love doing pure AS3 ... or at least I did ... it is however uneffective and unrewarding over time ... it is good for developping frameworks and libraries ... but nor for apps ... Haxe is good for distributed cross-plattform apps (and for geeks), Flash Authoring approach for "fancy" visual apps, and Flex for classic apps featuring prevalant UI concepts ...

Balancing HTML/CSS Between Designers and Engineers [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have a development process question.
Background: I work for a modest sized website where, historically, the designers created mockups/screenshots of what they wanted pages and components to look like, and the engineering team (myself included) turned them into html/css.
This works relatively well from a code cleanliness perspective, and helps significantly when it comes to writing javascript. It has fails, however, in helping to maintain consistency from one page/component to another. On one page, a header font might be 12px and on another 11px, largely because its a complicated site with lots to keep track of (and we've cycled through 4 designers.) We have only a few truly universal styles, and they only get used when the engineers recognizes the style - not when the designer tells them to.
Our most recent designer is a relatively capable HTML/CSS coder. We thought we might have him create mockups in HTML/CSS and hand us off the code for quick integration. Our hope was that the designer would be better at being consistent in his style and that it might save us some development time up front.
What we've discovered is that our designer is not quite as good as CSS as we had hoped and that his code is often slightly bloated and incompatible with what we need to do. Also, his style of coding is fundamentally different from the rest of the engineering team and isn't jiving terribly well with our established coding practices.
Question: How do you do the hand off from design to engineering? I know I've heard of companies that let their design team do all of the template coding, but I'm curious how that works. Does the design team actually incorporate members of the engineering team in those scenarios?
As we're structured right now, there's not a chance in hell we'd let our designer write the final templates and check them into SVN, even if he was a proficient HTML wiz. There's too much in the templates that requires knowledge of our codebase and of potential performance issues.
How do we get this process working? Is it a pipe dream?
Specifically - personally - since I come from a web-dev, small-shop background I do the CSS work and slicing the PSD (typically) myself. But then I like to think I'm well rounded like that :)
Generally, the best experience I've had of this was a largish company with very defined groups of developers including the design team who produced the gfx, the apps team who did the vast bulk of server-side coding and app architecture, and the UE (user experience) team who sewed the two together, producing XSLT/JSP/HTML markup in general, and the CSS and JS for the client-side.
There was a very structured process of:
userstory ->
"wireframe" (documents) ->
design (PSD) ->
"flat" markup (DHTML only) ->
integrated markup (with web-app)
Where "wireframe" would be close to a spec for UE, produced with UML or maybe visio. I have heard the term applied to step 4 which I think fits better, but this is what it was referred to as there.
Whilst this works well for the question at hand, I found it had other problems built in. It was very hard to work across teams, and because of the timescales the design team rarely involved UE in decision making (which put UE in some awkward positions), the apps team and design could be working at cross-purposes, and there wasn't a lot of scope to learn in these boxed in teams.
My suspicion (and I think ideal scenario) is that the developers on a project would each be capable of working with, say, 80% of the technology involved (be it CSS, SQL, whatever) to spread the decision control and risk, but each domain would have one (more?) "czar" who could act as authority and oversight within the domain. Actually producing those designs is to my mind a strange and magical skill in it's own right so I see no real overlap with developers there, but I think a pool of artists and project teams of cross-skilled programmers would be very powerful.
Apols for the long-windedness. I could go on at considerable length on this, I've spent a lot of time thinking about it.
btw, it seems like you could do with some serious web-devs there, (no offence). Having problems to "maintain consistency from one page/component to another" screams failure to grok CSS
In my experience, unless you limit your design severely, you need real coding skills to build a web page with interaction. Let me elaborate some. If you have built your pages very modular (think of GUI toolkit widgets) you can give your designer a handful of them, he can build the basic structure like playing toy blocks with a nice finishing paint.
Often, modularization alone is not enough for desired interactivity. So, some blocks needs their interactions to be designed carefully as well (like animation, fluid layout to accommodate indeterminate content, customized behaviour via extra javascript, caching to eliminate redundant requests and speeding up things) or ability to accommodate minor presentation variations, which brings us to the realm of programming, where you calculate dimensions, enable/disable parts, keep track of time, preload stuff, invalidate preloaded stuff and so on.
Enter HTML/CSS/JS. They are more of a product of evolution than intelligent design. You cannot always declare your intent and be done with it. You need attributes declared in your html, stupid hacks in CSS combined with extra markup, ridiculous amounts of js to smooth rough edges, duplicate rendering code on the server side. These tool were never meant to build applications.
I don't think one can achieve a complete separation of design and application development in these tools at hand. The effort required is too high to justify the marginal returns.
If you end up heavily modifying designer's code (which is othen the case if he is not one of the developers also), there is no point in making him suffer trying to express his intent using the wrong tools, nor developers breaking the design while modifying it and consequently fixing it. I don't even mention user experience.
In my opinion, no small internet businesses who want to ship a product in a reasonable time should spend their scarce resources to go against the grain. Let people do what they do best in collaboration if necessary. If you can't divide design process at an arbitrary satisfying point, you may as well not bother to separate at all. Pipelining works well for machines whose goal is determined to the last detail and not changing. I can't say the same for humans building and designing things be it software or hardware.
Where I work it's basically the same. Designers create mock-ups and specifications of the UI design, right down to the pixel, and the developer creates HTML/CSS/code out of that.
The reason I say code, is that we use UI frameworks (namely, GWT), and as much as we would want to, code and CSS styles are still very coupled. I do not believe there exists one UI framework in which code can be completely decoupled from the UI design.
So I guess for now it's still entirely the developers job. Though I would like to hear about organization which are able to hand off some of the work to designers.
The problem with handoffs is that the idea and implementation of one group is not going to match the abilities and implementation of the next group. Just by their nature handoffs are going to be wrought with problems. So what is an alternative to the ubiquitous handoff scenario? I think that integrating the user experience (UX) into an agile and iterative development process makes sure that what is really important occurs:
The customer's needs are researched then validated.
Early and continuous collaborating between usability experts, designers and programmers.
The actual process works by having everyone collaborate with the customer up-front on their needs. Then the design is researched and prototyped in the iteration before coding begins. Thus when coding is occurring, the next set of designs are being worked on. Programmers should be looking forward at what designers are doing and the designers look back to be sure programmers are on target. Once a design is coded, it goes to the customer for acceptance, by that time the programmers are working on the next set of interfaces.
Jeff Patton did a podcast on Agile UX recently that goes into some of the implementation concepts and common problems.
There is a whole group on Yahoo dedicated to agile usability (which mostly involves interface design).
For the CSS inconsistencies... I'd just suggest making a style guide then trying to stick to it. Have someone be in charge of "design consistency" that way the can spank anyone inventing yet another way to display the user.
At my company, my ideal work flow doesn't work very often, but sometimes it does. I löve when this happens: The engineers write the webapp and output semantic html with only minimal CSS. then you have the designers do the CSS.
I like it when it goes this way, because:
It is easy for me to write semantic
HTML.
I am not very good at coming up
with a good design for my semantic
html.
It is entirely possible to do
the CSS without asking me questions.
The markup just speaks for itself.
However, this rarely works. Because:
The CSS has to be modified whenever the HTML changes and the designers' time is sparse.
Moreover, our designers don't enjoy styling my markup, and fighting for their time is not pleasant.
Our designers often want to change the markup. Mostly because they believe some layouts cannot be done without changing the markup or because they believe that it's the only way to make IE obey. They are technically not able to change the markup, though.
I have my doubts about many of their cases. Many times they claim IE incompatibility, I strongly doubt they really know IE that well. There are neat CSS hacks to make IE obey without resorting to
<br clear="all">
So, sadly, usually this ideal is a little off for me.
A separate designer - developer workflow is the best way to go. Designing a website and coding it are altogether different jobs. There are issues of cross browser compatability, CSS, XHTML, apart from coding standards to deal with.
You could also opt for outsourcing your HTML to a specialized PSD to HTML conversion expert like us (ButterflyHTML). It may work out cost effective in the long run