Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
In the Reinforcement Learning book by Sutton & Barto, version 2018, authors provided a formula (Eq. 7.14, page 151) of the off-policy form with control variate :
How can I understand this equation? I can understand if we are on-policy, the later two terms inside the gamma part cancels out. But anyone why do we have to multiply the rho with G_{t+1:h}? How does this formula make any sense?
The \rho_{t+1} should always be there, even without control variate. This is because this equation concerns state-action values. It becomes clearer when you write out the arguments of the returns:
where , with pi and b the target and behavior policies respectively, and A_{t+1} was sampled according to the behavior policy.
The book in this section 7.4 is a bit confusing, because it goes straight from on-policy to off-policy with covariates, skipping over the recursive expressions of the returns for off-policy without covariates.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
I am going through this paper to have a better understanding of how deep learning for Vision-related task works. I am not able to completely understand what exactly is "semantically-driven reconstruction" in this paragraph. -
The model is trained sequentially, starting from the lowest layer.
This allows to achieve good semantically-driven reconstruction results
at smaller scales that are working with images of very low resolution
and thus performing mostly global image manipulations.
Can anyone paraphrase this paragraph for easy understanding?
Thank You.
This PDF presentation should help, but the basic idea is that rather than simply parsing the image on pixels (e.g. boundaries, contours, depth), a semantic reconstruction would allow for image features to be labeled and the scene to be "understood" in a more human sense. In the context of this paragraph, then, they claim that by training their each model layer separately (rather than training all layers at once), they can successfully label and manipulate image features, even if the image resolution is low.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I am creating a chess variant. The rules and pieces are the same with classic chess. The only different is the size of the board (12x12 instead of 8x8).
My goal is to validate and apply moves only. What options I have aside from writing everything from scratch?
Most popular formats seem to be limited to 8x8 boards only.
I am fine with any popular programming language.
There are three general approaches chess engines take in move generation. In chess programming jargon these are commonly known as:
1)Bitboards
2)Mailbox (chess jargon for arrays with padding)
3)Piece lists
The most common method used today is Bitboards, which unfortunately is not easily modifiable to larger boards. This shouldn't be too bad for you, however. The reason bitboards are the de-facto standard is not because the are the easiest to implement (they are in fact the most complex), but because they are much faster for move generation (and by extension validation). However, this is only pertinent for use in a search function that needs to validate moves tens of millions of times per second. If you just want good old simple move validation, the method two should be more than adequate, and easily adaptable to larger boards. If you want to see chess engines that utilize this method, look up engiines that use a mailbox or oX88 board representation. I think the didactic CPW engine uses mailbox.
https://chessprogramming.wikispaces.com/CPW-Engine
and here is an article about move generation:
https://chessprogramming.wikispaces.com/Move+Generation
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
Let me tell you my requirements and see if anyone has some recommendations...
Cross browser (as much as realistically possible)
Be able to drag labeled objects around the screen (boxes and circles with labels in them)
Attach objects to each other by drawing lines (arrows) between them
Be able to "pick" any object on the screen (including those lines from #3)
I am not looking for a tool that just gives me all of that, I can program the logistics on top of it. But is seems to me I have basically described a scene graph. I know of cakejs - but was wondering of any other solutions out there.
Any help is greatly appreciated.
UmlCanvas might do what you need. It has a data definition language to describe graphs and relationships as well as a canvas to render those entities. Unfortunately it looks like something open-sourced by a startup before they decided to pursue other interests.
http://umlcanvas.org
http://blog.thesoftwarefactory.be
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I want to extract important terms from a text and create a domain specific term set. Then I want to learn how these words are used in text, positively or negatively.
Do you know any open source project which will help me to accomplish this tasks?
Edit:
Example Text:
"Although car is not comfortable, I like the design of it."
From this text, I want to extract something like these:
design: positive
comfort(able): negative
For parsing the text and getting the parts of speech you want, there are lots of toolkits
http://incubator.apache.org/opennlp/
http://www.nltk.org/
etc.
Check out http://en.wikipedia.org/wiki/Sentiment_analysis for ideas about finding how words are used positively or negatively, if what you mean by that is connotation. I don't know of any solid platforms for doing this, but maybe you can tell us more about your problem for some ideas.
In absence of a toolkit that'll do this for you, you might find that getting NPs and the ADJs linked to them would be sufficient. You'd also need negation detection. I've used this ohnlp.sourceforge.net (build on Apache UIMA) and it comes with a negation detection algorithm that is moderately decent.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
I am trying to see if my understanding of "othello" fame is correct or not. According to the rules, we flip the dark/light sides if we get some sequence like X000X => XXXXX. The question I have is if in the process of flipping 0->X or X-> 0, do we also need to consider the rows/columns/diagonals of newly flipped elements? e.g. consider board state as shown in above image(New element X is placed # 2,3)
When we update board, we mark elements from 2,3 to 6,3 as Xs but in this process elements like horizontal 4,3 to 4,5 and diagonal 2,3 to 4,5 are also eligible for update? so do we update those elements as well? or just the elements which have starting as 2,3 (i.e update rows/column/diagonal whose starting point is the element we are dealing with, in our case 2,3?)
Please help me understand it
No. Newly flipped pieces are not considered recursively.
No, this would unbalance the game significantly. Some friends and I actually tried playing like this a long time ago. The game turned out to be barely playable. It was less a matter of strategy and more a matter of luck and who happened to go first, reach a side first, etc.
In any given turn, tiles are flipped only outward from the piece which was placed during that turn.
(Note also in the example board in your question, O has seriously given up the game to X. He has no chance at this point :)