Related
I’m trying to use alpha zero general to apply on a different game (kind of like Chess), this is the original code for othello:
https://github.com/suragnair/alpha-zero-general
However, after a few iterations (about 300 self play), it still seems like doing nonsense moves. So I’m wondering whether my code is wrong. Here are some few questions I came up with:
should num_channels be modified?
note: I’m actually confused about the parameter “number of channels”. In my opinion, it should be at most 3 for othello (there are only “black”, “white” and “none” type of pieces), however, num_channels is set to 128 in the original case.
Another question is about the “board”, which will be the input of the nnet. The original code uses a 2D array to present a Othello board, which uses 1 to represent a white piece, -1 for the black piece, 0 for none.
However, there are 6, for instance, kind of pieces in chess. Normally it should be input as a 3D dimension, 2D for the board, another dimension for different kinds of pieces. (or channel, just like an RGB 128*128 picture should be a 3D input (3*128*128))
I now use a 2D array only, where 1,2,3,... represents king, queen, bishop, etc. I’m not sure if this causes a problem.
I’ve tried to figure it out from the written code but I couldn’t find an answer.
Yes, you will need to have a channel for each type of piece, or another more compact representation of the board. The reason has got to do with the 'squashing' functions (non-linearities) and the fact that the network is learning a continuous map. You put too much of a 'burden' on a single input neuron. You will need a channel for each type of piece for each player, if you have 8 types of piece and 2 players you will need 16 channels. Use 0-1 matracies for each channel.
Think about what will happen to the input values if they are integers when we run them though this function.
I am writing a children learning game, whereby you move letter blocks around in physics and snap them in to the correct place.
I have 3 static sensors which work great and when another box touches them I can run code on that instance, but for the life of me I cannot get the moving box to take position of the sensor.
3 days I have tried different ways, with set transform.
All I need to do is move a dynamic box that hits my sensor in to the exact same position as the sensor.
I thought it would be easy but its giving me a real headache...
I had similar problems and I found this page with a lot of good tutorials, when I learned box2d... Maybe lesson 2.25 and 2.26 point out what you desiring for.
good luck
I have a simple question regarding the Minimax algorithm: for example for the tic-tac-toe game, how do I determine the utility function's for each player plays? It doesn't do that automatically, does it? I must hard-code the values in the game, it can't learn them by itself, does it?
No, a MiniMax does not learn. It is a smarter version of a brute-force tree search.
Typically you would implement the utility function directly. In this case the algorithm would not learn how to play the game, it would use the information that you had explicitly hard-coded in the implementation.
However, it would be possible to use genetic programming (GP) or some equivalent technique to automatically derive a utility function. In this case you would not have to encode any explicit strategy. Instead the evolution would discover its own way of playing the game well.
You could either combine your minimax code and the GP code into a single (probably very slow) adaptive program, or you could run the GP first, find a good utility function and then add this function to your minimax code just as you would any hand-coded function.
Tic-Tac-Toe is small enough to run the game to the end and assign 1 for win, 0 for draw and -1 for lose.
Otherwise you have to provide a function which determines the value of a position heuristically. In chess for example a big factor is the value of the material, but also who controls the center or how easily the pieces can move.
As for learning, you can add weight factors to different aspects of the position and try to optimize those by repeatedly playing games.
How do determine the utility function for each play?
Carefully ;-) This article shows how a slightly flawed evaluation function (one for ex. which either doesn't go "deep" enough in looking ahead in the tree of possible plys, or one which fails to capture the relative strengh of some board positions) results in an overall weak algorithm (one that looses more often).
it can't learn them by itself, does it?
No, it doesn't. There are ways, however, to make the computer learn the relative strength of board positions. For example by looking into Donald Mitchie and his MENACE program you'll see how a stochastic process can be used to learn the board without any a priori knowledge but the rules of the game. The funny part is that while this can be implemented in computers, a few hundred colored beads and match boxes are all that is required, thanks to the relatively small size of the game space, and also thanks to various symmetries.
After learning such a cool way of teaching the computer how to play, we may not be so interested in going back to MinMax as applied to Tic-Tac-Toe. After all MinMax is a relatively simple way of pruning a decision tree, which is hardly needed with tic-tac-toe's small game space. But, if we must ;-) [go back to MinMax]...
We can look into the "matchbox" associated with the next play (i.e. not going deep at all), and use the percentage of beads associated with each square, as an additional factor. We can then evaluate a traditional tree, but only going, say 2 or 3 moves deep (a shallow look-ahead depth which would typically end in usually in losses or draws) and rate each next move on the basis of the simple -1 (loss), 0 (draw/unknown), +1 (win) rating. By then combining the beads percentage and the simple rating (by say addition, certainly not by multiplication), we are able to effectively use MinMax in a fashion that is more akin to the way it is used in cases when it is not possible to evaluate the game tree to its end.
Bottom line: In the case of Tic-Tac-Toe, MinMax only becomes more interesting (for example in helping us explore the effectiveness of a particular utility function) when we remove the deterministic nature of the game, associated with the easy evaluation the full tree. Another way of making the game [mathematically] interesting is to play with a opponent which makes mistakes...
Does anyone have any good references for equations which can be implemented relatively easily for how to compute the transfer of angular momentum between two rigid bodies?
I've been searching for this sort of thing for a while, and I haven't found any particularly comprehensible explanations of the problem.
To be precise, the question comes about as this; two rigid bodies are moving on a frictionless (well, nearly) surface; think of it as air hockey. The two rigid bodies come into contact, and then move away. Now, without considering angular momentum, the equations are relatively simple; the problem becomes, what happens with the transfer of angular momentum between the bodies?
As an example, assume the two bodies have no angular momentum whatsoever; they're not rotating. When they interact at an oblique angle (vector of travel does not align with the line of their centers of mass), obviously a certain amount of their momentum gets transferred into angular momentum (i.e. they each get a certain amount of spin), but how much and what are the equations for such?
This can probably be solved by using a many-body rigid system to calculate, but I want to get a much more optimized calculation going, so I can calculate this stuff in real-time. Does anyone have any ideas on the equations, or pointers to open-source implementations of these calculations for inclusion in a project? To be precise, I need this to be a rather well-optimized calculation, because of the number of interactions that need to be simulated within a single "tick" of the simulation.
Edit: Okay, it looks like there's not a lot of precise information about this topic out there. And I find the "Physics for Programmers" type of books to be a bit too... dumbed down to really get; I don't want code implementation of an algorithm; I want to figure out (or at least have sketched out for me) the algorithm. Only in that way can I properly optimize it for my needs. Does anyone have any mathematic references on this sort of topic?
If you're interested in rotating non-spherical bodies then http://www.myphysicslab.com/collision.html shows how to do it. The asymmetry of the bodies means that the normal contact force during the collision can create a torque about their respective CGs, and thus cause the bodies to start spinning.
In the case of a billiard ball or air hockey puck, things are a bit more subtle. Since the body is spherical/circular, the normal force is always right through the CG, so there's no torque. However, the normal force is not the only force. There's also a friction force that is tangential to the contact normal which will create a torque about the CG. The magnitude of the friction force is proportional to the normal force and the coefficient of friction, and opposite the direction of relative motion. Its direction is opposing the relative motion of the objects at their contact point.
Well, my favorite physics book is Halliday and Resnick. I never ever feel like that book is dumbing down anything for me (the dumb is inside the skull, not on the page...).
If you set up a thought problem, you can start to get a feeling for how this would play out.
Imagine that your two rigid air hockey pucks are frictionless on the bottom but have a maximal coefficient of friction around the edges. Clearly, if the two pucks head towards each other with identical kinetic energy, they will collide perfectly elastically and head back in opposite directions.
However, if their centers are offset by 2*radius - epsilon, they'll just barely touch at one point on the perimeter. If they had an incredibly high coefficient of friction around the edge, you can imagine that all of their energy would be transferred into rotation. There would have to be a separation after the impact, of course, or they'd immediately stop their own rotations as they stuck together.
So, if you're just looking for something plausible and interesting looking (ala game physics), I'd say that you could normalize the coefficient of friction to account for the tiny contact area between the two bodies (pick something that looks interesting) and use the sin of the angle between the path of the bodies and the impact point. Straight on, you'd get a bounce, 45 degrees would give you bounce and spin, 90 degrees offset would give you maximal spin and least bounce.
Obviously, none of the above is an accurate simulation. It should be a simple enough framework to cause interesting behaviors to happen, though.
EDIT: Okay, I came up with another interesting example that is perhaps more telling.
Imagine a single disk (as above) moving towards a motionless, rigid, near one-dimensional pin tip that provides the previous high friction but low stickiness. If the disk passes at a distance that it just kisses the edge, you can imagine that a fraction of its linear energy will be converted to rotational energy.
However, one thing you know for certain is that there is a maximum rotational energy after this touch: the disk cannot end up spinning at such a speed that it's outer edge is moving at a speed higher than the original linear speed. So, if the disk was moving at one meter per second, it can't end up in a situation where its outer edge is moving at more than one meter per second.
So, now that we have a long essay, there are a few straightforward concepts that should aid intuition:
The sine of the angle of the impact will affect the resulting rotation.
The linear energy will determine the maximum possible rotational energy.
A single parameter can simulate the relevant coefficients of friction to the point of being interesting to look at in simulation.
You should have a look at Physics for Game Developers - it's hard to go wrong with an O'Reilly book.
Unless you have an excellent reason for reinventing the wheel,
I'd suggest taking a good look at the source code of some open source physics engines, like Open Dynamics Engine or Bullet. Efficient algorithms in this area are an artform, and the best implementations no doubt are found in the wild, in throroughly peer-reviewed projects like these.
Please have a look at this references!
If you want to go really into Mecanics, this is the way to go, and its the correct and mathematically proper way!
Glocker Ch., Set-Valued Force Laws: Dynamics of Non-Smooth Systems. Lecture Notes in Applied Mechanics 1, Springer Verlag, Berlin, Heidelberg 2001, 222 pages. PDF (Contents, 149 kB)
Pfeiffer F., Glocker Ch., Multibody Dynamics with Unilateral Contacts. JohnWiley & Sons, New York 1996, 317 pages. PDF (Contents, 398 kB)
Glocker Ch., Dynamik von Starrkörpersystemen mit Reibung und Stößen. VDI-Fortschrittberichte Mechanik/Bruchmechanik, Reihe 18, Nr. 182, VDI-Verlag, Düsseldorf, 1995, 220 pages. PDF (4094 kB)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I'm teaching a kid programming, and am introducing some basic artificial intelligence concepts at the moment. To begin with we're going to implement a tic-tac-toe game that searches the entire game tree and as such plays perfectly. Once we finish that I want to apply the same concepts to a game that has too many positions to evaluate every single one, so that we need to implement a heuristic to evaluate intermediate positions.
The best thing I could think of was Dots and Boxes. It has the advantage that I can set the board size arbitrarily large to stop him from searching the entire tree, and I can make a very basic scoring function be the number of my boxes minus the number of opponent boxes. Unfortunately this means that for most of the beginning of the game every position will be evaluated equivalently with a score of 0, because it takes quite a few moves before players actually start making boxes.
Does anyone have any better ideas for games? (Or a better scoring function for dots and boxes)?
Another game choice could be Reversi aka Othello.
A naive heuristic would be to simply count the number of tiles gained by each valid move and choose the greatest. From there you can factor in board position and minimizing vulnerably to the opponent.
One game you may consider is Connect Four. Simple game with straightforward rules but more complicated that Tic-Tac-Toe.
Checkers will let you teach several methods. Simple lookahead, depth search of best-case-worst-case decisions, differences between short-term and long-term gains, and something they could continue to work on after learning what you want to teach them.
Personally I think that last bit is the most critical -- there are natural points in the AI development which are good to stop at, see if you can beat it, and then delve into deeper AI mechanisms. It keeps your student interested without being horribly frustrated, and gives them more to do on their own if they want to continue the project.
How about Reversi? It has a pretty nice space of heuristics based on number of pieces, number of edge pieces, and number of corner pieces.
How about Mancala? Only 6 possible moves each turn, and it's easy to calculate the resulting score for each, but it's important to consider the opponent's response, and the game tree gets big pretty fast.
Gomoku is a nice, simple game, and fun one to write AI for.
Rubik's Infinity's quite fun, it's a little bit like Connect Four but subtly different. Evauluating a position is pretty easy.
I knocked together a Perl script to play it a while back, and actually had to reduce the number of moves ahead it looked, or it beat me every time, usually with quite surprising tactics.
Four in a line Hard enough, but easy enough to come up with an easy working evaluation function, for example, (distance to four from my longest line - distance to four from my opponent's longest line)
I really like Connect Four. Very easy to program using a Minimax algorithm. A good evaluation function could be:
eval_score = 0
for all possible rows/lines/diagonals of length 4 on the board:
if (#player_pieces = 0) // possible to connect four here?
if (#computer_pieces = 4)
eval_score = 10000
break for loop
else
eval_score = eval_score + #computer_pieces
(less pieces to go -> higher score)
end if
else if (#player_pieces = 4)
eval_score = -10000
break for loop
end if
end for
To improve the program you can add:
If computer moves first, play in the middle column (this has been proven to be optimal)
Alpha-Beta Pruning
Move Ordering
Zobrist Hashes
How about starting your Dots and Boxes game with random lines already added. This can get you into the action quickly. Just need to make sure you don't start the game with any boxes.
Take a look at Go.
Simple enough for kid on very small boards.
Complexity scales infinitely.
Has a lot of available papers, algorithms and programs to use either as a scale or basis.
Update: reversi was mentioned, which is a simplified variant of Go. Might be a better choice.
In regards to a better heuristic for dots and boxes, I suggest looking at online strategy guides for the game. The first result on Google for "dots and boxes strategy" is quite helpful.
Knowing how to use the chain rule separates an OK player from a good one. Knowing when the chain rule will work against you is what separates the best players from the good ones.