I have a simple question regarding the Minimax algorithm: for example for the tic-tac-toe game, how do I determine the utility function's for each player plays? It doesn't do that automatically, does it? I must hard-code the values in the game, it can't learn them by itself, does it?
No, a MiniMax does not learn. It is a smarter version of a brute-force tree search.
Typically you would implement the utility function directly. In this case the algorithm would not learn how to play the game, it would use the information that you had explicitly hard-coded in the implementation.
However, it would be possible to use genetic programming (GP) or some equivalent technique to automatically derive a utility function. In this case you would not have to encode any explicit strategy. Instead the evolution would discover its own way of playing the game well.
You could either combine your minimax code and the GP code into a single (probably very slow) adaptive program, or you could run the GP first, find a good utility function and then add this function to your minimax code just as you would any hand-coded function.
Tic-Tac-Toe is small enough to run the game to the end and assign 1 for win, 0 for draw and -1 for lose.
Otherwise you have to provide a function which determines the value of a position heuristically. In chess for example a big factor is the value of the material, but also who controls the center or how easily the pieces can move.
As for learning, you can add weight factors to different aspects of the position and try to optimize those by repeatedly playing games.
How do determine the utility function for each play?
Carefully ;-) This article shows how a slightly flawed evaluation function (one for ex. which either doesn't go "deep" enough in looking ahead in the tree of possible plys, or one which fails to capture the relative strengh of some board positions) results in an overall weak algorithm (one that looses more often).
it can't learn them by itself, does it?
No, it doesn't. There are ways, however, to make the computer learn the relative strength of board positions. For example by looking into Donald Mitchie and his MENACE program you'll see how a stochastic process can be used to learn the board without any a priori knowledge but the rules of the game. The funny part is that while this can be implemented in computers, a few hundred colored beads and match boxes are all that is required, thanks to the relatively small size of the game space, and also thanks to various symmetries.
After learning such a cool way of teaching the computer how to play, we may not be so interested in going back to MinMax as applied to Tic-Tac-Toe. After all MinMax is a relatively simple way of pruning a decision tree, which is hardly needed with tic-tac-toe's small game space. But, if we must ;-) [go back to MinMax]...
We can look into the "matchbox" associated with the next play (i.e. not going deep at all), and use the percentage of beads associated with each square, as an additional factor. We can then evaluate a traditional tree, but only going, say 2 or 3 moves deep (a shallow look-ahead depth which would typically end in usually in losses or draws) and rate each next move on the basis of the simple -1 (loss), 0 (draw/unknown), +1 (win) rating. By then combining the beads percentage and the simple rating (by say addition, certainly not by multiplication), we are able to effectively use MinMax in a fashion that is more akin to the way it is used in cases when it is not possible to evaluate the game tree to its end.
Bottom line: In the case of Tic-Tac-Toe, MinMax only becomes more interesting (for example in helping us explore the effectiveness of a particular utility function) when we remove the deterministic nature of the game, associated with the easy evaluation the full tree. Another way of making the game [mathematically] interesting is to play with a opponent which makes mistakes...
Related
I'm attempting to write a short mini-program in Python that plays around with force-based algorithms for graph drawing.
I'm trying to minimize the number of times lines intersect. Wikipedia suggests giving the lines an electrical charge so that they repel each other. I asked my physics teacher how I might simulate this, and she mentioned using calculus with Coulomb's Law, but I'm uncertain how to start.
Could somebody give me a hint on how I could do this? (Or alternatively, another way to tweak a force-based graph drawing algorithm to minimize the number of times the lines cross?) I'm just looking for a hint; no source code please.
In case anybody's interested, my source code and a youtube vid I made about it.
You need to explicitly include a term in your cost function that minimizes the number of edge crossings. For example, for every pair of edges that cross, you incur a fixed penalty or, if the edges are weighted, you incur a penalty that is the product of the two weights.
I am working on a simple drawing application, and i need an algorithm to make flood fills.
The user workflow will look like this (similar to Flash CS, just more simpler):
the user draws straight lines on the workspace. These are treated as vectors, and can be selected and moved after they are drawn.
user selects the fill tool, and clicks on the drawing area. If the area is surrounded by lines in every direction a fill is applied to the area.
if the lines are moved after the fill is applied, the area of fill is changed accordingly.
Anyone has a nice idea, how to implement such algorithm? The main task is basically to determine the line segments surrounding a point. (and storing this information somehow, incase the lines are moved)
EDIT: an explanation image: (there can be other lines of course in the canvas, that do not matter for the fill algorithm)
EDIT2: a more difficult situation:
EDIT3: I have found a way to fill polygons with holes http://alienryderflex.com/polygon_fill/ , now the main question is, how do i find my polygons?
You're looking for a point location algorithm. It's not overly complex, but it's not simple enough to explain here. There's a good chapter on it in this book: http://www.cs.uu.nl/geobook/
When I get home I'll get my copy of the book and see if I can try anyway. There's just a lot of details you need to know about. It all boils down to building a DCEL of the input and maintain a datastructure as lines are added or removed. Any query with a mouse coord will simply return an inner halfedge of the component, and those in particular contain pointers to all of the inner components, which is exactly what you're asking for.
One thing though, is that you need to know the intersections in the input (because you cannot build the trapezoidal map if you have intersecting lines) , and if you can get away with it (i.e. input is few enough segments) I strongly suggest that you just use the naive O(n²) algorithm (simple, codeable and testable in less than 1 hour). The O(n log n) algorithm takes a few days to code and use a clever and very non-trivial data structure for the status. It is however also mentioned in the book, so if you feel up to the task you have 2 reasons to buy it. It is a really good book on geometric problems in general, so for that reason alone any programmer with interest in algorithms and datastructures should have a copy.
Try this:
http://keith-hair.net/blog/2008/08/04/find-intersection-point-of-two-lines-in-as3/
The function returns the intersection (if any) between two lines in ActionScript. You'll need to loop through all your lines against each other to get all of them.
Of course the order of the points will be significant if you're planning on filling them - that could be harder!
With ActionScript you can use beginFill and endFill, e.g.
pen_mc.beginFill(0x000000,100);
pen_mc.lineTo(400,100);
pen_mc.lineTo(400,200);
pen_mc.lineTo(300,200);
pen_mc.lineTo(300,100);
pen_mc.endFill();
http://www.actionscript.org/resources/articles/212/1/Dynamic-Drawing-Using-ActionScript/Page1.html
Flash CS4 also introduces support for paths:
http://www.flashandmath.com/basic/drawpathCS4/index.html
If you want to get crazy and code your own flood fill then Wikipedia has a decent primer, but I think that would be reinventing the atom for these purposes.
Here's the current dependency graph (with freehand circles for TheTXI)
A game has players and a single board shared between them.
A player also has access to the board to be able to add/move/remove units from it.
A player has access to the units it owns, both on and off the board (A unit also knows it's owner, but that can probably be removed and just do a lookup).
A board has units on it and knows units' positions.
Units have abilities (Players possibly can as well)
The big one that I'm having trouble figuring out is Unit's Abilities. They should be able to affect anything in the game, healing/damaging a player/unit, reposition things on the board, possibly even the game itself (so far there isn't a need, but it could come up).
How can I have abilities that can affect anything without them having a reference to everything? I realize each ability can and should have references to only what it needs, but a unit class has abilities built in to them, so if a unit's ability affects the board, it will need to get a reference to the board from the unit somehow?
I'm trying to keep the design as flexible as possible, because the rules aren't set in stone yet (we're creating a game, and still pretty early on, so it's try something, see how it feels, change the rules until the game feels right)
Even whether there's a board/map at all is still up in the air, so the units should be decoupled from that, which they currently are. There is no global state or any "god object" (yet), and I'd like to keep it that way.
Specifically it's in Python, webapp, so while the question itself is language agnostic, any specifics based on a dynamic language with first class functions are certainly welcome.
First, let me point you to the wonderful gamedev StackExchange. You might have more luck posting this question there.
As far as your question itself goes, I think one solution would be to pass notifications from the object triggering the ability up to the game object, which would then parse these notifications and hand them out to specific players, or boards, and from there to specific units.
It's hard to explain, so let me try and write it out in psuedocode...
Game {
getMe(); //Returns reference to singleton class.
list of players
list of boards
}
Board {
list of units
}
Unit {
int health
}
function Unit.notifyAbility(source, targeting-condition, ability-code) {
Game::getMe()->sendNotification(source, targeting-condition, ability-code);
}
function Game.sendNotification(source, targeting-condition, ability-code) {
for each unit in list of units {
if(unit matches targeting condition) {
apply ability-code
}
}
}
Targeting-condition and ability-code could themselves be data structures that pass relevant information. Even better, make them virtual classes and use some form of polymorphism to handle unique cases.
Example:
AbilityCode {
virtual function applyToUnit(target Unit)
virtual function applyToPlayer(target Unit)
}
AbilityGainHP: child of AbilityCode {
function applyToUnit(target Unit) { target.hp+= gainAmt; }
int gainAmt;
}
Hope this makes sense.
You might overthink this a bit.
Keep in mind, you want to be able to work with your code base, so it's important that you can easily understand your code, even if you haven't worked on that particular part for a bit.
If you describe your game in words, you'll most likely the use something like:
It's a game for X players, played on a board of (specfics).
Each player controls Y units.
So, start your design like this.
The game (either your complete code, or just the PLAYING state of your program) needs to have references to the board and the players.
Now, the board will handle most of your problems.
If a player interacts with the game, he'll most likely do something on the board.
Start with a simple system that allows you to move stuff around you should be able to move around, while not being able to move the stuff you shouldn't.
The abilities will most likely be a list containing the currently available selection out of the whole list.
These will depend on the unit and maybe other elements of the board.
Depending on the type of game, you can either use events to inform the units that something has changed, or simply get the information when you need it.
Don't add more layers of abstraction than you actually need.
From your description, the abilities seem to be the part you want to spend most time on, to ensure you can easily change and add to them.
Start with a simple design for the other parts, and refactor if you see that you need an extra layer at certain points.
This will avoid analysis paralysis. Esp. when you want to make a "neat design" it's very easy to overcomplicate things.
You want to get the game running. You want to be able to read (and understand) your code easily.
Your coding style and knowledge will change even while working on the project, and you'll also get new ideas while coding.
while playing to this game I wondered how an AI controlling either the detectives either the criminal could work.
For lazy people the aim of the game is simple:
the board game is an undirected graphs that has 4 kinds of edges (that can also overlap for same pair or vertices), each kind is a type of transport that requires a specific kind of ticket
detectives have a bunch of tickets to move around this graph, one move per turn (which means from a node to another node). The criminal can do the same set of moves (plus 3 exclusive paths) but with no limits on tickes
the criminal is usually hidden to detectives but it has to show up himself in 5 specific turns (and then hide again)
if detectives are able to catch him (one of them must occupy the same cell of the criminal) before 24 moves then they win, otherwise the criminal wins
the criminal has to show which ticket he uses each turn but he also has 1 black ticket per detective (let's assume 5) that can be used to vanify this thing
the criminal also has two 2x tickets that allow him to use two tickets (and so two movements) in the same turn
I can think effectively about an AI for the criminal that it would be just a minmax tree that tries to choose movements that maximize the number of moves needed by detectives to reach him (it seems to be a good metric) but I cannot think anything enough cool for detectives which should cooperate and try to guess where the criminal can be by looking at tickets it uses.
It's just for fun but do you now any cool ideas to work out something quite clever?
You've asked how to model this, not how to solve this efficiently:
It can be easily modeled as a partially observable markov decision process (wiki link). This works both for the detectives and the criminal. POMDPs are a very generic model.
I love this game, and I think for the detectives you want to model the probability that the criminal is at each location. Every once in a while you know the exact position of the criminal, and then you can take into account the following moves he makes to determine which spots he could possibly be at.
Once you have this, I'm not quite sure how to optimize the detectives moves. You can move the detectives to reduce the set of possibilities, effectively corraling the criminal. But I'm sure there is also some higher level strategy needed surrounding the tickets and not running out of them.
I'd imagine some kind of a monte carlo implementation would be an excellent candidate for this, ie. simulating thousands of combinations and choosing the one that ends with the best result most of the time. Since the criminal has to be visible for 5 turns, the branching factor should stay well under control, although MC has also been shown to be a very good technique in games of high branching factor, ie. Go.
In order to get teamwork going between the detectives you need to model them as a team rather than as individuals. Minimax is still a good way to go but (sadly) your branching factor is going to soar.
Instead of stepping through all the detectives making what appears to be the best for each instead for your team of detectives you work out each permutation of moves they could make. If teamwork helps in this game then the minimax will favour the permutations in which the detectives are working together.
I'm not sure if it will be practical, 5 detectives for 24 ply might be too much work but it'd be fun to try and that's the point right?
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
I'm teaching a kid programming, and am introducing some basic artificial intelligence concepts at the moment. To begin with we're going to implement a tic-tac-toe game that searches the entire game tree and as such plays perfectly. Once we finish that I want to apply the same concepts to a game that has too many positions to evaluate every single one, so that we need to implement a heuristic to evaluate intermediate positions.
The best thing I could think of was Dots and Boxes. It has the advantage that I can set the board size arbitrarily large to stop him from searching the entire tree, and I can make a very basic scoring function be the number of my boxes minus the number of opponent boxes. Unfortunately this means that for most of the beginning of the game every position will be evaluated equivalently with a score of 0, because it takes quite a few moves before players actually start making boxes.
Does anyone have any better ideas for games? (Or a better scoring function for dots and boxes)?
Another game choice could be Reversi aka Othello.
A naive heuristic would be to simply count the number of tiles gained by each valid move and choose the greatest. From there you can factor in board position and minimizing vulnerably to the opponent.
One game you may consider is Connect Four. Simple game with straightforward rules but more complicated that Tic-Tac-Toe.
Checkers will let you teach several methods. Simple lookahead, depth search of best-case-worst-case decisions, differences between short-term and long-term gains, and something they could continue to work on after learning what you want to teach them.
Personally I think that last bit is the most critical -- there are natural points in the AI development which are good to stop at, see if you can beat it, and then delve into deeper AI mechanisms. It keeps your student interested without being horribly frustrated, and gives them more to do on their own if they want to continue the project.
How about Reversi? It has a pretty nice space of heuristics based on number of pieces, number of edge pieces, and number of corner pieces.
How about Mancala? Only 6 possible moves each turn, and it's easy to calculate the resulting score for each, but it's important to consider the opponent's response, and the game tree gets big pretty fast.
Gomoku is a nice, simple game, and fun one to write AI for.
Rubik's Infinity's quite fun, it's a little bit like Connect Four but subtly different. Evauluating a position is pretty easy.
I knocked together a Perl script to play it a while back, and actually had to reduce the number of moves ahead it looked, or it beat me every time, usually with quite surprising tactics.
Four in a line Hard enough, but easy enough to come up with an easy working evaluation function, for example, (distance to four from my longest line - distance to four from my opponent's longest line)
I really like Connect Four. Very easy to program using a Minimax algorithm. A good evaluation function could be:
eval_score = 0
for all possible rows/lines/diagonals of length 4 on the board:
if (#player_pieces = 0) // possible to connect four here?
if (#computer_pieces = 4)
eval_score = 10000
break for loop
else
eval_score = eval_score + #computer_pieces
(less pieces to go -> higher score)
end if
else if (#player_pieces = 4)
eval_score = -10000
break for loop
end if
end for
To improve the program you can add:
If computer moves first, play in the middle column (this has been proven to be optimal)
Alpha-Beta Pruning
Move Ordering
Zobrist Hashes
How about starting your Dots and Boxes game with random lines already added. This can get you into the action quickly. Just need to make sure you don't start the game with any boxes.
Take a look at Go.
Simple enough for kid on very small boards.
Complexity scales infinitely.
Has a lot of available papers, algorithms and programs to use either as a scale or basis.
Update: reversi was mentioned, which is a simplified variant of Go. Might be a better choice.
In regards to a better heuristic for dots and boxes, I suggest looking at online strategy guides for the game. The first result on Google for "dots and boxes strategy" is quite helpful.
Knowing how to use the chain rule separates an OK player from a good one. Knowing when the chain rule will work against you is what separates the best players from the good ones.