Hello, it has been some time since I wrote on this... And I have had several set backs, but at last I have created the framework for my poker AI. The program pick random cards for each of the possible card positions, it calculates the best hand between the player and one opponent (This will be the AI). This is just the set up for the AI however.
At the current moment in time, the plan is to create a decision tree which will create a "decision score" and using fuzzy logic find out the correct way to play, whether it be fold, checking or raising (and by how much). This will be kept up-to-date from now and on the current situation of the program and the written dissertation.
Tuesday 22 March 2011
Tuesday 25 January 2011
Research Proposal
Hello! This is my final research proposal that I have sumitted.
Introduction
Poker has been played for years, the origins are disputed, but it has been about for a long time. It has been the past time and profession of many people all over the world. Poker first made it’s way into the mainstream, only in the past few years. Since the invention of the internet, all forms of poker have exploded since any one can play, and for very little.
Poker games have been created since the rise of the video games industry, with it possible to play other players online or even against an artificial intelligence (AI). Since these games can be played for no money they can be used like most other games, just played for entertainment purposes. However the games with AI are simple and just use basic betting strategies and are very repetitive and it becomes very simple to see the patterns in the way that they play and for a decent player, it is easy enough to exploit these weaknesses.
Real poker players change their strategy according to their opponent, rather than just stick to one formula. This is essential for any poker player to make a profit and avoid losing all their money very quickly. Obviously a computer program doesn’t need to worry about how much money they lose, as long as the user is having fun. The goal of this project is to make the computer think more like a human on the poker table and vary its style of play depending on opponent. Which leads us to the question “Can a no-limit Texas Hold’em poker AI learn from other players and change strategy accordingly?”
Motivation
Poker video games, even at an advanced level tend to be easily read and not pose a challenge or any sort of real learning to the game and the thought processes involved, even at the highest difficulties the games are not challenging enough. Simple randomising algorithms tend to be used rather than exact calculations on bluffing strategies and pot-odd calculations that a real professional player would do.
The term artificial intelligence is “the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.” According to the oxford English dictionary. When a program uses non-exact methods of working out what is the correct play in certain situations, this goes against the actual meaning of AI, as a human wouldn’t “randomly” do something just because they want to be beaten out of their money and for the other player to have fun.
The goal for this project is to create an AI agent; capable of playing Texas hold’em poker, and to play well and to be able to pose a challenge or even beat professional players. Obviously everyone is not a professional poker player, so difficulties should be taking into consideration.
Research Topic
The topics that will need to be researched for this project will be
Current methods
Game theory
AI methods
Current methods – The methods currently used in the industry to create challenging AI poker players
Game theory – The theory behind all strategies used in games of this nature, such as pot-odd calculations (working out if the strength of a hand is worth the risk of the money involved.), bluffing calculations (when to bluff, when not to bluff, how often to bluff, how much on each bet per bluff)
AI methods – The way that an AI agent is structured. Such as fuzzy logic, finite state machines.
The current methods will need to be researched as this will provide a background and a starting point for making the AI. Game theory is important for poker, as it provides the knowledge that a professional poker would have to know which action is the best, i.e. calling, betting, raising, checking or folding. The AI methods that could possibly be used will need to be researched and the best methodology selected for this project.
Research Question
The research question for this project is:
“Can a no-limit Texas Hold’em poker AI learn from other players and change strategy accordingly?”
The objectives are as follows:
To produce a working Poker agent.
For this agent to be able to play competently using game theory and betting strategies.
For the agent to be able to change it strategy depending on how it’s opponent plays.
Project Methodology
There are several different methods in which this can be accomplished. Since poker is a game of “Imperfect Information” (some of the cards are concealed), it cannot be treated the same way in which one would treat a chess AI, for example where all the information is available.
All poker decisions are made via a neural network. All the information must be taken into consideration to create a truly realistic poker AI, use the information work out the probabilities and use these to make the correct decision on what play to make.
The input layer would consist of the number of players left in the pot, how much is in the pot, the current bet, the previous bets, the position, the community cards and the hole cards. This is information that a professional poker player would use. The hidden layer would consist of the thought processes that human would do. Such as pot odd calculations, hand probabilities, using such algorithms as Bayes’ theorem, the Nash Equilibrium and the Monte Carlo method, the AI should be able to run complex computations, over and over so that all of the different possibilities can be worked out and each action given a percentage of what should be done. For example, 40% chance that the AI will fold and 60% on the AI to raise a bet. These will be used to work out the output layer, which will be either fold, check, call, bet or raise. Each action will be given an additional weight, depending on how the opponent has been playing up to this point. They will be separated into different categories. The main categories will include: Tight-Aggressive, Tight-Passive, Loose-Aggressive and Loose-Passive. (To clarify, a Tight player is one that only plays the possible hand, a Loose player is one that plays a lot of hands, good or bad, an Aggressive player will bet and raise, rather than check and call, and a Passive player will check and call, rather than bet and raise).
Project Evaluation
The main point that needs to be stated in this section is that the final product will be focused on the AI and the algorithms used. It will most likely have a simple UI. Graphics are not a priority, and will only be considered if time allows for it. The main aim of the construction of the AI is to use proven winning betting strategies to take advantage of opponent’s weaknesses and exploit them.
The first test that will be used on this program to see if can actually play poker and if the program will deal hands and the AI will use reasonable actions. The second test will be if the AI agent can use game theory and betting strategies, to find optimum method for its own hand. The third and final test will be to test if it can change its strategies according to the way that an opponent plays.
It will also need to be tested against several different types of opponents, so the AI will be tested against various people of different skill levels, and it will be able to take advantage of each player and win consistently. The main target for all of the tests will be that will more often than they lose. Even the world’s best poker players do not win on every single hand, as long as the AI agent makes a net profit, then it will be a success.
Issues
The same issues will apply here as they would with any project and these problems need to be dealt with as soon as they arise. Any issue that can be foreseen in advanced would drop the time taken to sort these inconveniences.
Resource Requirements
The project will need an access to academic resources such as books, white papers in the aspects of Artificial Intelligence and Poker strategies, as much as the normal research requirement.
There are no exceptional resources needed for this project.
References
lab Joy Woller (1996) The basics of Monte Carlo Simulations [Online] Available from :
http://www.chem.unl.edu/zeng/joy/mclab/mcintro.html [Accessed on 20/12/2010]
Unknown Author (Date unknown) [Online] Available from : http://pokerai.org/pj2/index.php?option=com_frontpage&Itemid=1 [Accessed on 24/12/2010]
Eliezer S. Yudkowsky (2003) An Intuitive Explantion of Bayes Theorem [Online] Available from: http://yudkowsky.net/rational/bayes [Accessed on : 19/12/2010]
Unknown Author (2010) CPRG Homepage [Online]Available from : http://poker.cs.ualberta.ca/ [Accessed on : 18/11/2010]
Roger A. McCain (Date Unknown) Game theory : An Introductory Sketch [Online] Available From : http://faculty.lebow.drexel.edu/McCainR//top/eco/game/nash.html [Accessed on : 18/11/2010]
Unknown Author (Date Unknown) [Online] Available from : http://www.gametheory.net/ [Accessed on : 18/11/2010]
Introduction
Poker has been played for years, the origins are disputed, but it has been about for a long time. It has been the past time and profession of many people all over the world. Poker first made it’s way into the mainstream, only in the past few years. Since the invention of the internet, all forms of poker have exploded since any one can play, and for very little.
Poker games have been created since the rise of the video games industry, with it possible to play other players online or even against an artificial intelligence (AI). Since these games can be played for no money they can be used like most other games, just played for entertainment purposes. However the games with AI are simple and just use basic betting strategies and are very repetitive and it becomes very simple to see the patterns in the way that they play and for a decent player, it is easy enough to exploit these weaknesses.
Real poker players change their strategy according to their opponent, rather than just stick to one formula. This is essential for any poker player to make a profit and avoid losing all their money very quickly. Obviously a computer program doesn’t need to worry about how much money they lose, as long as the user is having fun. The goal of this project is to make the computer think more like a human on the poker table and vary its style of play depending on opponent. Which leads us to the question “Can a no-limit Texas Hold’em poker AI learn from other players and change strategy accordingly?”
Motivation
Poker video games, even at an advanced level tend to be easily read and not pose a challenge or any sort of real learning to the game and the thought processes involved, even at the highest difficulties the games are not challenging enough. Simple randomising algorithms tend to be used rather than exact calculations on bluffing strategies and pot-odd calculations that a real professional player would do.
The term artificial intelligence is “the theory and development of computer systems able to perform tasks normally requiring human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.” According to the oxford English dictionary. When a program uses non-exact methods of working out what is the correct play in certain situations, this goes against the actual meaning of AI, as a human wouldn’t “randomly” do something just because they want to be beaten out of their money and for the other player to have fun.
The goal for this project is to create an AI agent; capable of playing Texas hold’em poker, and to play well and to be able to pose a challenge or even beat professional players. Obviously everyone is not a professional poker player, so difficulties should be taking into consideration.
Research Topic
The topics that will need to be researched for this project will be
Current methods
Game theory
AI methods
Current methods – The methods currently used in the industry to create challenging AI poker players
Game theory – The theory behind all strategies used in games of this nature, such as pot-odd calculations (working out if the strength of a hand is worth the risk of the money involved.), bluffing calculations (when to bluff, when not to bluff, how often to bluff, how much on each bet per bluff)
AI methods – The way that an AI agent is structured. Such as fuzzy logic, finite state machines.
The current methods will need to be researched as this will provide a background and a starting point for making the AI. Game theory is important for poker, as it provides the knowledge that a professional poker would have to know which action is the best, i.e. calling, betting, raising, checking or folding. The AI methods that could possibly be used will need to be researched and the best methodology selected for this project.
Research Question
The research question for this project is:
“Can a no-limit Texas Hold’em poker AI learn from other players and change strategy accordingly?”
The objectives are as follows:
To produce a working Poker agent.
For this agent to be able to play competently using game theory and betting strategies.
For the agent to be able to change it strategy depending on how it’s opponent plays.
Project Methodology
There are several different methods in which this can be accomplished. Since poker is a game of “Imperfect Information” (some of the cards are concealed), it cannot be treated the same way in which one would treat a chess AI, for example where all the information is available.
All poker decisions are made via a neural network. All the information must be taken into consideration to create a truly realistic poker AI, use the information work out the probabilities and use these to make the correct decision on what play to make.
The input layer would consist of the number of players left in the pot, how much is in the pot, the current bet, the previous bets, the position, the community cards and the hole cards. This is information that a professional poker player would use. The hidden layer would consist of the thought processes that human would do. Such as pot odd calculations, hand probabilities, using such algorithms as Bayes’ theorem, the Nash Equilibrium and the Monte Carlo method, the AI should be able to run complex computations, over and over so that all of the different possibilities can be worked out and each action given a percentage of what should be done. For example, 40% chance that the AI will fold and 60% on the AI to raise a bet. These will be used to work out the output layer, which will be either fold, check, call, bet or raise. Each action will be given an additional weight, depending on how the opponent has been playing up to this point. They will be separated into different categories. The main categories will include: Tight-Aggressive, Tight-Passive, Loose-Aggressive and Loose-Passive. (To clarify, a Tight player is one that only plays the possible hand, a Loose player is one that plays a lot of hands, good or bad, an Aggressive player will bet and raise, rather than check and call, and a Passive player will check and call, rather than bet and raise).
Project Evaluation
The main point that needs to be stated in this section is that the final product will be focused on the AI and the algorithms used. It will most likely have a simple UI. Graphics are not a priority, and will only be considered if time allows for it. The main aim of the construction of the AI is to use proven winning betting strategies to take advantage of opponent’s weaknesses and exploit them.
The first test that will be used on this program to see if can actually play poker and if the program will deal hands and the AI will use reasonable actions. The second test will be if the AI agent can use game theory and betting strategies, to find optimum method for its own hand. The third and final test will be to test if it can change its strategies according to the way that an opponent plays.
It will also need to be tested against several different types of opponents, so the AI will be tested against various people of different skill levels, and it will be able to take advantage of each player and win consistently. The main target for all of the tests will be that will more often than they lose. Even the world’s best poker players do not win on every single hand, as long as the AI agent makes a net profit, then it will be a success.
Issues
The same issues will apply here as they would with any project and these problems need to be dealt with as soon as they arise. Any issue that can be foreseen in advanced would drop the time taken to sort these inconveniences.
Resource Requirements
The project will need an access to academic resources such as books, white papers in the aspects of Artificial Intelligence and Poker strategies, as much as the normal research requirement.
There are no exceptional resources needed for this project.
References
lab Joy Woller (1996) The basics of Monte Carlo Simulations [Online] Available from :
http://www.chem.unl.edu/zeng/joy/mclab/mcintro.html [Accessed on 20/12/2010]
Unknown Author (Date unknown) [Online] Available from : http://pokerai.org/pj2/index.php?option=com_frontpage&Itemid=1 [Accessed on 24/12/2010]
Eliezer S. Yudkowsky (2003) An Intuitive Explantion of Bayes Theorem [Online] Available from: http://yudkowsky.net/rational/bayes [Accessed on : 19/12/2010]
Unknown Author (2010) CPRG Homepage [Online]Available from : http://poker.cs.ualberta.ca/ [Accessed on : 18/11/2010]
Roger A. McCain (Date Unknown) Game theory : An Introductory Sketch [Online] Available From : http://faculty.lebow.drexel.edu/McCainR//top/eco/game/nash.html [Accessed on : 18/11/2010]
Unknown Author (Date Unknown) [Online] Available from : http://www.gametheory.net/ [Accessed on : 18/11/2010]
Subscribe to:
Posts (Atom)