GAAD 09 - AI Techniques for Games
Quote
"Much learnings does not teach insight"
What are Rule Based Systems (RBS)`?
- Behavior choice on the basis of several if-then rules - Suitable for low-level control & decision planning - Easy to implement & extend - Extendable by fuzzy logic - Disadvantages: implicit knowledge, no backtracking, complex rules, hard to debug
Breadth first & Depth first
- Breadth first search: Finds optimal solution (=minimal depth) - Depth fist search can result in non-optimal solutions - Breadth first search is complete Depth fist search is not complete if the search tree is (infinitely) big -> Has a comlexity
Discussion of Local Heuristics
- Constant (Storage) or linear (run time) complexity - Easy to implement - Incomplete (local minima) - Visual unfavorable results - Don't minimize the total costs
Hill Climbing Strategy
- Determine all successors of the current state - Assess the costs of al successors - Choose the successor with the lowest cost - Cancel if the current state has lower cost than all succesors Problem: Incomplete, goal can possibly not be reached
Probabilistic FSM More
- Different actions are possible - expanded scpe of action (several transitions for the same event) - Transitions are marked with probabilisties - Random choise at run time
Probabilistic Finite State Machines
- FSM expose easy predicable behavior - Non-deterministic finite state machines - Probability distribution in transfer function d - Markov-Models: Considered context (earlier transitions) Application: - Consideration of the personality - Specifiy Behavior (Group, Culture, Avatar Biography)
Example of Probabilistic FSM
- Finite State Machine: -> IF enemy sighted THEN go into attack state - Probabilistic finite state machine - IF enemy sighted THEN add 10 to attack state probability
Normal Division (Grids, Combs)
- Grid Size: smallest agent - immense memory demands - bumpind movement - Improvement needed -> need to adopt movement to that
Examples?
- Mostly C# or JS, also Lua, Python in Engines like Unity - Also: Unreal: Visual Scripting Language Blueprint
Navigation Meshes
- Polygonal mesh, describes walkable areas - Integrated approach for path planning & collision avoidance - Avoids collision with static obstacles: -> Common implementations rely on connected convex polygons -> Extends the radius of the obstacles by the agent's size to avoid wall hugging => Not good for dynamic encounters -> Fines grids -> better subdivision
Potential Fields
- Smooth movments e.g. for following camera - Difficult to implement for a realistic movement - Poor performance in narrow passages & dynamic environments - Prone to get stuck in local minima situations - Problems in dealing with the symmetrical obstacles
Game Tree for Round Based 2 Person Games
- Start Configuration (Blue) - Decision function to assert the final configurations - Finite amount of allowed action/game turns: -> Transition: Configuration -> Configuration Blue: Start Configuration Black: Negative Assessment (Mistake/Game over) Green: Perfect soluation (Game won)
How are scripts used?
- Total control over behavior model - Ideal for the implementation of short-term goals - triggered by events - complex behavior possible - usually fast foreseeable - deterministic - support many engine & game modding editors
What are informed/heuristic search strategies?
- Use context information to prefer the most promising candidate in the search space - Cost function: know effort & estimated cost for the completion of the hypothesis - Local heuristics (passing a local obstacle) - Global heuristics (go in that direction, it is over there) Cost function: e.g. number of moves, cost of a movement, ...?
Why & How Should one use AI in Games?
- possibility (computing time) <- is that available? - Necessity (unique feature) - AI-Developer (2/10) over the total projekt time - Requirement for AI-Engines: - Real-Time capability: Efficient algorithms & highly optimized code - Anytime algorithms - Incremental algorithms - Modular architecture & synchronization in game-engine - Authoring tools - integration in modelling tools & enginges (Maya, 3ds Max, Blender) - Cross platform SDKs (PC, XBox, PS, Mobile)
Finite State Machines
-> Can be described as a tuple -> Input alphabet ETA - Events, Actions, Time - Internal States/Attributes -> Sates Q (?) - Actions, Goals, Requirements, Motivations -> Markes States: - Starting state q0 - Final state qe - Current state -> Transfer function -> Actions or animtions: connected with states and/or transitions
Hierarchical Finite State Machines
-> Complex behavior: Seperate finite state machines -> Independent behavior models: 1. Reduction of complexity 2. better modelling & mentainance 3. multiple use Note: Is not a complete FSM ! - All links between top level states still exist - need more states for wandering
Kasparov vs. Deep Blue & Go
-> Deep Blue never understood Chess -> Defeated Kasparov with Bruteforce Go: - Deep Learning -> Changed a lot how AI works -> Now a lot of AI models for computers
Decision Tree
-> Evaluated from root to leafs -> Child need to represent all possible decision outcomes -> For closure Yes/No -> 2 Children Yes/No/Maybe 3 Children
What optimal playing abilities should an NPC have?
-> It is always easy to create an NPC which is better than the player But: Frustration for player But NPS should not be too dumb Goal of the game: - Entertainment, no frustration -> Enemies not too intelligent -> NPCs not too dumb -> Mistakes increase plasubility -> Fairness expected Paradox of game AI: "Every game-AI should eventually loose...but pretty close"
Random Avoidance
-> Random turn to get away from obstacle & reaching goal
Comparison of the Search Strategies
-> Uninfromed search is always more expensive than heuristical search
What are the requirements for AI in Games?
1) -/+ Accessible: - Environment can be perceived completely - not all states of the environment are known -> Important for balance of the game 2) Static/Dynamic - Environment is changed reclusively by simulated actions - additional unknown determining factors 3) +/- Deterministic - Action's effects are unknown - Action's effects are represented by probability distribution 4) Discreet/Continous: - Finite amount of states - States are represented by continous paramenters
AI Engines, Which kinds of Skill do they have? + Graphic
1) Basic Skill -> Path Planning 2) Agents' control: - Decision & learning techniques - Animation controll 3) Team control: - coordinated group control
Content
1. Artificial Intelligence in Digital Games 2. Behavior Control - 2.1 Scripts - 2.2 Finite State Machines - 2.3 Rule-Based Fuzzy Systesm - 2.4 Decision Trees 3. Searching & Planning Methods - 3.1 Search Strategies - 3.2 Uninformed Search Strategies - 3.3 Informed & Heuristic Search Strategies - 3.4 Further Movement Strategies 4. References
What are the assessment criteria for search strategies?
1. Completeness -> Will all available solutions be found 2. Halting problem -> will the algorithm stop if no solution is found (make a condition to make it stop?) 3. Efficiency -> will the solution with the lowest cost be found 4. Complexity -> Run time & disk space demand
Local heuristics
1. Orientation towards a goal 2. Avoid or bypass obstables
What are the 11 ways to loose glamoroulsy?
1. Realistic perception for NPC (visibility constraints) 2. Don't attack the player immediately 3. Don't hit the player with the first try 4. Warn the player 5. Respond to player's mistakes 6. Attack with "Kung-Fu" style 7. Make the NPC intentionally vulnarable and exposed 8. NPC follow non-optimal strategies 9. High additional cost for NPC 10. Signal intentions of NPC 11. Only move back in the last minute
Which two kinds of search algorithms exist?
1. Uninformed search strategies 2. Informed/heuristic search strategies
Which FSM Expansions exists?
4 Types: 1. Probabilistic finite state machines 2. Pushdown automation 3. Hierarchical finite state machines 4. Parallel finite state machines
Turing Test in Game Environments
=> Differentiate between humans & autonomous agents Criteria: - Rational, goal oriented behavior - Adequate reaction (especially to new situations) - adaptation to the player's behavior - Unique individual behavior (Personality & emotions, mistakes) Team: - Task sharing - Specialization - Coordination => Possible to construct a good agent if you have a limited game
Representation of the World
=> Game world structured by: - Grid (uninformed/adaptive) - Navigation meshes - Potential fields => Path construction influenced by - Performance - Memory consumption - Visual qualitiy of the paths
A* Alg Problems
=> Reduce Complexity = Size of the search space - Maximize cell size - Use of adaptive spatial data structures - Path planning for groups of agents - hierarchical path planning => Memory allocation => Real-Time capable planning - Path planning with priorities - Time-Sharing: Maximal number of iterations for each planning cycle =>Immediate feedback: - Determine start segment without complete planning - Combine start segment & path planning with A*
3.4 Global heuristics
A* Algorithm -> Widely used in path finding -> one of the most optimal path finding algortihms -> Uses Cost Function -> But under certain conditions outperformed Idea: - Analyze the path, which will lead you most likely to the goal based on estimations
1.
AI in Digital Games
FSM (Finite state Machines) Pros & Cons
Advantages: - Intuitive - Expressive - Efficient - Easy to implement - Easy to debug - Visual Editors Disadvantages: - Not adaptive - Limited repertoire of behaviors - Foreseable behavor - Number of states and transtions grows extremly fast (exponential) - Comparing propositions is difficult "take the better armor", "attack only at the third time"
Machine Learning in Digital Games *
Application in NPC Controll: 1. Challangeing enemies 2. Cooperative NPC 3. Believable world Challange: - Actions in unknow situations - Tactics which fits the situation - Adaption to the player's tactic Model building based on: - Experience and/or example data => Using AI an be a more entertaining experience
Chapter 2
Behavior Control
Techniques for agent control -> Which Behavior controls exist?
Behavior control: - Scripts - Finite state machines - Fuzzy-Logic - Decision trees - Planning
AI as a Search Problem for Building Opponents What are common search examples?
Common search examples: - Optimal turn in a round based 2 person game - Adequate reaction of autonomous characters - Optimal path in a complex surrounding -...
Path Planning for NPCs What are the cost factors? What are the influence factors?
Cost factors: - Distance - use of resources (Time, Fuel) - Tactical aspects (Covering, height advantage, movement advantage, surprise) Influence factors: - The agent's size - Possible ways of shifting (running, jumping, climing, flying) - Collision avoidance with static & dynamic obstacles - Discover world (Fog-Of-War) - Dynamic aspects (Status of doors & elevators, amount of damage)
2.4
Decision Trees: - Decision Support Tool using Trees
Decision Trees Explanation
Decition Trees: - Classification Alg - Traverse from the root to a terminating leaf - Easy & compact representation -> High coordination cost if manually consturcted -> Can be learned from examples (expandable) Nodes -> represent the attributes One Child for every possible outcome Finally: outcome is in the final leaf -> To a certain degree: classification alg. -> Sometimes complicated DT can be extended by new leaves Close Relative: Behavior Tree: -> But Traverses from left to right & far more complex things can be described
Goals of Game AI ?
Demand: - Goal oriented approchan -> AI decides what to do, how to do it & realizes it - Reactive -> the AI immediately reacts to changes in the environment & adapts knowledge - Knowledge -> The AI knows about its environment & its own behavior - Characteristic -> AI represents believable, consistent characters - Fast & easy development - Low demands of system resources -> All goals can't be fulfilled simulatneously (especially in real time)
Extension of Depth-First Search
Depth first search with an interative extension: - Search until depth i is reached - if in depth i no solution can be found, increase i Backtracking: - Only follow the current path - Save the remaining alternatives in alle nodes - Failure: Reset to last open alternative Search Direction: - Top Down - Bottom Up - Bidirectional search
Throwback to Techniques for Agent Contoll
E.g. Learning Methods
Agent Model -> Parameters which we need
Environment: Game World Perception: - Sensors/Receptors - Intenal states Selective behavior: - Action selection (goals, strategy) Expression of the behavior: - Action execution (action) - Path planning (steering) - Realization (locomotion) Actuators: - Changing the environemnt
Scripting Simple AI in Unity
Example for Unity
2.2
Finite State Machines
Rule Based Fuzzy Systems
Fuzzy != chaotic -> Using Fuzzy Logic: "FL Systems produce acceptable but definite output in response to defined ?? distorted or inaccurate & incomplete data In the picture: Example of membership functions In Fuzzy Systems: Representation of degrees T0-T6 At some point: can be nominal & warm or warm & hot -> with some expression in number => Represents ambigous system Works like that: For one ot the thngs you have the highest number: e.g. 0.4 -> retreat => Conclusion -> Decision -> Next Action: retreat Sometimes Fuzzy Logic is the solution to complex problems But sometimes: not suitable for high accuracy -> Because: Ambigous terms & Only understandable when set of rules is not complex
AI in Digital Games What is the goal?
Graphics already very advanced -> But better gameplay through AI is still a challange Goal: Better gameplay: - Challenging enemies - interesting & diversified story - Long term entertainment - new game concepts Goal-oriented realistic behavior: - Robustness in new situations - acquisition of novel behaviors - adapt to the player's strategy - personality & emotion - social behavior
What is a Quake Bot?
Has several states & move through those when events happen -> State: - Attack enemy (attack) - Discover world (wander) - Chase enemy (chase) - Spawn in the world (spawn) -> Events: - E: Enemy contact (enemy) - S: Sound perceived (sound) - D: Die (die)
Chapter 3.3
Informed/Heuristic Search Strategies
Techniques for Agent Control -> Learning methods
Learning methods: - NN - bayesian networks - genetic algorithms - decision trees - reinforcement learning - supervised learning - (Deep learning)
Path Planning as Graph Search
Looking at 3D Represenation -> need to transform in 2D Layout -> Between this nodes player can move Nodes: Space segmentation of the game world Simplification 3D -> 2D Layout - Uniform or variable grids: Size of the smallest agent - Corner graph, path points - Navigation meshes Edge: Reachability
ML
Make AI think intelligently, in the same way humans thinks Breaks down "world" & than searches a certain state -> a weight to the state or position in an unknown field of data => Searching is the universal technique of problem sovling in AI Tiktakto -> Search Tree Path finding problems: NPC search Decision Tree: looking for optimal state as an answer to a questin what would be the next action ! Problem of Searching & Representation Learning through exp: -> Wide usage, but complex & not easy to understand & apply Building Problemspace -> finding optimal state
Using Game Tree
Node: Reference to configurations of greater nodes Edges: Operator, that generates nodes; Costs Depth: - i possible game turns of player A - i + 1 possible game turns of player B Leafs: Assessment (Vicotry, Defeat, Draw)
What do autonomous agents do in digital games?
Non-player characters (NPC): - Enemies - Team colleagues - Uninvolved creatures Additional as: - Reporter - Camera controll - Author/Director -> "Hidden agents"
Overview
Play strategies & path planning as a search problem - Uninformed search strategies - Informed & heuristic search strategies Path Planning as an optimization problem Tremendous challenge -> References
Using the Min Max Alg. for that?
Requirements: - Comlete Game Tree Player: - Maximize benefit Enemies: Minimize the enemies' chances - Assess nodes - Propagate assessment
2.3
Rule Based Fuzzy Systems
2.1
Scripts
Chapter 3.1
Search Strategies
Chapter 3
Searching & Planning Methods
Contour Tracing
Trace the countour
Pushdown Automation
Typical FSM for guardian tasks: - Idle, Patrol, Search, Attack Storage of earlier states: - memory, return to earlier behavior - State stack Transitions: 1. Push 2. Pop 3. Replace
Chapter 3.2
Uninformed Search Strategies
Chapter 3.5 Further Movement Strategies
new Chapter