: A Guide For Machine Teaching Anderson K. creator SPD 2022 eng Eng lis h 204 Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Introduction: The Right Brain in the Right Place (Why We Need Autonomous AI). . . . . . . xxv Part I. When Automation Doesn’t Work 1. Sometimes Machines Make Bad Decisions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Math, Menus, and Manuals: How Machines Make Automated Decisions 5 Control Theory Uses Math to Calculate Decisions 5 Optimization Algorithms Use Menus of Options to Evaluate Decisions 9 Expert Systems Recall Stored Expertise 21 2. The Quest for More Human-Like Decision-Making. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Augmenting Human Intelligence 28 How Humans Make Decisions and Acquire Skills 29 Humans Act on What They Perceive 30 Humans Build Complex Correlations into Their Intuition with Practice 31 Humans Abstract to Strategy for Complex Tasks 31 There’s a New Kind of AI in Town 36 The Superpowers of Autonomous AI 40 Autonomous AI Makes More Human-Like Decisions 41 Autonomous AI Perceives, Then Acts 41 The Difference Between Perception and Action in AI 42 Autonomous AI Learns and Adapts When Things Change 43 Autonomous AI Can Spot Patterns 43 vii Autonomous AI Infers from Experience 44 Autonomous AI Improvises and Strategizes 44 Autonomous AI Can Plan for the Long-Term Future 45 Autonomous AI Brings Together the Best of All Decision-Making Technologies 46 When Should You Use Autonomous AI? 46 Autonomous AI Is like a Brilliant, Curious Toddler That Needs to Be Taught 47 Part II. What Is Machine Teaching? 3. How Brains Learn Best: Teaching Humans and AI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Learning Multiple Skills Simultaneously Is Hard for Humans and AI 53 Teaching Skills and Strategies Explicitly 54 Teaching Allows Us to Trust AI 58 The Mindset of a Machine Teacher 60 Teacher More Than Programmer 60 Learner More Than Expert 62 What Is a Brain Design? 62 How Decision-Making Works 63 Acquiring Skill Is like Learning to Navigate by Exploring 68 A Brain Design Is a Mental Map That Guides Exploration with Landmarks 69 4. Building Blocks for Machine Teaching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Case Study: Learning to Walk Is Hard to Evolve, Easier to Teach 76 So, Why Do We Walk? 77 Strategy Versus Evolution 78 Teaching Walking as Three Skills 81 Concepts Capture Knowledge 84 Skills Are Specialized Concepts 84 Brains Are Built from Skills 86 Building Skills 86 Expert Rules Inflate into Skills 87 Perceptive Concepts Discern or Recognize 91 Directive Concepts Decide and Act 96 Selective Concepts Supervise and Assign 97 Brains Are Organized by Functions and Strategies 99 Sequences or Parallel Execution for Functional Skills 100 Hierarchies for Strategies 108 Visual Language of Brain Design 113 viii | Table of Contents Part III. How Do You Teach a Machine? Understanding the Process 117 Meet with Experts 118 Ask the Right Questions 118 Case Study: Let’s Design a Smart Thermostat 119 5. Teaching Your AI Brain What to Do. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Determining Which Actions the Brain Will Take 122 Perception Is Required, but It’s Not All We Need 122 Sequential Decisions 123 Triggering the Action in Your AI Brain 124 Setting the Decision Frequency 125 Handling Delayed Consequences for Brain Actions 125 Actions for Smart Thermostat 127 6. Setting Goals for Your AI Brain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 There’s Always a Trade-off 129 Throughput Versus Efficiency 131 Supervisors Have Different Goals Than Crews Do 132 Don’t Prioritize Goals; Balance Them Instead 133 Watch Out for Expert Rules Disguised as Goals 133 Ideal Versus Available 134 Setting Goals 135 Step 1: Identify Scenarios 135 Step 2: Match Goals to Scenarios 136 Step 3: Teach Strategies for Each Scenario 137 Goal Objectives 137 Maximize 137 Minimize 137 Reach, like the Finish Line for a Race 137 Drive, like the Temperature for a Thermostat 138 Avoid, like Dangerous Conditions 138 Standardize, like the Heat in an Oven 139 Smooth, like a Line 139 Expanding Task Algebra to Include Goal Objectives 140 Setting Goals for a Smart Thermostat 141 7. Teaching Skills to Your AI Brain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Teaching Focuses and Guides Practice (Exploration) 144 Skills Can Evolve and Transform 147 Skills Adapt to the Scenario 148 Levels of Teaching Sophistication 148 Table of Contents | ix The Introductory Teacher Conveys the Facts and Goals 149 The Coach Sequences Skills to Practice 149 The Mentor Teaches Strategy 150 The Maestro Democratizes New Paradigms 151 How Maestros Democratize Technology 153 Levels of Autonomous AI Architecture 154 Machine Learning Adds Perception 155 Monolithic Brains Are Advanced Beginners 156 Concept Networks Are Competent Learners 157 Massive Concept Networks Are Proficient Learners 159 Pursuing Expert Skill Acquisition in Autonomous AI 160 Brains That Come with Hardwired Skills 161 Brains That Define Skills as They Learn 162 Brains That Assemble Themselves 164 Brains with Skills That Coordinate 165 Steps to Architect an AI Brain 166 Step 1: Identify the Skills That You Want to Teach 166 Step 2: Orchestrate How the Skills Work Together 168 Step 3: Select Which Technology Should Perform Each Skill 168 Pitfalls to Avoid When Teaching Skills 168 Pitfall 1: Confusing the solution for the problem 169 Pitfall 2: Losing the forest for the trees 169 Example of Teaching Skills to an AI Brain: Rubber Factory 169 Brain Design for Our Smart Thermostat 171 8. Giving Your AI Brain the Information It Needs to Learn and Decide. . . . . . . . . . . . . . 173 Sensors: The Five Senses for Your AI Brain 174 Variables 174 Proxy Variables 175 Trends 176 Simulators: A Gym for Your Autonomous AI to Practice In 176 Simulating Reality Using Physics and Chemistry 179 Simulating Reality Using Statistics and Events 179 Simulating Reality Using Machine Learning 179 Simulating Reality Using Expert Rules 180 Sensor Variables for Smart Thermostat 180 Part IV. Tools for the Machine Teacher 9. Designing AI Brains That Someone Can Actually Build. . . . . . . . . . . . . . . . . . . . . . . . . 185 Designers and Builders Working Together in Harmony (Mostly) 185 x | Table of Contents The Autonomous AI Design Fallacy Designs but Won’t Iterate 187 The Autonomous AI Implementation Fallacy Skips Design Altogether 188 Specification for Documenting AI Brain Designs 188 Platform for Machine Teaching 190 Platform for Wiring Multiple Skills Together as Modules 190 What Difference Will You Make with Machine Teaching? 191 Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Table of Contents | xi 9789355422866 241018 20250509114327.0