Table of Contents
Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Introduction: The Right Brain in the Right Place (Why We Need Autonomous AI). . . . . . . xxv
Part I. When Automation Doesn’t Work
1. Sometimes Machines Make Bad Decisions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Math, Menus, and Manuals: How Machines Make Automated Decisions 5
Control Theory Uses Math to Calculate Decisions 5
Optimization Algorithms Use Menus of Options to Evaluate Decisions 9
Expert Systems Recall Stored Expertise 21
2. The Quest for More Human-Like Decision-Making. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Augmenting Human Intelligence 28
How Humans Make Decisions and Acquire Skills 29
Humans Act on What They Perceive 30
Humans Build Complex Correlations into Their Intuition with Practice 31
Humans Abstract to Strategy for Complex Tasks 31
There’s a New Kind of AI in Town 36
The Superpowers of Autonomous AI 40
Autonomous AI Makes More Human-Like Decisions 41
Autonomous AI Perceives, Then Acts 41
The Difference Between Perception and Action in AI 42
Autonomous AI Learns and Adapts When Things Change 43
Autonomous AI Can Spot Patterns 43
vii
Autonomous AI Infers from Experience 44
Autonomous AI Improvises and Strategizes 44
Autonomous AI Can Plan for the Long-Term Future 45
Autonomous AI Brings Together the Best of All Decision-Making
Technologies 46
When Should You Use Autonomous AI? 46
Autonomous AI Is like a Brilliant, Curious Toddler That Needs to Be Taught 47
Part II. What Is Machine Teaching?
3. How Brains Learn Best: Teaching Humans and AI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Learning Multiple Skills Simultaneously Is Hard for Humans and AI 53
Teaching Skills and Strategies Explicitly 54
Teaching Allows Us to Trust AI 58
The Mindset of a Machine Teacher 60
Teacher More Than Programmer 60
Learner More Than Expert 62
What Is a Brain Design? 62
How Decision-Making Works 63
Acquiring Skill Is like Learning to Navigate by Exploring 68
A Brain Design Is a Mental Map That Guides Exploration with Landmarks 69
4. Building Blocks for Machine Teaching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Case Study: Learning to Walk Is Hard to Evolve, Easier to Teach 76
So, Why Do We Walk? 77
Strategy Versus Evolution 78
Teaching Walking as Three Skills 81
Concepts Capture Knowledge 84
Skills Are Specialized Concepts 84
Brains Are Built from Skills 86
Building Skills 86
Expert Rules Inflate into Skills 87
Perceptive Concepts Discern or Recognize 91
Directive Concepts Decide and Act 96
Selective Concepts Supervise and Assign 97
Brains Are Organized by Functions and Strategies 99
Sequences or Parallel Execution for Functional Skills 100
Hierarchies for Strategies 108
Visual Language of Brain Design 113
viii | Table of Contents
Part III. How Do You Teach a Machine?
Understanding the Process 117
Meet with Experts 118
Ask the Right Questions 118
Case Study: Let’s Design a Smart Thermostat 119
5. Teaching Your AI Brain What to Do. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
Determining Which Actions the Brain Will Take 122
Perception Is Required, but It’s Not All We Need 122
Sequential Decisions 123
Triggering the Action in Your AI Brain 124
Setting the Decision Frequency 125
Handling Delayed Consequences for Brain Actions 125
Actions for Smart Thermostat 127
6. Setting Goals for Your AI Brain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
There’s Always a Trade-off 129
Throughput Versus Efficiency 131
Supervisors Have Different Goals Than Crews Do 132
Don’t Prioritize Goals; Balance Them Instead 133
Watch Out for Expert Rules Disguised as Goals 133
Ideal Versus Available 134
Setting Goals 135
Step 1: Identify Scenarios 135
Step 2: Match Goals to Scenarios 136
Step 3: Teach Strategies for Each Scenario 137
Goal Objectives 137
Maximize 137
Minimize 137
Reach, like the Finish Line for a Race 137
Drive, like the Temperature for a Thermostat 138
Avoid, like Dangerous Conditions 138
Standardize, like the Heat in an Oven 139
Smooth, like a Line 139
Expanding Task Algebra to Include Goal Objectives 140
Setting Goals for a Smart Thermostat 141
7. Teaching Skills to Your AI Brain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Teaching Focuses and Guides Practice (Exploration) 144
Skills Can Evolve and Transform 147
Skills Adapt to the Scenario 148
Levels of Teaching Sophistication 148
Table of Contents | ix
The Introductory Teacher Conveys the Facts and Goals 149
The Coach Sequences Skills to Practice 149
The Mentor Teaches Strategy 150
The Maestro Democratizes New Paradigms 151
How Maestros Democratize Technology 153
Levels of Autonomous AI Architecture 154
Machine Learning Adds Perception 155
Monolithic Brains Are Advanced Beginners 156
Concept Networks Are Competent Learners 157
Massive Concept Networks Are Proficient Learners 159
Pursuing Expert Skill Acquisition in Autonomous AI 160
Brains That Come with Hardwired Skills 161
Brains That Define Skills as They Learn 162
Brains That Assemble Themselves 164
Brains with Skills That Coordinate 165
Steps to Architect an AI Brain 166
Step 1: Identify the Skills That You Want to Teach 166
Step 2: Orchestrate How the Skills Work Together 168
Step 3: Select Which Technology Should Perform Each Skill 168
Pitfalls to Avoid When Teaching Skills 168
Pitfall 1: Confusing the solution for the problem 169
Pitfall 2: Losing the forest for the trees 169
Example of Teaching Skills to an AI Brain: Rubber Factory 169
Brain Design for Our Smart Thermostat 171
8. Giving Your AI Brain the Information It Needs to Learn and Decide. . . . . . . . . . . . . . 173
Sensors: The Five Senses for Your AI Brain 174
Variables 174
Proxy Variables 175
Trends 176
Simulators: A Gym for Your Autonomous AI to Practice In 176
Simulating Reality Using Physics and Chemistry 179
Simulating Reality Using Statistics and Events 179
Simulating Reality Using Machine Learning 179
Simulating Reality Using Expert Rules 180
Sensor Variables for Smart Thermostat 180
Part IV. Tools for the Machine Teacher
9. Designing AI Brains That Someone Can Actually Build. . . . . . . . . . . . . . . . . . . . . . . . . 185
Designers and Builders Working Together in Harmony (Mostly) 185
x | Table of Contents
The Autonomous AI Design Fallacy Designs but Won’t Iterate 187
The Autonomous AI Implementation Fallacy Skips Design Altogether 188
Specification for Documenting AI Brain Designs 188
Platform for Machine Teaching 190
Platform for Wiring Multiple Skills Together as Modules 190
What Difference Will You Make with Machine Teaching? 191
Glossary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Table of Contents | xi