Table of Contents Preface page xiii 1. Introduction 1 Part I Background 2. Basic Neuroscience 7 2.1 Neurons 7 2.2 Action Potentials or Spikes 8 2.3 Dendrites and Axons 9 2.4 Synapses 9 2.5 Spike Generation 10 2.6 Adapting the Connections: Synaptic Plasticity 11 2.6.1 LTP 11 2.6.2 LTD 11 2.6.3 STDP 11 2.6.4 Short-Term Facilitation and Depression 13 2.7 Brain Organization, Anatomy, and Function 13 2.8 Summary 16 2.9 Questions and Exercises 17 3. Recording and Stimulating the Brain 18 3.1 Recording Signals from the Brain 18 3.1.1 Invasive Techniques 18 3.1.2 Noninvasive Techniques 26 3.2 Stimulating the Brain 32 3.2.1 Invasive Techniques 32 3.2.2 Noninvasive Techniques 33 3.3 Simultaneous Recording and Stimulation 34 3.3.1 Multielectrode Arrays 35 3.3.2 Neurochip 35 3.4 Summary 36 3.5 Questions and Exercises 37 4. Signal Processing 39 4.1 Spike Sorting 39 4.2 Frequency Domain Analysis 40 4.2.1 Fourier Analysis 40 4.2.2 Discrete Fourier Transform (DFT) 43 4.2.3 Fast Fourier Transform (FFT) 45 4.2.4 Spectral Features 45 4.3 Wavelet Analysis 45 4.4 Time Domain Analysis 46 4.4.1 Hjorth Parameters 46 4.4.2 Fractal Dimension 48 4.4.3 Autoregressive (AR) Modeling 49 4.4.4 Bayesian Filtering 49 4.4.5 Kalman Filtering 52 4.4.6 Particle Filtering 54 4.5 Spatial Filtering 54 4.5.1 Bipolar, Laplacian, and Common Average Referencing 55 4.5.2 Principal Component Analysis (PCA) 56 4.5.3 Independent Component Analysis (ICA) 60 4.5.4 Common Spatial Patterns (CSP) 61 4.6 Artifact Reduction Techniques 63 4.6.1 Thresholding 64 4.6.2 Band-Stop and Notch Filtering 65 4.6.3 Linear Modeling 65 4.6.4 Principal Component Analysis (PCA) 66 4.6.5 Independent Component Analysis (ICA) 66 4.7 Summary 68 4.8 Questions and Exercises 68 5. Machine Learning 71 5.1 Classification Techniques 72 5.1.1 Binary Classification 72 5.1.2 Ensemble Classification Techniques 78 5.1.3 Multi-Class Classification 80 5.1.4 Evaluation of Classification Performance 84 5.2 Regression 87 5.2.1 Linear Regression 88 5.2.2 Neural Networks and Backpropagation 89 5.2.3 Radial Basis Function (RBF) Networks 92 5.2.4 Gaussian Processes 93 5.3 Summary 96 5.4 Questions and Exercises 96 Part II Putting It All Together 6. Building a BCI 101 6.1 Major Types of BCIs 101 6.2 Brain Responses Useful for Building BCIs 101 6.2.1 Conditioned Responses 101 6.2.2 Population Activity 102 6.2.3 Imagined Motor and Cognitive Activity 103 6.2.4 Stimulus-Evoked Activity 103 6.3 Summary 104 6.4 Questions and Exercises 105 Part III Major Types of BCIs 7. Invasive BCIs 109 7.1 Two Major Paradigms in Invasive Brain-Computer Interfacing 109 7.1.1 BCIs Based on Operant Conditioning 109 7.1.2 BCIs Based on Population Decoding 111 7.2 Invasive BCIs in Animals 113 7.2.1 BCIs for Prosthetic Arm and Hand Control 113 7.2.2 BCIs for Lower-Limb Control 126 7.2.3 BCIs for Cursor Control 129 7.2.4 Cognitive BCIs 132 7.3 Invasive BCIs in Humans 137 7.3.1 Cursor and Robotic Control Using a Multielectrode Array Implant 138 7.3.2 Cognitive BCIs in Humans 143 7.4 Long-Term Use of Invasive BCIs 143 7.4.1 Long-Term BCI Use and Formation of a Stable Cortical Representation 144 7.4.2 Long-Term Use of a Human BCI Implant 144 7.5 Summary 146 7.6 Questions and Exercises 147 8. Semi-Invasive BCIs 149 8.1 Electrocorticographic (ECoG) BCIs 149 8.1.1 ECoG BCIs in Animals 150 8.1.2 ECoG BCIs in Humans 151 8.2 BCIs Based on Peripheral Nerve Signals 169 8.2.1 Nerve-Based BCIs 170 8.2.2 Targeted Muscle Reinnervation (TMR) 173 8.3 Summary 174 8.4 Questions and Exercises 175 9. Noninvasive BCIs 177 9.1 Electroencephalographic (EEG) BCIs 177 9.1.1 Oscillatory Potentials and ERD 178 9.1.2 Slow Cortical Potentials 187 9.1.3 Movement-Related Potentials 189 9.1.4 Stimulus Evoked Potentials 193 9.1.5 BCIs Based on Cognitive Tasks 199 9.1.6 Error Potentials in BCIs 200 9.1.7 Coadaptive BCIs 201 9.1.8 Hierarchical BCIs 203 9.2 Other Noninvasive BCIs: fMRI, MEG, and fNIR 203 9.2.1 Functional Magnetic Resonance Imaging Based BCIs 204 9.2.2 Magnetoencephalography Based BCIs 205 9.2.3 Functional Near Infrared and Optical BCIs 206 9.3 Summary 206 9.4 Questions and Exercises 207 10. BCIs that Stimulate 210 10.1 Sensory Restoration 210 10.1.1 Restoring Hearing: Cochlear Implants 210 10.1.2 Restoring Sight: Cortical and Retinal Implants 213 10.2 Motor Restoration 216 10.2.1 Deep Brain Stimulation (DBS) 216 10.3 Sensory Augmentation 217 10.4 Summary 219 10.5 Questions and Exercises 219 11. Bidirectional and Recurrent BCIs 221 11.1 Cursor Control with Direct Cortical Instruction via Stimulation 221 11.2 Active Tactile Exploration Using a BCI and Somatosensory Stimulation 224 11.3 Bidirectional BCI Control of a Mini-Robot 226 11.4 Cortical Control of Muscles via Functional Electrical Stimulation 229 11.5 Establishing New Connections between Brain Regions 230 11.6 Summary 234 11.7 Questions and Exercises 234 Part IV Applications and Ethics 12. Applications of BCIs 239 12.1 Medical Applications 239 12.1.1 Sensory Restoration 239 12.1.2 Motor Restoration 240 12.1.3 Cognitive Restoration 240 12.1.4 Rehabilitation 240 12.1.5 Restoring Communication with Menus, Cursors, and Spellers 241 12.1.6 Brain-Controlled Wheelchairs 241 12.2 Nonmedical Applications 242 12.2.1 Web Browsing and Navigating Virtual Worlds 243 12.2.2 Robotic Avatars 245 12.2.3 High Throughput Image Search 248 12.2.4 Lie Detection and Applications in Law 249 12.2.5 Monitoring Alertness 253 12.2.6 Estimating Cognitive Load 256 12.2.7 Education and Learning 258 12.2.8 Security, Identification, and Authentication 260 12.2.9 Physical Amplification with Exoskeletons 261 12.2.10 Mnemonic and Cognitive Amplification 262 12.2.11 Applications in Space 263 12.2.12 Gaming and Entertainment 265 12.2.13 Brain-Controlled Art 267 12.3 Summary 269 12.4 Questions and Exercises 269 13. Ethics of Brain-Computer Interfacing 272 13.1 Medical, Health, and Safety Issues 272 13.1.1 Balancing Risks versus Benefits 272 13.1.2 Informed Consent 273 13.2 Abuse of BCI Technology 273 13.3 BCI Security and Privacy 274 13.4 Legal Issues 275 13.5 Moral and Social-Justice Issues 276 13.6 Summary 277 13.7 Questions and Exercises 277 14. Conclusion 279 Appendix: Mathematical Background 281 A.1 Basic Mathematical Notation and Units of Measurement 281 A.2 Vectors, Matrices, and Linear Algebra 282 A2.1 Matrices 284 A2.2 Eigenvectors and Eigenvalues 287 A2.3 Lines, Planes, and Hyperplanes 288 A.3 Probability Theory 288 A3.1 Random Variables and Axioms of Probability 288 A3.2 Joint and Conditional Probability 289 A3.3 Mean, Variance, and Covariance 290 A3.4 Probability Density Function 291 A3.5 Uniform Distribution 291 A3.6 Bernoulli Distribution 291 A3.7 Binomial Distribution 292 A3.8 Poisson Distribution 292 A3.9 Gaussian Distribution 293 A3.10 Multivariate Gaussian Distribution 293 References 295 Index 307