Philipp O. J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3© Springer International Publishing Switzerland 2013

Graduate Texts in Physics

Series EditorsRichard Needs, William T. Rhodes, Susan Scott, H. Eugene Stanley and Martin Stutzmann

For further volumes: www.​springer.​com/​series/​8431

Graduate Texts in Physics publishes core learning/teaching material for graduate- and advanced-level undergraduate courses on topics of current and emerging fields within physics, both pure and applied. These textbooks serve students at the MS- or PhD-level and their instructors as comprehensive sources of principles, definitions, derivations, experiments and applications (as relevant) for their mastery and teaching, respectively. International in scope and relevance, the textbooks correspond to course syllabi sufficiently to serve as required reading. Their didactic style, comprehensiveness and coverage of fundamental material also make them suitable as introductions or references for scientists entering, or requiring timely knowledge of, a research field.

Philipp O. J. Scherer

Computational PhysicsSimulation of Classical and Quantum Systems

Additional material to this book can be downloaded from http://extras.springer.com.

Philipp O. J. Scherer

Physikdepartment T38, Technische Universität München, Garching, Germany

ISSN 1868-4513e-ISSN 1868-4521

ISBN 978-3-319-00400-6e-ISBN 978-3-319-00401-3

Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number: 2013944508

© Springer International Publishing Switzerland 2010, 2013

2nd edition. First Edition was published under the same title in 2010.

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

To Christine

Preface to the Second Edition

This textbook introduces the main principles of computational physics, which include numerical methods and their application to the simulation of physical systems. The first edition was based on a one-year course in computational physics where I presented a selection of only the most important methods and applications. Approximately one-third of this edition is new. I tried to give a larger overview of the numerical methods, traditional ones as well as more recent developments. In many cases it is not possible to pin down the “best” algorithm, since this may depend on subtle features of a certain application, the general opinion changes from time to time with new methods appearing and computer architectures evolving, and each author is convinced that his method is the best one. Therefore I concentrated on a discussion of the prevalent methods and a comparison for selected examples. For a comprehensive description I would like to refer the reader to specialized textbooks like “Numerical Recipes” or elementary books in the field of the engineering sciences.

The major changes are as follows.

A new chapter is dedicated to the discretization of differential equations and the general treatment of boundary value problems. While finite differences are a natural way to discretize differential operators, finite volume methods are more flexible if material properties like the dielectric constant are discontinuous. Both can be seen as special cases of the finite element methods which are omnipresent in the engineering sciences. The method of weighted residuals is a very general way to find the “best” approximation to the solution within a limited space of trial functions. It is relevant for finite element and finite volume methods but also for spectral methods which use global trial functions like polynomials or Fourier series.

Traditionally, polynomials and splines are very often used for interpolation. I included a section on rational interpolation which is useful to interpolate functions with poles but can also be an alternative to spline interpolation due to the recent development of barycentric rational interpolants without poles.

The chapter on numerical integration now discusses Clenshaw-Curtis and Gaussian methods in much more detail, which are important for practical applications due to their high accuracy.

Besides the elementary root finding methods like bisection and Newton-Raphson, also the combined methods by Dekker and Brent and a recent extension by Chandrupatla are discussed in detail. These methods are recommended in most text books. Function minimization is now discussed also with derivative free methods, including Brent’s golden section search method. Quasi-Newton methods for root finding and function minimizing are thoroughly explained.

Eigenvalue problems are ubiquitous in physics. The QL-method, which is very popular for not too large matrices is included as well as analytic expressions for several differentiation matrices.

The discussion of the singular value decomposition was extended and its application to low rank matrix approximation and linear fitting is discussed.

For the integration of equations of motion (i.e. of initial value problems) many methods are available, often specialized for certain applications. For completeness, I included the predictor-corrector methods by Nordsieck and Gear which have been often used for molecular dynamics and the backward differentiation methods for stiff problems.

A new chapter is devoted to molecular mechanics, since this is a very important branch of current computational physics. Typical force field terms are discussed as well as the calculation of gradients which are necessary for molecular dynamics simulations.

The simulation of waves now includes three additional two-variable methods which are often used in the literature and are based on generally applicable schemes (leapfrog, Lax-Wendroff, Crank-Nicolson).

The chapter on simple quantum systems was rewritten. Wave packet simulation has become very important in theoretical physics and theoretical chemistry. Several methods are compared for spatial discretization and time integration of the one-dimensional Schrödinger equation. The dissipative two-level system is used to discuss elementary operations on a qubit.

The book is accompanied by many computer experiments. For those readers who are unable to try them out, the essential results are shown by numerous figures.

This book is intended to give the reader a good overview over the fundamental numerical methods and their application to a wide range of physical phenomena. Each chapter now starts with a small abstract, sometimes followed by necessary physical background information. Many references, original work as well as specialized text books, are helpful for more deepened studies.

Philipp O. J. Scherer

Garching, Germany

February 2013

Preface to the First Edition

Computers have become an integral part of modern physics. They help to acquire, store and process enormous amounts of experimental data. Algebra programs have become very powerful and give the physician the knowledge of many mathematicians at hand. Traditionally physics has been divided into experimental physics which observes phenomena occurring in the real world and theoretical physics which uses mathematical methods and simplified models to explain the experimental findings and to make predictions for future experiments. But there is also a new part of physics which has an ever growing importance. Computational physics combines the methods of the experimentalist and the theoretician. Computer simulation of physical systems helps to develop models and to investigate their properties.

This book is a compilation of the contents of a two-part course on computational physics which I have given at the TUM (Technische Universität München) for several years on a regular basis. It attempts to give the undergraduate physics students a profound background in numerical methods and in computer simulation methods but is also very welcome by students of mathematics and computational science who want to learn about applications of numerical methods in physics. This book may also support lecturers of computational physics and bio-computing. It tries to bridge between simple examples which can be solved analytically and more complicated but instructive applications which provide insight into the underlying physics by doing computer experiments.

The first part gives an introduction into the essential methods of numerical mathematics which are needed for applications in physics. Basic algorithms are explained in detail together with limitations due to numerical inaccuracies. Mathematical explanations are supplemented by numerous numerical experiments.

The second part of the book shows the application of computer simulation methods for a variety of physical systems with a certain focus on molecular biophysics. The main object is the time evolution of a physical system. Starting from a simple rigid rotor or a mass point in a central field, important concepts of classical molecular dynamics are discussed. Further chapters deal with partial differential equations, especially the Poisson-Boltzmann equation, the diffusion equation, nonlinear dynamic systems and the simulation of waves on a 1-dimensional string. In the last chapters simple quantum systems are studied to understand e.g. exponential decay processes or electronic transitions during an atomic collision. A two-state quantum system is studied in large detail, including relaxation processes and excitation by an external field. Elementary operations on a quantum bit (qubit) are simulated.

Basic equations are derived in detail and efficient implications are discussed together with numerical accuracy and stability of the algorithms. Analytical results are given for simple test cases which serve as a benchmark for the numerical methods. Many computer experiments are provided realized as Java applets which can be run in the web browser. For a deeper insight the source code can be studied and modified with the free “netbeans” ¹ environment.

Philipp O. J. Scherer

Garching, Germany

April 2010

Contents

Part I Numerical Methods

1 Error Analysis 3

1.​1 Machine Numbers and Rounding Errors 3

1.​2 Numerical Errors of Elementary Floating Point Operations 6

1.​2.​1 Numerical Extinction 7

1.​2.​2 Addition 8

1.​2.​3 Multiplication 9

1.​3 Error Propagation 9

1.​4 Stability of Iterative Algorithms 11

1.​5 Example:​ Rotation 12

1.​6 Truncation Error 13

1.​7 Problems 14

2 Interpolation 15

2.​1 Interpolating Functions 15

2.​2 Polynomial Interpolation 16

2.​2.​1 Lagrange Polynomials 17

2.​2.​2 Barycentric Lagrange Interpolation 17

2.​2.​3 Newton’s Divided Differences 18

2.​2.​4 Neville Method 20

2.​2.​5 Error of Polynomial Interpolation 21

2.​3 Spline Interpolation 22

2.​4 Rational Interpolation 25

2.​4.​1 Padé Approximant 25

2.​4.​2 Barycentric Rational Interpolation 27

2.​5 Multivariate Interpolation 32

2.​6 Problems 33

3 Numerical Differentiation 37

3.​1 One-Sided Difference Quotient 37

3.​2 Central Difference Quotient 38

3.​3 Extrapolation Methods 39

3.​4 Higher Derivatives 41

3.​5 Partial Derivatives of Multivariate Functions 42

3.​6 Problems 43

4 Numerical Integration 45

4.​1 Equidistant Sample Points 46

4.​1.​1 Closed Newton-Cotes Formulae 46

4.​1.​2 Open Newton-Cotes Formulae 48

4.​1.​3 Composite Newton-Cotes Rules 48

4.​1.​4 Extrapolation Method (Romberg Integration) 49

4.​2 Optimized Sample Points 50

4.​2.​1 Clenshaw-Curtis Expressions 50

4.​2.​2 Gaussian Integration 52

4.​3 Problems 56

5 Systems of Inhomogeneous Linear Equations 59

5.​1 Gaussian Elimination Method 60

5.​1.​1 Pivoting 63

5.​1.​2 Direct LU Decomposition 63

5.​2 QR Decomposition 64

5.​2.​1 QR Decomposition by Orthogonalizatio​n 64

5.​2.​2 QR Decomposition by Householder Reflections 66

5.​3 Linear Equations with Tridiagonal Matrix 69

5.​4 Cyclic Tridiagonal Systems 71

5.​5 Iterative Solution of Inhomogeneous Linear Equations 73

5.​5.​1 General Relaxation Method 73

5.​5.​2 Jacobi Method 73

5.​5.​3 Gauss-Seidel Method 74

5.​5.​4 Damping and Successive Over-Relaxation 75

5.​6 Conjugate Gradients 76

5.​7 Matrix Inversion 77

5.​8 Problems 78

6 Roots and Extremal Points 83

6.​1 Root Finding 83

6.​1.​1 Bisection 84

6.​1.​2 Regula Falsi (False Position) Method 85

6.​1.​3 Newton-Raphson Method 85

6.​1.​4 Secant Method 86

6.​1.​5 Interpolation 87

6.​1.​6 Inverse Interpolation 88

6.​1.​7 Combined Methods 91

6.​1.​8 Multidimensional​ Root Finding 97

6.​1.​9 Quasi-Newton Methods 98

6.​2 Function Minimization 99

6.​2.​1 The Ternary Search Method 99

6.​2.​2 The Golden Section Search Method (Brent’s Method) 101

6.​2.​3 Minimization in Multidimensions 106

6.​2.​4 Steepest Descent Method 106

6.​2.​5 Conjugate Gradient Method 107

6.​2.​6 Newton-Raphson Method 107

6.​2.​7 Quasi-Newton Methods 108

6.​3 Problems 110

7 Fourier Transformation 113

7.​1 Fourier Integral and Fourier Series 113

7.​2 Discrete Fourier Transformation 114

7.​2.​1 Trigonometric Interpolation 116

7.​2.​2 Real Valued Functions 118

7.​2.​3 Approximate Continuous Fourier Transformation 119

7.​3 Fourier Transform Algorithms 120

7.​3.​1 Goertzel’s Algorithm 120

7.​3.​2 Fast Fourier Transformation 121

7.​4 Problems 125

8 Random Numbers and Monte Carlo Methods 127

8.​1 Some Basic Statistics 127

8.​1.​1 Probability Density and Cumulative Probability Distribution 127

8.​1.​2 Histogram 128

8.​1.​3 Expectation Values and Moments 129

8.​1.​4 Example:​ Fair Die 130

8.​1.​5 Normal Distribution 131

8.​1.​6 Multivariate Distributions 132

8.​1.​7 Central Limit Theorem 133

8.​1.​8 Example:​ Binomial Distribution 133

8.​1.​9 Average of Repeated Measurements 134

8.​2 Random Numbers 135

8.​2.​1 Linear Congruent Mapping 135

8.​2.​2 Marsaglia-Zamann Method 135

8.​2.​3 Random Numbers with Given Distribution 136

8.​2.​4 Examples 136

8.​3 Monte Carlo Integration 138

8.3.1 Numerical Calculation of π 138

8.​3.​2 Calculation of an Integral 139

8.​3.​3 More General Random Numbers 140

8.​4 Monte Carlo Method for Thermodynamic Averages 141

8.​4.​1 Simple Sampling 141

8.​4.​2 Importance Sampling 142

8.​4.​3 Metropolis Algorithm 142

8.​5 Problems 144

9 Eigenvalue Problems 147

9.​1 Direct Solution 148

9.​2 Jacobi Method 148

9.​3 Tridiagonal Matrices 150

9.​3.​1 Characteristic Polynomial of a Tridiagonal Matrix 151

9.​3.​2 Special Tridiagonal Matrices 151

9.3.3 The QL Algorithm 156

9.​4 Reduction to a Tridiagonal Matrix 157

9.​5 Large Matrices 159

9.​6 Problems 160

10 Data Fitting 161

10.​1 Least Square Fit 162

10.​1.​1 Linear Least Square Fit 163

10.​1.​2 Linear Least Square Fit with Orthogonalizatio​n 165

10.​2 Singular Value Decomposition 167

10.​2.​1 Full Singular Value Decomposition 168

10.​2.​2 Reduced Singular Value Decomposition 168

10.​2.​3 Low Rank Matrix Approximation 170

10.​2.​4 Linear Least Square Fit with Singular Value Decomposition 172

10.​3 Problems 175

11 Discretization of Differential Equations 177

11.​1 Classification of Differential Equations 178

11.​1.​1 Linear Second Order PDE 178

11.​1.​2 Conservation Laws 179

11.​2 Finite Differences 180

11.​2.​1 Finite Differences in Time 181

11.​2.​2 Stability Analysis 182

11.​2.​3 Method of Lines 183

11.​2.​4 Eigenvector Expansion 183

11.​3 Finite Volumes 185

11.​3.​1 Discretization of fluxes 188

11.​4 Weighted Residual Based Methods 190

11.​4.​1 Point Collocation Method 191

11.​4.​2 Sub-domain Method 191

11.​4.​3 Least Squares Method 192

11.​4.​4 Galerkin Method 192

11.​5 Spectral and Pseudo-spectral Methods 193

11.​5.​1 Fourier Pseudo-spectral Methods 193

11.​5.​2 Example:​ Polynomial Approximation 194

11.​6 Finite Elements 196

11.​6.​1 One-Dimensional Elements 196

11.​6.​2 Two- and Three-Dimensional Elements 197

11.​6.​3 One-Dimensional Galerkin FEM 201

11.​7 Boundary Element Method 204

12 Equations of Motion 207

12.​1 The State Vector 208

12.​2 Time Evolution of the State Vector 209

12.​3 Explicit Forward Euler Method 210

12.​4 Implicit Backward Euler Method 212

12.​5 Improved Euler Methods 213

12.​6 Taylor Series Methods 215

12.​6.​1 Nordsieck Predictor-Corrector Method 215

12.​6.​2 Gear Predictor-Corrector Methods 217

12.​7 Runge-Kutta Methods 217

12.​7.​1 Second Order Runge-Kutta Method 218

12.​7.​2 Third Order Runge-Kutta Method 218

12.​7.​3 Fourth Order Runge-Kutta Method 219

12.​8 Quality Control and Adaptive Step Size Control 220

12.​9 Extrapolation Methods 221

12.​10 Linear Multistep Methods 222

12.​10.​1 Adams-Bashforth Methods 222

12.​10.​2 Adams-Moulton Methods 223

12.​10.​3 Backward Differentiation (Gear) Methods 223

12.​10.​4 Predictor-Corrector Methods 224

12.​11 Verlet Methods 225

12.​11.​1 Liouville Equation 225

12.​11.​2 Split-Operator Approximation 226

12.​11.​3 Position Verlet Method 227

12.​11.​4 Velocity Verlet Method 227

12.​11.​5 Störmer-Verlet Method 228

12.​11.​6 Error Accumulation for the Störmer-Verlet Method 229

12.​11.​7 Beeman’s Method 230

12.​11.​8 The Leapfrog Method 231

12.​12 Problems 232

Part II Simulation of Classical and Quantum Systems

13 Rotational Motion 239

13.​1 Transformation to a Body Fixed Coordinate System 239

13.​2 Properties of the Rotation Matrix 240

13.3 Properties of W , Connection with the Vector of Angular Velocity 242

13.​4 Transformation Properties of the Angular Velocity 244

13.​5 Momentum and Angular Momentum 246

13.​6 Equations of Motion of a Rigid Body 246

13.​7 Moments of Inertia 247

13.​8 Equations of Motion for a Rotor 248

13.​9 Explicit Methods 248

13.​10 Loss of Orthogonality 250

13.​11 Implicit Method 251

13.​12 Kinetic Energy of a Rotor 255

13.​13 Parametrization by Euler Angles 255

13.​14 Cayley-Klein Parameters, Quaternions, Euler Parameters 256

13.​15 Solving the Equations of Motion with Quaternions 259

13.​16 Problems 260

14 Molecular Mechanics 263

14.​1 Atomic Coordinates 264

14.​2 Force Fields 266

14.​2.​1 Intramolecular Forces 267

14.​2.​2 Intermolecular Interactions 269

14.​3 Gradients 270

14.​4 Normal Mode Analysis 274

14.​4.​1 Harmonic Approximation 274

14.​5 Problems 276

15 Thermodynamic Systems 279

15.​1 Simulation of a Lennard-Jones Fluid 279

15.​1.​1 Integration of the Equations of Motion 280

15.​1.​2 Boundary Conditions and Average Pressure 281

15.​1.​3 Initial Conditions and Average Temperature 281

15.​1.​4 Analysis of the Results 282

15.​2 Monte Carlo Simulation 287

15.​2.​1 One-Dimensional Ising Model 287

15.​2.​2 Two-Dimensional Ising Model 289

15.​3 Problems 290

16 Random Walk and Brownian Motion 293

16.​1 Markovian Discrete Time Models 293

16.​2 Random Walk in One Dimension 294

16.​2.​1 Random Walk with Constant Step Size 295

16.​3 The Freely Jointed Chain 296

16.​3.​1 Basic Statistic Properties 297

16.​3.​2 Gyration Tensor 299

16.​3.​3 Hookean Spring Model 300

16.​4 Langevin Dynamics 301

16.​5 Problems 303

17 Electrostatics 305

17.​1 Poisson Equation 305

17.​1.​1 Homogeneous Dielectric Medium 306

17.​1.​2 Numerical Methods for the Poisson Equation 307

17.​1.​3 Charged Sphere 309

17.1.4 Variable ε 311

17.1.5 Discontinuous ε 313

17.​1.​6 Solvation Energy of a Charged Sphere 314

17.​1.​7 The Shifted Grid Method 314

17.​2 Poisson-Boltzmann Equation 315

17.​2.​1 Linearization of the Poisson-Boltzmann Equation 317

17.​2.​2 Discretization of the Linearized Poisson-Boltzmann Equation 318

17.​3 Boundary Element Method for the Poisson Equation 318

17.​3.​1 Integral Equations for the Potential 318

17.​3.​2 Calculation of the Boundary Potential 321

17.​4 Boundary Element Method for the Linearized Poisson-Boltzmann Equation 324

17.​5 Electrostatic Interaction Energy (Onsager Model) 325

17.​5.​1 Example:​ Point Charge in a Spherical Cavity 326

17.​6 Problems 327

18 Waves 329

18.​1 Classical Waves 329

18.​2 Spatial Discretization in One Dimension 332

18.​3 Solution by an Eigenvector Expansion 334

18.​4 Discretization of Space and Time 337

18.​5 Numerical Integration with a Two-Step Method 338

18.​6 Reduction to a First Order Differential Equation 340

18.​7 Two-Variable Method 343

18.​7.​1 Leapfrog Scheme 343

18.​7.​2 Lax-Wendroff Scheme 345

18.​7.​3 Crank-Nicolson Scheme 347

18.​8 Problems 349

19 Diffusion 351

19.​1 Particle Flux and Concentration Changes 351

19.​2 Diffusion in One Dimension 353

19.​2.​1 Explicit Euler (Forward Time Centered Space) Scheme 353

19.​2.​2 Implicit Euler (Backward Time Centered Space) Scheme 355

19.​2.​3 Crank-Nicolson Method 357

19.​2.​4 Error Order Analysis 358

19.​2.​5 Finite Element Discretization 360

19.​3 Split-Operator Method for Multidimensions 360

19.​4 Problems 362

20 Nonlinear Systems 363

20.​1 Iterated Functions 364

20.​1.​1 Fixed Points and Stability 364

20.​1.​2 The Lyapunov Exponent 366

20.​1.​3 The Logistic Map 367

20.​1.​4 Fixed Points of the Logistic Map 367

20.​1.​5 Bifurcation Diagram 369

20.​2 Population Dynamics 370

20.​2.​1 Equilibria and Stability 370

20.​2.​2 The Continuous Logistic Model 371

20.​3 Lotka-Volterra Model 372

20.​3.​1 Stability Analysis 372

20.​4 Functional Response 373

20.​4.​1 Holling-Tanner Model 375

20.​5 Reaction-Diffusion Systems 378

20.​5.​1 General Properties of Reaction-Diffusion Systems 378

20.​5.​2 Chemical Reactions 378

20.​5.​3 Diffusive Population Dynamics 379

20.​5.​4 Stability Analysis 379

20.​5.​5 Lotka-Volterra Model with Diffusion 380

20.​6 Problems 382

21 Simple Quantum Systems 385

21.​1 Pure and Mixed Quantum States 386

21.​1.​1 Wavefunctions 387

21.​1.​2 Density Matrix for an Ensemble of Systems 387

21.​1.​3 Time Evolution of the Density Matrix 388

21.​2 Wave Packet Motion in One Dimension 389

21.​2.​1 Discretization of the Kinetic Energy 390

21.​2.​2 Time Evolution 392

21.​2.​3 Example:​ Free Wave Packet Motion 402

21.​3 Few-State Systems 403

21.​3.​1 Two-State System 405

21.​3.​2 Two-State System with Time Dependent Perturbation 408

21.​3.​3 Superexchange Model 410

21.​3.​4 Ladder Model for Exponential Decay 412

21.​3.​5 Landau-Zener Model 414

21.​4 The Dissipative Two-State System 416

21.​4.​1 Equations of Motion for a Two-State System 416

21.​4.​2 The Vector Model 417

21.4.3 The Spin- ${1 \over 2}$ System 418

21.​4.​4 Relaxation Processes—The Bloch Equations 420

21.​4.​5 The Driven Two-State System 421

21.​4.​6 Elementary Qubit Manipulation 428

21.​5 Problems 430

Appendix I Performing the Computer Experiments433

Appendix II Methods and Algorithms435

References441

Index449

Footnotes

1

www.​netbeans.​org .

Part 1
Numerical Methods

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_1© Springer International Publishing Switzerland 2013

1. Error Analysis

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Input data as well as the results of elementary operations have to be represented by machine numbers, the subset of real numbers which is used by the arithmetic unit of today’s computers. Generally this generates rounding errors. This kind of numerical error can be avoided in principle by using arbitrary precision arithmetics or symbolic algebra programs. But this is unpractical in many cases due to the increase in computing time and memory requirements. Results from more complex operations like square roots or trigonometric functions can have even larger errors since series expansions have to be truncated and iterations accumulate the errors of the individual steps. In addition, the precision of input data from an experiment is limited. In this chapter we study the influence of numerical errors on the uncertainties of the calculated results and the stability of simple algorithms.

Several sources of errors are important for numerical data processing:

Experimental uncertainty:

Input data from an experiment have a limited precision. Instead of the vector of exact values x the calculation uses x+Δx, with an uncertainty Δx. This can lead to large uncertainties of the calculated results if an unstable algorithm is used or if the unavoidable error inherent to the problem is large.

Rounding errors:

The arithmetic unit of a computer uses only a subset of the real numbers, the so called machine numbers A⊂ℜ. The input data as well as the results of elementary operations have to be represented by machine numbers whereby rounding errors can be generated. This kind of numerical error can be avoided in principle by using arbitrary precision arithmetics¹ or symbolic algebra programs. But this is unpractical in many cases due to the increase in computing time and memory requirements.

Truncation errors:

Results from more complex operations like square roots or trigonometric functions can have even larger errors since series expansions have to be truncated and iterations can accumulate the errors of the individual steps.

1.1 Machine Numbers and Rounding Errors

Floating point numbers are internally stored as the product of sign, mantissa and a power of 2. According to the IEEE754 standard [130] single, double and quadruple precision numbers are stored as 32, 64 or 128 bits (Table 1.1):

Table 1.1

Binary floating point formats

Format

Sign

Exponent

Hidden bit

Fraction

Precision ε M

Float

s

b ₀⋯b ₇

1

a ₀⋯a ₂₂

2⁻²⁴=5.96E⁻⁸

Double

s

b ₀⋯b ₁₀

1

a ₀⋯a ₅₁

2⁻⁵³=1.11E⁻¹⁶

Quadruple

s

b ₀⋯b ₁₄

1

a ₀⋯a ₁₁₁

2⁻¹¹³=9.63E⁻³⁵

The sign bit s is 0 for positive and 1 for negative numbers. The exponent b is biased by adding E which is half of its maximum possible value (Table 1.2).² The value of a number is given by

(1.1)

The mantissa a is normalized such that its first bit is 1 and its value is between 1 and 2

(1.2)

Table 1.2

Exponent bias E

Decimal value

Binary value

Hexadecimal value

Data type

127₁₀

1111111₂

Single

1023₁₀

1111111111₂

Double

16383₁₀

11111111111111₂

$3FFF

Quadruple

Since the first bit of a normalized floating point number always is 1, it is not necessary to store it explicitly (hidden bit or J-bit). However, since not all numbers can be normalized, only the range of exponents from $001⋯$7FE is used for normalized numbers. An exponent of $000 signals that the number is not normalized (zero is an important example, there exist even two zero numbers with different sign) whereas the exponent $7FF is reserved for infinite or undefined results (Table 1.3).

Table 1.3

Special double precision numbers

Hexadecimal value

Symbolic value

+0

−0

+inf

−inf

NAN

Min_Normal

Max_Normal

Min_Subnormal

Max_Subnormal

The range of normalized double precision numbers is between

and

Example

Consider the following bit pattern which represents a double precision number:

The exponent is 100 0000 0101₂−011 1111 1111₂=110₂ and the mantissa including the J-bit is 1 1001 0000 0000⋯₂. Hence the decimal value is

Input numbers which are not machine numbers have to be rounded to the nearest machine number. This is formally described by a mapping ℜ→A

with the property³

(1.3)

For the special case that x is exactly in the middle between two successive machine numbers, a tie-breaking rule is necessary. The simplest rules are to round up always (round-half-up) or always down (round-half-down). However, these are not symmetric and produce a bias in the average round-off error. The IEEE754 standard [130] recommends the round-to-nearest-even method, i.e. the least significant bit of the rounded number should always be zero. Alternatives are round-to-nearest-odd, stochastic rounding and alternating rounding.

The cases of exponent overflow and exponent underflow need special attention:

Whenever the exponent b has the maximum possible value b=b max and a=1.11⋯11 has to be rounded to a′=10.00⋯0, the rounded number is not a machine number and the result is ±inf.

Numbers in the range $2^{b_{\min}}>|x|\geq2^{b_{\min}-t}$ have to be represented with loss of accuracy by denormalized machine numbers. Their mantissa cannot be normalized since it is a<1 and the exponent has the smallest possible value b=b min. Even smaller numbers with $|x|<2^{-t+b_{\min}}$ have to be rounded to ±0.

The maximum rounding error for normalized numbers with t binary digits

(1.4)

is given by (Fig. 1.1)

(1.5)

and the relative error is bounded by

(1.6)

The error bound determines the relative machine precision⁴

(1.7)

and the rounding operation can be described by

(1.8)

Fig. 1.1

(Round to nearest)  Normalized machine numbers with t=3 binary digits are shown. Rounding to the nearest machine number produces a round-off error which is bounded by half the spacing of the machine numbers

The round-off error takes its maximum value if the mantissa is close to 1. Consider a number

If ε<ε M then $\operatorname{rd}(x)=1$ whereas for ε>ε M rounding gives $\operatorname{rd}(x)=1+2^{1-t}$ (Fig. 1.2). Hence ε M is given by the largest number ε for which $\operatorname{rd}(1.0+\varepsilon)=1.0$ and is therefore also called unit round off.

Fig. 1.2

(Unit round off)

1.2 Numerical Errors of Elementary Floating Point Operations

Even for two machine numbers x,y∈A the results of addition, subtraction, multiplication or division are not necessarily machine numbers. We have to expect some additional round-off errors from all these elementary operations [244]. We assume that the results of elementary operations are approximated by machine numbers as precisely as possible. The IEEE754 standard [130] requires that the exact operations x+y,x−y,x×y,x÷y are approximated by floating point operations A→A with the property:

(1.9)

1.2.1 Numerical Extinction

For an addition or subtraction one summand has to be denormalized to line up the exponents (for simplicity we consider only the case x>0,y>0)

(1.10)

If the two numbers differ much in their magnitude, numerical extinction can happen. Consider the following case:

(1.11)

The mantissa of the exact sum is

(1.12)

Rounding to the nearest machine number gives

(1.13)

since

(1.14)

Consider now the case

(1.15)

For normalized numbers the mantissa is in the interval

(1.16)

hence we have

(1.17)

Especially for x=1 we have

(1.18)

2−t could be rounded to 0 or to 21−t since the distance is the same |2−t −0|=|2−t −21−t |=2−t .

The smallest machine number with fl ₊(1,ε)>1 is either ε=0.00⋯1 t 0⋯=2−t or ε=0.00⋯1 t 0⋯012t−1=2−t (1+21−t ). Hence the machine precision ε M can be determined by looking for the smallest (positive) machine number ε for which fl ₊(1,ε)>1.

1.2.2 Addition

Consider the sum of two floating point numbers

(1.19)

First the input data have to be approximated by machine numbers:

(1.20)

The addition of the two summands may produce another error α since the result has to be rounded. The numerical result is

(1.21)

Neglecting higher orders of the error terms we have in first order

(1.22)

and the relative error of the numerical sum is

(1.23)

If x ₁≈−x ₂ then numerical extinction can produce large relative errors and uncertainties of the input data can be strongly enhanced.

1.2.3 Multiplication

Consider the multiplication of two floating point numbers

(1.24)

The numerical result is

(1.25)

with the relative error

(1.26)

The relative errors of the input data and of the multiplication just add up to the total relative error. There is no enhancement. Similarly for a division

(1.27)

the relative error is

(1.28)

1.3 Error Propagation

Consider an algorithm consisting of a sequence of elementary operations. From the set of input data which is denoted by the vector

(1.29)

a set of output data is calculated

(1.30)

Formally this can be denoted by a vector function

(1.31)

which can be written as a product of r simpler functions representing the elementary operations

(1.32)

Starting with x intermediate results $\mathbf{x}_{i}=(x_{i1},\ldots x_{in_{i}})$ are calculated until the output data y result from the last step:

(1.33)

In the following we analyze the influence of numerical errors onto the final results. We treat all errors as small quantities and neglect higher orders. Due to round-off errors and possible experimental uncertainties the input data are not exactly given by x but by

(1.34)

The first step of the algorithm produces the result

(1.35)

Taylor series expansion gives in first order

(1.36)

with the partial derivatives

(1.37)

and the round-off errors of the first step

(1.38)

The error of the first intermediate result is

(1.39)

The second intermediate result is

(1.40)

with the error

(1.41)

Finally the error of the result is

(1.42)

The product of the matrices Dφ (r)⋯Dφ ⁽¹⁾ is the matrix which contains the derivatives of the output data with respect to the input data (chain rule)

(1.43)

The first two contributions to the total error do not depend on the way in which the algorithm is divided into elementary steps in contrary to the remaining summands. Hence the inevitable error which is inherent to the problem can be estimated as [244]

(1.44)

or in case the error of the input data is dominated by the round-off errors |Δx j |≤ε M |x j |

(1.45)

Additional errors which are smaller than this inevitable error can be regarded as harmless. If all errors are harmless, the algorithm can be considered well behaved.

1.4 Stability of Iterative Algorithms

Often iterative algorithms are used which generate successive values starting from an initial value x ₀ according to an iteration method

(1.46)

for instance to solve a large system of equations or to approximate a time evolution x j ≈x(jΔt). Consider first a linear iteration equation which can be written in matrix form as

(1.47)

If the matrix A is the same for all steps we have simply

(1.48)

Consider the unavoidable error originating from errors Δx of the start values:

(1.49)

The initial errors Δx can be enhanced exponentially if A has at least one eigenvalue⁵ λ with |λ|>1. On the other hand the algorithm is conditionally stable if for all eigenvalues |λ|≤1 holds. For a more general nonlinear iteration

(1.50)

the error propagates according to

(1.51)

The algorithm is conditionally stable if all eigenvalues of the derivative matrix Dφ have absolute values |λ|≤1.

1.5 Example: Rotation

Consider a simple rotation in the complex plane. The equation of motion

(1.52)

obviously has the exact solution

(1.53)

As a simple algorithm for numerical integration we use a time grid

(1.54)

(1.55)

and iterate the function values

(1.56)

Since

(1.57)

uncertainties in the initial condition will grow exponentially and the algorithm is not stable. A stable method is obtained by taking the derivative in the middle of the time interval (page 213)

and making the approximation (page 214)

This gives the implicit equation

(1.58)

which can be solved by

(1.59)

Now we have

(1.60)

and the calculated orbit is stable.

1.6 Truncation Error

The algorithm in the last example is stable but of course not perfect. Each step produces an error due to the finite time step. The exact solution

(1.61)

is approximated by

(1.62)

(1.63)

which deviates from the exact solution by a term of the order O(Δt ³), hence the local error order of this algorithm is O(Δt ³) which is indicated by writing

(1.64)

Integration up to a total time T=NΔt accumulates a global error of the order NΔt ³=TΔt ².

1.7 Problems

Problem 1.1

(Machine precision)

In this computer experiment we determine the machine precision ε M . Starting with a value of 1.0, x is divided repeatedly by 2 until numerical addition of 1 and x=2−M gives 1. Compare single and double precision calculations.

Problem 1.2

(Maximum and minimum integers)

Integers are used as counters or to encode elements of a finite set like characters or colors. There are different integer formats available which store signed or unsigned integers of different length (Table 1.4). There is no infinite integer and addition of 1 to the maximum integer gives the minimum integer.

Table 1.4

Maximum and minimum integers

Java format

Bit length

Minimum

Maximum

Byte

8

−128

127

Short

16

−32768

32767

Integer

32

−2147483647

2147483648

Long

64

−9223372036854775808

9223372036854775807

Char

16

0

65535

In this computer experiment we determine the smallest and largest integer numbers. Beginning with I=1 we add repeatedly 1 until the condition I+1>I becomes invalid or subtract repeatedly 1 until I−1<I becomes invalid. For the 64 bit long integer format this takes to long. Here we multiply alternatively I by 2 until I−1<I becomes invalid. For the character format the corresponding ordinal number is shown which is obtained by casting the character to an integer.

Problem 1.3

(Truncation error)

This computer experiment approximates the cosine function by a truncated Taylor series

(1.65)

in the interval −π/2<x<π/2. The function mycos(x,n max) is numerically compared to the intrinsic cosine function.

References

130.

IEEE 754-2008, Standard for Floating-Point Arithmetic, IEEE Standards Association, 2008

244.

J. Stör, R. Bulirsch, Introduction to Numerical Analysis, 3rd revised edn. (Springer, New York, 2010). ISBN 978-1441930064

Footnotes

1

For instance the open source GNU MP bignum library.

2

In the following the usual hexadecimal notation is used which represents a group of 4 bits by one of the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F.

3

Sometimes rounding is replaced by a simpler truncation operation which, however leads to significantly larger rounding errors.

4

Also known as machine epsilon.

5

The eigenvalues of A are solutions of the eigenvalue equation A x=λ x (Chap. 9).

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_2© Springer International Publishing Switzerland 2013

2. Interpolation

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Interpolation is necessary, if additional data points are needed, for instance to draw a continuous curve. It can also be helpful to represent a complicated function by a simpler one or to develop more sophisticated numerical methods for the calculation of numerical derivatives and integrals. In the following we concentrate on the most important interpolating functions which are polynomials, splines and rational functions. The interpolating polynomial can be explicitly constructed with the Lagrange method. Newton’s method is numerically efficient if the polynomial has to be evaluated at many interpolating points and Neville’s method has advantages if the polynomial is not needed explicitly and has to be evaluated only at one interpolation point.

For interpolation over a larger range, Spline functions can be superior which are piecewise defined polynomials. Especially cubic splines are often used to draw smooth curves. Curves with poles can be represented by rational interpolating functions whereas a special class of rational interpolants without poles provides a rather new alternative to spline interpolation.

Experiments usually produce a discrete set of data points (x i ,f i ) which represent the value of a function f(x) for a finite set of arguments {x ₀⋯x n }. If additional data points are needed, for instance to draw a continuous curve, interpolation is necessary. Interpolation also can be helpful to represent a complicated function by a simpler one or to develop more sophisticated numerical methods for the calculation of numerical derivatives and integrals. In the following we concentrate on the most important interpolating functions which are polynomials, splines and rational functions. Trigonometric interpolation is discussed in Chap. 7. An interpolating function reproduces the given function values at the interpolation points exactly (Fig. 2.1). The more general procedure of curve fitting, where this requirement is relaxed, is discussed in Chap. 10.

Fig. 2.1

(Interpolating function) The interpolating function Φ(x) reproduces a given data set Φ(x i )=f i and provides an estimate of the function f(x) between the data points

The interpolating polynomial can be explicitly constructed with the Lagrange method. Newton’s method is numerically efficient if the polynomial has to be evaluated at many interpolating points and Neville’s method has advantages if the polynomial is not needed explicitly and has to be evaluated only at one interpolation point.

Polynomials are not well suited for interpolation over a larger range. Spline functions can be superior which are piecewise defined polynomials. Especially cubic splines are often used to draw smooth curves. Curves with poles can be represented by rational interpolating functions whereas a special class of rational interpolants without poles provides a rather new alternative to spline interpolation.

2.1 Interpolating Functions

Consider the following problem: Given are n+1 sample points (x i ,f i ), i=0⋯n and a function of x which depends on n+1 parameters a i :

(2.1)

The parameters are to be determined such that the interpolating function has the proper values at all sample points (Fig. 2.1)

(2.2)

An interpolation problem is called linear if the interpolating function is a linear combination of functions

(2.3)

Important examples are

polynomials

(2.4)

trigonometric functions

(2.5)

spline functions which are piecewise polynomials, for instance the cubic spline

(2.6)

Important examples for nonlinear interpolating functions are

rational functions

(2.7)

exponential functions

(2.8)

where amplitudes a i as well as exponents λ i have to be optimized.

2.2 Polynomial Interpolation

For n+1 sample points (x i ,f i ), i=0⋯n, x i ≠x j there exists exactly one interpolating polynomial of degree n with

(2.9)

2.2.1 Lagrange Polynomials

Lagrange polynomials [137] are defined as

(2.10)

They are of degree n and have the property

(2.11)

The interpolating polynomial is given in terms of Lagrange polynomials by

(2.12)

2.2.2 Barycentric Lagrange Interpolation

With the polynomial

(2.13)

the Lagrange polynomial can be written as

(2.14)

which, introducing the Barycentric weights [24]

(2.15)

becomes the first form of the barycentric interpolation formula

(2.16)

The interpolating polynomial can now be evaluated according to

(2.17)

Having computed the weights u i , evaluation of the polynomial only requires O(n) operations whereas calculation of all the Lagrange polynomials requires O(n ²) operations. Calculation of ω(x) can be avoided considering that

(2.18)

is a polynomial of degree n with

(2.19)

But this is only possible if

(2.20)

Therefore

(2.21)

which is known as the second form of the barycentric interpolation formula.

2.2.3 Newton’s Divided Differences

Newton’s method of divided differences [138] is an alternative for efficient numerical calculations [271]. Rewrite

(2.22)

With the first order divided difference

(2.23)

this becomes

(2.24)

and with the second order divided difference

(2.25)

we have

(2.26)

Higher order divided differences are defined recursively by

(2.27)

They are invariant against permutation of the arguments which can be seen from the explicit formula

(2.28)

Finally we have

(2.29)

with a polynomial of degree n

(2.30)

and the function

(2.31)

Obviously q(x i )=0, i=0⋯n, hence p(x) is the interpolating polynomial.

Algorithm

The divided differences are arranged in the following way:

(2.32)

Since only the diagonal elements are needed, a one-dimensional data array t[0]⋯t[n] is sufficient for the calculation of the polynomial coefficients:

The value of the polynomial is then evaluated by

2.2.4 Neville Method

The Neville method [180] is advantageous if the polynomial is not needed explicitly and has to be evaluated only at one point. Consider the interpolating polynomial for the points x ₀⋯x k , which will be denoted as P 0,1…k (x). Obviously

(2.33)

since for x=x ₁⋯x k−1 the right hand side is

(2.34)

For x=x ₀ we have

(2.35)

and finally for x=x k

(2.36)

Algorithm

We use the following scheme to calculate P 0,1⋯n (x) recursively:

(2.37)

The first column contains the function values P i (x)=f i . The value P 01⋯n can be calculated using a 1-dimensional data array p[0]⋯p[n]:

2.2.5 Error of Polynomial Interpolation

The error of polynomial interpolation [12] can be estimated with the help of the following theorem:

If f(x) is n+1 times differentiable then for each $\overline{x}$ there exists ξ within the smallest interval containing $\overline{x}$ as well as all the x i with

(2.38)

From a discussion of the function

(2.39)

it can be seen that the error increases rapidly outside the region of the sample points (extrapolation is dangerous!). As an example consider the sample points (Fig. 2.2)

(2.40)

Fig. 2.2

(Interpolating polynomial) The interpolated function (solid curve) and the interpolating polynomial (broken curve) for the example (2.40) are compared

The maximum interpolation error is estimated by (|f (n+1)|≤1)

(2.41)

whereas the error increases rapidly outside the interval 0<x<2π (Fig. 2.3).

Fig. 2.3

(Interpolation error) The polynomial ω(x) is shown for the example (2.40). Its roots x i are given by the x values of the sample points (circles). Inside the interval x ₀⋯x ₄ the absolute value of ω is bounded by |ω(x)|≤35 whereas outside the interval it increases very rapidly

2.3 Spline Interpolation

Polynomials are not well suited for interpolation over a larger range. Often spline functions are superior which are piecewise defined polynomials [186, 228]. The simplest case is a linear spline which just connects the sampling points by straight lines:

(2.42)

(2.43)

The most important case is the cubic spline which is given in the interval x i ≤x<x i+1 by

(2.44)

We want to have a smooth interpolation and assume that the interpolating function and their first two derivatives are continuous. Hence we have for the inner boundaries:

(2.45)

(2.46)

(2.47)

We have to specify boundary conditions at x ₀ and x n . The most common choice are natural boundary conditions s″(x ₀)=s″(x n )=0, but also periodic boundary conditions s″(x ₀)=s″(x n ), s′(x ₀)=s′(x n ), s(x ₀)=s(x n ) or given derivative values s′(x ₀) and s′(x n ) are often used. The second derivative is a linear function [244]

(2.48)

which can be written using h i+1=x i+1−x i and M i =s″(x i ) as

(2.49)

since

(2.50)

(2.51)

Integration gives with the two constants A i and B i

(2.52)

(2.53)

From s(x i )=y i and s(x i+1)=y i+1 we have

(2.54)

(2.55)

and hence

(2.56)

(2.57)

Now the polynomial is

(2.58)

Comparison with

(2.59)

gives

(2.60)

(2.61)

(2.62)

(2.63)

Finally we calculate M i from the continuity of s′(x). Substituting for A i in $p'_{i}(x)$ we have

(2.64)

and from $p'_{i-1}(x_{i})=p'_{i}(x_{i})$ it follows

(2.65)

(2.66)

which is a system of linear equations for the M i . Using the abbreviations

(2.67)

(2.68)

(2.69)

we have

(2.70)

We define for natural boundary conditions

(2.71)

and in case of given derivative values

(2.72)

The system of equation has the form

(2.73)

For periodic boundary conditions we define

(2.74)

and the system of equations is (with M n =M ₀)

(2.75)

All this tridiagonal systems can be easily solved with a special Gaussian elimination method (Sects. 5.​3, 5.​4).

2.4 Rational Interpolation

The use of rational approximants allows to interpolate functions with poles, where polynomial interpolation can give poor results [244]. Rational approximants without poles [90] are also well suited for the case of equidistant x i , where higher order polynomials tend to become unstable. The main disadvantages are additional poles which are difficult to control and the appearance of unattainable points. Recent developments using the barycentric form of the interpolating function [25, 90, 227] helped to overcome these difficulties.

2.4.1 Padé Approximant

The Padé approximant [13] of order [M/N] to a function f(x) is the rational function

(2.76)

which reproduces the McLaurin series (the Taylor series at x=0) of

(2.77)

up to order M+N, i.e.

(2.78)

Multiplication gives

(2.79)

and collecting powers of x we find the system of equations

(2.80)

where

(2.81)

(2.82)

Example

(Calculate the [3,3] approximant to tan(x))

The Laurent series of the tangent is

(2.83)

We set q ₀=1. Comparison of the coefficients of the polynomial

(2.84)

gives the equations

(2.85)

We easily find

(2.86)

and the approximant of order [3,3] is

(2.87)

This expression reproduces the tangent quite well (Fig. 2.4). Its pole at $\sqrt{10}/2\approx1.581$ is close to the pole of the tangent function at π/2≈1.571.

Fig. 2.4

(Padé approximation to tan(x)) The Padé approximant ((2.87), dash-dotted curve) reproduces the tangent (full curve) quite well

2.4.2 Barycentric Rational Interpolation

If the weights of the barycentric form of the interpolating polynomial (2.21) are taken as general parameters u i ≠0 it becomes a rational function

(2.88)

which obviously interpolates the data points since

(2.89)

With the polynomials¹

a rational interpolating function is given by²

Obviously there are infinitely different rational interpolating functions which differ by the weights u=(u ₀,u ₁⋯u n ) (an example is shown in Fig. 2.5). To fix the parameters u i , additional conditions have to be imposed.

Fig. 2.5

(Rational interpolation) The data points $(1,\frac{1}{2}),(2,\frac{1}{5}),(3,\frac{1}{10})$ are interpolated by several rational functions. The [1,1] approximant (2.95) corresponding to u=(5,−20,15) is shown by the solid curve, the dashed curve shows the function $R(x)=\frac{8x^{2}-36x+38}{10(3x^{2}-12x+11)}$ which is obtained for u=(1,1,1) and the dash-dotted curve shows the function $R(x)=\frac{4x^{2}-20x+26}{10(5-4x+x^{2})}$ which follows for u=(1,−1,1) and has no real poles

2.4.2.1 Rational Interpolation of Order [M,N]

One possibility is to assume that P(x) and Q(x) are of order ≤M and ≤N, respectively with M+N=n. This gives n additional equations for the 2(n+1) polynomial coefficients. The number of unknown equals n+1 and the rational interpolant is uniquely determined up to a common factor in numerator and denominator.

Example

(Consider the data points $f(1)=\frac{1}{2},f(2)=\frac{1}{5},f(3)=\frac{1}{10}$)

The polynomials are

(2.90)

(2.91)

To obtain a [1,1] approximant we have to solve the equations

(2.92)

(2.93)

which gives

(2.94)

and thus

(2.95)

General methods to obtain the coefficients u i for a given data set are described in [25, 227]. They also allow to determine unattainable points corresponding to u i =0 and to locate the poles. Without loss of generality it can be assumed [227] that M≥N.³

Let P(x) be the unique polynomial which interpolates the product f(x)Q(x)

(2.96)

Then from (2.31) we have

(2.97)

Setting

(2.98)

we have

(2.99)

which is zero if P(x i )/Q(x i )=f i for i=0⋯n. But then

(2.100)

The polynomial Q(x) can be written in Newtonian form (2.30)

(2.101)

With the abbreviation

(2.102)

we find

(2.103)

which we apply repeatedly to (2.100) to get the system of n−M=N equations for N+1 unknowns

(2.104)

from which the coefficients ν j can be found by Gaussian elimination up to a scaling factor. The Newtonian form of Q(x) can then be converted to the barycentric form as described in [271].

2.4.2.2 Rational Interpolation Without Poles

Polynomial interpolation of larger data sets can be ill behaved, especially for the case of equidistant x-values. Rational interpolation without poles can be a much better choice here (Fig. 2.6).

Fig. 2.6

(Interpolation of a step function) A step function with uniform x-values (circles) is interpolated by a polynomial (full curve), a cubic spline (dashed curve) and with the rational Floater-Hormann d=1 function ((2.105), dash-dotted curve). The rational function behaves similar to the spline function but provides in addition an analytical function with continuous derivatives

Berrut [23] suggested to choose the following weights

With this choice Q(x) has no real roots. Floater and Hormann [90] used the different choice

(2.105)

which becomes very similar for equidistant x-values.

Floater and Hormann generalized this expression and found a class of rational interpolants without poles given by the weights

(2.106)

where 0≤d≤n and the approximation order increases with d. In the uniform case this simplifies to (Table 2.1)

(2.107)

Table 2.1

Floater-Hormann weights for uniform data

|u k |

d

1,1,1,…,1,1,1

0

1,2,2,2,…,2,2,2,1

1

1,3,4,4,4,…,4,4,4,3,1

2

1,4,7,8,8,8,…,8,8,8,7,4,1

3

1,5,11,15,16,16,16,…,16,16,16,15,11,5,1

4

2.5 Multivariate Interpolation

The simplest 2-dimensional interpolation method is bilinear interpolation.⁴ It uses linear interpolation for both coordinates within the rectangle x i ≤x≤x i+1, y i ≤y i ≤y i+1:

(2.108)

which can be written as a two dimensional polynomial

(2.109)

with

(2.110)

Application of higher order polynomials is straightforward. For image processing purposes bicubic interpolation is often used.

If high quality is needed more sophisticated interpolation methods can be applied. Consider for instance two-dimensional spline interpolation on a rectangular mesh of data to create a new data set with finer resolution⁵

(2.111)

First perform spline interpolation in x-direction for each data row j to calculate new data sets

(2.112)

and then interpolate in y direction to obtain the final high resolution data (Fig. 2.7)

(2.113)

Fig. 2.7

Bispline interpolation

2.6 Problems

Problem 2.1

(Polynomial interpolation)

This computer experiment interpolates a given set of n data points by

a polynomial

(2.114)

a linear spline which connects successive points by straight lines

(2.115)

a cubic spline with natural boundary conditions

(2.116)

(2.117)

a rational function without poles

(2.118)

with weights according to Berrut

(2.119)

or Floater-Hormann

(2.120)

(2.121)

Interpolate the data (Table 2.2) in the range −1.5<x<0.

Table 2.2

Zener diode voltage/current data

Voltage

−1.5

−1.0

−0.5

0.0

Current

−3.375

−1.0

−0.125

0.0

Now add some more sample points (Table 2.3) for −1.5<x<4.5.

Table 2.3

Additional voltage/current data

Voltage

1.0

2.0

3.0

4.0

4.1

4.2

4.5

Current

0.0

0.0

0.0

0.0

1.0

3.0

10.0

Interpolate the function f(x)=sin(x) at the points $x=0,\frac{\pi}{2},\pi,\frac{3\pi}{2},2\pi$. Take more sample points and check if the quality of the fit is improved.

Investigate the oscillatory behavior for a discontinuous pulse or step function as given by the data (Table 2.4).

Table 2.4

Pulse and step function data

x

−3

−2

−1

0

1

2

3

y pulse

0

0

0

1

0

0

0

y step

0

0

0

1

1

1

1

Problem 2.2

(Two-dimensional interpolation)

This computer experiment uses bilinear interpolation or bicubic spline interpolation to interpolate the data (Table 2.5) on a finer grid Δx=Δy=0.1.

Table 2.5

Data set for two-dimensional interpolation

x

0

1

2

0

1

2

0

1

2

y

0

0

0

1

1

1

2

2

2

f

1

0

−1

0

0

0

−1

0

1

References

12.

C.T.H. Baker, Numer. Math. 15, 315 (1970) MathSciNetMATHCrossRef

13.

G.A. Baker Jr., P. Graves-Morris, Padé Approximants (Cambridge University Press, New York, 1996) MATHCrossRef

23.

J.P. Berrut, Comput. Math. Appl. 14, 1 (1988) MathSciNetCrossRef

24.

J.-P. Berrut, L.N. Trefethen, Barycentric Lagrange interpolation. SIAM Rev. 46(3), 501–517 (2004) MathSciNetCrossRef

25.

J.P. Berrut, R. Baltensperger, H.D. Mittelmann, Int. Ser. Numer. Math. 151, 27 (2005) MathSciNetCrossRef

90.

M.S. Floater, K. Hormann, Numer. Math. 107, 315 (2007) MathSciNetMATHCrossRef

137.

H. Jeffreys, B.S. Jeffreys, Lagrange’s interpolation formula, in Methods of Mathematical Physics, 3rd edn. (Cambridge University Press, Cambridge, 1988), p. 260, § 9.011

138.

H. Jeffreys, B.S. Jeffreys, Divided differences, in Methods of Mathematical Physics, 3rd edn. (Cambridge University Press, Cambridge, 1988), pp. 260–264, § 9.012

180.

E.H. Neville, J. Indian Math. Soc. 20, 87 (1933)

186.

G. Nürnberger, Approximation by Spline Functions (Springer, Berlin, 1989). ISBN 3-540-51618-2 MATHCrossRef

227.

C. Schneider, W. Werner, Math. Comput. 47, 285 (1986) MathSciNetMATHCrossRef

228.

I.J. Schoenberg, Q. Appl. Math. 4, 45–99 and 112–141 (1946) MathSciNet

244.

J. Stör, R. Bulirsch, Introduction to Numerical Analysis, 3rd revised edn. (Springer, New York, 2010). ISBN 978-1441930064

271.

W. Werner, Math. Comput. 43, 205 (1984) MATHCrossRef

Footnotes

1

$\omega(x)=\prod_{i=0}^{n}(x-x_{i})$ as in (2.39).

2

It can be shown that any rational interpolant can be written in this form.

3

The opposite case can be treated by considering the reciprocal function values 1/f(x i ).

4

Bilinear means linear interpolation in two dimensions. Accordingly linear interpolation in three dimensions is called trilinear.

5

A typical task of image processing.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_3© Springer International Publishing Switzerland 2013

3. Numerical Differentiation

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

For more complex problems analytical derivatives are not always available and have to be approximated by numerical methods. Numerical differentiation is also very important for the discretization of differential equations. The simplest approximation uses a forward difference quotient and is not very accurate. A symmetric difference quotient improves the quality. Even higher precision is obtained with the extrapolation method. Approximations to higher order derivatives can be obtained systematically with the help of polynomial interpolation.

For more complex problems analytical derivatives are not always available and have to be approximated by numerical methods. Numerical differentiation is also very important for the discretization of differential equations (Sect. 11.​2). The simplest approximation uses a forward difference quotient (Fig. 3.1) and is not very accurate. A symmetric difference quotient improves the quality. Even higher precision is obtained with the extrapolation method. Approximations to higher order derivatives can be obtained systematically with the help of polynomial interpolation.

Fig. 3.1

(Numerical differentiation) Numerical differentiation approximates the differential quotient by a difference quotient $\frac{\mathrm{d} f}{\mathrm{d} x}\approx\frac{\Delta f}{\Delta x}$. However, approximation by a simple forward difference $\frac{\mathrm{d} f}{\mathrm{d} x}(x_{0})\approx\frac{f(x_{0}+h)-f(x_{0})}{h}$, is not very accurate

3.1 One-Sided Difference Quotient

The simplest approximation of a derivative is the ordinary difference quotient which can be taken forward

(3.1)

or backward

(3.2)

Its truncation error can be estimated from the Taylor series expansion

(3.3)

The error order is O(h). The step width should not be too small to avoid rounding errors. Error analysis gives

(3.4)

(3.5)

The errors are uncorrelated and the relative error of the result can be estimated by

(3.6)

Numerical extinction produces large relative errors for small step width h. The optimal value of h gives comparable errors from rounding and truncation. It can be found from

(3.7)

Assuming that the magnitude of the function and the derivative are comparable, we have the rule of thumb

(double precision). The corresponding relative error is of the same order.

3.2 Central Difference Quotient

Accuracy is much higher if a symmetric central difference quotient is used (Fig. 3.2):

(3.8)

Fig. 3.2

(Difference quotient) The central difference quotient (right side) approximates the derivative (dotted) much more accurately than the one-sided difference quotient (left side)

The error order is O(h ²). The optimal step width is estimated from

(3.9)

again with the assumption that function and derivatives are of similar magnitude as

(3.10)

The relative error has to be expected in the order of $\frac{h_{0}^{2}}{24}\approx10^{-11}$.

3.3 Extrapolation Methods

The Taylor series of the symmetric difference quotient contains only even powers of h:

(3.11)

The Extrapolation method [217] uses a series of step widths, e.g.

(3.12)

and calculates an estimate of D(0) by polynomial interpolation (Fig. 3.3). Consider D ₀=D(h ₀) and $D_{1}=D(\frac{h_{0}}{2})$. The polynomial of degree 1 (with respect to h ²) p(h)=a+bh ² can be found by the Lagrange method

(3.13)

Extrapolation for h=0 gives

(3.14)

Taylor series expansion shows

(3.15)

(3.16)

that the error order is $O(h_{0}^{4})$. For 3 step widths h ₀=2h ₁=4h ₂ we obtain the polynomial of second order (in h ²)

(3.17)

and the improved expression

(3.18)

Fig. 3.3

(Numerical differentiation) The derivative $\frac{\mathrm{d}}{\mathrm{d} x}\sin(x)$ is calculated numerically using algorithms with increasing error order (3.1, 3.8, 3.14, 3.18). For very small step sizes the error increases as h ⁻¹ due to rounding errors

Often used is the following series of step widths:

(3.19)

The Neville method

(3.20)

gives for h=0

(3.21)

which can be written as

(3.22)

and can be calculated according to the following scheme:

(3.23)

Here the values of the polynomials are arranged in matrix form

(3.24)

with the recursion formula

(3.25)

3.4 Higher Derivatives

Difference quotients for higher derivatives can be obtained systematically using polynomial interpolation. Consider equidistant points

(3.26)

From the second order polynomial

(3.27)

we calculate the derivatives

(3.28)

(3.29)

which are evaluated at x ₀:

(3.30)

(3.31)

Higher order polynomials can be evaluated with an algebra program. For five sample points

we find

(3.32)

(3.33)

(3.34)

3.5 Partial Derivatives of Multivariate Functions

Consider polynomials of more than one variable. In two dimensions we use the Lagrange polynomials

(3.35)

The interpolating polynomial is

(3.36)

For the nine sample points

(3.37)

we obtain the polynomial

(3.38)

which gives an approximation to the gradient

(3.39)

the Laplace operator

(3.40)

and the mixed second derivative

(3.41)

3.6 Problems

Problem 3.1

(Numerical differentiation)

In this computer experiment we calculate the derivative of f(x)=sin(x) numerically with

the single sided difference quotient

(3.42)

the symmetrical difference quotient

(3.43)

higher order approximations which can be derived using the extrapolation method

(3.44)

(3.45)

The error of the numerical approximation is shown on a log-log plot as a function of the step width h.

References

217.

L.F. Richardson, Philos. Trans. R. Soc. Lond. Ser. A 210, 307–357 (1911) CrossRef

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_4© Springer International Publishing Switzerland 2013

4. Numerical Integration

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Integrals over complicated functions often have to be calculated numerically. Integration is also the elementary step in solving equations of motion. An integral over an infinite interval may have to be transformed into an integral over an infinite interval first. A definite integral can be approximated numerically as the weighted average over a finite number of function values.

Specific sets of quadrature points and quadrature weights are known as “integral rules”. Newton-Cotes rules use equidistant points and are easy to apply. Accuracy can be improved by dividing the integration range into sub-intervals and applying composite Newton-Cotes rules. Extrapolation methods reduce the error almost to machine precision but need many function evaluations. Equidistant sample points are convenient but not the best choice. Clenshaw-Curtis expressions use non uniform sample points and a rapidly converging Chebyshev expansion. Gaussian integration fully optimizes the sample points with the help of orthogonal polynomials.

Physical simulations often involve the calculation of definite integrals over complicated functions, for instance the Coulomb interaction between two electrons. Integration is also the elementary step in solving equations of motion.

An integral over a finite interval [a,b] can always be transformed into an integral over [0,1] or [−1,1]

(4.1)

An Integral over an infinite interval may have to be transformed into an integral over a finite interval by substitution of the integration variable, for example

(4.2)

(4.3)

In general a definite integral can be approximated numerically as the weighted average over a finite number of function values

(4.4)

Specific sets of quadrature points x i and quadrature weights w i are known as “integral rules”. Newton-Cotes rules like the trapezoidal rule, the midpoint rule or Simpson’s rule, use equidistant points x i and are easy to apply. Accuracy can be improved by dividing the integration range into sub-intervals and applying composite Newton-Cotes rules. Extrapolation methods reduce the error almost to machine precision but need many function evaluations. Equidistant sample points are convenient but not the best choice. Clenshaw-Curtis expressions use non uniform sample points and a rapidly converging Chebyshev expansion. Gaussian integration fully optimizes the sample points with the help of orthogonal polynomials.

4.1 Equidistant Sample Points

For equidistant points

(4.5)

the interpolating polynomial of order N with p(x i )=f(x i ) is given by the Lagrange method

(4.6)

Integration of the polynomial gives

(4.7)

After substituting

(4.8)

we have

(4.9)

and hence

(4.10)

The weight factors are given by

(4.11)

4.1.1 Closed Newton-Cotes Formulae

For N=1 the polynomial is

(4.12)

and the integral is

(4.13)

which is known as the trapezoidal rule (Fig. 4.1). N=2 gives Simpson’s rule

(4.14)

Larger N give further integration rules

(4.15)

For even larger N negative weight factors appear and the formulas are not numerically stable.

Fig. 4.1

(Trapezoidal rule and midpoint rule) The trapezoidal rule (left) approximates the integral by the average of the function values at the boundaries. The midpoint rule (right) evaluates the function in the center of the interval and has the same error order

4.1.2 Open Newton-Cotes Formulae

Alternatively, the integral can be computed from only interior points

(4.16)

The simplest case is the midpoint rule (Fig. 4.1)

(4.17)

The next two are

(4.18)

(4.19)

4.1.3 Composite Newton-Cotes Rules

Newton-Cotes formulae are only accurate, if the step width is small. Usually the integration range is divided into small sub-intervals

(4.20)

for which a simple quadrature formula can be used. Application of the trapezoidal rule for each interval

(4.21)

gives the composite trapezoidal rule

(4.22)

with error order O(h ²). Repeated application of Simpson’s rule for [a,a+2h],[a+2h,a+4h],⋯ gives the composite Simpson’s rule

(4.23)

with error order O(h ⁴).¹

Repeated application of the midpoint rule gives the composite midpoint rule

(4.24)

with error order O(h ²).

4.1.4 Extrapolation Method (Romberg Integration)

For the trapezoidal rule the Euler-McLaurin expansion exists which for a 2m times differentiable function has the form

(4.25)

Therefore extrapolation methods are applicable. From the composite trapezoidal rule for h and h/2 an approximation of error order O(h ⁴) results:

(4.26)

(4.27)

(4.28)

More generally, for the series of step widths

(4.29)

the Neville method gives the recursion for the interpolating polynomial

(4.30)

which for h=0 becomes the higher order approximation to the integral (Fig. 4.2)

(4.31)

The polynomial values can again be arranged in matrix form

(4.32)

with

(4.33)

and the recursion formula

(4.34)

(4.35)

Fig. 4.2

(Romberg integration) The integral $\int_{0}^{\pi^{2}}\sin(x^{2})dx$ is calculated numerically. Circles show the absolute error of the composite trapezoidal rule (4.22) for the step size sequence h i+1=h i /2. Diamonds show the absolute error of the extrapolated value (4.31). The error order of the trapezoidal rule is O(h ²) whereas the error order of the Romberg method increases by factors of h ². For very small step sizes the rounding errors dominate which increase as h ⁻¹

4.2 Optimized Sample Points

The Newton-Cotes method integrates polynomials of order up to N−1 exactly, using N equidistant sample points. Unfortunately the polynomial approximation converges slowly, at least for not so well behaved integrands. The accuracy of the integration can be improved by optimizing the sample point positions. Gaussian quadrature determines the N positions and N weights such, that a polynomial of order 2N−1 is integrated exactly. The Clenshaw-Curtis and the related Fejer methods use the roots or the extrema of the Chebyshev polynomials as nodes and determine the weights to integrate polynomials of order N. However, since the approximation by Chebyshev polynomials usually converges very fast, the accuracy is in many cases comparable to the Gaussian method [185, 253]. In the following we restrict the integration interval to [−1,1]. The general case [a,b] is then given by a simple change of variables.

4.2.1 Clenshaw-Curtis Expressions

Clenshaw and Curtis [60] make the variable substitution

(4.36)

for the integral

(4.37)

and approximate the function by the trigonometric polynomial ((7.​19) with N=2M, T=2π)

(4.38)

which interpolates (Sect. 7.​2.​1) f(cost) at the sample points

(4.39)

(4.40)

and where the Fourier coefficients are given by (7.​17)

(4.41)

The function cos(jt) is related to the Chebyshev polynomials of the first kind which for −1≤x≤1 are given by the trigonometric definition

(4.42)

and can be calculated recursively

(4.43)

(4.44)

(4.45)

Substituting x=cost we find

(4.46)

Hence the Fourier series (4.38) corresponds to a Chebyshev approximation

(4.47)

and can be used to approximate the integral

(4.48)

(4.49)

where, in fact, only the even j contribute.

Example

(Clenshaw Curtis quadrature for M=5)

The function has to be evaluated at the sample points $x_{k}=\cos(\frac{\pi}{5}k)= (1,0.80902,0.30902,-0.30902, -0.80902, -1)$. The Fourier coefficients are given by

(4.50)

and the integral is approximately

(4.51)

Clenshaw Curtis weights of very high order can be calculated efficiently [240, 267] using the FFT algorithm (fast Fourier transformation, Sect. 7.​3.​2)

4.2.2 Gaussian Integration

Now we will optimize the positions of the N quadrature points x i to obtain the maximum possible accuracy. We approximate the integral by a sum

(4.52)

and determine the 2N parameters x i and w i such that a polynomial of order 2N−1 is integrated exactly. This can be achieved with the help of a set of polynomials which are orthogonal with respect to the scalar product

(4.53)

where the weight function w(x) and the interval [a,b] determine a particular set of orthogonal polynomials.

4.2.2.1 Gauss-Legendre Integration

Again we restrict the integration interval to [−1,1] in the following. For integrals with one or two infinite boundaries see Sect. 4.2.2.2. The simplest choice for the weight function is

(4.54)

An orthogonal system of polynomials on the interval [−1,1] can be found using the Gram-Schmidt method:

(4.55)

(4.56)

(4.57)

(4.58)

The P n are known as Legendre-polynomials. Consider now a polynomial p(x) of order 2N−1. It can be interpolated at the N quadrature points x i using the Lagrange method by a polynomial $\widetilde{p}(x)$ of order N−1:

(4.59)

Then p(x) can be written as

(4.60)

Obviously q(x) is a polynomial of order (2N−1)−N=N−1. Now choose the positions x i as the roots of the Legendre polynomial of order N

(4.61)

Then we have

(4.62)

since P N is orthogonal to the polynomial of lower order. But now

(4.63)

with the weight factors

(4.64)

Example

(Gauss-Legendre integration with 2 quadrature points)

The second order Legendre polynomial

(4.65)

has two roots

(4.66)

The Lagrange polynomials are

(4.67)

and the weights

(4.68)

(4.69)

This gives the integral rule

(4.70)

For a general integration interval we substitute

(4.71)

and find the approximation

(4.72)

The next higher order Gaussian rule is given by

(4.73)

4.2.2.2 Other Types of Gaussian Integration

Further integral rules can be obtained by using other sets of orthogonal polynomials, for instance

Chebyshev polynomials

(4.74)

(4.75)

(4.76)

(4.77)

Hermite polynomials

(4.78)

(4.79)

Laguerre polynomials

(4.80)

(4.81)

(4.82)

4.2.2.3 Connection with an Eigenvalue Problem

The determination of quadrature points and weights can be formulated as an eigenvalue problem [109, 133]. Any set of orthogonal polynomials satisfies a three term recurrence relation

(4.83)

with a n >0,c n >0 which can be written in matrix form [272]

(4.84)

or shorter

(4.85)

with a tridiagonal matrix T. Obviously P N (x)=0 if and only if

(4.86)

hence the roots of P N (x) are given by the eigenvalues of T. The matrix T is symmetric if the polynomials are orthonormal, otherwise it can be transformed into a symmetric tridiagonal matrix by an orthogonal transformation [272]. Finally the quadrature weight corresponding to the eigenvalue x j can be calculated from the first component of the corresponding eigenvector [109].

4.3 Problems

Problem 4.1

(Romberg integration)

Use the trapezoidal rule

(4.87)

with the step sequence

(4.88)

and calculate the elements of the triangular matrix

(4.89)

(4.90)

to obtain the approximations

(4.91)

calculate

(4.92)

and compare the absolute error of the trapezoidal sums T(h i )=T i,0 and the extrapolated values T 0,i .

calculate

(4.93)

for ε=10⁻³. Compare with the composite midpoint rule

(4.94)

References

60.

C.W. Clenshaw, A.R. Curtis, Numer. Math. 2, 197 (1960) MathSciNetMATHCrossRef

109.

G.H. Golub, J.H. Welsch, Math. Comput. 23, 221–230 (1969) MathSciNetMATHCrossRef

133.

R.Z. Iqbal, Master thesis, School of Mathematics, University of Birmingham, 2008

185.

M. Novelinkova, in: WDS’11, Proceedings of Contributed Papers, Part I (2011), p. 67

240.

A. Sommariva, Comput. Math. Appl. 65(4), 682–693 (2013). doi:10.​1016/​j.​camwa.​2012.​12.​004 [MATLAB CODES (zip file)] MathSciNetCrossRef

253.

L.N. Trefethen, SIAM Rev. 50, 67–87 (2008) MathSciNetADSMATHCrossRef

267.

J. Waldvogel, BIT Numer. Math. 46, 195 (2006) MathSciNetMATHCrossRef

272.

H. Wilf, Mathematics for the Physical Sciences (Wiley, New York, 1962) MATH

Footnotes

1

The number of sample points must be even.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_5© Springer International Publishing Switzerland 2013

5. Systems of Inhomogeneous Linear Equations

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Many problems in physics and especially computational physics involve systems of linear equations which arise e.g. from linearization of a general nonlinear problem or from discretization of differential equations. If the dimension of the system is not too large standard methods like Gaussian elimination or QR decomposition are sufficient. Systems with a tridiagonal matrix are important for cubic spline interpolation and numerical second derivatives. They can be solved very efficiently with a specialized Gaussian elimination method. Practical applications often involve very large dimensions and require iterative methods. Convergence of Jacobi and Gauss-Seidel methods is slow and can be improved by relaxation or over-relaxation. An alternative for large systems is the method of conjugate gradients.

Many problems in physics and especially computational physics involve systems of linear equations

(5.1)

or shortly in matrix form

(5.2)

which arise e.g. from linearization of a general nonlinear problem like (Sect. 20.​2)

(5.3)

or from discretization of differential equations like

(5.4)

If the dimension of the system is not too large standard methods like Gaussian elimination or QR decomposition are sufficient. Systems with a tridiagonal matrix are important for cubic spline interpolation and numerical second derivatives. They can be solved very efficiently with a specialized Gaussian elimination method. Practical applications often involve very large dimensions and require iterative methods. Convergence of Jacobi and Gauss-Seidel methods is slow and can be improved by relaxation or over-relaxation. An alternative for large systems is the method of conjugate gradients.

5.1 Gaussian Elimination Method

A series of linear combinations of the equations transforms the matrix A into an upper triangular matrix. Start with subtracting a i1/a ₁₁ times the first row from rows 2⋯n

(5.5)

which can be written as a multiplication

(5.6)

with the Frobenius matrix

(5.7)

The result has the form

(5.8)

Now subtract $\frac{a_{i2}}{a_{22}}$ times the second row from rows 3⋯n. This can be formulated as

(5.9)

with

(5.10)

The result is

(5.11)

Continue until an upper triangular matrix results after n−1 steps:

(5.12)

(5.13)

(5.14)

(5.15)

The transformed system of equations

(5.16)

can be solved easily by backward substitution:

(5.17)

(5.18)

(5.19)

Alternatively the matrices L i can be inverted:

(5.20)

This gives

(5.21)

The product of the inverted matrices is a lower triangular matrix:

(5.22)

Hence the matrix A becomes decomposed into a product of a lower and an upper triangular matrix

(5.23)

which can be used to solve the system of equations (5.2).

(5.24)

in two steps:

(5.25)

which can be solved from the top

(5.26)

(5.27)

(5.28)

and

(5.29)

which can be solved from the bottom

(5.30)

(5.31)

(5.32)

5.1.1 Pivoting

To improve numerical stability and to avoid division by zero pivoting is used. Most common is partial pivoting. In every step the order of the equations is changed in order to maximize the pivoting element a k,k in the denominator. This gives LU decomposition of the matrix PA where P is a permutation matrix. P is not needed explicitly. Instead an index vector is used which stores the new order of the equations

(5.33)

Total pivoting exchanges rows and columns of A. This can be time consuming for larger matrices.

If the elements of the matrix are of different orders of magnitude it can be necessary to balance the matrix, for instance by normalizing all rows of A. This can be also achieved by selecting the maximum of

(5.34)

as the pivoting element.

5.1.2 Direct LU Decomposition

LU decomposition can be also performed in a different order [203]. For symmetric positive definite matrices there exists the simpler and more efficient Cholesky method decomposes the matrix into the product LL T of a lower triangular matrix and its transpose [204].

5.2 QR Decomposition

The Gaussian elimination method can become numerically unstable [254]. An alternative method to solve a system of linear equations uses the decomposition [108]

(5.35)

with a unitary matrix Q † Q=1 (an orthogonal matrix Q T Q=1 if A is real) and an upper right triangular matrix R. The system of linear equations (5.2) is simplified by multiplication with Q †=Q ⁻¹

(5.36)

(5.37)

Such a system with upper triangular matrix is easily solved (see (5.29)).

5.2.1 QR Decomposition by Orthogonalization

Gram-Schmidt orthogonalization [108, 244] provides a simple way to perform a QR decomposition. It is used for symbolic calculations and also for least square fitting (10.​1.​2) but can become numerically unstable.

From the decomposition A=QR we have

(5.38)

(5.39)

which gives the k-th column vector a k of A as a linear combination of the orthonormal vectors q ₁⋯q k . Similarly q k is a linear combination of the first k columns of A. With the help of the Gram-Schmidt method r jk and q j are calculated as follows:

(5.40)

(5.41)

For k=2⋯n:

(5.42)

(5.43)

(5.44)

(5.45)

Obviously now

(5.46)

since per definition

(5.47)

and

(5.48)

Hence

(5.49)

Orthogonality gives

(5.50)

In matrix notation we have finally

(5.51)

If the columns of A are almost linearly dependent, numerical stability can be improved by an additional orthogonalization step

(5.52)

after (5.43) which can be iterated several times to improve the results [66, 244].

5.2.2 QR Decomposition by Householder Reflections

Numerically stable algorithms use a series of transformations with unitary matrices, mostly Householder reflections (Fig. 5.1) [244]¹ which have the form

(5.53)

with a unit vector

(5.54)

Obviously P is an orthogonal matrix since

(5.55)

In the first step we try to find a vector u such that the first column vector of A

(5.56)

is transformed into a vector along the 1-axis

(5.57)

Multiplication with the transpose vector gives

(5.58)

and

(5.59)

can have both signs. From (5.57) we have

(5.60)

Multiplication with $\mathbf{a}_{1}^{T}$ gives

(5.61)

and since

(5.62)

we have

(5.63)

and from (5.60) we find

(5.64)

To avoid numerical extinction the sign of k is chosen such that

(5.65)

Then,

(5.66)

(5.67)

and the Householder transformation of the first column vector of A gives

(5.68)

Thus after the first step a matrix results of the form

In the following (n−2) steps further Householder reflections are applied in the subspace k≤i,j≤n to eliminate the elements

of the k-th row vector below the diagonal of the matrix

(5.69)

Finally an upper triangular matrix results

(5.70)

If the orthogonal matrix Q is needed explicitly additional numerical operations are necessary to form the product

(5.71)

Fig. 5.1

(Householder transformation) Geometrically the Householder transformation (5.53) is a mirror operation with respect to a plane with normal vector u

5.3 Linear Equations with Tridiagonal Matrix

Linear equations with the form

(5.72)

(5.73)

(5.74)

can be solved very efficiently with a specialized Gaussian elimination method.² They are important for cubic spline interpolation or second derivatives. We begin by eliminating a ₂. To that end we multiply the first line with a ₂/b ₁ and subtract it from the first line. The result is the equation

(5.75)

with the abbreviations

(5.76)

We iterate this procedure

(5.77)

(5.78)

until we reach the n-th equation, which becomes simply

(5.79)

(5.80)

Now we immediately have

(5.81)

and backward substitution gives

(5.82)

and finally

(5.83)

This algorithm can be formulated as LU decomposition: Multiplication of the matrices

(5.84)

gives

(5.85)

which coincides with the matrix

(5.86)

if we choose

(5.87)

since then from (5.78)

(5.88)

and

(5.89)

5.4 Cyclic Tridiagonal Systems

Periodic boundary conditions lead to a small perturbation of the tridiagonal matrix

(5.90)

The system of equations

(5.91)

can be reduced to a tridiagonal system [205] with the help of the Sherman-Morrison formula [236], which states that if A ₀ is an invertible matrix and u,v are vectors and

(5.92)

then the inverse of the matrix³

(5.93)

is given by

(5.94)

We choose

(5.95)

Then

(5.96)

is tridiagonal. The free parameter α has to be chosen such that the diagonal elements do not become too small. We solve the system (5.91) by solving the two tridiagonal systems

(5.97)

and compute x from

(5.98)

5.5 Iterative Solution of Inhomogeneous Linear Equations

Discretized differential equations often lead to systems of equations with large sparse matrices, which have to be solved by iterative methods.

5.5.1 General Relaxation Method

We want to solve

(5.99)

iteratively. To that end we divide the matrix A into two (non singular) parts [244]

(5.100)

and rewrite (5.99) as

(5.101)

which we use to define the iteration

(5.102)

(5.103)

A fixed point x of this equation fulfills

(5.104)

and is obviously a solution of (5.99). The iteration can be written as

(5.105)

or

(5.106)

5.5.2 Jacobi Method

Jacobi divides the matrix A into its diagonal and two triangular matrices [38]:

(5.107)

For A ₁ the diagonal part is chosen

(5.108)

giving

(5.109)

which reads explicitly

(5.110)

This method is stable but converges rather slowly. Reduction of the error by a factor of 10−p needs about $\frac{pN}{2}$ iterations. N grid points have to be evaluated in each iteration and the method scales with O(N ²) [206].

5.5.3 Gauss-Seidel Method

With

(5.111)

the iteration becomes

(5.112)

which has the form of a system of equations with triangular matrix [139]. It reads explicitly

(5.113)

Forward substitution starting from x ₁ gives

(5.114)

This looks very similar to the Jacobi method. But here all changes are made immediately. Convergence is slightly better (roughly a factor of 2) and the numerical effort is reduced [206].

5.5.4 Damping and Successive Over-Relaxation

Convergence can be improved [206] by combining old and new values. Starting from the iteration

(5.115)

we form a linear combination with

(5.116)

giving the new iteration equation

(5.117)

In case of the Jacobi method with D=A ₁ we have

(5.118)

or explicitly

(5.119)

The changes are damped (0<ω<1) or exaggerated⁴ (1<ω<2).

In case of the Gauss-Seidel method with A ₁=D+L the new iteration (5.117) is

(5.120)

or explicitly

(5.121)

It can be shown that the successive over-relaxation method converges only for 0<ω<2.

For optimal choice of ω about $\frac{1}{3}p\sqrt{N}$ iterations are needed to reduce the error by a factor of 10−p . The order of the method is $O(N^{\frac{3}{2}})$ which is comparable to the most efficient matrix inversion methods [206].

5.6 Conjugate Gradients

At the minimum of the quadratic function

(5.122)

the gradient

(5.123)

is zero and therefore the minimum of h is also a solution of the linear system of equations

(5.124)

The stationary point can be found especially efficient with the method of conjugate gradients (page 107). The function h is minimized along the search direction

by solving

(5.125)

(5.126)

The parameter β is chosen as

(5.127)

The gradient of h is the residual vector and is iterated according to

(5.128)

This method [121] solves a linear system without storing the matrix A itself. Only the product A s is needed. In principle the solution is reached after N=dim(A) steps, but due to rounding errors more steps can be necessary and the method has to be considered as an iterative one.

5.7 Matrix Inversion

LU and QR decomposition can be also used to calculate the inverse of a nonsingular matrix

(5.129)

The decomposition is performed once and then the column vectors of A ⁻¹ are calculated similar to (5.24)

(5.130)

or (5.37)

(5.131)

Consider now a small variation of the right hand side of (5.2)

(5.132)

Instead of

(5.133)

the resulting vector is

(5.134)

and the deviation can be measured by⁵

(5.135)

and since

(5.136)

the relative error becomes

(5.137)

The relative error of b is multiplied by the condition number for inversion

(5.138)

5.8 Problems

Problem 5.1

(Comparison of different direct Solvers, Fig. 5.2)

In this computer experiment we solve the system of equations

(5.139)

with several methods:

Gaussian elimination without pivoting (Sect. 5.1),

Gaussian elimination with partial pivoting (Sect. 5.1.1),

QR decomposition with Householder reflections (Sect. 5.2.2),

QR decomposition with Gram-Schmidt orthogonalization (Sect. 5.2.1),

QR decomposition with Gram-Schmidt orthogonalization with extra orthogonalization step (5.52).

The right hand side is chosen as

(5.140)

hence the exact solution is

(5.141)

Several test matrices can be chosen:

Gaussian elimination is theoretically unstable but is stable in many practical cases. The instability can be demonstrated with the example [254]

(5.142)

No pivoting takes place in the LU decomposition of this matrix and the entries in the last column double in each step:

(5.143)

Since the machine precision is ϵ M =2⁻⁵³ for double precision calculations we have to expect numerical inaccuracy for dimension n>53.

Especially well conditioned are matrices [273] which are symmetric

(5.144)

and also diagonal dominant

(5.145)

We use the matrix

(5.146)

which can be inverted explicitly by

(5.147)

and has a condition number⁶ which is proportional to the dimension n (Fig. 5.3).

The Hilbert matrix [22, 122]

(5.148)

is especially ill conditioned [16] even for moderate dimension. It is positive definite and therefore the inverse matrix exists and even can be written down explicitly [222]. Its column vectors are very close to linearly dependent and the condition number grows exponentially with its dimension (Fig. 5.3). Numerical errors are large for all methods compared (Fig. 5.2).

Fig. 5.2

(Comparison of different direct solvers) Gaussian elimination without (circles) and with (x) pivoting, QR decomposition with Householder reflections (squares), with Gram-Schmidt orthogonalization (diamonds) and including extra orthogonalization (+) are compared. The maximum difference $\max_{i=1\cdots n}(|x_{i}-x_{i}^{\mathit{exact}}|)$ increases only slightly with the dimension n for the well behaved matrix ((5.146), a) but quite dramatically for the ill conditioned Hilbert matrix ((5.148), b)

Fig. 5.3

(Condition numbers) The condition number $\operatorname{cond}(A)$ increases only linearly with the dimension n for the well behaved matrix ((5.146), full circles) but exponentially for the ill conditioned Hilbert matrix ((5.148), open circles)

Random matrices

(5.149)

References

16.

B. Beckermann, Numer. Math. 85, 553 (2000) MathSciNetMATHCrossRef

22.

A. Bermann, S. Gueron, Math. Gaz. 86, 274 (2002) CrossRef

38.

I.N. Bronshtein, K.A. Semendyayev, Handbook of Mathematics, 3rd edn. (Springer, New York, 1997), p. 892 MATH

66.

J.W. Daniel, W.B. Gragg, L. Kaufmann, G.W. Stewart, Math. Comput. 30, 772 (1976) MATH

108.

G.H. Golub, C.F. Van Loan, Matrix Computations, 3rd edn. (Johns Hopkins University Press, Baltimore, 1976). ISBN 978-0-8018-5414-9

121.

M.R. Hestenes, E. Stiefel, J. Res. Natl. Bur. Stand. 49, 435 (1952) MathSciNetCrossRef

122.

D. Hilbert, L. Nordheim, J. von Neumann, Math. Ann. 98, 1 (1927) MATHCrossRef

139.

H. Jeffreys, B.S. Jeffreys, Methods of Mathematical Physics, 3rd edn. (Cambridge University Press, Cambridge, 1988), pp. 305–306

203.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, LU decomposition and its applications, in Numerical Recipes: The Art of Scientific Computing, 3rd edn. (Cambridge University Press, Cambridge, 2007), pp. 48–55

204.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Cholesky decomposition, in Numerical Recipes: The Art of Scientific Computing, 3rd edn. (Cambridge University Press, Cambridge, 2007), pp. 100–101

205.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Cyclic tridiagonal systems, in Numerical Recipes: The Art of Scientific Computing, 3rd edn. (Cambridge University Press, Cambridge, 2007), p. 79

206.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Relaxation mehods for boundary value problems, in Numerical Recipes: The Art of Scientific Computing, 3rd edn. (Cambridge University Press, Cambridge, 2007), pp. 1059–1065

222.

I.R. Savage, E. Lukacs, in Contributions to the Solution of Systems of Linear Equations and the Determination of Eigenvalues, ed. by O. Taussky. National Bureau of Standards, Applied Mathematics Series, vol. 39 (1954)

236.

J. Sherman, W.J. Morrison, Ann. Math. Stat. 20, 621 (1949)

244.

J. Stör, R. Bulirsch, Introduction to Numerical Analysis, 3rd revised edn. (Springer, New York, 2010). ISBN 978-1441930064

254.

L.N. Trefethen, D. Bau III, Numerical Linear Algebra (Society for Industrial and Applied Mathematics, Philadelphia, 1997), p. 1 MATHCrossRef

273.

J.H. Wilkinson, J. ACM 8, 281 (1961) MathSciNetMATHCrossRef

Footnotes

1

Alternatively Givens rotations [108] can be employed which need slightly more floating point operations.

2

This algorithm is only well behaved if the matrix is diagonal dominant |b i |>|a i |+|c i |.

3

Here uv T is the outer or matrix product of the two vectors.

4

This is also known as the method of successive over-relaxation (SOR).

5

The vector norm used here is not necessarily the Euclidian norm.

6

Using the Frobenius norm $\|A\|=\sqrt{\sum_{ij}A_{ij}^{2}}$.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_6© Springer International Publishing Switzerland 2013

6. Roots and Extremal Points

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

In computational physics very often roots and local extrema of a function have to be determined. In one dimension bisection is a very robust but rather inefficient root finding method. If a good starting point close to the root is available and the function is smooth enough, the Newton-Raphson method converges much faster. Special strategies are necessary to find roots of not so well behaved functions or higher order roots. The combination of bisection and interpolation as by the methods of Dekker, Brent and more recently Chandrupatla provides generally applicable algorithms. In multidimensions Quasi-Newton methods are a good choice. Whereas local extrema can be found as the roots of the gradient, at least in principle, direct optimization can be more efficient. In one dimension the ternary search method or Brent’s more efficient golden section search method can be used. In multidimensions the class of direction set search methods is very popular which includes the methods of steepest descent and conjugate gradients, the Newton-Raphson method and, if calculation of the full Hessian matrix is too expensive, the Quasi-Newton methods.

In computational physics very often roots of a function, i.e. solutions of an equation like

(6.1)

have to be determined. A related problem is the search for local extrema (Fig. 6.1)

(6.2)

which for a smooth function are solutions of the equations

(6.3)

In one dimension bisection is a very robust but rather inefficient root finding method. If a good starting point close to the root is available and the function smooth enough, the Newton-Raphson method converges much faster. Special strategies are necessary to find roots of not so well behaved functions or higher order roots. The combination of bisection and interpolation like in Dekker’s and Brent’s methods provides generally applicable algorithms. In multidimensions calculation of the Jacobian matrix is not always possible and Quasi-Newton methods are a good choice. Whereas local extrema can be found as the roots of the gradient, at least in principle, direct optimization can be more efficient. In one dimension the ternary search method or Brent’s more efficient golden section search method can be used. In multidimensions the class of direction set search methods is very popular which includes the methods of steepest descent and conjugate gradients, the Newton-Raphson method and, if calculation of the full Hessian matrix is too expensive, the Quasi-Newton methods.

Fig. 6.1

Roots and local extrema of a function

6.1 Root Finding

If there is exactly one root in the interval a ₀<x<b ₀ then one of the following methods can be used to locate the position with sufficient accuracy. If there are multiple roots, these methods will find one of them and special care has to be taken to locate the other roots.

6.1.1 Bisection

The simplest method [45] to solve

(6.4)

uses the following algorithm (Fig. 6.2):

(1)

Determine an interval [a ₀,b ₀], which contains a sign change of f(x). If no such interval can be found then f(x) does not have any zero crossings.

(2)

Divide the interval into $[a_{0},a_{0}+\frac{b_{0}-a_{0}}{2}]$ $[a_{0}+\frac{b_{0}-a_{0}}{2},b_{0}]$ and choose that interval [a ₁,b ₁], where f(x) changes its sign.

(3)

repeat until the width b n −a n <ε is small enough.¹

Fig. 6.2

Root finding by bisection

The bisection method needs two starting points which bracket a sign change of the function. It converges but only slowly since each step reduces the uncertainty by a factor of 2.

6.1.2 Regula Falsi (False Position) Method

The regula falsi [92] method (Fig. 6.3) is similar to the bisection method [45]. However, polynomial interpolation is used to divide the interval [x r ,a r ] with f(x r )f(a r )<0. The root of the linear polynomial

(6.5)

is given by

(6.6)

which is inside the interval [x r ,a r ]. Choose the sub-interval which contains the sign change:

(6.7)

Then ξ r provides a series of approximations with increasing precision to the root of f(x)=0.

Fig. 6.3

Regula falsi method

6.1.3 Newton-Raphson Method

Consider a function which is differentiable at least two times around the root ξ. Taylor series expansion around a point x ₀ in the vicinity

(6.8)

gives for x=ξ

(6.9)

Truncation of the series and solving for ξ gives the first order Newton-Raphson [45, 252] method (Fig. 6.4)

(6.10)

and the second order Newton-Raphson method (Fig. 6.4)

(6.11)

Fig. 6.4

Newton-Raphson method

The Newton-Raphson method converges fast if the starting point is close enough to the root. Analytic derivatives are needed. It may fail if two or more roots are close by.

6.1.4 Secant Method

Replacing the derivative in the first order Newton Raphson method by a finite difference quotient gives the secant method [45] (Fig. 6.5) which has been known for thousands of years before [196]

(6.12)

Round-off errors can become important as |f(x r )−f(x r−1)| gets small. At the beginning choose a starting point x ₀ and determine

(6.13)

using a symmetrical difference quotient.

Fig. 6.5

Secant method

6.1.5 Interpolation

The secant method is also obtained by linear interpolation

(6.14)

The root of the polynomial p(x r+1)=0 determines the next iterate x r+1

(6.15)

Quadratic interpolation of three function values is known as Muller’s method [177]. Newton’s form of the interpolating polynomial is

(6.16)

which can be rewritten

(6.17)

and has the roots

(6.18)

To avoid numerical cancellation, this is rewritten

(6.19)

The sign in the denominator is chosen such that x r+1 is the root closer to x r . The roots of the polynomial can become complex valued and therefore this method is useful to find complex roots.

6.1.6 Inverse Interpolation

Complex values of x r can be avoided by interpolation of the inverse function instead

(6.20)

Using the two points x r ,x r−1 the Lagrange method gives

(6.21)

and the next approximation of the root corresponds again to the secant method (6.12)

(6.22)

Inverse quadratic interpolation needs three starting points x r ,x r−1,x r−2 together with the function values f r ,f r−1,f r−2 (Fig. 6.6). The inverse function x=f ⁻¹(y) is interpolated with the Lagrange method

(6.23)

For y=0 we find the next iterate

(6.24)

Fig. 6.6

Root finding by interpolation of the inverse function

Inverse quadratic interpolation is only a good approximation if the interpolating parabola is single valued and hence if it is a monotonous function in the range of f r ,f r−1,f r−2. For the following discussion we assume that the three values of x are renamed such that x ₁<x ₂<x ₃.

Consider the limiting case (a) in Fig. 6.7 where the polynomial has a horizontal tangent at y=f ₃ and can be written as

(6.25)

Its value at y=0 is

(6.26)

If f ₁ and f ₃ have different sign and |f ₁|<|f ₃| (Sect. 6.1.7.2) we find

(6.27)

Fig. 6.7

(Validity of inverse quadratic interpolation)  Inverse quadratic interpolation is only applicable if the interpolating polynomial p(y) is monotonous in the range of the interpolated function values f ₁…f ₃. (a) and (b) show the limiting cases where the polynomial has a horizontal tangent at f ₁ or f ₃. (c) shows the case where the extremum of the parabola is inside the interpolation range and interpolation is not feasible

Brent [36] used this as a criterion for the applicability of the inverse quadratic interpolation. However, this does not include all possible cases where interpolation is applicable. Chandrupatla [54] gave a more general discussion. The limiting condition is that the polynomial p(y) has a horizontal tangent at one of the boundaries x 1,3. The derivative values are

(6.28)

(6.29)

with [54]

(6.30)

(6.31)

Since for a parabola either f ₁<f ₂<f ₃ or f ₁>f ₂>f ₃ the conditions for applicability of inverse interpolation finally become

(6.32)

(6.33)

which can be combined into

(6.34)

This method is usually used in combination with other methods (Sect. 6.1.7.2).

6.1.7 Combined Methods

Bisection converges slowly. The interpolation methods converge faster but are less reliable. The combination of both gives methods which are reliable and converge faster than pure bisection.

6.1.7.1 Dekker’s Method

Dekker’s method [47, 70] combines bisection and secant method. The root is bracketed by intervals [c r ,b r ] with decreasing width where b r is the best approximation to the root found and c r is an earlier guess for which f(c r ) and f(b r ) have different sign. First an attempt is made to use linear interpolation between the points (b r ,f(b r )) and (a r ,f(a r )) where a r is usually the preceding approximation a r =b r−1 and is set to the second interval boundary a r =c r−1 if the last iteration did not lower the function value (Fig. 6.8).

Fig. 6.8

Dekker’s method

Starting from an initial interval [x ₀,x ₁] with sign(f(x ₀))≠sign(f(x ₁)) the method proceeds as follows [47]:

initialization

iteration

If x s is very close to the last b then increase the distance to avoid too small steps else choose x s if it is between b and x m , otherwise choose x m (thus choosing the smaller interval)

Determine x k as the latest of the previous iterates x ₀…x r−1 for which sign(f(x k ))≠sign(f(x r )).

If the new function value is lower update the approximation to the root

otherwise keep the old approximation and update the second interval boundary

repeat until |c−b|<ε or f r =0.

6.1.7.2 Brent’s Method

In certain cases Dekker’s method converges very slowly making many small steps of the order ε. Brent [36, 37, 47] introduced some modifications to reduce such problems and tried to speed up convergence by the use of inverse quadratic interpolation (Sect. 6.1.6). To avoid numerical problems the iterate (6.24) is written with the help of a quotient

(6.35)

with

(6.36)

(6.37)

If only two points are available linear interpolation is used. The iterate (6.22) then is written as

(6.38)

with

(6.39)

The division is only performed if interpolation is appropriate and division by zero cannot happen. Brent’s method is fast and robust at the same time. It is often recommended by text books. The algorithm is summarized in the following [209].

Start with an initial interval [x ₀,x ₁] with f(x ₀)f(x ₁)≤0

initialization

iteration

If c is a better approximation than b exchange values

calculate midpoint relative to b

stop if accuracy is sufficient

use bisection if the previous step width e was too small or the last step did not improve

otherwise try interpolation

make p a positive quantity

update previous step width

use interpolation if applicable, otherwise use bisection

calculate new function value

be sure to bracket the root

6.1.7.3 Chandrupatla’s method

In 1997 Chandrupatla [54] published a method which tries to use inverse quadratic interpolation whenever possible according to (6.34). He calculates the relative position of the new iterate as (Fig. 6.9)

(6.40)

Fig. 6.9

Chandrupatla’s method

The algorithm proceeds as follows:

Start with an initial interval [x ₀,x ₁] with f(x ₀)f(x ₁)≤0

initialization

iteration

Chandrupatlas’s method is more efficient than Dekker’s and Brent’s, especially for higher order roots (Figs. 6.10, 6.11, 6.12).

Fig. 6.10

(Comparison of different solvers)  The root of the equation f(x)=x ²−2 is determined with different methods: Newton-Raphson (a, squares), Chandrupatla (b, circles), Brent (c, triangle up), Dekker (d, diamonds), regula falsi (e, stars), pure bisection (f, dots). Starting values are x ₁=−1,x ₂=2. The absolute error is shown as function of the number of iterations. For x ₁=−1, the Newton-Raphson method converges against $-\sqrt{2}$

Fig. 6.11

(Comparison of different solvers for a third order root)  The root of the equation f(x)=(x−1)³ is determined with different methods: Newton-Raphson (a), Chandrupatla (b), Brent (c), Dekker (d), regula falsi (e), pure bisection (f). Starting values are x ₁=0, x ₂=1.8. The absolute error is shown as function of the number of iterations

Fig. 6.12

(Comparison of different solvers for a high order root)  The root of the equation f(x)=x ²⁵ is determined with different methods: Newton-Raphson (a), Chandrupatla (b, circles), Brent (c), Dekker (d), regula falsi (e), pure bisection (f, dots). Starting values are x ₁=−1, x ₂=2. The absolute error is shown as function of the number of iterations

6.1.8 Multidimensional Root Finding

The Newton-Raphson method can be easily generalized for functions of more than one variable. We search for the solution of a system of n nonlinear equations in n variables x i

(6.41)

The first order Newton-Raphson method results from linearization of

(6.42)

with the Jacobian matrix

(6.43)

If the Jacobian matrix is not singular the equation

(6.44)

can be solved by

(6.45)

This can be repeated iteratively

(6.46)

6.1.9 Quasi-Newton Methods

Calculation of the Jacobian matrix can be very time consuming. Quasi-Newton methods use instead an approximation to the Jacobian which is updated during each iteration. Defining the differences

(6.47)

(6.48)

we obtain from the truncated Taylor series

(6.49)

the so called Quasi-Newton or secant condition

(6.50)

We attempt to construct a family of successive approximation matrices J r so that, if J were a constant, the procedure would become consistent with the Quasi-Newton condition. Then for the new update J r+1 we have

(6.51)

Since d (r),y (r) are already known, these are only n equations for the n ² elements of J r+1. To specify J r+1 uniquely, additional conditions are required. For instance, it is reasonable to assume, that

(6.52)

Then J r+1 differs from J r only by a rank one updating matrix

(6.53)

From the secant condition we obtain

(6.54)

hence

(6.55)

This gives Broyden’s update formula [42]

(6.56)

To update the inverse Jacobian matrix, the Sherman-Morrison formula [236]

(6.57)

can be applied to have

(6.58)

6.2 Function Minimization

Minimization or maximization of a function² is a fundamental task in numerical mathematics and closely related to root finding. If the function f(x) is continuously differentiable then at the extremal points the derivative is zero

(6.59)

Hence, in principle root finding methods can be applied to locate local extrema of a function. However, in some cases the derivative cannot be easily calculated or the function even is not differentiable. Then derivative free methods similar to bisection for root finding have to be used.

6.2.1 The Ternary Search Method

Ternary search is a simple method to determine the minimum of a unimodal function f(x). Initially we have to find an interval [a ₀,b ₀] which is certain to contain the minimum. Then the interval is divided into three equal parts [a ₀,c ₀], [c ₀,d ₀], [d ₀,b ₀] and either the first or the last of the three intervals is excluded (Fig. 6.13). The procedure is repeated with the remaining interval [a ₁,b ₁]=[a ₀,d ₀] or [a ₁,b ₁]=[c ₀,b ₀].

Fig. 6.13

Ternary search method

Each step needs two function evaluations and reduces the interval width by a factor of 2/3 until the maximum possible precision is obtained. It can be determined by considering a differentiable function which can be expanded around the minimum x ₀ as

(6.60)

Numerically calculated function values f(x) and f(x ₀) only differ, if

(6.61)

which limits the possible numerical accuracy to

(6.62)

and for reasonably well behaved functions (Fig. 6.14) we have the rule of thumb [207]

(6.63)

Fig. 6.14

(Ternary search method)  The minimum of the function f(x)=1+0.01x ²+0.1x ⁴ is determined with the ternary search method. Each iteration needs two function evaluations. After 50 iterations the function minimum f min=1 is reached to machine precision ε M ≈10⁻¹⁶. The position of the minimum x min cannot be determined with higher precision than $\sqrt{\varepsilon_{M}}\approx10^{-8}$ (6.63)

However, it may be impossible to reach even this precision, if the quadratic term of the Taylor series vanishes (Fig. 6.15).

Fig. 6.15

(Ternary search method for a higher order minimum)  The minimum of the function f(x)=1+0.1 x ⁴ is determined with the ternary search method. Each iteration needs two function evaluations. After 30 iterations the function minimum f min=1 is reached to machine precision ε M ≈10⁻¹⁶. The position of the minimum x min cannot be determined with higher precision than $\sqrt[4]{\varepsilon_{M}}\approx10^{-4}$

The algorithm can be formulated as follows:

iteration

6.2.2 The Golden Section Search Method (Brent’s Method)

To bracket a local minimum of a unimodal function f(x) three points a,b,c are necessary (Fig. 6.16) with

(6.64)

Fig. 6.16

(Golden section search method)  A local minimum of the function f(x) is bracketed by three points a, b, c. To reduce the uncertainty of the minimum position a new point ξ is chosen in the interval a<ξ<c and either a or c is dropped according to the relation of the function values. For the example shown a has to be replaced by ξ

The position of the minimum can be determined iteratively by choosing a new value ξ in the interval a<ξ<c and dropping either a or c, depending on the ratio of the function values. A reasonable choice for ξ can be found as follows (Fig. 6.17) [147, 207]. Let us denote the relative positions of the middle point and the trial point as

(6.65)

The relative width of the new interval will be

(6.66)

(6.67)

The golden search method requires that

(6.68)

Otherwise it would be possible that the larger interval width is selected many times slowing down the convergence. The value of t is positive if c−b>b−a and negative if c−b<b−a, hence the trial point always is in the larger of the two intervals. In addition the golden search method requires that the ratio of the spacing remains constant. Therefore we set

(6.69)

(6.70)

Eliminating t we obtain for a<ξ<b the equation

(6.71)

Besides the trivial solution β=1 there is only one positive solution

(6.72)

For b<ξ<c we end up with

(6.73)

which has the nontrivial positive solution

(6.74)

Hence the lengths of the two intervals [a,b], [b,c] have to be in the golden ratio $\varphi=\frac{1+\sqrt{5}}{2}\approx1.618$ which gives the method its name. Using the golden ratio the width of the interval bracketing the minimum reduces by a factor of 0.618 (Figs. 6.18, 6.19).

Fig. 6.17

Golden section search method

Fig. 6.18

(Golden section search method)  The minimum of the function f(x)=1+0.01x ²+0.1x ⁴ is determined with the golden section search method. Each iteration needs only one function evaluation. After 40 iterations the function minimum f min=1 is reached to machine precision ε M ≈10⁻¹⁶. The position of the minimum x min cannot be determined to higher precision than $\sqrt{\varepsilon_{M}}\approx10^{-8}$ (6.63)

Fig. 6.19

(Golden section search for a higher order minimum)  The minimum of the function f(x)=1+0.1 x ⁴ is determined with the golden section search method. Each iteration needs only one function evaluation. After 28 iterations the function minimum f min=1 is reached to machine precision ε M ≈10⁻¹⁶. The position of the minimum x min cannot be determined to higher precision than $\sqrt[4]{\varepsilon_{M}}\approx10^{-4}$

The algorithm can be formulated as follows:

To start the method we need three initial points which can be found by Brent’s exponential search method (Fig. 6.20). Begin with three points

(6.75)

where h ₀ is a suitable initial step width which depends on the problem. If the minimum is not already bracketed then if necessary exchange a ₀ and b ₀ to have

(6.76)

Then replace the three points by

(6.77)

and repeat this step until

(6.78)

or n exceeds a given maximum number. In this case no minimum can be found and we should check if the initial step width was too large.

Fig. 6.20

Brent’s exponential search

Brent’s method can be improved by making use of derivatives and by combining the golden section search with parabolic interpolation [207].

6.2.3 Minimization in Multidimensions

We search for local minima (or maxima) of a function

which is at least two times differentiable. In the following we denote the gradient vector by

(6.79)

and the matrix of second derivatives (Hessian) by

(6.80)

The very popular class of direction set methods proceeds as follows (Fig. 6.21). Starting from an initial guess x ₀ a set of direction vectors s r and step lengths λ r is determined such that the series of vectors

(6.81)

approaches the minimum of h(x). The method stops if the norm of the gradient becomes sufficiently small or if no lower function value can be found.

Fig. 6.21

(Direction set minimization)  Starting from an initial guess x ₀ a local minimum is approached by making steps along a set of direction vectors s r

6.2.4 Steepest Descent Method

The simplest choice, which is known as the method of gradient descent or steepest descent³ is to go in the direction of the negative gradient

(6.82)

and to determine the step length by minimizing h along this direction

(6.83)

Obviously two consecutive steps are orthogonal to each other since

(6.84)

This can lead to a zig-zagging behavior and a very slow convergence of this method (Figs. 6.22, 6.23).

Fig. 6.22

(Function minimization)  The minimum of the Rosenbrock function h(x,y)=100(y−x ²)²+(1−x)² is determined with different methods. Conjugate gradients converge much faster than steepest descent. Starting at (x,y)=(0,2), Newton-Raphson reaches the minimum at x=y=1 within only 5 iterations to machine precision

Fig. 6.23

(Minimization of the Rosenbrock function)  Left: Newton-Raphson finds the minimum after 5 steps within machine precision. Middle: conjugate gradients reduce the gradient to 4×10⁻¹⁴ after 265 steps. Right: The steepest descent method needs 20000 steps to reduce the gradient to 5×10⁻¹⁴. Red lines show the minimization pathway. Colored areas indicate the function value (light blue: <0.1, grey: 0.1…0.5, green: 5…50, pink: >100). Screenshots taken from Problem 6.2

6.2.5 Conjugate Gradient Method

This method is similar to the steepest descent method but the search direction is iterated according to

(6.85)

(6.86)

(6.87)

where λ r is chosen to minimize h(x r+1) and the simplest choice for β is made by Fletcher and Rieves [89]

(6.88)

6.2.6 Newton-Raphson Method

The first order Newton-Raphson method uses the iteration

(6.89)

The search direction is

(6.90)

and the step length is λ=1. This method converges fast if the starting point is close to the minimum. However, calculation of the Hessian can be very time consuming (Fig. 6.22).

6.2.7 Quasi-Newton Methods

Calculation of the full Hessian matrix as needed for the Newton-Raphson method can be very time consuming. Quasi-Newton methods use instead an approximation to the Hessian which is updated during each iteration. From the Taylor series

(6.91)

we obtain the gradient

(6.92)

Defining the differences

(6.93)

(6.94)

and neglecting higher order terms we obtain the Quasi-Newton or secant condition

(6.95)

We want to approximate the true Hessian by a series of matrices H r which are updated during each iteration to sum up all the information gathered so far. Since the Hessian is symmetric and positive definite, this also has to be demanded for the H r .⁴ This cannot be achieved by a rank one update matrix. Popular methods use a symmetric rank two update of the form

(6.96)

The Quasi-Newton condition then gives

(6.97)

hence H r d r −y r must be a linear combination of u and v. Making the simple choice

(6.98)

and assuming that these two vectors are linearly independent, we find

(6.99)

(6.100)

which together defines the very popular BFGS (Broyden, Fletcher, Goldfarb, Shanno) method [43, 86, 105, 235]

(6.101)

Alternatively the DFP method by Davidon, Fletcher and Powell, directly updates the inverse Hessian matrix B=H ⁻¹ according to

(6.102)

Both of these methods can be inverted with the help of the Sherman-Morrison formula to give

(6.103)

(6.104)

6.3 Problems

Problem 6.1

(Root finding methods)

This computer experiment searches roots of several test functions:

You can vary the initial interval or starting value and compare the behavior of different methods:

bisection

regula falsi

Dekker’s method

Brent’s method

Chandrupatla’s method

Newton-Raphson method

Problem 6.2

(Stationary points)

This computer experiment searches a local minimum of the Rosenbrock function⁵

(6.105)

The method of steepest descent minimizes h(x,y) along the search direction

(6.106)

(6.107)

Conjugate gradients make use of the search direction

(6.108)

(6.109)

The Newton-Raphson method needs the inverse Hessian

(6.110)

(6.111)

(6.112)

and iterates according to

(6.113)

You can choose an initial point (x ₀,y ₀). The iteration stops if the gradient norm falls below 10⁻¹⁴ or if the line search fails to find a lower function value.

References

36.

R.P. Brent, Comput. J. 14, 422 (1971) MathSciNetMATHCrossRef

37.

R.P. Brent, Algorithms for Minimization without Derivatives (Prentice Hall, Englewood Cliffs, 1973), Chap. 4 MATH

42.

C.G. Broyden, Math. Comput. 19, 577 (1965) MathSciNetMATHCrossRef

43.

C.G. Broyden, J. Inst. Math. Appl. 6, 76 (1970) MathSciNetMATHCrossRef

45.

R.L. Burden, J.D. Faires, Numerical Analysis (Brooks Cole, Boston, 2010), Chap. 2

47.

J.C.P. Bus, T.J. Dekker, ACM Trans. Math. Softw. 1, 330 (1975) MathSciNetMATHCrossRef

54.

T.R. Chandrupatla, Adv. Eng. Softw. 28, 145 (1997) CrossRef

70.

T.J. Dekker, Finding a zero by means of successive linear interpolation, in Constructive Aspects of the Fundamental Theorem of Algebra, ed. by B. Dejon, P. Henrici (Wiley-Interscience, London, 1969)

86.

R. Fletcher, Comput. J. 13, 317 (1970) MATHCrossRef

89.

R. Fletcher, C. Reeves, Comput. J. 7, 149 (1964) MathSciNetMATHCrossRef

92.

J.A. Ford, Improved Algorithms of Illinois-Type for the Numerical Solution of Nonlinear Equations (University of Essex Press, Essex, 1995), Technical Report CSM-257

105.

D. Goldfarb, Math. Comput. 24, 23 (1970) MathSciNetMATHCrossRef

147.

J. Kiefer, Proc. Am. Math. Soc. 4, 502–506 (1953) MathSciNetMATHCrossRef

177.

D.E. Muller, Math. Tables Other Aids Comput. 10, 208 (1956) MATHCrossRef

196.

J.M. Papakonstantinou, A historical development of the (n+1)-point secant method. M.A. thesis, Rice University Electronic Theses and Dissertations, 2007

207.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Golden section search in one dimension, in Numerical Recipes: The Art of Scientific Computing, 3rd edn. (Cambridge University Press, New York, 2007). ISBN 978-0-521-88068-8, Sect. 10.2

209.

A. Quarteroni, R. Sacco, F. Saleri, Numerical Mathematics, 2nd edn. (Springer, Berlin, 2007) MATH

235.

D.F. Shanno, Math. Comput. 24, 647 (1970) MathSciNetCrossRef

236.

J. Sherman, W.J. Morrison, Ann. Math. Stat. 20, 621 (1949)

252.

J.Y. Tjalling, Historical development of the Newton-Raphson method. SIAM Rev. 37, 531 (1995) MathSciNetMATHCrossRef

Footnotes

1

Usually a combination like ε=2ε M +|b n |ε r of an absolute and a relative tolerance is taken.

2

In the following we consider only a minimum since a maximum could be found as the minimum of −f(x).

3

Which should not be confused with the method of steepest descent for the approximate calculation of integrals.

4

This is a major difference to the Quasi-Newton methods for root finding (6.1.9).

5

A well known test function for minimization algorithms.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_7© Springer International Publishing Switzerland 2013

7. Fourier Transformation

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Fourier transformation is a very important tool for signal analysis but also helpful to simplify the solution of differential equations or the calculation of convolution integrals. An important numerical method is the discrete Fourier transformation which can be used for trigonometric interpolation and also as a numerical approximation to the continuous Fourier integral. It can be realized efficiently by Goertzel’s algorithm or the family of fast Fourier transformation methods. For real valued even functions the computationally simpler discrete cosine transformation can be applied. Several computer experiments demonstrate the principles of trigonometric interpolation and nonlinear filtering.

Fourier transformation is a very important tool for signal analysis but also helpful to simplify the solution of differential equations or the calculation of convolution integrals. In this chapter we discuss the discrete Fourier transformation as a numerical approximation to the continuous Fourier integral. It can be realized efficiently by Goertzel’s algorithm or the family of fast Fourier transformation methods. Computer experiments demonstrate trigonometric interpolation and nonlinear filtering as applications.

7.1 Fourier Integral and Fourier Series

We use the symmetric definition of the Fourier transformation:

(7.1)

The inverse Fourier transformation

(7.2)

decomposes f(t) into a superposition of oscillations. The Fourier transform of a convolution integral

(7.3)

becomes a product of Fourier transforms:

(7.4)

A periodic function with f(t+T)=f(t)¹ is transformed into a Fourier series

(7.5)

For a periodic function which in addition is real valued f(t)=f(t)∗ and even f(t)=f(−t), the Fourier series becomes a cosine series

(7.6)

with real valued coefficients

(7.7)

7.2 Discrete Fourier Transformation

We divide the time interval 0≤t<T by introducing a grid of N equidistant points

(7.8)

The function values (samples)

(7.9)

are arranged as components of a vector

With respect to the orthonormal basis

(7.10)

f is expressed as a linear combination

(7.11)

The discrete Fourier transformation is the transformation to an orthogonal base in frequency space

(7.12)

with

(7.13)

These vectors are orthogonal

(7.14)

(7.15)

Alternatively a real valued basis can be defined:

(7.16)

The components of f in frequency space are given by the scalar product

(7.17)

From

(7.18)

we find the inverse transformation

(7.19)

7.2.1 Trigonometric Interpolation

The last equation can be interpreted as an interpolation of the function f(t) at the sampling points t n by a linear combination of trigonometric functions

(7.20)

which is a polynomial of

(7.21)

Since

(7.22)

the frequencies ω j and ω N−j are equivalent (Fig. 7.1)

(7.23)

Fig. 7.1

(Equivalence of ω ₁ and ω N−1) The two functions cosωt and cos(N−1)ωt have the same values at the sample points t n but are very different in between

If we use trigonometric interpolation to approximate f(t) between the grid points, the two frequencies are no longer equivalent and we have to restrict the frequency range to avoid unphysical high frequency components (Fig. 7.2):

(7.24)

The interpolating function (N even) is

(7.25)

(7.26)

The maximum frequency is

(7.27)

and hence

(7.28)

This is known as the sampling theorem which states that the sampling frequency f s must be larger than twice the maximum frequency present in the signal.

Fig. 7.2

(Trigonometric interpolation) For trigonometric interpolation the high frequencies have to be replaced by the corresponding negative frequencies to provide meaningful results between the sampling points. The circles show sampling points which are fitted using only positive frequencies (full curve) or replacing the unphysical high frequency by its negative counterpart (broken curve). The squares show the calculated Fourier spectrum

7.2.2 Real Valued Functions

For a real valued function

(7.29)

and hence

(7.30)

Here it is sufficient to calculate the sums for j=0⋯N/2. If the function is real valued and also even

(7.31)

(7.32)

and the Fourier sum (7.19) turns into a cosine sum

(7.33)

(7.34)

which correspond to two out of eight different versions [268] of the discrete cosine transformation [2, 211].

Equation (7.34) can be used to define the interpolating function

(7.35)

The real valued Fourier coefficients are given by

(7.36)

(7.37)

7.2.3 Approximate Continuous Fourier Transformation

We continue the function f(t) periodically by setting

(7.38)

and write

(7.39)

Comparing with the trapezoidal rule (4.​13) for the integral

(7.40)

we find

(7.41)

which shows that the discrete Fourier transformation is an approximation to the Fourier series of a periodic function with period T which coincides with f(t) in the interval 0<t<T. The range of the integral can be formally extended to ±∞ by introducing a windowing function

(7.42)

The discrete Fourier transformation approximates the continuous Fourier transformation but windowing leads to a broadening of the spectrum. For practical purposes smoother windowing functions are used like a triangular window or one of the following [119]:

7.3 Fourier Transform Algorithms

Straightforward evaluation of the sum

(7.43)

needs O(N ²) additions, multiplications and trigonometric functions.

7.3.1 Goertzel’s Algorithm

Goertzel’s method [103] is very useful if not the whole Fourier spectrum is needed but only some of the Fourier components, for instance to demodulate a frequency shift key signal or the dial tones which are used in telephony.

The Fourier transform can be written as

(7.44)

which can be evaluated recursively

(7.45)

to give the result

(7.46)

Equation (7.45) is a simple discrete filter function. Its transmission function is obtained by application of the z-transform [144]

(7.47)

(the discrete version of the Laplace transform) which yields

(7.48)

One disadvantage of this method is that it uses complex numbers. This can be avoided by the following more complicated recursion

(7.49)

with the transmission function

(7.50)

A second filter removes one factor in the denominator

(7.51)

which in the time domain corresponds to the simple expression

The overall filter function finally again is (7.48).

(7.52)

Hence the Fourier component of f is given by

(7.53)

The order of the iteration (7.44) can be reversed by writing

(7.54)

which is very useful for real time filter applications.

7.3.2 Fast Fourier Transformation

If the number of samples is N=2 p , the Fourier transformation can be performed very efficiently by this method.² The phase factor

(7.55)

can take only N different values. The number of trigonometric functions can be reduced by reordering the sum. Starting from a sum with N samples

(7.56)

we separate even and odd powers of the unit root

(7.57)

This division is repeated until only sums with one summand remain

(7.58)

For example, consider the case N=8:

Expansion gives

(7.59)

Generally a summand of the Fourier sum can be written using the binary representation of n

(7.60)

in the following way:

(7.61)

The function values are reordered according to the following algorithm

(i)

count from 0 to N−1 using binary numbers m=000,001,010,⋯

(ii)

bit reversal gives the binary numbers n=000,100,010,⋯

(iii)

store f n at the position m. This will be denoted as s m =f n

As an example for N=8 the function values are in the order

(7.62)

Now calculate sums with two summands. Since $W_{2}^{j}$ can take only two different values

(7.63)

a total of 8 sums have to be calculated which can be stored again in the same workspace:

(7.64)

Next calculate sums with four summands. $W_{4}^{j}$ can take one of four values

(7.65)

The following combinations are needed:

(7.66)

The next step gives the sums with eight summands. With

(7.67)

we calculate

(7.68)

which is the final result.

The following shows a simple Fast Fourier Transformation algorithm. The number of trigonometric function evaluations can be reduced but this reduces the readability. At the beginning Data[k] are the input data in bit reversed order.

7.4 Problems

Problem 7.1

(Discrete Fourier transformation)

In this computer experiment for a given set of input samples

(7.69)

the Fourier coefficients

(7.70)

are calculated with Goertzel’s method (7.3.1).

The results from the inverse transformation

(7.71)

are compared with the original function values f(t n ).

The Fourier sum is used for trigonometric interpolation with only positive frequencies

(7.72)

Finally the unphysical high frequencies are replaced by negative frequencies (7.24). The results can be studied for several kinds of input data.

Problem 7.2

(Noise filter)

This computer experiment demonstrates a nonlinear filter.

First a noisy input signal is generated.

The signal can be chosen as

monochromatic sin(ωt)

the sum of two monochromatic signals a ₁sinω ₁ t+a ₂sinω ₂ t

a rectangular signal with many harmonic frequencies sign(sinωt)

Different kinds of white noise can be added

dichotomous ±1

constant probability density in the range [−1,1]

Gaussian probability density

The amplitudes of signal and noise can be varied. All Fourier components are removed which are below a threshold value and the filtered signal is calculated by inverse Fourier transformation.

References

2.

N. Ahmed, T. Natarajan, K.R. Rao, IEEE Trans. Comput. 23, 90 (1974) MathSciNetMATHCrossRef

62.

J.W. Cooley, J.W. Tukey, Math. Comput. 19, 297 (1965) MathSciNetMATHCrossRef

74.

P. Duhamel, M. Vetterli, Signal Process. 19, 259 (1990) MathSciNetMATHCrossRef

103.

G. Goertzel, Am. Math. Mon. 65, 34 (1958) MathSciNetMATHCrossRef

119.

F.j. Harris, Proc. IEEE 66, 51 (1978) ADSCrossRef

144.

E.I. Jury, Theory and Application of the Z-Transform Method (Krieger, Melbourne, 1973). ISBN 0-88275-122-0

187.

H.J. Nussbaumer, Fast Fourier Transform and Convolution Algorithms (Springer, Berlin, 1990)

211.

K.R. Rao, P. Yip, Discrete Cosine Transform: Algorithms, Advantages, Applications (Academic Press, Boston, 1990) MATH

268.

Z. Wang, B.R. Hunt, Appl. Math. Comput. 16, 19 (1985) MathSciNetMATHCrossRef

Footnotes

1

This could also be the periodic continuation of a function which is only defined for 0<t<T.

2

There exist several Fast Fourier Transformation algorithms [74, 187]. We consider only the simplest one here [62].

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_8© Springer International Publishing Switzerland 2013

8. Random Numbers and Monte Carlo Methods

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Many-body problems often involve the calculation of integrals of very high dimension which cannot be treated by standard methods. For the calculation of thermodynamic averages Monte Carlo methods are very useful which sample the integration volume at randomly chosen points. After summarizing some basic statistics, we discuss algorithms for the generation of pseudo-random numbers with given probability distribution which are essential for all Monte Carlo methods. We show how the efficiency of Monte Carlo integration can be improved by sampling preferentially the important configurations. Finally the famous Metropolis algorithm is applied to classical many-particle systems. Computer experiments visualize the central limit theorem and apply the Metropolis method to the traveling salesman problem.

Many-body problems often involve the calculation of integrals of very high dimension which cannot be treated by standard methods. For the calculation of thermodynamic averages Monte Carlo methods [49, 85, 174, 220] are very useful which sample the integration volume at randomly chosen points. In this chapter we discuss algorithms for the generation of pseudo-random numbers with given probability distribution which are essential for all Monte Carlo methods. We show how the efficiency of Monte Carlo integration can be improved by sampling preferentially the important configurations. Finally the famous Metropolis algorithm is applied to classical many-particle systems and nonlinear optimization problems.

8.1 Some Basic Statistics

In the following we discuss some important concepts which are used to analyze experimental data sets [218]. Repeated measurements of some observable usually give slightly different results due to fluctuations of the observable in time and errors of the measurement process. The distribution of the measured data is described by a probability distribution, which in many cases approximates a simple mathematical form like the Gaussian normal distribution. The moments of the probability density give important information about the statistical properties, especially the mean and the standard deviation of the distribution. If the errors of different measurements are uncorrelated, the average value of a larger number of measurements is a good approximation to the “exact” value.

8.1.1 Probability Density and Cumulative Probability Distribution

Consider an observable ξ, which is measured in a real or a computer experiment. Repeated measurements give a statistical distribution of values.

The cumulative probability distribution (Fig. 8.1) is given by the function

(8.1)

and has the following properties:

F(x) is monotonously increasing

F(−∞)=0, F(∞)=1

F(x) can be discontinuous (if there are discrete values of ξ)

The probability to measure a value in the interval x ₁<ξ≤x ₂ is

(8.2)

The height of a jump gives the probability of a discrete value

(8.3)

In regions where F(x) is continuous, the probability density can be defined as

(8.4)

Fig. 8.1

(Cumulative probability distribution of transition energies)  The figure shows schematically the distribution of transition energies for an atom which has a discrete and a continuous part

8.1.2 Histogram

From an experiment F(x) cannot be determined directly. Instead a finite number N of values x i are measured. By

we denote the number of measurements with x i ≤x. The cumulative probability distribution is the limit

(8.5)

A histogram (Fig. 8.2) counts the number of measured values which are in the interval x i <x≤x i+1:

(8.6)

Contrary to Z N (x) itself, the histogram depends on the choice of the intervals.

Fig. 8.2

(Histogram)  The cumulative distribution of 100 Gaussian random numbers is shown together with a histogram with bin width Δx=0.6

8.1.3 Expectation Values and Moments

The expectation value of the random variable ξ is defined by

(8.7)

with the Riemann-Stieltjes integral [202]

(8.8)

Higher moments are defined as

(8.9)

if these integrals exist. Most important are the expectation value

(8.10)

and the variance, which results from the first two moments

(8.11)

The standard deviation σ is a measure of the width of the distribution. The expectation value of a function φ(x) is defined by

(8.12)

For continuous F(x) we have with dF(x)=f(x) dx the ordinary integral

(8.13)

(8.14)

whereas for a pure step function F(x) (only discrete values x i are observed with probabilities p(x i )=F(x i +0)−F(x i −0))

(8.15)

(8.16)

8.1.4 Example: Fair Die

When a six-sided fair die is rolled, each of its sides shows up with the same probability of 1/6. The cumulative probability distribution F(x) is a pure step function (Fig. 8.3) and

(8.17)

(8.18)

(8.19)

Fig. 8.3

Cumulative probability distribution of a fair die

8.1.5 Normal Distribution

The Gaussian normal distribution is defined by the cumulative probability distribution

(8.20)

and the probability density

(8.21)

with the properties

(8.22)

(8.23)

(8.24)

Since $\varPhi(0)=\frac{1}{2}$ and with the definition

(8.25)

we have

(8.26)

which can be expressed in terms of the error function¹

(8.27)

as

(8.28)

A general Gaussian distribution with mean value $\overline{x}$ and standard deviation σ has the probability distribution

(8.29)

and the cumulative distribution

(8.30)

(8.31)

8.1.6 Multivariate Distributions

Consider now two quantities which are measured simultaneously. ξ and η are the corresponding random variables. The cumulative distribution function is

(8.32)

Expectation values are defined as

(8.33)

For continuous F(x,y) the probability density is

(8.34)

and the expectation value is simply

(8.35)

The moments of the distribution are the expectation values

(8.36)

Most important are the averages

(8.37)

and the covariance matrix

(8.38)

The correlation coefficient is defined as

(8.39)

If there is no correlation then ρ=0 and F(x,y)=F ₁(x)F ₂(y).

8.1.7 Central Limit Theorem

Consider N independent random variables ξ i with the same cumulative distribution function F(x), for which E[ξ]=0 and E[ξ ²]=1. Define a new random variable

(8.40)

with the cumulative distribution function F N (x). In the limit N→∞ this distribution approaches (Fig. 8.4) a cumulative normal distribution [215]

(8.41)

Fig. 8.4

(Central limit theorem)  The cumulative distribution function of η (8.42) is shown for N=4 and compared to the normal distribution (8.20)

8.1.8 Example: Binomial Distribution

Toss a coin N times giving ξ i =1 (heads) or ξ i =−1 (tails) with equal probability $P=\frac{1}{2}$. Then E[ξ i ]=0 and $E[\xi_{i}^{2}]=1$. The distribution of

(8.42)

can be derived from the binomial distribution

(8.43)

where p counts the number of tosses with ξ=+1. Since

(8.44)

the probability of finding $\eta=\frac{n}{\sqrt{N}}$ is given by the binomial coefficient

(8.45)

or

(8.46)

8.1.9 Average of Repeated Measurements

A quantity X is measured N times. The results X ₁⋯X N are independent random numbers with the same distribution function f(X i ). Their expectation value is the exact value E[X i ]=∫dX i  X i f(X i )=X and the standard deviation due to measurement uncertainties is $\sigma _{X}=\sqrt{E[X_{i}^{2}]-X^{2}}$. The new random variables

(8.47)

have zero mean

(8.48)

and unit standard deviation

(8.49)

Hence the quantity

(8.50)

obeys a normal distribution

(8.51)

From

(8.52)

we obtain

(8.53)

The average of N measurements obeys a Gaussian distribution around the exact value X with a reduced standard deviation of

(8.54)

8.2 Random Numbers

True random numbers of high quality can be generated using physical effects like thermal noise in a diode or from a light source. Special algorithms are available to generate pseudo random numbers which have comparable statistical properties but are not unpredictable since they depend on some initial seed values. Often an iteration

(8.55)

is used to calculate a series of pseudo random numbers. Using 32 Bit integers there are 2³² different numbers, hence the period cannot exceed 2³².

8.2.1 Linear Congruent Mapping

A simple algorithm is the linear congruent mapping

(8.56)

with maximum period m. Larger periods can be achieved if the random number depends on several predecessors. A function of the type

(8.57)

using 32 Bit integers has a maximum period of 232t .

Example

For t=2 and generating 10⁶ numbers per second the period is 584942 years.

8.2.2 Marsaglia-Zamann Method

A high quality random number generator can be obtained from the combination of two generators [168]. The first one

(8.58)

with

(8.59)

has a period of 2⁹⁵. The second one

(8.60)

has a period of 2³². The period of the combination is 2¹²⁷. Here is a short subroutine in C:

#define N 100000

typedef unsigned long int unlong /* 32 Bit */

unlong x=521288629, y=362436069, z=16163801, c=1, n=1131199209;

unlong mzran()

{ unlong s;

 if (y>x+c) {s=y-(x+c)-18; c=0;}

 else {s=y-(x+c)-18;c=1;}

 x=y; y=z; z=s; n=69069*n+1013904243;

 return(z+n);

}

8.2.3 Random Numbers with Given Distribution

Assume we have a program that generates random numbers in the interval [0,1] like in C:

The corresponding cumulative distribution function is

(8.61)

Random numbers with cumulative distribution F(x) can be obtained as follows:

F(x) increases monotonously and therefore

(8.62)

but since 0≤F(x)≤1 we have

(8.63)

This method of course is applicable only if F ⁻¹ can be expressed analytically.

8.2.4 Examples

8.2.4.1 Fair Die

A six-sided fair die can be simulated as follows:

choose a random number r∈[0,1]

Let

8.2.4.2 Exponential Distribution

The cumulative distribution function

(8.64)

which corresponds to the exponential probability density

(8.65)

can be inverted by solving

(8.66)

for x:

8.2.4.3 Random Points on the Unit Sphere

We consider the surface element

(8.67)

Our aim is to generate points on the unit sphere (θ,φ) with the probability density

(8.68)

The corresponding cumulative distribution is

(8.69)

Since this factorizes, the two angles can be determined independently:

8.2.4.4 Gaussian Distribution (Box Muller)

For a Gaussian distribution the inverse F ⁻¹ has no simple analytical form. The famous Box Muller method [34] is based on a 2-dimensional normal distribution with probability density

(8.70)

which reads in polar coordinates

(8.71)

Hence

(8.72)

and the cumulative distribution factorizes:

(8.73)

The inverse of F ρ is

(8.74)

and the following algorithm generates Gaussian random numbers:

8.3 Monte Carlo Integration

Physical problems often involve high dimensional integrals (for instance path integrals, thermodynamic averages) which cannot be evaluated by standard methods. Here Monte Carlo methods can be very useful. Let us start with a very basic example.

8.3.1 Numerical Calculation of π

The area of a unit circle (r=1) is given by r ² π=π. Hence π can be calculated by numerical integration. We use the following algorithm:

choose N points randomly in the first quadrant, for instance N independent pairs x,y∈[0,1]

calculate r ²=x ²+y ²

count the number of points within the circle, i.e. the number of points Z(r ²≤1).

$\frac{\pi}{4}$ is approximately given by $\frac{Z(r^{2}\leq1)}{N}$.

The result converges rather slowly (Figs. 8.5, 8.6)

Fig. 8.5

Convergence of the numerical integration

Fig. 8.6

Error of the numerical integration

8.3.2 Calculation of an Integral

Let ξ be a random variable in the interval [a,b] with the distribution

(8.75)

The expectation value of a function g(x) is

(8.76)

hence the average of N randomly taken function values approximates the integral

(8.77)

To estimate the error we consider the new random variable

(8.78)

Its average is

(8.79)

and the variance follows from

(8.80)

(8.81)

The width of the distribution and hence the uncertainty falls off as $1/\sqrt{N}$.

8.3.3 More General Random Numbers

Consider now random numbers ξ∈[a,b] with arbitrary (but within [a,b] not vanishing) probability density f(x). The integral is approximated by

(8.82)

The new random variable

(8.83)

according to (8.81) has a standard deviation given by

(8.84)

which can be reduced by choosing f similar to g. Then preferentially ξ are generated in regions where the integrand is large (importance sampling).

8.4 Monte Carlo Method for Thermodynamic Averages

Consider the partition function of a classical N particle system

(8.85)

with an energy function

(8.86)

If the potential energy does not depend on the momenta and for equal masses the partition function simplifies to

(8.87)

and it remains the calculation of the configuration integral

(8.88)

In the following we do not need the partition function itself but only averages of some quantity A(q) given by

(8.89)

8.4.1 Simple Sampling

Let ξ be a random variable with probability distribution

(8.90)

(8.91)

We choose M random numbers ξ and calculate the expectation value of A(ξ) from

(8.92)

Consider now the case of random numbers ξ equally distributed over the range q min⋯q max:

(8.93)

Define a sample by choosing one random value ξ for each of the 3N coordinates. The average over a large number M of samples gives the expectation value

(8.94)

as

(8.95)

Hence

(8.96)

is an approximation to the average of A(q i ), if the range of the q i is sufficiently large. However, many of the samples will have small weight and contribute only little.

8.4.2 Importance Sampling

Let us try to sample preferentially the most important configurations. Choose the distribution function as

(8.97)

The expectation value of A(q) approximates the thermal average

(8.98)

and the partition function is not needed explicitly for the calculation.

8.4.3 Metropolis Algorithm

The algorithm by Metropolis [175] can be used to select the necessary configurations. Starting from an initial configuration $\mathbf{q}_{0}=(q_{1}^{(0)}\cdots q_{3N}^{(0)})$ a chain of configurations is generated. Each configuration depends only on its predecessor, hence the configurations form a Markov chain.

The transition probabilities

(8.99)

are chosen to fulfill the condition of detailed balance (Fig. 8.7)

(8.100)

This is a sufficient condition that the configurations are generated with probabilities given by their Boltzmann factors. This can be seen from consideration of an ensemble of such Markov chains: Let N n (q i ) denote the number of chains which are in the configuration q i after n steps. The changes during the following step are

(8.101)

In thermal equilibrium

and the changes (8.101) vanish:

(8.102)

A solution of

(8.103)

corresponds to a zero eigenvalue of the system of equations

(8.104)

One solution of this eigenvalue equation is given by

(8.105)

However, there may be other solutions. For instance if not all configurations are connected by possible transitions and some isolated configurations are occupied initially.

Fig. 8.7

Principle of detailed balance

Metropolis Algorithm

This famous algorithm consists of the following steps:

(a)

choose a new configuration randomly (trial step) with probability

(b)

calculate

(c)

if R≥1 the trial step is accepted q i+1=q trial

(d)

if R<1 the trial step is accepted only with probability R. Choose a random number ξ∈[0,1] and the next configuration according to

The transition probability is the product

(8.106)

of the probability T i→j to select i→j as a trial step and the probability A i→j to accept the trial step. Now we have

(8.107)

Since T i→j =T j→i , in both cases

(8.108)

8.5 Problems

Problem 8.1

(Central limit theorem)

This computer experiment draws a histogram for the random variable τ, which is calculated from N random numbers ξ ₁⋯ξ N :

(8.109)

The ξ i are random numbers with zero mean and unit variance and can be chosen as

ξ i =±1 (coin tossing)

Gaussian random numbers

Investigate how a Gaussian distribution is approached for large N.

Problem 8.2

(Nonlinear optimization)

MC methods can be used for nonlinear optimization (Traveling salesman problem, structure optimization etc.) [31]. Consider an energy function depending on many coordinates

(8.110)

Introduce a fictitious temperature T and generate configurations with probabilities

(8.111)

Slow cooling drives the system into a local minimum. By repeated heating and cooling other local minima can be reached (simulated annealing).

In this computer experiment we try to find the shortest path which visits each of up to N=50 given points. The fictitious Boltzmann factor for a path with total length L is

(8.112)

Starting from an initial path S=(i ₁,i ₂⋯i N ) n<5 and p are chosen randomly and a new path S′=(i ₁⋯i p−1,i p+n ⋯i p ,i p+n+1⋯i N ) is generated by reverting the sub-path

Start at high temperature T>L and cool down slowly.

References

31.

E. Bonomi, J.-L. Lutton, SIAM Rev. 26, 551 (1984) MathSciNetMATHCrossRef

34.

G.E.P. Box, M.E. Muller, Ann. Math. Stat. 29, 610 (1958) MATHCrossRef

49.

R.E. Caflisch, Monte Carlo and quasi-Monte Carlo methods. Acta Numer. 7, 1–49 (1998) MathSciNetCrossRef

85.

G.S. Fishman, Monte Carlo: Concepts, Algorithms, and Applications (Springer, New York, 1995). ISBN 038794527X

168.

G. Marsaglia, A. Zaman, Ann. Appl. Probab. 1, 462 (1991) MathSciNetMATHCrossRef

174.

N. Metropolis, S. Ulam, J. Am. Stat. Assoc. 44, 335 (1949) MathSciNetMATHCrossRef

175.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, J. Chem. Phys. 21, 1087 (1953) ADSCrossRef

202.

H. Pollard, Q. J. Pure Appl. Math. 49, 1 (1920)

215.

J. Rice, Mathematical Statistics and Data Analysis, 2nd edn. (Duxbury Press, N. Scituate, 1995). ISBN 0-534-20934-3 MATH

218.

R.D. Richtmyer, Principles of Modern Mathematical Physics I (Springer, New York, 1978) CrossRef

220.

C.P. Robert, G. Casella, Monte Carlo Statistical Methods, 2nd edn. (Springer, New York, 2004). ISBN 0387212396 MATHCrossRef

Footnotes

1

erf(x) is an intrinsic function in FORTRAN or C.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_9© Springer International Publishing Switzerland 2013

9. Eigenvalue Problems

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Eigenvalue problems are omnipresent in physics. Important examples are the time independent Schrödinger equation in a finite orthogonal basis or the harmonic motion of a molecule around its equilibrium structure. Most important are ordinary eigenvalue problems, which involve the solution of a homogeneous system of linear equations with a Hermitian (or symmetric, if real) matrix.

Matrices of small dimension can be diagonalized directly by determining the roots of the characteristic polynomial and solving a homogeneous system of linear equations. The Jacobi method uses successive rotations to diagonalize a matrix with a unitary transformation. A very popular method for not too large symmetric matrices reduces the matrix to tridiagonal form which can be diagonalized efficiently with the QL algorithm. Certain tridiagonal matrices can be diagonalized analytically. Special algorithms are available for matrices of very large dimension, for instance the famous Lanczos method.

Eigenvalue problems are omnipresent in physics. Important examples are the time independent Schrödinger equation in a finite orthogonal basis (Chap. 9)

(9.1)

or the harmonic motion of a molecule around its equilibrium structure (Sect. 14.​4.​1)

(9.2)

Most important are ordinary eigenvalue problems,¹ which involve the solution of a homogeneous system of linear equations

(9.3)

with a Hermitian (or symmetric, if real) matrix [198]

(9.4)

Matrices of small dimension can be diagonalized directly by determining the roots of the characteristic polynomial and solving a homogeneous system of linear equations. The Jacobi method uses successive rotations to diagonalize a matrix with a unitary transformation. A very popular method for not too large symmetric matrices reduces the matrix to tridiagonal form which can be diagonalized efficiently with the QL algorithm. Some special tridiagonal matrices can be diagonalized analytically. Special algorithms are available for matrices of very large dimension, for instance the famous Lanczos method.

9.1 Direct Solution

For matrices of very small dimension (2,3) the determinant

(9.5)

can be written explicitly as a polynomial of λ. The roots of this polynomial are the eigenvalues. The eigenvectors are given by the system of equations

(9.6)

9.2 Jacobi Method

Any symmetric 2×2 matrix

(9.7)

can be diagonalized by a rotation of the coordinate system. Rotation by the angle φ corresponds to an orthogonal transformation with the rotation matrix

(9.8)

In the following we use the abbreviations

(9.9)

The transformed matrix is

(9.10)

It is diagonal if

(9.11)

or

(9.12)

Calculation of φ is not necessary since only its cosine and sine appear in (9.10). From [198]

(9.13)

we see that t is a root of

(9.14)

hence

(9.15)

For reasons of convergence [198] the solution with smaller magnitude is chosen which can be written as

(9.16)

again for reasons of convergence the smaller solution φ is preferred and therefore we take

(9.17)

The diagonal elements of the transformed matrix are

(9.18)

(9.19)

The trace of the matrix is invariant

(9.20)

whereas the difference of the diagonal elements is

(9.21)

and the transformed matrix has the simple form

(9.22)

For larger dimension N>2 the Jacobi method uses the following algorithm:

(1)

look for the dominant non-diagonal element max i≠j |a ij |

(2)

perform a rotation in the (ij)-plane to cancel the element $\tilde{a}_{ij}$ of the transformed matrix $\tilde{A}=R^{(ij)}\cdot A\cdot R^{(ij)-1}$. The corresponding rotation matrix has the form

(9.23)

(3)

repeat (1–2) until convergence (if possible).

The sequence of Jacobi rotations gives the over all transformation

(9.24)

Hence

(9.25)

and the column vectors of R ⁻¹=(v ₁,v ₂⋯v N ) are the eigenvectors of A:

(9.26)

9.3 Tridiagonal Matrices

A tridiagonal matrix has nonzero elements only in the main diagonal and the first diagonal above and below. Many algorithms simplify significantly when applied to tridiagonal matrices.

9.3.1 Characteristic Polynomial of a Tridiagonal Matrix

The characteristic polynomial of a tridiagonal matrix

(9.27)

can be calculated recursively:

(9.28)

9.3.2 Special Tridiagonal Matrices

Certain classes of tridiagonal matrices can be diagonalized exactly [55, 151, 281].

9.3.2.1 Discretized Second Derivatives

Discretization of a second derivative involves, under Dirichlet boundary conditions f(x ₀)=f(x N+1)=0, the differentiation matrix (Sect. 18.​2)

(9.29)

Its eigenvectors have the form

(9.30)

This can be seen by inserting (9.30) into the n-th line of the eigenvalue equation (9.31)

(9.31)

(9.32)

with the eigenvalue

(9.33)

The first line of the eigenvalue equation (9.31) reads

(9.34)

and from the last line we have

(9.35)

which holds if

(9.36)

This simplifies to

(9.37)

Hence the possible values of k are

(9.38)

and the eigenvectors are explicitly (Fig. 9.1)

(9.39)

Fig. 9.1

(Lowest eigenvector for fixed and open boundaries) Top: for fixed boundaries f n =sin(nk) which is zero at the additional points x ₀,x N+1. For open boundaries f n =cos((n−1)k) with horizontal tangent at x ₁,x N due to the boundary conditions f ₂=f ₀, f N−1=f N+1

For Neumann boundary conditions $\frac{\partial f}{\partial x}(x_{1})=\frac{\partial f}{\partial x}(x_{N})=0$ the matrix is slightly different (Sect. 18.​2)

(9.40)

Its eigenvalues are also given by the expression (9.33). To obtain the eigenvectors, we try a more general ansatz with a phase shift

(9.41)

Obviously

(9.42)

The first and last lines of the eigenvalue equation give

(9.43)

and

(9.44)

which is solved by

(9.45)

hence finally the eigenvector is (Fig. 9.1)

(9.46)

Even simpler is the case of the corresponding cyclic tridiagonal matrix

(9.47)

which has eigenvectors

(9.48)

and eigenvalues

(9.49)

where the possible k−values again follow from the first and last line

(9.50)

(9.51)

which both lead to

(9.52)

(9.53)

9.3.2.2 Discretized First Derivatives

Using symmetric differences to discretize a first derivative in one dimension leads to the matrix²

(9.54)

The characteristic polynomial of the Hermitian matrix iD is given by the recursion (9.3.1)

(9.55)

which after the substitution x=−λ/2 is exactly the recursion for the Chebyshev polynomial of the second kind U N (x). Hence the eigenvalues of D are given by the roots x k of U N (x) as

(9.56)

The eigenvalues of the corresponding cyclic tridiagonal matrix

(9.57)

are easy to find. Inserting the ansatz for the eigenvector

(9.58)

we find the eigenvalues

(9.59)

(9.60)

and from the first and last equation

(9.61)

(9.62)

the possible k-values

(9.63)

9.3.3 The QL Algorithm

Any real matrix A can be decomposed into the product of a lower triangular matrix L and an orthogonal matrix Q T =Q ⁻¹ (this is quite similar to the QR factorization with an upper triangular matrix which is discussed in Sect. 5.​2)

(9.64)

For symmetric tridiagonal matrices this factorization can be efficiently realized by multiplication with a sequence of rotation matrices which eliminate the off-diagonal elements in the lower part

(9.65)

An orthogonal transformation of A is given by

(9.66)

The QL algorithm is an iterative algorithm. It uses the transformation

(9.67)

where the shift parameter σ n was introduced to improve convergence and the QL factorization is applied to A n −σ n . This transformation conserves symmetry and tridiagonal form. Repeated transformation gives a sequence of tridiagonal matrices, which converge to a diagonal matrix if the shifts σ n are properly chosen. A very popular choice [198] is Wilkinson’s shift

(9.68)

which is that eigenvalue of the matrix which is closer to a ₁₁ (the smaller one if a ₁₁=a ₂₂).

9.4 Reduction to a Tridiagonal Matrix

Any symmetric matrix can be transformed to a tridiagonal matrix by a series of Householder transformations (5.​53)

(9.69)

The following orthogonal transformation P ₁ brings the first row and column to tridiagonal form. We divide the matrix A according to

(9.70)

with the (N−1)-dimensional vector

Now let

(9.71)

Then

(9.72)

and

(9.73)

The first row of A is transformed by multiplication with P ₁ according to

(9.74)

The elements number 3⋯N are eliminated if we choose³

(9.75)

because then

(9.76)

and

(9.77)

Finally we have

(9.78)

as desired.

For the next step we choose

(9.79)

to eliminate the elements a ₂₄⋯a 2N . Note that P ₂ does not change the first row and column of A ⁽²⁾ and therefore

(9.80)

After N−1 transformations finally a tridiagonal matrix is obtained.

9.5 Large Matrices

Special algorithms are available for matrices of very large dimension to calculate only some eigenvalues and eigenvectors. The famous Lanczos method [153] diagonalizes the matrix in a subspace which is constructed from the vectors

(9.81)

which, starting from an initial normalized guess vector x ₀ are orthonormalized to obtain a tridiagonal matrix,

(9.82)

This series is truncated by setting

(9.83)

and neglecting

(9.84)

Within the subspace of the x ₁⋯x N the matrix A is represented by the tridiagonal matrix

(9.85)

which can be diagonalized with standard methods. The whole method can be iterated using an eigenvector of T as the new starting vector and increasing N until the desired accuracy is achieved. The main advantage of the Lanczos method is that the matrix A will not be stored in memory. It is sufficient to calculate scalar products with A.

9.6 Problems

Problem 9.1

(Computer experiment: disorder in a tight-binding model)

We consider a two-dimensional lattice of interacting particles. Pairs of nearest neighbors have an interaction V and the diagonal energies are chosen from a Gaussian distribution

(9.86)

The wave function of the system is given by a linear combination

(9.87)

where on each particle (i,j) one basis function ψ ij is located. The nonzero elements of the interaction matrix are given by

(9.88)

(9.89)

The matrix H is numerically diagonalized and the amplitudes C ij of the lowest state are shown as circles located at the grid points. As a measure of the degree of localization the quantity

(9.90)

is evaluated. Explore the influence of coupling V and disorder Δ.

References

55.

H.W. Chang, S.E. Liu, R. Burridge, Linear Algebra Appl. 430, 999 (2009) MathSciNetMATHCrossRef

151.

S. Kouachi, Electron. J. Linear Algebra 15, 115 (2006) MathSciNetMATH

153.

C. Lanczos, J. Res. Natl. Bur. Stand. 45, 255 (1951) MathSciNetCrossRef

198.

B.N. Parlett, The Symmetric Eigenvalue Problem (Society for Industrial and Applied Mathematics, Philadelphia, 1998) MATHCrossRef

281.

W.C. Yueh, Appl. Math. E-Notes 5, 66 (2005) MATH

Footnotes

1

We do not consider general eigenvalue problems here.

2

This matrix is skew symmetric, hence iT is Hermitian and has real eigenvalues iλ.

3

To avoid numerical extinction we choose the sign to be that of A ₁₂.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_10© Springer International Publishing Switzerland 2013

10. Data Fitting

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Often a set of data points has to be fitted by a continuous function, either to obtain approximate function values in between the data points or to describe a functional relationship between two or more variables by a smooth curve, i.e. to fit a certain model to the data. If uncertainties of the data are negligibly small, an exact fit is possible, for instance with polynomials, spline functions or trigonometric functions. If the uncertainties are considerable, a curve has to be constructed that fits the data points approximately. Usually a model function is used which depends on a set of parameters which have to be determined such that the overall error is minimized, which in most practical cases is measured by the mean square difference. The optimal parameters are determined by solving the system of normal equations. If the model function depends linearly on the parameters, orthogonalization offers a numerically more stable method. The dimensionality of a data matrix can be reduced with the help of singular value decomposition, which allows to approximate a matrix by another matrix of lower rank and is also useful for linear regression, especially if the columns of the data matrix are linearly dependent.

Often a set of data points has to be fitted by a continuous function, either to obtain approximate function values in between the data points or to describe a functional relationship between two or more variables by a smooth curve, i.e. to fit a certain model to the data. If uncertainties of the data are negligibly small, an exact fit is possible, for instance with polynomials, spline functions or trigonometric functions (Chap. 2). If the uncertainties are considerable, a curve has to be constructed that fits the data points approximately. Consider a two-dimensional data set

(10.1)

and a model function

(10.2)

which depends on the variable x and n≤m additional parameters a j . The errors of the fitting procedure are given by the residuals

(10.3)

The parameters a j have to be determined such, that the overall error is minimized, which in most practical cases is measured by the mean square difference¹

(10.4)

The optimal parameters are determined by solving the system of normal equations. If the model function depends linearly on the parameters, orthogonalization offers a numerically more stable method. The dimensionality of a data matrix can be reduced with the help of singular value decomposition, which allows to approximate a matrix by another matrix of lower rank and is also useful for linear regression, especially if the columns of the data matrix are linearly dependent.

10.1 Least Square Fit

A (local) minimum of (10.4) corresponds to a stationary point with zero gradient. For n model parameters there are n, generally nonlinear, equations which have to be solved [275]. From the general condition

(10.5)

we find

(10.6)

which can be solved with the methods discussed in Chap. 6. For instance, the Newton-Raphson method starts from a suitable initial guess of parameters

(10.7)

and tries to improve the fit iteratively by making small changes to the parameters

(10.8)

The changes $\Delta a_{j}^{s}$ are determined approximately by expanding the model function

(10.9)

to approximate the new residuals

(10.10)

and the derivatives

(10.11)

Equation (10.6) now becomes

(10.12)

which is a system of n (usually overdetermined) linear equations for the Δa j , the so-called normal equations:

(10.13)

With the definition

(10.14)

(10.15)

the normal equations can be written as

(10.16)

10.1.1 Linear Least Square Fit

Especially important are model functions which depend linearly on all parameters (Fig. 10.1 shows an example which is discussed in Problem 10.1)

(10.17)

The derivatives are

(10.18)

and the minimum of (10.4) is given by the solution of the normal equations

(10.19)

which for a linear fit problem become

(10.20)

with

(10.21)

(10.22)

Fig. 10.1

(Least square fit) The polynomial C(T)=aT+bT ³ (full curve) is fitted to a set of data points which are distributed randomly around the “exact” values C(T)=a ₀ T+b ₀ T ³ (dashed curve). For more details see Problem 10.1

Example

(Linear regression)

For a linear fit function

(10.23)

the mean square difference is

(10.24)

and we have to solve the equations

(10.25)

which can be done here with determinants

(10.26)

(10.27)

10.1.2 Linear Least Square Fit with Orthogonalization

With the definitions

(10.28)

and the m×n matrix

(10.29)

the linear least square fit problem (10.20) can be formulated as a search for the minimum of

(10.30)

In the last section we calculated the gradient

(10.31)

and solved the normal equations

(10.32)

This method can become numerically unstable. Alternatively we use orthogonalization of the n column vectors a k of A to have

(10.33)

where a k and q k are now vectors of dimension m. Since the q k are orthonormal $\mathbf{q}_{i}^{T}\mathbf{q}_{k}=\delta_{ik}$ we have

(10.34)

The q k can be augmented by another (m−n) vectors to provide an orthonormal basis of $\mathbb{R}^{m}$. These will not be needed explicitly. They are orthogonal to the first n vectors and hence to the column vectors of A. All vectors q k together form an orthogonal matrix

(10.35)

and we can define the transformation of the matrix A:

(10.36)

The vector b transforms as

(10.37)

Since the norm of a vector is not changed by unitary transformations

(10.38)

which is minimized if

(10.39)

The error of the fit is given by

(10.40)

Example

(Linear regression)

Consider again the fit function

(10.41)

for the measured data (x i ,y i ). The fit problem is to determine

(10.42)

Orthogonalization of the column vectors

(10.43)

with the Schmidt method gives:

(10.44)

(10.45)

(10.46)

(10.47)

(10.48)

(10.49)

Transformation of the right hand side gives

(10.50)

and we have to solve the system of linear equations

(10.51)

The solution

(10.52)

(10.53)

coincides with the earlier results since

(10.54)

10.2 Singular Value Decomposition

Computational physics often has to deal with large amounts of data. Singular value decomposition is a very useful tool to reduce redundancies and to extract the most important information from data. It has been used for instance for image compression [216], it is very useful to extract the essential dynamics from molecular dynamics simulations [99, 221] and it is an essential tool of bio-informatics [71].

10.2.1 Full Singular Value Decomposition

For m≥n,² any real³ m×n matrix A of rank r≤n can be decomposed into a product

(10.55)

(10.56)

where U is an m×m orthogonal matrix, Σ is an m×n matrix, in which the upper part is an n×n diagonal matrix and V is an orthogonal n×n matrix.

The diagonal elements s i are called singular values. Conventionally, they are sorted in descending order and the last n−r of them are zero. For a square n×n matrix singular value decomposition (10.56) is equivalent to diagonalization

(10.57)

10.2.2 Reduced Singular Value Decomposition

We write

(10.58)

with the m×n matrix U n and the m×(m−n) matrix U m−n and

(10.59)

with the diagonal n×n matrix S. The singular value decomposition then becomes

(10.60)

which is known as reduced singular value decomposition. U n (usually simply denoted by U) is not unitary but its column vectors, called the left singular vectors, are orthonormal

(10.61)

as well as the column vectors of V which are called the right singular vectors

(10.62)

Hence the products

(10.63)

give the n×n unit matrix.

In principle, U and V can be obtained from diagonalization of A T A and AA T , since

(10.64)

(10.65)

However, calculation of U by diagonalization is very inefficient, since usually only the first n rows are needed (i.e. U n ). To perform a reduced singular value decomposition, we first diagonalize

(10.66)

which has positive eigenvalues d i ≥0, sorted in descending order and obtain the singular values

(10.67)

Now we determine a matrix U such, that

(10.68)

or, since V is unitary

(10.69)

The last n−r singular values are zero if r<n. Therefore we partition the matrices (indices denote the number of rows)

(10.70)

We retain only the first r columns and obtain a system of equations

(10.71)

which can be easily solved to give the first r rows of U

(10.72)

The remaining n−r column vectors of U have to be orthogonal to the first r columns but are otherwise arbitrary. They can be obtained for instance by the Gram Schmidt method.

For larger matrices direct decomposition algorithms are available, for instance [72], which is based on a reduction to bidiagonal form and a variant of the QL algorithm as first introduced by Golub and Kahan [107].

10.2.3 Low Rank Matrix Approximation

Component-wise (10.60) reads

(10.73)

Approximations to A of lower rank are obtained by reducing the sum to only the largest singular values (the smaller singular values are replaced by zero). It can be shown [243] that the matrix of rank l≤r

(10.74)

is the rank-l matrix which minimizes

(10.75)

If only the largest singular value is taken into account, A is approximated by the rank-1 matrix

(10.76)

As an example, consider an m×n matrix

(10.77)

which contains the values of certain quantities x ₁⋯x n observed at different times t ₁⋯t m . For convenience, we assume that the average values have been subtracted, such that $\sum_{j=1}^{m}x_{i}=0$. Approximation (10.76) reduces the dimensionality to 1, i.e. a linear relation between the data. The i-th row of A,

(10.78)

is approximated by

(10.79)

which describes a direct proportionality of different observables

(10.80)

According to (10.75) this linear relation minimizes the mean square distance between the data points (10.78) and their approximation (10.79).

Example

(Linear approximation [165])

Consider the data matrix

(10.81)

First subtract the row averages

(10.82)

to obtain

(10.83)

Diagonalization of

(10.84)

gives the eigenvalues

(10.85)

and the eigenvectors

(10.86)

Since there are no zero singular values we find

(10.87)

This gives the decomposition⁴

(10.88)

If we neglect the second contribution corresponding to the small singular value s ₂ we have an approximation of the data matrix by a rank-1 matrix. The column vectors of the data matrix, denoted as x and y, are approximated by

(10.89)

which describes a proportionality between x and y (Fig. 10.2).

Fig. 10.2

(Linear approximation by singular value decomposition) The data set (10.81) is shown as circles. The linear approximation which is obtained by retaining only the dominant singular value is shown by the squares and the full line. It minimizes the mean square distance to the data points. Stars and the dashed line show the approximation by linear regression, which minimizes the mean square distance in vertical direction

10.2.4 Linear Least Square Fit with Singular Value Decomposition

The singular value decomposition can be used for linear regression [165]. Consider a set of data, which have to be fitted to a linear function

(10.90)

with the residual

(10.91)

Let us subtract the averages

(10.92)

which we write in matrix notation as

(10.93)

or shorter

(10.94)

Now let us insert the full decomposition of X

(10.95)

Since U is orthogonal

(10.96)

where we introduce the abbreviations

(10.97)

The sum of squared residuals has the form

(10.98)

Hence a n−r is arbitrary and one minimum of S D is given by

(10.99)

which can be written more compactly as

(10.100)

with the Moore-Penrose pseudoinverse [20] of Σ

(10.101)

Finally we have

(10.102)

where

(10.103)

is the Moore-Penrose pseudoinverse of X.

Example

The following data matrix has rank 2

(10.104)

A solution to the linear fit problem is given by

(10.105)

The fit function is

(10.106)

and the residuals are

(10.107)

10.3 Problems

Problem 10.1

(Least square fit)

At temperatures far below Debye and Fermi temperatures the specific heat of a metal contains contributions from electrons and lattice vibrations and can be described by

(10.108)

The computer experiment generates data

(10.109)

(10.110)

with relative error

(10.111)

Random numbers ξ j are taken from a Gaussian normal distribution function (Sect. 8.​2.​4.​4).

The fit parameters a,b are determined from minimization of the sum of squares

(10.112)

Compare the “true values” a ₀,b ₀ with the fitted values a,b.

References

20.

A. Ben-Israel, T.N.E. Greville, Generalized Inverses (Springer, Berlin, 2003). ISBN 0-387-00293-6 MATH

71.

D.P. Derrarr et al., in A Practical Approach to Microarray Data Analysis (Kluwer Academic, Norwell, 2003), p. 91

72.

DGESVD routine from the freely available software package LAPACK, http:​/​/​www.​netlib.​org/​lapack

99.

A.E. Garcia, Phys. Rev. Lett. 86, 2696 (1992) ADSCrossRef

107.

G. Golub, W. Kahan, J. Soc. Ind. Appl. Math., Ser. B Numer. Anal. 2, 205 (1965) MathSciNetCrossRef

165.

J. Mandel, Am. Stat. 36, 15 (1982)

216.

J.A. Richards, Remote Sensing Digital Image Analysis (Springer, Berlin, 1993) CrossRef

221.

T.D. Romo et al., Proteins 22, 311 (1995) CrossRef

243.

G.W. Stewart, SIAM Rev. 35, 551 (1993) MathSciNetMATHCrossRef

275.

J. Wolberg, Data Analysis Using the Method of Least Squares: Extracting the Most Information from Experiments (Springer, Berlin, 2005)

Footnotes

1

Minimization of the sum of absolute errors ∑|r i | is much more complicated.

2

Otherwise consider the transpose matrix.

3

Generalization to complex matrices is straightforward.

4

$\mathbf{u}_{i}\mathbf{v}_{i}^{T}$ is the outer or matrix product of two vectors.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_11© Springer International Publishing Switzerland 2013

11. Discretization of Differential Equations

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

The simplest approach to discretize a differential equation replaces differential quotients by quotients of finite differences. For the space variables this method works best on a regular grid. Finite volume methods, which are very popular in computational fluid dynamics, take averages over small control volumes and can be easily used with irregular grids. Finite elements and finite volumes belong to the general class of finite element methods which are prominent in the engineering sciences and use an expansion in piecewise polynomials with small support. Spectral methods, on the other hand, expand the solution as a linear combination of global basis functions. A general concept is the method of weighted residuals. Most popular is Galerkin’s method. The simpler point collocation and sub-domain collocation methods fulfill the differential equation only at certain points or averaged over certain control volumes. The more demanding least-squares method has become popular in computational fluid dynamics and computational electrodynamics.

If the Green’s function is available for a problem, the method of boundary elements is an interesting alternative. It reduces the dimensionality and is, for instance, very popular in chemical physics to solve the Poisson-Boltzmann equation.

Many processes in science and technology can be described by differential equations involving the rate of changes in time or space of a continuous variable, the unknown function. While the simplest differential equations can be solved exactly, a numerical treatment is necessary in most cases and the equations have to be discretized to turn them into a finite system of equations which can be solved by computers [6, 155, 200]. In this chapter we discuss different methods to discretize differential equations. The simplest approach is the method of finite differences, which replaces the differential quotients by difference quotients (Chap. 3). It is often used for the discretization of time. Finite difference methods for the space variables work best on a regular grid. Finite volume methods are very popular in computational fluid dynamics. They take averages over small control volumes and can be easily used with irregular grids. Finite differences and finite volumes belong to the general class of finite element methods which are prominent in the engineering sciences and use an expansion in piecewise polynomials with small support. Spectral methods, on the other hand, expand the solution as a linear combination of global basis functions like polynomials or trigonometric functions. A general concept for the discretization of differential equations is the method of weighted residuals which minimizes the weighted residual of a numerical solution. Most popular is Galerkin’s method which uses the expansion functions also as weight functions. Simpler are the point collocation and sub-domain collocation methods which fulfill the differential equation only at certain points or averaged over certain control volumes. More demanding is the least-squares method which has become popular in computational fluid dynamics and computational electrodynamics. The least-square integral provides a measure for the quality of the solution which can be used for adaptive grid size control.

If the Green’s function is available for a problem, the method of boundary elements is an interesting alternative. It reduces the dimensionality and is, for instance, very popular in chemical physics to solve the Poisson-Boltzmann equation.

11.1 Classification of Differential Equations

An ordinary differential equation (ODE) is a differential equation for a function of one single variable, like Newton’s law for the motion of a body under the influence of a force field

(11.1)

a typical initial value problem where the solution in the domain t ₀≤t≤T is determined by position and velocity at the initial time

(11.2)

Such equations of motion are discussed in Chap. 12. They also appear if the spatial derivatives of a partial differential equation have been discretized. Usually this kind of equation is solved by numerical integration over finite time steps Δt=t n+1−t n . Boundary value problems, on the other hand, require certain boundary conditions¹ to be fulfilled, for instance the linearized Poisson-Boltzmann equation in one dimension (Chap. 17)

(11.3)

where the value of the potential is prescribed on the boundary of the domain x ₀≤x≤x ₁

(11.4)

Partial differential equations (PDE) finally involve partial derivatives with respect to at least two different variables, in many cases time and spatial coordinates.

11.1.1 Linear Second Order PDE

A very important class are second order linear partial differential equations of the general form

(11.5)

where the coefficients a ij ,b i ,c,d are functions of the variables x ₁…x N but do not depend on the function f itself. The equation is classified according to the eigenvalues of the coefficient matrix a ij as [141]

elliptical if all eigenvalues are positive or all eigenvalues are negative, like for the Poisson equation (Chap. 17)

(11.6)

hyperbolic if one eigenvalue is negative and all the other eigenvalues are positive or vice versa, for example the wave equation in one spatial dimension (Chap. 18)

(11.7)

parabolic if at least one eigenvalue is zero, like for the diffusion equation (Chap. 19)

(11.8)

ultra-hyperbolic if there is no zero eigenvalue and more than one positive as well as more than one negative eigenvalue. Obviously the dimension then must be 4 at least.

11.1.2 Conservation Laws

One of the simplest first order partial differential equations is the advection equation

(11.9)

which describes transport of a conserved quantity with density f (for instance mass, number of particles, charge etc.) in a medium streaming with velocity u. This is a special case of the class of conservation laws (also called continuity equations)

(11.10)

which are very common in physics. Here J describes the corresponding flux and g is an additional source (or sink) term. For instance the advection-diffusion equation (also known as convection equation) has this form which describes quite general transport processes:

(11.11)

where one contribution to the flux

(11.12)

is proportional to the gradient of the concentration C (Fick’s first law) and the second part depends on the velocity field u of a streaming medium. The source term S represents the effect of chemical reactions. Equation (11.11) is also similar to the drift-diffusion equation in semiconductor physics and closely related to the Navier Stokes equations which are based on the Cauchy momentum equation [1]

(11.13)

where σ denotes the stress tensor. Equation (11.10) is the strong or differential form of the conservation law. The requirements on the smoothness of the solution are reduced by using the integral form which is obtained with the help of Gauss’ theorem

(11.14)

An alternative integral form results from Galerkin’s [98] method of weighted residuals which introduces a weight function w(x) and considers the equation

(11.15)

or after applying Gauss’ theorem

(11.16)

The so called weak form of the conservation law states that this equation holds for arbitrary weight functions w.

11.2 Finite Differences

The simplest method to discretize a differential equation is to introduce a grid of equidistant points and to discretize the differential operators by finite differences (FDM) as described in Chap. 3. For instance, in one dimension the first and second derivatives can be discretized by

(11.17)

(11.18)

(11.19)

(11.20)

These expressions are not well defined at the boundaries of the grid m=1,M unless the boundary conditions are taken into account. For instance, in case of a Dirichlet problem f ₀ and f M+1 are given boundary values and

(11.21)

(11.22)

Other kinds of boundary conditions can be treated in a similar way.

11.2.1 Finite Differences in Time

Time derivatives can be treated similarly using an independent time grid

(11.23)

(11.24)

and finite differences like the first order forward difference quotient

(11.25)

or the symmetric difference quotient

(11.26)

to obtain a system of equations for the function values at the grid points $f_{m}^{n}$. For instance for the diffusion equation in one spatial dimension

(11.27)

the simplest discretization is the FTCS (forward in time, centered in space) scheme

(11.28)

which can be written in matrix notation as

(11.29)

with

(11.30)

11.2.2 Stability Analysis

Fully discretized linear differential equations provide an iterative algorithm of the type²

(11.31)

which propagates numerical errors according to

(11.32)

(11.33)

Errors are amplified exponentially if the absolute value of at least one eigenvalue of A is larger than one. The algorithm is stable if all eigenvalues of A are smaller than one in absolute value (Sect. 1.​4). If the eigenvalue problem is difficult to solve, the von Neumann analysis is helpful which decomposes the errors into a Fourier series and considers the Fourier components individually by setting

(11.34)

and calculating the amplification factor

(11.35)

The algorithm is stable if |g(k)|≤1 for all k.

Example

For the discretized diffusion equation (11.28) we find

(11.36)

(11.37)

(11.38)

hence stability requires

(11.39)

11.2.3 Method of Lines

Alternatively time can be considered as a continuous variable. The discrete values of the function then are functions of time (so called lines)

(11.40)

and a set of ordinary differential equations has to be solved. For instance for diffusion in one dimension (11.27) the equations

(11.41)

which can be written in matrix notation as

(11.42)

or briefly

(11.43)

Several methods to integrate such a semi-discretized equation will be discussed in Chap. 12. If eigenvectors and eigenvalues of A are easy available, an eigenvector expansion can be used.

11.2.4 Eigenvector Expansion

A homogeneous system

(11.44)

where the matrix A is obtained from discretizing the spatial derivatives, can be solved by an eigenvector expansion. From the eigenvalue problem

(11.45)

we obtain the eigenvalues λ and eigenvectors f λ which provide the particular solutions:

(11.46)

(11.47)

These can be used to expand the general solution

(11.48)

The coefficients C λ follow from the initial values by solving the linear equations

(11.49)

If the differential equation is second order in time

(11.50)

the particular solutions are

(11.51)

(11.52)

and the eigenvector expansion is

(11.53)

The coefficients C λ± follow from the initial amplitudes and velocities

(11.54)

For a first order inhomogeneous system

(11.55)

the expansion coefficients have to be time dependent

(11.56)

and satisfy

(11.57)

After taking the scalar product with f μ ³

(11.58)

can be solved by a simple time integration. For a second order system

(11.59)

we introduce the first time derivative as a new variable

(11.60)

to obtain a first order system of double dimension

(11.61)

where eigenvectors and eigenvalues can be found from those of A (11.45)

(11.62)

(11.63)

Insertion of

gives

(11.64)

and taking the scalar product with one of the left-eigenvectors we end up with

(11.65)

(11.66)

11.3 Finite Volumes

Whereas the finite differences method uses function values

(11.67)

at the grid points

(11.68)

the finite volume method (FVM) [79] averages function values and derivatives over small control volumes V r which are disjoint and span the domain V (Fig. 11.1)

(11.69)

The averages are

(11.70)

or in the simple case of cubic control volumes of equal size h ³

(11.71)

Fig. 11.1

(Finite volume method)  The domain V is divided into small control volumes V r , in the simplest case cubes around the grid points r ijk

Such average values have to be related to discrete function values by numerical integration (Chap. 4). The midpoint rule (4.​17), for instance replaces the average by the central value

(11.72)

whereas the trapezoidal rule (4.​13) implies the average over the eight corners of the cube

(11.73)

In (11.73) the function values refer to a dual grid [79] centered around the vertices of the original grid (11.68) (Fig. 11.2),

(11.74)

Fig. 11.2

(Dual grid)  The dual grid (black) is centered around the vertices of the original grid (red)

The average gradient can be rewritten using the generalized Stokes’ theorem as

(11.75)

For a cubic grid we have to integrate over the six faces of the control volume

(11.76)

The integrals have to be evaluated numerically. Applying as the simplest approximation the midpoint rule (4.​17)

(11.77)

this becomes

(11.78)

which involves symmetric difference quotients. However, the function values in (11.78) refer neither to the original nor to the dual grid. Therefore we interpolate (Fig. 11.3)

(11.79)

(11.80)

or

(11.81)

Fig. 11.3

(Interpolation between grid points)  Interpolation is necessary to relate the averaged gradient (11.78) to the original or dual grid

The finite volume method is capable of treating discontinuities and is very flexible concerning the size and shape of the control volumes.

11.3.1 Discretization of fluxes

Integration of (11.10) over a control volume and application of Gauss’ theorem gives the integral form of the conservation law

(11.82)

which involves the flux J of some property like particle concentration, mass, energy or momentum density or the flux of an electromagnetic field. The total flux through a control volume is given by the surface integral

(11.83)

which in the special case of a cubic volume element of size h ³ becomes the sum over the six faces of the cube (Fig. 11.4)

(11.84)

Fig. 11.4

Flux through a control volume

The surface integral can be evaluated numerically (Chap. 4). Using the midpoint approximation (11.77) we obtain

(11.85)

The trapezoidal rule (4.​13) introduces an average over the four corners (Fig. 11.3)

(11.86)

which replaces the flux values in (11.85) by the averages

(11.87)

(11.88)

(11.89)

One advantage of the finite volume method is that the flux is strictly conserved.

11.4 Weighted Residual Based Methods

A general method to discretize partial differential equations is to approximate the solution within a finite dimensional space of trial functions.⁴ The partial differential equation is turned into a finite system of equations or a finite system of ordinary differential equations if time is treated as a continuous variable. This is the basis of spectral methods which make use of polynomials or Fourier series but also of the very successful finite element methods. Even finite difference methods and finite volume methods can be formulated as weighted residual based methods.

Consider a differential equation⁵ on the domain V which is written symbolically with the differential operator $\mathcal{T}$

(11.90)

and corresponding boundary conditions which are expressed with a boundary operator $\mathcal{B}$ ⁶

(11.91)

The basic principle to obtain an approximate solution $\tilde{u}(\mathbf {r})$ is to choose a linear combination of expansion functions N i (r) i=1…r as a trial function which fulfills the boundary conditions⁷

(11.92)

(11.93)

In general (11.92) is not an exact solution and the residual

(11.94)

will not be zero throughout the whole domain V. The function $\tilde{u}$ has to be determined such that the residual becomes “small” in a certain sense. To that end weight functions⁸ w j j=1…r are chosen to define the weighted residuals

(11.95)

The optimal parameters u i are then obtained from the solution of the equations

(11.96)

In the special case of a linear differential operator these equations are linear

(11.97)

Several strategies are available to choose suitable weight functions.

11.4.1 Point Collocation Method

The collocation method uses the weight functions w j (r)=δ(r−r j ), with certain collocation points r j ∈V. The approximation $\tilde{u}$ obeys the differential equation at the collocation points

(11.98)

and for a linear differential operator

(11.99)

The point collocation method is simple to use, especially for nonlinear problems. Instead of using trial functions satisfying the boundary conditions, extra collocation points on the boundary can be added (mixed collocation method).

11.4.2 Sub-domain Method

This approach uses weight functions which are the characteristic functions of a set of control volumes V i which are disjoint and span the whole domain similar as for the finite volume method

(11.100)

(11.101)

The residuals then are integrals over the control volumes and

(11.102)

respectively

(11.103)

11.4.3 Least Squares Method

Least squares methods have become popular for first order systems of differential equations in computational fluid dynamics and computational electrodynamics [30, 140].

The L2-norm of the residual (11.94) is given by the integral

(11.104)

It is minimized by solving the equations

(11.105)

which is equivalent to choosing the weight functions

(11.106)

or for a linear differential operator simply

(11.107)

Advantages of the least squares method are that boundary conditions can be incorporated into the residual and that S provides a measure for the quality of the solution which can be used for adaptive grid size control. On the other hand S involves a differential operator of higher order and therefore much smoother trial functions are necessary.

11.4.4 Galerkin Method

Galerkin’s widely used method [87, 98] chooses the basis functions as weight functions

(11.108)

and solves the following system of equations

(11.109)

or in the simpler linear case

(11.110)

11.5 Spectral and Pseudo-spectral Methods

Spectral methods use basis functions which are nonzero over the whole domain, the trial functions being mostly polynomials or Fourier sums [35]. They can be used to solve ordinary as well as partial differential equations. The combination of a spectral method with the point collocation method is also known as pseudo-spectral method.

11.5.1 Fourier Pseudo-spectral Methods

Linear differential operators become diagonal in Fourier space. Combination of Fourier series expansion and point collocation leads to equations involving a discrete Fourier transformation, which can be performed very efficiently with the Fast Fourier Transform methods.

For simplicity we consider only the one-dimensional case. We choose equidistant collocation points

(11.111)

and expansion functions

(11.112)

For a linear differential operator

(11.113)

and the condition on the residual becomes

(11.114)

or

(11.115)

which is nothing but a discrete Fourier back transformation (Sect. 7.​2, (7.​19)) which can be inverted to give

(11.116)

Instead of exponential expansion functions, sine and cosine functions can be used to satisfy certain boundary conditions, for instance to solve the Poisson equation within a cube (Sect. 17.​1.​2).

11.5.2 Example: Polynomial Approximation

Let us consider the initial value problem (Fig. 11.5)

(11.117)

with the well known solution

(11.118)

We choose a polynomial trial function with the proper initial value

(11.119)

The residual is

(11.120)

Fig. 11.5

(Approximate solution of a simple differential equation)  The initial value problem $\frac{\mathrm {d}}{\mathrm {d}x}u(x)-u(x)=0$ u(0)=1 for 0≤x≤1 is approximately solved with a polynomial trial function $\tilde {u}(x)=1+u_{1}x+u_{2}x^{2}$. The parameters u 1,2 are optimized with the method of weighted residuals using point collocation (full curve), sub-domain collocation (dotted curve), Galerkin’s method (dashed curve) and least squares (dash-dotted curve). The absolute error $\tilde{u}(x)-\mathrm {e}^{x}$ (top) and the residual $R(x)=\frac{\mathrm {d}}{\mathrm {d}x}\tilde{u}(x)- \tilde {u}(x)= (u_{1}-1)+(2u_{2}-u_{1})x -u_{2}x^{2}$ both are smallest for the least squares and sub-domain collocation methods

11.5.2.1 Point Collocation Method

For our example we need two collocation points to obtain two equations for the two unknowns u 1,2. We choose $x_{1}=0,x_{2}=\frac{1}{2}$. Then we have to solve the equations

(11.121)

(11.122)

which gives

(11.123)

(11.124)

11.5.2.2 Sub-domain Method

We need two sub-domains to obtain two equations for the two unknowns u 1,2. We choose $V_{1}=\{x,0<x<\frac{1}{2}\}$, $V_{2}=\{x,\frac{1}{2}<x<1\}$. Integration gives

(11.125)

(11.126)

(11.127)

(11.128)

11.5.2.3 Galerkin Method

Galerkin’s method uses the weight functions w ₁(x)=x,w ₂(x)=x ². The equations

(11.129)

(11.130)

have the solution

(11.131)

(11.132)

11.5.2.4 Least Squares Method

The integral of the squared residual

(11.133)

is minimized by solving

(11.134)

(11.135)

which gives

(11.136)

(11.137)

11.6 Finite Elements

The method of finite elements (FEM) is a very flexible method to discretize partial differential equations [84, 210]. It is rather dominant in a variety of engineering sciences. Usually the expansion functions N i are chosen to have compact support. The integration volume is divided into disjoint sub-volumes

(11.138)

The N i (x) are piecewise continuous polynomials which are nonzero only inside V i and a few neighbor cells.

11.6.1 One-Dimensional Elements

In one dimension the domain is an interval V={x;a≤x≤b} and the sub-volumes are small sub-intervals V i ={x;x i ≤x≤x i+1}. The one-dimensional mesh is the set of nodes {a=x ₀,x ₁…x r =b}. Piecewise linear basis functions (Fig. 11.6) are in the 1-dimensional case given by

(11.139)

and the derivatives are (except at the nodes x i )

(11.140)

Fig. 11.6

(Finite elements in one dimension)  The basis functions N i are piecewise continuous polynomials and have compact support. In the simplest case they are composed of two linear functions over the sub-intervals x i−1≤x≤x i and x i ≤x≤x i+1

11.6.2 Two- and Three-Dimensional Elements

In two dimensions the mesh is defined by a finite number of points (x i ,y i )∈V (the nodes of the mesh). There is considerable freedom in the choice of these points and they need not be equally spaced.

11.6.2.1 Triangulation

The nodes can be regarded as forming the vertices of a triangulation⁹ of the domain V (Fig. 11.7).

Fig. 11.7

(Triangulation of a two-dimensional domain)  A two-dimensional mesh is defined by a set of node points which can be regarded to form the vertices of a triangulation

The piecewise linear basis function in one dimension (11.139) can be generalized to the two-dimensional case by constructing functions N i (x,y) which are zero at all nodes except (x i ,y i )

(11.141)

These functions are linear over each triangle which contains the vertex i and can be combined as the sum of small pyramids (Fig. 11.8). Let one of the triangles be denoted by its three vertices as T ijk .¹⁰ The corresponding linear function then is

(11.142)

where the coefficients follow from the conditions

(11.143)

as

(11.144)

with

(11.145)

which, apart from sign, is the area of the triangle T ijk . The basis function N i now is given by

Fig. 11.8

(Finite elements in two dimensions)  The simplest finite elements in two dimensions are piecewise linear functions N i (x,y) which are non-vanishing only at one node (x i ,y i ) (right side). They can be constructed from small pyramids built upon one of the triangles that contains this node (left side)

In three dimensions we consider tetrahedrons (Fig. 11.9) instead of triangles. The corresponding linear function of three arguments has the form

(11.146)

and from the conditions n i,j,k,l (x i ,y i ,z i )=1 and n i,j,k,l =0 on all other nodes we find (an algebra program is helpful at that point)

(11.147)

where V ijkl is, apart from sign, the volume of the tetrahedron

(11.148)

Fig. 11.9

(Tetrahedron)  The tetrahedron is the three-dimensional case of a Euclidean simplex, i.e. the simplest polytope

11.6.2.2 Rectangular Elements

For a rectangular grid rectangular elements offer a practical alternative to triangles. Since equations for four nodes have to be fulfilled, the basic element needs four parameters, which is the case for a bilinear expression. Let us denote one of the rectangles which contains the vertex i as R i,j,k,l . The other three edges are

(11.149)

where b x =±h x ,b y =±h y corresponding to the four rectangles with the common vertex i (Fig. 11.10).

Fig. 11.10

(Rectangular elements around one vertex)  The basis function N i is a bilinear function on each of the four rectangles containing the vertex (x i ,y i )

The bilinear function (Fig. 11.11) corresponding to R ijkl is

(11.150)

It has to fulfill

(11.151)

from which we find

(11.152)

(11.153)

The basis function centered at node i then is

(11.154)

Fig. 11.11

(Bilinear elements on a rectangular grid)  The basis functions N i (x,y) on a rectangular grid (right side) are piecewise bilinear functions (left side), which vanish at all nodes except (x i ,y i ) (right side)

Generalization to a three-dimensional grid is straightforward (Fig. 11.12). We denote one of the eight cuboids containing the node (x i ,y i ,z i ) as $C_{i,j_{1}\dots j_{7}}$ with $(x_{j_{1}},y_{j_{1}},z_{j_{1}})=(x_{i}+b_{x},y_{i},z_{i})\dots (x_{j_{7}},y_{j_{7}},z_{j_{7}})=(x_{i}+b_{x},y_{i}+b_{y},z_{i}+b_{z})$. The corresponding trilinear function is

(11.155)

Fig. 11.12

(Three-dimensional rectangular grid)  The basis function N i is trilinear on each of the eight cuboids containing the vertex i. It vanishes on all nodes except (x i ,y i ,z i )

11.6.3 One-Dimensional Galerkin FEM

As an example we consider the one-dimensional linear differential equation (11.5)

(11.156)

in the domain 0≤x≤1 with boundary conditions

(11.157)

We use the basis functions from (11.139) on a one-dimensional grid with

(11.158)

and apply the Galerkin method [88]. The boundary conditions require

(11.159)

The weighted residual is

(11.160)

First we integrate

(11.161)

Integration of the first derivative gives

(11.162)

For the second derivative partial integration gives

(11.163)

where the first two summands are zero due to the boundary conditions. Since $N_{i}\mbox{ and }N_{i}'$ are nonzero only for x i−1<x<x i+1 we find

(11.164)

Integration of the last term in (11.160) gives

(11.165)

Applying the trapezoidal rule¹¹ for both integrals we find

(11.166)

The discretized equation finally reads

(11.167)

which can be written in matrix notation as

(11.168)

where the matrix A is tridiagonal as a consequence of the compact support of the basis functions

For equally spaced nodes h i =h i−1=h and after division by h (11.167) reduces to a system of equations where the derivatives are replaced by finite differences (11.20)

(11.169)

and the function u is replaced by a certain average

(11.170)

The corresponding matrix in (11.169) is the so called mass matrix. Within the framework of the finite differences method the last expression equals

(11.171)

hence replacing it by u j (this is called mass lumping) introduces an error of the order O(h ²).

11.7 Boundary Element Method

The boundary element method (BEM) [18, 276] is a method for linear partial differential equations which can be brought into boundary integral form¹² like Laplace’s equation (Chap. 17)¹³

(11.172)

for which the fundamental solution

is given by

(11.173)

(11.174)

We apply Gauss’s theorem to the expression [277]

(11.175)

Integration over a volume V gives

(11.176)

This integral equation determines the potential self-consistently by its value and normal derivative on the surface of the cavity. It can be solved numerically by dividing the surface into a finite number of boundary elements. The resulting system of linear equations often has smaller dimension than corresponding finite element approaches. However, the coefficient matrix is in general full and not necessarily symmetric.

References

1.

D.J. Acheson, Elementary Fluid Dynamics (Oxford University Press, London, 1990) MATH

6.

W. Ames, Numerical Methods for Partial Differential Equations (Academic Press, San Diego, 1992) MATH

18.

G. Beer, I. Smith, C. Duenser, The Boundary Element Method with Programming: For Engineers and Scientists (Springer, Berlin, 2008)

30.

P. Bochev, M. Gunzburger, Least Squares Finite Element Methods (Springer, Berlin, 2009) MATH

35.

J.P. Boyd, Chebyshev and Fourier Spectral Methods (Dover, New York, 2001) MATH

79.

R. Eymard, T. Galloue, R. Herbin, Finite volume methods, in Handbook of Numerical Analysis, vol. 7, ed. by P.G. Ciarlet, J.L. Lions (Elsevier, Amsterdam, 2000), pp. 713–1020

84.

J. Fish, T. Belytschko, A First Course in Finite Elements (Wiley, New York, 2007) MATHCrossRef

87.

C.A.J. Fletcher, Computational Galerkin Methods (Springer, Berlin, 1984) MATHCrossRef

88.

C.A.J. Fletcher, Computational Techniques for Fluid Dynamics, vol. I, 2nd edn. (Springer, Berlin, 1991) MATH

98.

B.G. Galerkin, On electrical circuits for the approximate solution of the Laplace equation. Vestnik Inzh. 19, 897–908 (1915)

140.

B.-n. Jiang, The Least-Squares Finite Element Method (Springer, Berlin, 1998) MATHCrossRef

141.

F. John, Duke Math. J. 4, 300 (1938) MathSciNetCrossRef

155.

H.P. Langtangen, Computational Partial Differential Equations: Numerical Methods and Diffpack Programming (Springer, Berlin, 2003) CrossRef

195.

E.L. Ortiz, SIAM J. Numer. Anal. 6, 480 (1969) MathSciNetADSMATHCrossRef

200.

J. Peiro, S. Sherwin, in Handbook of Materials Modeling, vol. 1, ed. by S. Yip (Springer, Berlin, 2005), pp. 1–32

210.

S.S. Rao, The Finite Element Method in Engineering (Elsevier, Amsterdam, 2011) MATH

276.

L.C. Wrobel, M.H. Aliabadi, The Boundary Element Method (Wiley, New York, 2002)

277.

G. Wunsch, Feldtheorie (VEB Technik, Berlin, 1973)

Footnotes

1

Dirichlet b.c. concern the function values, Neumann b.c. the derivative, Robin b.c. a linear combination of both, Cauchy b.c. the function value and the normal derivative and mixed b.c. have different character on different parts of the boundary.

2

Differential equations which are higher order in time can be always brought to first order by introducing the time derivatives as additional variables.

3

If A is not Hermitian we have to distinguish left- and right-eigenvectors.

4

Also called expansion functions.

5

Generalization to systems of equations is straightforward.

6

One or more linear differential operators, usually a combination of the function and its first derivatives.

7

This requirement can be replaced by additional equations for the u i , for instance with the tau method [195].

8

Also called test functions.

9

The triangulation is not determined uniquely by the nodes.

10

The order of the indices does matter.

11

Higher accuracy can be achieved, for instance, by Gaussian integration.

12

This is only possible if the fundamental solution or Green’s function is available.

13

The minus sign is traditionally used.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_12© Springer International Publishing Switzerland 2013

12. Equations of Motion

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

We discuss several strategies for the time integration of first order initial value problems. The explicit Euler forward difference has low error order. The much more accurate symmetric difference quotient can be used as the corrector step in combination with an explicit Euler predictor step and is often used for the time integration of partial differential equations. Methods with higher error order can be obtained from a Taylor series expansion, like the Nordsieck and Gear predictor-corrector methods which have been often applied in molecular dynamics calculations. Runge-Kutta methods are very important for ordinary differential equations. They are robust and allow an adaptive control of the step size. Very accurate results can be obtained for ordinary differential equations with extrapolation methods like the famous Gragg-Bulirsch-Stör method. Multistep methods use information from several points. Best known are Adams-Bashforth-Moulton methods and Gear methods (also known as backward differentiation methods), which are especially useful for stiff problems. The class of Verlet methods has been developed for molecular dynamics calculations. These are symplectic and time reversible and conserve energy over long trajectories.

Simulation of a physical system means to calculate the time evolution of a model system in many cases. We consider a large class of models which can be described by a first order initial value problem

(12.1)

where Y is the state vector (possibly of very high dimension) which contains all information about the system. Our goal is to calculate the time evolution of the state vector Y(t) numerically. For obvious reasons this can be done only for a finite number of values of t and we have to introduce a grid of discrete times t n which for simplicity are assumed to be equally spaced:¹

(12.2)

Advancing time by one step involves the calculation of the integral

(12.3)

which can be a formidable task since f(Y(t),t) depends on time via the time dependence of all the elements of Y(t). In this chapter we discuss several strategies for the time integration. The explicit Euler forward difference has low error order but is useful as a predictor step for implicit methods. A symmetric difference quotient is much more accurate. It can be used as the corrector step in combination with an explicit Euler predictor step and is often used for the time integration of partial differential equations. Methods with higher error order can be obtained from a Taylor series expansion, like the Nordsieck and Gear predictor-corrector methods which have been often applied in molecular dyamics calculations. Runge-Kutta methods are very important for ordinary differential equations. They are robust and allow an adaptive control of the step size. Very accurate results can be obtained for ordinary differential equations with extrapolation methods like the famous Gragg-Bulirsch-Stör method. If the solution is smooth enough, multistep methods are applicable, which use information from several points. Most known are Adams-Bashforth-Moulton methods and Gear methods (also known as backward differentiation methods), which are especially useful for stiff problems. The class of Verlet methods has been developed for molecular dynamics calculations. They are symplectic and time reversible and conserve energy over long trajectories.

12.1 The State Vector

The state of a classical N-particle system is given by the position in phase space, or equivalently by specifying position and velocity for all the N particles

(12.4)

The concept of a state vector is not restricted to a finite number of degrees of freedom. For instance a diffusive system can be described by the particle concentrations as a function of the coordinate, i.e. the elements of the state vector are now indexed by the continuous variable x

(12.5)

Similarly, a quantum particle moving in an external potential can be described by the amplitude of the wave function

(12.6)

Numerical treatment of continuous systems is not feasible since even the ultimate high end computer can only handle a finite number of data in finite time. Therefore discretization is necessary (Chap. 11), by introducing a spatial mesh (Sects. 11.​2, 11.​3, 11.​6), which in the simplest case means a grid of equally spaced points

(12.7)

(12.8)

(12.9)

or by expanding the continuous function with respect to a finite set of basis functions (Sect. 11.​5). The elements of the state vector then are the expansion coefficients

(12.10)

(12.11)

If the density matrix formalism is used to take the average over a thermodynamic ensemble or to trace out the degrees of freedom of a heat bath, the state vector instead is composed of the elements of the density matrix

(12.12)

(12.13)

12.2 Time Evolution of the State Vector

We assume that all information about the system is included in the state vector. Then the simplest equation to describe the time evolution of the system gives the change of the state vector

(12.14)

as a function of the state vector (or more generally a functional in the case of a continuous system). Explicit time dependence has to be considered for instance to describe the influence of an external time dependent field.

Some examples will show the universality of this equation of motion:

N-particle system

The motion of N interacting particles is described by

(12.15)

where the acceleration of a particle is given by the total force acting upon this particle and thus depends on all the coordinates and eventually time (velocity dependent forces could be also considered but are outside the scope of this book)

(12.16)

Diffusion

Heat transport and other diffusive processes are described by the diffusion equation

(12.17)

which in its simplest spatially discretized version for 1-dimensional diffusion reads

(12.18)

Waves

Consider the simple 1-dimensional wave equation

(12.19)

which by introducing the velocity $g(\mathbf{x})=\frac{\partial}{\partial t}f(\mathbf{x})$ as an independent variable can be rewritten as

(12.20)

Discretization of space gives

(12.21)

Two-state quantum system

The Schrödinger equation for a two level system (for instance a spin-$\frac{1}{2}$ particle in a magnetic field) reads

(12.22)

12.3 Explicit Forward Euler Method

The simplest method which is often discussed in elementary physics textbooks approximates the integrand by its value at the lower bound (Fig. 12.1):

(12.23)

The truncation error can be estimated from a Taylor series expansion

(12.24)

The explicit Euler method has several serious drawbacks

Low error order

Suppose you want to integrate from the initial time t ₀ to the final time t ₀+T. For a time step of Δt you have to perform N=T/Δt steps. Assuming comparable error contributions from all steps the global error scales as NΔt ²=O(Δt). The error gets smaller as the time step is reduced but it may be necessary to use very small Δt to obtain meaningful results.

Loss of orthogonality and normalization

The simple Euler method can produce systematic errors which are very inconvenient if you want, for instance, to calculate the orbits of a planetary system. This can be most easily seen from a very simple example. Try to integrate the following equation of motion (see Example 1.​5 on page 12):

(12.25)

The exact solution is obviously given by a circular orbit in the complex plane:

(12.26)

(12.27)

Application of the Euler method gives

(12.28)

and you find immediately

(12.29)

which shows that the radius increases continually even for the smallest time step possible (Fig. 12.2).

Fig. 12.1

Explicit Euler method

Fig. 12.2

Systematic errors of the Euler method

The same kind of error appears if you solve the Schrödinger equation for a particle in an external potential or if you calculate the rotational motion of a rigid body. For the N-body system it leads to a violation of the conservation of phase space volume. This can introduce an additional sensitivity of the calculated results to the initial conditions. Consider a harmonic oscillator with the equation of motion

(12.30)

Application of the explicit Euler method gives

(12.31)

The change of the phase space volume (Fig. 12.3) is given by the Jacobi determinant

(12.32)

Fig. 12.3

Time evolution of the phase space volume

In this case the phase space volume increases continuously.

12.4 Implicit Backward Euler Method

Alternatively let us make a step backwards in time

(12.33)

which can be written as (Fig. 12.4)

(12.34)

Taylor series expansion gives

(12.35)

which shows that the error order again is O(Δt ²). The implicit method is sometimes used to avoid the inherent instability of the explicit method. For the examples in Sect. 12.3 it shows the opposite behavior. The radius of the circular orbit as well as the phase space volume decrease in time. The gradient at future time has to be estimated before an implicit step can be performed.

Fig. 12.4

Implicit backward Euler method

12.5 Improved Euler Methods

The quality of the approximation can be improved significantly by employing the midpoint rule (Fig. 12.5)

(12.36)

The error is smaller by one order of Δt:

(12.37)

The future value $Y(t+\frac{\Delta t}{2})$ can be obtained by two different approaches:

Predictor-corrector method

Since $f(Y(t+\frac{\Delta t}{2}),t_{n}+\frac{\Delta t}{2})$ is multiplied with Δt, it is sufficient to use an approximation with lower error order. Even the explicit Euler step is sufficient. Together the following algorithm results:

(12.38)

Averaging (Heun method)

The average of f(Y(t n ),t n ) and f(Y(t n +Δt),t+Δt) is another approximation to the midpoint value of comparable quality (Fig. 12.6).

Fig. 12.5

Improved Euler method

Fig. 12.6

Improved polygon (or Heun) method

Expansion around t n +Δt/2 gives

(12.39)

Inserting the average in (12.36) gives the following algorithm, which is also known as improved polygon method and corresponds to the trapezoidal rule for the integral (4.​13) or to a combination of explicit and implicit Euler step:

(12.40)

In the special case of a linear function f(Y(t),t)=FY(t) (for instance rotational motion or diffusion) this can be solved formally by

(12.41)

Numerically it is not necessary to perform the matrix inversion. Instead a linear system of equations is solved:

(12.42)

In certain cases the Heun method conserves the norm of the state vector, for instance if F has only imaginary eigenvalues (as for the 1-dimensional Schrödinger equation, see page 393).

In the general case a predictor step has to be made to estimate the state vector at t n +Δt before the Heun expression (12.40) can be evaluated:

(12.43)

12.6 Taylor Series Methods

Higher order methods can be obtained from a Taylor series expansion

(12.44)

The total time derivative can be expressed as

(12.45)

where the partial derivatives have been abbreviated in the usual way by $\frac{\partial f}{\partial t}=\dot{f}$ and $\frac{\partial f}{\partial Y}=f'$. Higher derivatives are given by

(12.46)

(12.47)

12.6.1 Nordsieck Predictor-Corrector Method

Nordsieck [184] determines an interpolating polynomial of degree m. As variables he uses the 0th to mth derivatives² evaluated at the current time t, for instance for m=5 he uses the variables

(12.48)

(12.49)

(12.50)

(12.51)

(12.52)

(12.53)

Taylor expansion gives approximate values at t+Δt

(12.54)

(12.55)

(12.56)

(12.57)

(12.58)

(12.59)

where the next term of the Taylor series $e(t)=\frac{\Delta t^{5}}{6!}\frac{\mathrm {d}^{6}}{\mathrm {d}t^{6}}Y(t)$ has been introduced as an approximation to the truncation error of the predicted values Y p ,g p , etc. It can be estimated from the second equation

(12.60)

This predictor-corrector method turns out to be rather unstable. However, stability can be achieved by slightly modifying the coefficients of the corrector step. Nordsieck suggested to use

(12.61)

(12.62)

(12.63)

(12.64)

(12.65)

12.6.2 Gear Predictor-Corrector Methods

Gear [101] designed special methods for molecular dynamics simulations (Chap. 14) where Newton’s law (12.15) has to be solved numerically. He uses again a truncated Taylor expansion for the predictor step

(12.66)

(12.67)

(12.68)

(12.69)

to calculate new coordinates etc. $\mathbf{r}_{n+1}^{p},\mathbf{v}_{n+1}^{p},\mathbf{a}_{n+1}^{p}\dots$ (Fig. 12.7). The difference between the predicted acceleration and that calculated using the predicted coordinates

(12.70)

is then used as a measure of the error to correct the predicted values according to

(12.71)

(12.72)

The coefficients c i were determined to optimize stability and accuracy. For instance the fourth order Gear corrector reads

(12.73)

(12.74)

(12.75)

Fig. 12.7

(Gear predictor-corrector method)  The difference between predicted acceleration a p and acceleration calculated for the predicted coordinates a(r p ) is used as a measure of the error to estimate the correction δ r

Gear methods are generally not time reversible and show systematic energy drifts. A reversible symplectic predictor-corrector method has been presented recently by Martyna and Tuckerman [169].

12.7 Runge-Kutta Methods

If higher derivatives are not so easily available, they can be approximated by numerical differences. f is evaluated at several trial points and the results are combined to reproduce the Taylor series as close as possible [48].

12.7.1 Second Order Runge-Kutta Method

Let us begin with two function values. As common in the literature we will denote the function values as K ₁,K ₂,… . From the gradient at time t n

(12.76)

we estimate the state vector at time t n +Δt as

(12.77)

The gradient at time t n +Δt is approximately

(12.78)

which has the Taylor series expansion

(12.79)

and application of the trapezoidal rule (4.​13) gives the 2nd order Runge-Kutta method

(12.80)

which in fact coincides with the improved Euler or Heun method. Taylor series expansion shows how the combination of K ₁ and K ₂ leads to an expression of higher error order:

(12.81)

12.7.2 Third Order Runge-Kutta Method

The accuracy can be further improved by calculating one additional function value at mid-time. From (12.76) we estimate the gradient at mid-time by

(12.82)

The gradient at time t n +Δt is then estimated as

(12.83)

Inserting the expansion (12.82) gives the leading terms

(12.84)

Applying Simpson’s rule (4.​14) we combine the three gradients to get the 3rd order Runge-Kutta method

(12.85)

where the Taylor series

(12.86)

recovers the exact Taylor series (12.44) including terms of order O(Δt ³).

12.7.3 Fourth Order Runge-Kutta Method

The 4th order Runge-Kutta method (RK4) is often used because of its robustness and accuracy. It uses two different approximations for the midpoint

and Simpson’s rule (4.​14) to obtain

Expansion of the Taylor series is cumbersome but with the help of an algebra program one can easily check that the error is of order Δt ⁵.

12.8 Quality Control and Adaptive Step Size Control

For practical applications it is necessary to have an estimate for the local error and to adjust the step size properly. With the Runge-Kutta method this can be achieved by a step doubling procedure. We calculate y n+2 first by two steps Δt and then by one step 2Δt. This needs 11 function evaluations as compared to 8 for the smaller step size only (Fig. 12.8). For the 4th order method we estimate the following errors:

(12.87)

(12.88)

The local error can be estimated from

The step size Δt can now be adjusted to keep the local error within the desired limits.

Fig. 12.8

Step doubling with the fourth order Runge-Kutta method

12.9 Extrapolation Methods

Application of the extrapolation method to calculate the integral $\int_{t_{n}}^{t_{n+1}}f(t)\,\mathrm{d}t$ produces very accurate results but can also be time consuming. The famous Gragg-Bulirsch-Stör method [244] starts from an explicit midpoint rule with a special starting procedure. The interval Δt is divided into a sequence of N sub-steps

(12.89)

First a simple Euler step is performed

(12.90)

and then the midpoint rule is applied repeatedly to obtain

(12.91)

Gragg [111] introduced a smoothing procedure to remove oscillations of the leading error term by defining

(12.92)

He showed that both approximations (12.91, 12.92) have an asymptotic expansion in powers of h ² and are therefore well suited for an extrapolation method. The modified midpoint method can be summarized as follows:

(12.93)

The number of sub-steps N is increased according to a sequence like

(12.94)

or

After each successive N is tried, a polynomial extrapolation is attempted. This extrapolation returns both the extrapolated values and an error estimate. If the error is still too large then N has to be increased further. A more detailed discussion can be found in [233, 234].

12.10 Linear Multistep Methods

All methods discussed so far evaluated one or more values of the gradient f(Y(t),t) only within the interval t n ⋯t n +Δt. If the state vector changes sufficiently smooth then multistep methods can be applied. Linear multistep methods use a combination of function values Y n and gradients f n from several steps

(12.95)

where the coefficients α, β are determined such, that a polynomial of certain order r is integrated exactly. The method is explicit if β ₀=0 and implicit otherwise. Multistep methods have a small local error and need fewer function evaluations. On the other hand, they have to be combined with other methods (like Runge-Kutta) to start and end properly and it can be rather complicated to change the step size during the calculation. Three families of linear multistep methods are commonly used: explicit Adams-Bashforth methods, implicit Adams-Moulton methods and backward differentiation formulas (also known as Gear formulas [102]).

12.10.1 Adams-Bashforth Methods

The explicit Adams-Bashforth method of order r uses the gradients from the last r−1 steps (Fig. 12.9) to obtain the polynomial

(12.96)

and to calculate the approximation

which is generally a linear combination of f n ⋯f n−r+1. For example, the Adams-Bashforth formulas of order 2, 3, 4 are:

(12.97)

Fig. 12.9

Adams-Bashforth method

12.10.2 Adams-Moulton Methods

The implicit Adams-Moulton method also uses the yet not known value Y n+1 (Fig. 12.10) to obtain the polynomial

(12.98)

The corresponding Adams-Moulton formulas of order 2 to 4 are:

(12.99)

(12.100)

Fig. 12.10

Adams-Moulton method

12.10.3 Backward Differentiation (Gear) Methods

Gear methods [102] are implicit and usually combined with a modified Newton method. They make use of previous function values Y n ,Y n−1… and the gradient f n+1 at time t+Δt. Only methods of order r≤6 are stable and useful. The general formula (12.95) is

(12.101)

For r=1 this becomes

(12.102)

and all linear polynomials

(12.103)

are integrated exactly if

(12.104)

which is the case for

(12.105)

Hence the first order Gear method is

(12.106)

which coincides with the implicit Euler method. The higher order stable Gear methods are given by

(12.107)

(12.108)

(12.109)

(12.110)

(12.111)

This class of algorithms is useful also for stiff problems (differential equations with strongly varying eigenvalues).

12.10.4 Predictor-Corrector Methods

The Adams-Bashforth-Moulton method combines the explicit method as a predictor step to calculate an estimate $y_{n+1}^{p}$ with a corrector step using the implicit method of same order. The general class of linear multistep predictor-corrector methods [100] uses a predictor step

(12.112)

which is corrected using the formula

(12.113)

and further iterations

(12.114)

(12.115)

The coefficients α,β have to be determined to optimize accuracy and stability.

12.11 Verlet Methods

For classical molecular dynamics simulations it is necessary to calculate very long trajectories. Here a family of symplectic methods often is used which conserve the phase space volume [3, 116, 193, 255, 257, 265]. The equations of motion of a classical interacting N-body system are

(12.116)

where the force acting on atom i can be calculated once a specific force field is chosen. Let us write these equations as a system of first order differential equations

(12.117)

where x(t) and v(t) are functions of time and the forces m a(x(t)) are functions of the time dependent coordinates.

12.11.1 Liouville Equation

We rewrite (12.117) as

(12.118)

where the Liouville operator $\mathcal{L}$ acts on the vector containing all coordinates and velocities:

(12.119)

The Liouville equation (12.118) can be formally solved by

(12.120)

For a better understanding let us evaluate the first members of the Taylor series of the exponential:

(12.121)

(12.122)

(12.123)

But since

(12.124)

(12.125)

we recover

(12.126)

12.11.2 Split-Operator Approximation

We introduce a small time step Δt=t/N and write

(12.127)

For the small time step Δt the split-operator approximation can be used which approximately factorizes the exponential operator. For example, write the Liouville operator as the sum of two terms

and make the approximation

(12.128)

Each of the two factors simply shifts positions or velocities

(12.129)

since these two steps correspond to either motion with constant velocities or constant coordinates and forces.

12.11.3 Position Verlet Method

Often the following approximation is used which is symmetrical in time

(12.130)

The corresponding algorithm is the so called position Verlet method (Fig. 12.11):

(12.131)

(12.132)

(12.133)

Fig. 12.11

(Position Verlet method)  The exact integration path is approximated by two half-steps with constant velocities and one step with constant coordinates

12.11.4 Velocity Verlet Method

If we exchange operators A and B we have

(12.134)

which produces the velocity Verlet algorithm (Fig. 12.12):

(12.135)

(12.136)

(12.137)

Fig. 12.12

(Velocity Verlet method)  The exact integration path is approximated by two half-steps with constant coordinates and one step with constant velocities

12.11.5 Störmer-Verlet Method

The velocity Verlet method is equivalent to Störmer’s version [208] of the Verlet method which is a two step method given by

(12.138)

(12.139)

To show the equivalence we add two consecutive position vectors

(12.140)

which simplifies to

(12.141)

This can be expressed as the difference of two consecutive velocities:

(12.142)

Now we substitute

(12.143)

to get

(12.144)

which simplifies to

(12.145)

Thus the equations of the velocity Verlet algorithm have been recovered. However, since the Verlet method is a 2-step method, the choice of initial values is important. The Störmer-Verlet method starts from two coordinate sets x ₀,x ₁. The first step is

(12.146)

(12.147)

The velocity Verlet method, on the other hand, starts from one set of coordinates and velocities x ₁, v ₁. Here the first step is

(12.148)

(12.149)

The two methods give the same resulting trajectory if we choose

(12.150)

If, on the other hand, x ₀ is known with higher precision, the local error order of Störmer’s algorithm changes as can be seen from addition of the two Taylor series

(12.151)

(12.152)

which gives

(12.153)

(12.154)

12.11.6 Error Accumulation for the Störmer-Verlet Method

Equation (12.153) gives only the local error of one single step. Assume the start values x ₀ and x ₁ are exact. The next value x ₂ has an error with the leading term Δx ₂=αΔt ⁴. If the trajectory is sufficiently smooth and the time step not too large the coefficient α will vary only slowly and the error of the next few iterations is given by

(12.155)

This shows that the effective error order of the Störmer-Verlet method is only O(Δt ³) similar to the velocity Verlet method.

12.11.7 Beeman’s Method

Beeman and Schofield [17, 229] introduced a method which is very similar to the Störmer-Verlet method but calculates the velocities with higher precision. This is important if, for instance, the kinetic energy has to be calculated. Starting from the Taylor series

(12.156)

the derivative of the acceleration is approximated by a backward difference

(12.157)

This equation can be used as an explicit step to update the coordinates or as a predictor step in combination with the implicit corrector step

(12.158)

which can be applied repeatedly (usually two iterations are sufficient). Similarly, the Taylor series of the velocity is approximated by

(12.159)

Inserting the velocity from (12.158) we obtain the corrector step for the velocity

(12.160)

In combination with (12.157) this can be replaced by

(12.161)

Together, (12.157) and (12.161) provide an explicit method which is usually understood as Beeman’s method. Inserting the velocity (12.160) from the previous step

(12.162)

into (12.157) gives

(12.163)

which coincides with the Störmer-Verlet method (12.138). We conclude that Beeman’s method should produce the same trajectory as the Störmer-Verlet method if numerical errors can be neglected and comparable initial values are used. In fact, the Störmer-Verlet method may suffer from numerical extinction and Beeman’s method provides a numerically more favorable alternative.

12.11.8 The Leapfrog Method

Closely related to the Verlet methods is the so called leapfrog method [116]. It uses the simple decomposition

(12.164)

but introduces two different time grids for coordinates and velocities which are shifted by Δt/2 (Fig. 12.13).

Fig. 12.13

(Leapfrog method)  The exact integration path is approximated by one step with constant coordinates and one step with constant velocities. Two different grids are used for coordinates and velocities which are shifted by Δt/2

The leapfrog algorithm is given by

(12.165)

(12.166)

Due to the shifted arguments the order of the method is increased as can be seen from the Taylor series:

(12.167)

(12.168)

One disadvantage of the leapfrog method is that some additional effort is necessary if the velocities are needed. The simple expression

(12.169)

is of lower error order than (12.168).

12.12 Problems

Problem 12.1

(Circular orbits)

In this computer experiment we consider a mass point moving in a central field. The equation of motion can be written as the following system of first order equations:

(12.170)

For initial values

(12.171)

the exact solution is given by

(12.172)

The following methods are used to calculate the position x(t),y(t) and the energy

(12.173)

The explicit Euler method (12.3)

(12.174)

The 2nd order Runge-Kutta method (12.7.1)

which consists of the predictor step

(12.175)

(12.176)

(12.177)

(12.178)

and the corrector step

(12.179)

(12.180)

(12.181)

(12.182)

The fourth order Runge-Kutta method (12.7.3)

The Verlet method (12.11.5)

(12.183)

(12.184)

(12.185)

(12.186)

To start the Verlet method we need additional coordinates at time −Δt which can be chosen from the exact solution or from the approximation

(12.187)

(12.188)

The leapfrog method (12.11.8)

(12.189)

(12.190)

(12.191)

(12.192)

where the velocity at time t n is calculated from

(12.193)

(12.194)

To start the leapfrog method we need the velocity at time $t_{-\frac{1}{2}}$ which can be taken from the exact solution or from

(12.195)

(12.196)

Compare the conservation of energy for the different methods as a function of the time step Δt. Study the influence of the initial values for leapfrog and Verlet methods.

Problem 12.2

(N-body system)

In this computer experiment we simulate the motion of three mass points under the influence of gravity. Initial coordinates and velocities as well as the masses can be varied. The equations of motion are solved with the 4th order Runge-Kutta method with quality control for different step sizes. The local integration error is estimated using the step doubling method. Try to simulate a planet with a moon moving round a sun!

Problem 12.3

(Adams-Bashforth method)

In this computer experiment we simulate a circular orbit with the Adams-Bashforth method of order 2⋯7. The absolute error at time T

(12.197)

is shown as a function of the time step Δt in a log-log plot. From the slope

(12.198)

the leading error order s can be determined. For very small step sizes rounding errors become dominating which leads to an increase Δ∼(Δt)⁻¹.

Determine maximum precision and optimal step size for different orders of the method. Compare with the explicit Euler method.

References

3.

M.P. Allen, D.J. Tildesley, Computer Simulation of Liquids (Oxford University Press, London, 1989). ISBN 0-19-855645-4

17.

D. Beeman, J. Comp. Physiol. 20, 130 (1976) ADS

48.

J.C. Butcher, The Numerical Analysis of Ordinary Differential Equations: Runge-Kutta and General Linear Methods (Wiley, New York, 1987) MATH

100.

C.W. Gear, Math. Comput. 21, 146 (1967) MathSciNetMATHCrossRef

101.

C.W. Gear, Numerical Initial Value Problems in Ordinary Differential Equations (Prentice Hall, Englewood Cliffs, 1971) MATH

102.

C.W. Gear, IEEE Trans. Circuit Theory 18, 89–95 (1971) CrossRef

111.

W.B. Gragg, SIAM J. Numer. Anal. 2, 384 (1965) MathSciNetADS

116.

E. Hairer, C. Lubich, G. Wanner, Acta Numer. 12, 399 (2003) MathSciNetMATHCrossRef

169.

G.J. Martyna, M.E. Tuckerman, J. Chem. Phys. 102, 8071 (1995) ADSCrossRef

184.

A. Nordsieck, Math. Comput. 16, 22 (1962) MathSciNetADSMATHCrossRef

193.

I.P. Omelyan, I.M. Mryglod, R. Folk, Comput. Phys. Commun. 151, 272 (2003) MathSciNetADSMATHCrossRef

208.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Second-order conservative equations, in Numerical Recipes: The Art of Scientific Computing, 3rd edn. (Cambridge University Press, Cambridge, 2007), Sect. 17.4

229.

P. Schofield, Comput. Phys. Commun. 5, 17 (1973) ADSCrossRef

233.

L.F. Shampine, IMA J. Numer. Anal. 3, 383 (1983) MathSciNetMATHCrossRef

234.

L.F. Shampine, L.S. Baca, Numer. Math. 41, 165 (1983) MathSciNetMATHCrossRef

244.

J. Stör, R. Bulirsch, Introduction to Numerical Analysis, 3rd revised edn. (Springer, New York, 2010). ISBN 978-1441930064

255.

S.-H. Tsai et al., Braz. J. Phys. 34, 384 (2004) ADSCrossRef

257.

M. Tuckerman, B.J. Berne, J. Chem. Phys. 97, 1990 (1992) ADSCrossRef

265.

L. Verlet, Phys. Rev. 159, 98 (1967) ADSCrossRef

Footnotes

1

Control of the step width will be discussed later.

2

In fact the derivatives of the interpolating polynomial which exist even if higher derivatives of f do not exist.

Part 2
Simulation of Classical and Quantum Systems

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_13© Springer International Publishing Switzerland 2013

13. Rotational Motion

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

An asymmetric top under the influence of time dependent external forces is a rather complicated subject in mechanics. Efficient methods to describe the rotational motion are important as well in astrophysics as in molecular physics. The orientation of a rigid body relative to the laboratory system can be described by a 3×3 matrix. Instead of solving nine equations for all its components, the rotation matrix can be parametrized by the four real components of a quaternion. Euler angles use the minimum necessary number of three parameters but have numerical disadvantages. Care has to be taken to conserve the orthogonality of the rotation matrix. Omelyan’s implicit quaternion method is very efficient and conserves orthogonality exactly. In computer experiments we compare different explicit and implicit methods for a free rotor, we simulate a rotor in an external field and the collision of two rotating molecules.

An asymmetric top under the influence of time dependent external forces is a rather complicated subject in mechanics. Efficient methods to describe the rotational motion are important as well in astrophysics as in molecular physics. The orientation of a rigid body relative to the laboratory system can be described by a 3×3 matrix. Instead of solving nine equations for all its components, the rotation matrix can be parametrized by the four real components of a quaternion. Euler angles use the minimum necessary number of three parameters but have numerical disadvantages. Care has to be taken to conserve the orthogonality of the rotation matrix. Omelyan’s implicit quaternion method is very efficient and conserves orthogonality exactly. In computer experiments we compare different explicit and implicit methods for a free rotor, we simulate a rotor in an external field and the collision of two rotating molecules.

13.1 Transformation to a Body Fixed Coordinate System

Let us define a rigid body as a set of mass points m i with fixed relative orientation (described by distances and angles).

The position of m i in the laboratory coordinate system CS will be denoted by r i . The position of the center of mass (COM) of the rigid body is

(13.1)

and the position of m i within the COM coordinate system CS c (Fig. 13.1) is ρ i :

(13.2)

Let us define a body fixed coordinate system CS cb , where the position ρ ib of m i is time independent $\frac{\mathrm{d}}{\mathrm{d} t}\boldsymbol{\rho}_{ib}=0$. ρ i and ρ ib are connected by a linear vector function

(13.3)

where A is a 3×3 matrix

(13.4)

Fig. 13.1

(Coordinate systems) Three coordinate systems will be used: The laboratory system CS, the center of mass system CS c and the body fixed system CS cb

13.2 Properties of the Rotation Matrix

Rotation conserves the length of ρ:¹

(13.5)

Consider the matrix

(13.6)

for which

(13.7)

holds for all vectors ρ. Let us choose the unit vector in x-direction:

Then we have

(13.8)

Similarly by choosing a unit vector in y or z direction we find M ₂₂=M ₃₃=0.

Now choose :

(13.9)

Since the diagonal elements vanish we have M ₁₂=−M ₂₁. With

we find M ₁₃=−M ₃₁ and M ₂₃=−M ₃₂, hence M is skew symmetric and has three independent components

(13.10)

Inserting (13.6) we have

(13.11)

which shows that A T A=1 or equivalently A T =A ⁻¹. Hence (det(A))²=1 and A is an orthogonal matrix. For a pure rotation without reflection only det(A)=+1 is possible.

From

(13.12)

we calculate the velocity

(13.13)

but since ρ ib is constant by definition, the last summand vanishes

(13.14)

and in the center of mass system we have

(13.15)

with the matrix

(13.16)

13.3 Properties of W, Connection with the Vector of Angular Velocity

Since rotation does not change the length of ρ i , we have

(13.17)

or in matrix notation

(13.18)

This holds for arbitrary ρ i . Hence W is skew symmetric and has three independent components

(13.19)

Now consider an infinitesimal rotation by the angle dφ (Fig. 13.2).

Fig. 13.2

Infinitesimal rotation

Then we have (the index i is suppressed)

(13.20)

which can be written as a cross product:

(13.21)

with

(13.22)

But this can be expressed in terms of the angular velocity ω as

(13.23)

and finally we have

(13.24)

and the more common form of the equation of motion

(13.25)

Example

(Rotation around the z-axis)

For constant angular velocity ω the equation of motion

(13.26)

has the formal solution

(13.27)

The angular velocity vector for rotation around the z-axis is

(13.28)

and

(13.29)

Higher powers of W can be easily calculated since

(13.30)

(13.31)

etc., and the rotation matrix is obtained from the Taylor series

(13.32)

13.4 Transformation Properties of the Angular Velocity

Now imagine we are sitting on the rigid body and observe a mass point moving outside. Its position in the laboratory system is r ₁. In the body fixed system we observe it at

(13.33)

and its velocity in the body fixed system is

(13.34)

The time derivative of the inverse matrix follows from

(13.35)

(13.36)

and hence

(13.37)

Now we rewrite this using the angular velocity as it is observed in the body fixed system

(13.38)

where W transforms as like a rank-2 tensor

(13.39)

From this equation the transformation properties of ω can be derived. We consider only rotation around one axis explicitly, since a general rotation matrix can always be written as a product of three rotations around different axes. For instance, rotation around the z-axis gives:

(13.40)

which shows that

(13.41)

i.e. ω transforms like a vector under rotations. However, there is a subtle difference considering general coordinate transformations involving reflections. For example, under reflection at the xy-plane W is transformed according to

(13.42)

and the transformed angular velocity vector is

(13.43)

This is characteristic of a so called axial or pseudo-vector. Under a general coordinate transformation it transforms as

(13.44)

13.5 Momentum and Angular Momentum

The total momentum is

(13.45)

since by definition we have ∑ i m i ρ i =0.

The total angular momentum can be decomposed into the contribution of the center of mass motion and the contribution relative to the center of mass

(13.46)

The second contribution is

(13.47)

This is a linear vector function of ω, which can be expressed simpler by introducing the tensor of inertia

(13.48)

or component-wise

(13.49)

as

(13.50)

13.6 Equations of Motion of a Rigid Body

Let F i be an external force acting on m i . Then the equation of motion for the center of mass is

(13.51)

If there is no total external force F ext , the center of mass moves with constant velocity

(13.52)

The time derivative of the angular momentum equals the total external torque

(13.53)

which can be decomposed into

(13.54)

With the decomposition of the angular momentum

(13.55)

we have two separate equations for the two contributions:

(13.56)

(13.57)

13.7 Moments of Inertia

The angular momentum (13.50) is

(13.58)

where the tensor of inertia in the body fixed system is

(13.59)

Since I b does not depend on time (by definition of the body fixed system) we will use the principal axes of I b as the axes of the body fixed system. Then I b takes the simple form

(13.60)

with the principle moments of inertia I 1,2,3.

13.8 Equations of Motion for a Rotor

The following equations describe pure rotation of a rigid body:

(13.61)

(13.62)

(13.63)

(13.64)

(13.65)

(13.66)

13.9 Explicit Methods

Equation (13.61) for the rotation matrix and (13.62) for the angular momentum have to be solved by a suitable algorithm. The simplest integrator is the explicit Euler method (Fig. 13.3) [241]:

(13.67)

(13.68)

Expanding the Taylor series of A(t) to second order we have the second order approximation (Fig. 13.3)

(13.69)

Fig. 13.3

(Global error of the explicit methods) The equations of a free rotor (13.8) are solved using the explicit first order (full curves) and second order (dashed curves) method. The deviations |det(A)−1| (diamonds) and |E kin −E kin (0)| (circles) at t=10 are shown as a function of the time step Δt. Full circles show the energy deviation of the first order method with reorthogonalization. The principal moments of inertia are I b =diag(1,2,3) and the initial angular momentum is L=(1,1,1). See also Problem 13.1

A corresponding second order expression for the angular momentum involves the time derivative of the forces and is usually not practicable.

The time derivative of W can be expressed via the time derivative of the angular velocity which can be calculated as follows:

(13.70)

Alternatively, in the laboratory system

(13.71)

where the first summand vanishes due to

(13.72)

Substituting the angular momentum we have

(13.73)

which reads in components:

(13.74)

Evaluation of the product gives a set of equations which are well known as Euler’s equations:

(13.75)

13.10 Loss of Orthogonality

The simple methods above do not conserve the orthogonality of A. This is an effect of higher order but the error can accumulate quickly. Consider the determinant of A. For the simple explicit Euler scheme we have

(13.76)

The error is of order Δt ², but the determinant will continuously increase, i.e. the rigid body will explode. For the second order integrator we find

(13.77)

This can be simplified to give

(13.78)

The second order method behaves somewhat better since the product of angular velocity and acceleration can change in time. To assure that A remains a rotation matrix we must introduce constraints or reorthogonalize A at least after some steps (for instance every time when |det(A)−1| gets larger than a certain threshold). The following method with a symmetric correction matrix is a very useful alternative [127]. The non-singular square matrix A can be decomposed into the product of an orthonormal matrix $\widetilde{A}$ and a positive semi-definite matrix S

(13.79)

with the positive definite square root of the symmetric matrix A T A

(13.80)

and

(13.81)

which is orthonormal as can be seen from

(13.82)

Since the deviation of A from orthogonality is small, we make the approximations

(13.83)

(13.84)

(13.85)

(13.86)

which can be easily evaluated.

13.11 Implicit Method

The quality of the method can be significantly improved by taking the time derivative at midstep (Fig. 13.4) (12.​5):

(13.87)

(13.88)

Taylor series expansion gives

(13.89)

(13.90)

which has the same error order as the explicit second order method. The matrix $A(t+\frac{\Delta t}{2})$ at mid-time can be approximated by

(13.91)

which does not change the error order of the implicit integrator which now becomes

(13.92)

This equation can be formally solved by

(13.93)

Alternatively, using angular velocities in the laboratory system we have the similar expression

(13.94)

The angular velocities at midtime can be calculated with sufficient accuracy from

(13.95)

With the help of an algebra program we easily prove that

(13.96)

and therefore the determinant of the rotation matrix is conserved. The necessary matrix inversion can be easily done:

(13.97)

The matrix product is explicitly

(13.98)

With the help of an algebra program it can be proved that this matrix is even orthogonal

(13.99)

and hence the orthonormality of A is conserved. The approximation for the angular momentum

(13.100)

can be used in an implicit way

(13.101)

Alternatively Euler’s equations can be used in the form [190, 191]

(13.102)

where the product ω b2(t)ω b3(t) is approximated by

(13.103)

$\omega_{b1}(t+\frac{\Delta t}{2})$ is determined by iterative solution of the last two equations. Starting with $\omega_{b1}(t-\frac{\Delta t}{2})$ convergence is achieved after few iterations.

Fig. 13.4

(Global error of the implicit method) The equations of a free rotor (13.8) are solved using the implicit method. The deviations |det(A)−1| (diamonds) and |E kin −E kin (0)| (circles) at t=10 are shown as a function of the time step Δt. Initial conditions as in Fig. 13.3. See also Problem 13.1

Example

(Free symmetric rotor)

For the special case of a free symmetric rotor (I b2=I b3,N int =0) Euler’s equations simplify to:

(13.104)

(13.105)

(13.106)

(13.107)

Coupled equations of this type appear often in physics. The solution can be found using a complex quantity

(13.108)

which obeys the simple differential equation

(13.109)

with the solution

(13.110)

Finally

(13.111)

i.e. ω b rotates around the 1-axis with frequency λ.

13.12 Kinetic Energy of a Rotor

The kinetic energy of the rotor is

(13.112)

The second part is the contribution of the rotational motion. It can be written as

(13.113)

since

(13.114)

13.13 Parametrization by Euler Angles

So far we had to solve equations for all 9 components of the rotation matrix. But there are six constraints since the column vectors of the matrix have to be orthonormalized. Therefore the matrix can be parametrized with less than 9 variables. In fact it is sufficient to use only three variables. This can be achieved by splitting the full rotation into three rotations around different axis. Most common are Euler angles defined by the orthogonal matrix [106]

(13.115)

obeying the equations

(13.116)

(13.117)

(13.118)

Different versions of Euler angles can be found in the literature, together with the closely related cardanic angles. For all of them a sinθ appears in the denominator which causes numerical instabilities at the poles. One possible solution to this problem is to switch between two different coordinate systems.

13.14 Cayley-Klein Parameters, Quaternions, Euler Parameters

There exists another parametrization of the rotation matrix which is very suitable for numerical calculations. It is connected with the algebra of the so called quaternions. The vector space of the complex 2×2 matrices can be spanned using Pauli matrices by

(13.119)

Any complex 2×2 matrix can be written as a linear combination

(13.120)

Accordingly any vector x∈R ³ can be mapped onto a complex 2×2 matrix:

(13.121)

Rotation of the coordinate system leads to the transformation

(13.122)

where

(13.123)

is a complex 2×2 rotation matrix. Invariance of the length ($|\mathbf{x}|=\sqrt{-\det(P)}$) under rotation implies that Q must be unitary, i.e. Q †=Q ⁻¹ and its determinant must be 1. Explicitly

(13.124)

and Q has the form

(13.125)

Setting x ±=x±iy, the transformed matrix has the same form as P:

(13.126)

From comparison we find the transformed vector components:

(13.127)

(13.128)

(13.129)

This gives us the rotation matrix in terms of the Cayley-Klein parameters α and β:

(13.130)

For practical calculations one often prefers to have four real parameters instead of two complex ones. The so called Euler parameters q ₀,q ₁,q ₂,q ₃ are defined by

(13.131)

Now the matrix Q

(13.132)

becomes a so called quaternion which is a linear combination of the four matrices

(13.133)

which obey the following multiplication rules:

(13.134)

In terms of Euler parameters the rotation matrix reads

(13.135)

and from the equation $\dot{A}=WA$ we derive the equation of motion for the quaternion

(13.136)

or from $\dot{A}=AW_{b}$ the alternative equation

(13.137)

Both of these equations can be written briefly in the form

(13.138)

Example

(Rotation around the z-axis)

Rotation around the z-axis corresponds to the quaternion with Euler parameters

(13.139)

as can be seen from the rotation matrix

(13.140)

The time derivative of q obeys the equation

(13.141)

After a rotation by 2π the quaternion changes its sign, i.e. q and −q parametrize the same rotation matrix!

13.15 Solving the Equations of Motion with Quaternions

As with the matrix method we can obtain a simple first or second order algorithm from the Taylor series expansion

(13.142)

Now only one constraint remains, which is the conservation of the norm of the quaternion. This can be taken into account by rescaling the quaternion whenever its norm deviates too much from unity.

It is also possible to use Omelyan’s [192] method:

(13.143)

gives

(13.144)

where the inverse matrix is

(13.145)

and the matrix product

(13.146)

This method conserves the norm of the quaternion and works quite well.

13.16 Problems

Problem 13.1

(Free rotor, Fig. 13.5)

In this computer experiment we compare different methods for a free rotor (Sect. 13.8):

explicit first order method (13.67)

(13.147)

explicit second order method (13.69)

(13.148)

implicit second order method (13.93)

(13.149)

Fig. 13.5

Free asymmetric rotor

The explicit methods can be combined with reorthogonalization according to (13.79) or with the Gram-Schmidt method. Reorthogonalization threshold and time step can be varied and the error of kinetic energy and determinant are plotted as a function of the total simulation time.

Problem 13.2

(Rotor in a field, Fig. 13.6)

In this computer experiment we simulate a molecule with a permanent dipole moment in a homogeneous electric field E. We neglect vibrations and describe the molecule as a rigid body consisting of nuclei with masses m i and partial charges Q i . The total charge is ∑ i Q i =0. The dipole moment is

(13.150)

and external force and torque are

(13.151)

(13.152)

The angular momentum changes according to

(13.153)

where the dipole moment is constant in the body fixed system. We use the implicit integrator for the rotation matrix (13.93) and the equation

(13.154)

to solve the equations of motion numerically. Obviously the component of the angular momentum parallel to the field is constant. The potential energy is

(13.155)

Fig. 13.6

Rotor in an electric field

Problem 13.3

(Molecular collision)

This computer experiment simulates the collision of two rigid methane molecules (Fig. 13.7). The equations of motion are solved with the implicit quaternion method (13.143) and the velocity Verlet method (12.​11.​4). The two molecules interact by a standard 6–12 Lennard-Jones potential (14.​24) [3]. For comparison the attractive r ⁻⁶ part can be switched off. The initial angular momenta as well as the initial velocity v and collision parameter b can be varied. Total energy and momentum are monitored and the decomposition of the total energy into translational, rotational and potential energy are plotted as a function of time.

Fig. 13.7

Molecular collision

Study the exchange of momentum and angular momentum and the transfer of energy between translational and rotational degrees of freedom.

References

3.

M.P. Allen, D.J. Tildesley, Computer Simulation of Liquids (Oxford University Press, London, 1989). ISBN 0-19-855645-4

106.

H. Goldstein, Klassische Mechanik (Akademische Verlagsgesellschaft, Frankfurt am Main, 1974) MATH

127.

B.K.P. Horn, H.M. Hilden, S. Negahdaripour, J. Opt. Soc. Am. A 5, 1127 (1988) MathSciNetADSCrossRef

190.

I.P. Omelyan, Phys. Rev. 58, 1169 (1998) ADSCrossRef

191.

I.P. Omelyan, Comput. Phys. Commun. 109, 171 (1998) ADSMATHCrossRef

192.

I.P. Omelyan, Comput. Phys. 12, 97 (1998) ADSCrossRef

241.

R. Sonnenschein, A. Laaksonen, E. Clementi, J. Comput. Chem. 7, 645 (1986) CrossRef

Footnotes

1

ρ T ρ denotes the scalar product of two vectors whereas ρρ T is the outer or matrix product.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_14© Springer International Publishing Switzerland 2013

14. Molecular Mechanics

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Classical molecular mechanics simulations have become a very valuable tool for the investigation of atomic and molecular systems, mainly in the area of materials science and molecular biophysics. They use a classical energy function which treats the atoms as mass points interacting by classical forces. Simulations of larger molecules use empirical force fields, which approximate the potential energy surface of the electronic ground state. We discuss the most important interaction terms, which are conveniently expressed in internal coordinates, i.e. bond lengths, bond angles and dihedral angles. We derive expressions for the gradients of the force field with respect to Cartesian coordinates. In a computer experiment we simulate a glycine dipeptide and demonstrate the principles of energy minimization, normal mode analysis of small amplitude motions around an equilibrium geometry and the simulation of molecular dynamics.

Classical molecular mechanics simulations have become a very valuable tool for the investigation of atomic and molecular systems [97, 115, 157, 212, 225], mainly in the area of materials science and molecular biophysics. Based on the Born-Oppenheimer separation which assumes that the electrons move much faster than the nuclei, nuclear motion is described quantum mechanically by the Hamiltonian

(14.1)

Molecular mechanics uses the corresponding classical energy function

(14.2)

which treats the atoms as mass points interacting by classical forces

(14.3)

Stable structures, i.e. local minima of the potential energy can be found by the methods discussed in Chap. 6. Small amplitude motions around an equilibrium geometry are described by a harmonic normal mode analysis. Molecular dynamics (MD) simulations solve the classical equations of motion

(14.4)

numerically.

The potential energy function $U (\mathbf{r}_{j}^{\mathit{Nuc}} )$ can be calculated with simplified quantum methods for not too large systems [50, 152]. Classical MD simulations for larger molecules use empirical force fields, which approximate the potential energy surface of the electronic ground state. They are able to describe structural and conformational changes but not chemical reactions which usually involve more than one electronic state. Among the most popular classical force fields are AMBER [63], CHARMM [163] and GROMOS [58, 262].

In this chapter we discuss the most important interaction terms, which are conveniently expressed in internal coordinates, i.e. bond lengths, bond angles and dihedral angles. We derive expressions for the gradients of the force field with respect to Cartesian coordinates. In a computer experiment we simulate a glycine dipeptide and demonstrate the principles of energy minimization, normal mode analysis and dynamics simulation.

14.1 Atomic Coordinates

The most natural coordinates for the simulation of molecules are the Cartesian coordinates (Fig. 14.1) of the atoms,

(14.5)

which can be collected into a 3N-dimensional vector

(14.6)

The second derivatives of the Cartesian coordinates appear directly in the equations of motion (14.4)

(14.7)

Cartesian coordinates have no direct relation to the structural properties of molecules. For instance a protein is a long chain of atoms (the so called backbone) with additional side groups (Fig. 14.2).

Fig. 14.1

(Molecular coordinates)  Cartesian coordinates (left) are used to solve the equations of motion whereas the potential energy is more conveniently formulated in internal coordinates (right)

Fig. 14.2

(Conformation of a protein)  The relative orientation of two successive protein residues can be described by three angles (Ψ,Φ,ω)

The protein structure can be described more intuitively with the help of atomic distances and angles. Internal coordinates are (Fig. 14.3) distances between two bonded atoms (bond lengths)

Fig. 14.3

(Internal coordinates)  The structure of a molecule can be described by bond lengths, bond angles and dihedral angles

(14.8)

angles between two bonds (bond angles)

(14.9)

and dihedral angles which describe the planarity and torsions of the molecule. A dihedral angle (Fig. 14.4) is the angle between two planes which are defined by three bonds

(14.10)

(14.11)

where the conventional sign of the dihedral angle [136] is determined by

(14.12)

Fig. 14.4

Dihedral angle

Internal coordinates are very convenient for the formulation of a force field. On the other hand, the kinetic energy (14.2) becomes complicated if expressed in internal coordinates. Therefore both kinds of coordinates are used in molecular dynamics calculations. The internal coordinates are usually arranged in Z-matrix form. Each line corresponds to one atom i and shows its position relative to three atoms j, k, l in terms of the bond length b ij , the bond angle ϕ ijk and the dihedral angle θ ijkl (Fig. 14.5 and Table 14.1).

Fig. 14.5

(Glycine dipeptide model)  The glycine dipeptide is the simplest model for a peptide. It is simulated in Problem 14.1. Optimized internal coordinates are shown in Table 14.1

Table 14.1

(Z-matrix)  The optimized values of the internal coordinates from Problem 14.1 are shown in Z-matrix form. Except for the first three atoms the position of atom i is given by its distance b ij to atom j, the bond angle ϕ ijk and the dihedral angle θ ijkl

Number i

Label

j

k

l

Bond length (Å) b ij

Bond angle ϕ ijk

Dihedral θ ijkl

1

N1

2

C2

1

1.45

3

C3

2

1

1.53

108.6

4

N4

3

2

1

1.35

115.0

160.7

5

C5

4

3

2

1.44

122.3

−152.3

6

C6

5

4

3

1.51

108.7

−153.1

7

O7

3

2

1

1.23

121.4

−26.3

8

O8

6

5

4

1.21

124.4

123.7

9

O9

6

5

4

1.34

111.5

−56.5

10

H10

1

2

3

1.02

108.7

−67.6

11

H11

1

2

3

1.02

108.7

49.3

12

H12

2

3

4

1.10

109.4

−76.8

13

H13

2

3

4

1.10

109.4

38.3

14

H14

4

3

2

1.02

123.1

27.5

15

H15

5

4

3

1.10

111.2

−32.5

16

H16

5

4

3

1.10

111.1

86.3

17

H17

9

6

5

0.97

106.9

−147.4

14.2 Force Fields

Classical force fields are usually constructed as an additive combination of many interaction terms. Generally these can be divided into intramolecular contributions U bonded which determine the configuration and motion of a single molecule and intermolecular contributions U non-bonded describing interactions between different atoms or molecules

(14.13)

14.2.1 Intramolecular Forces

The most important intramolecular forces depend on the deviation of bond lengths, bond angles and dihedral angles from their equilibrium values. For simplicity a sum of independent terms is used as for the CHARMM force field [39, 40, 163]

(14.14)

The forces are derived from potential functions which are in the simplest case approximated by harmonic oscillator parabolas (Fig. 14.6), like the bond stretching energy

(14.15)

angle bending terms

(14.16)

together with the Urey-Bradly correction

(14.17)

and “improper dihedral” terms which are used to keep planarity

(14.18)

Torsional energy contributions are often described by a cosine function¹

(14.19)

where m=1,2,3,4,6 describes the symmetry. For instance m=3 for the three equivalent hydrogen atoms of a methyl group. In most cases the phase shift $\theta_{ijkl}^{0}=0$ or $\theta_{ijkl}^{0}=\pi$. Then the dihedral potential can be expanded as a polynomial of cosθ, for instance

(14.20)

(14.21)

(14.22)

Fig. 14.6

Intramolecular forces

For more general $\theta_{ijkl}^{0}$ the torsional potential can be written as a polynomial of cosθ ijkl and sinθ ijkl .

The atoms are classified by element and bonding environment. Atoms of the same atom type are considered equivalent and the parameters transferable (for an example see Tables 14.2, 14.3, 14.4).

Table 14.2

(Atom types of the glycine dipeptide)  Atom types for glycine oligopeptides according to Bautista and Seminario [15]. The atoms are classified by element and bonding environment. Atoms of the same atom type are considered equivalent

Atom type

Atoms

C

C3,

C₁

C2, C5

C₂

C6

N

N4

N₂

N1

O

O7

O₁

O9

O₂

O8

H

H14

H₁

H12, H13, H15, H16

H₂

H17

H₃

H10, H11

Table 14.3

(Bond stretching parameters)  Equilibrium bond lengths (Å) and force constants (kcal mol⁻¹ Å⁻²) for the glycine dipeptide from [15]

Bond type

b ⁰

k

Bonds

r C,N

1.346

1296.3

C3-N4

r C1,N

1.438

935.5

N4-C5

r C1,N2

1.452

887.7

N1-C2

r C2,C1

1.510

818.9

C5-C6

r C,C1

1.528

767.9

C2-C3

r C2,O2

1.211

2154.5

C6-O8

r C,O

1.229

1945.7

C3-O7

r C2,O1

1.339

1162.1

C6-O9

r N,H

1.016

1132.4

N4-H14

r N2,H3

1.020

1104.5

N1-H10, N1-H11

r C1,H1

1.098

900.0

C2-H12, C2-H13, C5-H15, C5-H16

r O1,H2

0.974

1214.6

O9-H17

Table 14.4

(Bond angle parameters)  Equilibrium bond angles (deg) and force constants (kcal mol⁻¹ rad⁻²) for the glycine dipeptide from [15]

Angle type

ϕ ⁰

k

Angles

ϕ N,C,C1

115.0

160.0

C2-C3-N4

ϕ C1,N,C

122.3

160.1

C3-N4-C5

ϕ C1,C2,O1

111.5

156.0

C5-C6-O9

ϕ C1,C2,O2

124.4

123.8

C5-C6-O8

ϕ C1,C,O

121.4

127.5

C2-C3-O7

ϕ O2,C2,O1

124.1

146.5

O8-C6-O9

ϕ N,C,O

123.2

132.7

N4-C3-O7

ϕ C,C1,H1

110.1

74.6

H12-C2-C3, H13-C2-C3

ϕ C2,C1,H1

109.4

69.6

H16-C5-C6, H15-C5-C6

ϕ C,N,H

123.1

72.0

C3-N4-H14

ϕ C1,N,H

114.6

68.3

C5-N4-H14

ϕ C1,N2,H3

108.7

71.7

H10-N1-C2, H11-N1-C2

ϕ H1,C1,H1

106.6

48.3

H13-C2-H12, H15-C5-H16

ϕ H3,N2,H3

107.7

45.2

H10-N1-H11

ϕ C,C1,N2

109.0

139.8

N1-C2-C3

ϕ C2,C1,N

108.6

129.0

N4-C5-C6

ϕ C2,O1,H2

106.9

72.0

H17-O9-C6

ϕ N,C1,H1

111.1

73.3

H15-C5-N4, H16-C5-N4

ϕ N2,C1,H1

112.6

80.1

H13-C2-N1, H12-C2-N1

14.2.2 Intermolecular Interactions

Interactions between non-bonded atoms

(14.23)

include the Coulomb interaction and the weak attractive van der Waals forces which are usually combined with a repulsive force at short distances to account for the Pauli principle. Very often a sum of pairwise Lennard-Jones potentials is used (Fig. 14.7) [3]

(14.24)

Fig. 14.7

(Lennard-Jones potential)  The 6–12 potential (14.24) has its minimum at $r_{\min}=\sqrt[6]{2}\sigma\approx1.12\sigma$ with U min=−ϵ

The charge distribution of a molecular system can be described by a set of multipoles at the position of the nuclei, the bond centers and further positions (lone pairs for example). Such distributed multipoles can be calculated quantum chemically for not too large molecules. In the simplest models only partial charges are taken into account giving the Coulomb energy as a sum of atom-atom interactions

(14.25)

More sophisticated force fields include higher charge multipoles and polarization effects.

14.3 Gradients

The equations of motion are usually solved in Cartesian coordinates and the gradients of the potential are needed in Cartesian coordinates. Since the potential depends only on relative position vectors r ij , the gradient with respect to a certain atom position r k can be calculated from

(14.26)

Therefore it is sufficient to calculate gradients with respect to the difference vectors. Numerically efficient methods to calculate first and second derivatives of many force field terms are given in [156, 259, 260]. The simplest potential terms depend only on the distance of two atoms. For instance bond stretching terms, Lennard-Jones and Coulomb energies have the form

(14.27)

where the gradient is

(14.28)

The most important gradients of this kind are

(14.29)

(14.30)

(14.31)

The gradient of the harmonic bond angle potential is

(14.32)

where the gradient of the angle can be calculated from the gradient of its cosine

(14.33)

(14.34)

In principle, the sine function in the denominator could lead to numerical problems which can be avoided by treating angles close to 0 or π separately or using a function of cosϕ ijk like the trigonometric potential

(14.35)

instead [5, 213, 224]. Alternatively, the gradient of ϕ can be brought to a form which is free of singularities by expressing the sine in the denominator by a cosine [59]

(14.36)

and similarly

(14.37)

Gradients of the dihedral potential are most easily calculated for $\theta_{ijkl}^{0}=0$ or π. In that case, the dihedral potential is a polynomial of cosθ ijkl only (14.20)–(14.22) and

(14.38)

whereas in the general case 0<θ ijkl <π application of the chain rule gives

(14.39)

If this is evaluated with the help of

(14.40)

singularities appear for θ=0 and π. The same is the case for the gradients of the harmonic improper potential

(14.41)

Again, one possibility which has been often used, is to treat angles close to 0 or π separately [39]. However, the gradient of the angle θ ijkl can be calculated directly, which is much more efficient [19].

The gradient of the cosine follows from application of the product rule

(14.42)

First we derive the differentiation rule

(14.43)

which helps us to find

(14.44)

(14.45)

(14.46)

and

(14.47)

(14.48)

(14.49)

(14.50)

Finally we collect terms to obtain the gradients of the cosine [59]

(14.51)

(14.52)

(14.53)

14.4 Normal Mode Analysis

The nuclear motion around an equilibrium configuration can be approximately described as the combination of independent harmonic normal modes. Equilibrium configurations can be found with the methods discussed in Sect. 6.​2. The convergence is usually rather slow (Fig. 14.8) except for the full Newton-Raphson method, which needs the calculation and inversion of the Hessian matrix.

Fig. 14.8

(Convergence of energy and gradient)  The energy of the glycine dipeptide is minimized with the methods of steepest descent and conjugate gradients

14.4.1 Harmonic Approximation

At an equilibrium configuration

(14.54)

the gradient of the potential energy vanishes. For small deviations from the equilibrium

(14.55)

approximation by a truncated Taylor series gives

(14.56)

and the equations of motion are approximately

(14.57)

Assuming periodic oscillations

(14.58)

we have

(14.59)

If mass weighted coordinates are used, defined as

(14.60)

this becomes an ordinary eigenvalue problem

(14.61)

The eigenvectors u r of the symmetric matrix

(14.62)

are the solutions of

(14.63)

and satisfy (14.61)

(14.64)

with normal mode frequencies

(14.65)

Finally, the Cartesian coordinates are linear combinations of all normal modes

(14.66)

In a true local energy minimum the Hessian matrix H ij is positive definite and all frequencies are real valued. The six lowest frequencies are close to zero and correspond to translations and rotations of the whole system (Fig. 14.9).

Fig. 14.9

(Normal mode distribution for the dipeptide model)  The cumulative distribution (Sect. 8.​1.​2) of normal mode frequencies is shown for the glycine dipeptide. Translations and rotations of the molecule correspond to the lowest 6 frequencies which are close to zero. The highest frequencies between 3100 cm⁻¹ and 3600 cm⁻¹ correspond to the stretching modes of the 8 hydrogen atoms

14.5 Problems

Problem 14.1

(Simulation of a glycine dipeptide)

In this computer experiment a glycine dipeptide (Fig. 14.5) is simulated. Parameters for bond stretching (Table 14.3) and bond angle (Table 14.4) terms have been derived from quantum calculations by Bautista and Seminario [15].

Torsional potential terms (Table 14.5) can be added to make the structure more rigid. This is especially important for the O9-H17, N4-H14 and N1-H10 bonds, which rotate almost freely without torsional potentials.

Table 14.5

(Torsional potential terms)  Torsional potential terms $V_{ijkl}=k_{ijkl} (1-\cos(\theta_{ijkl}- \theta_{ijkl}^{0}))$, which can be added to the forcefield. Minimum angles are from the optimized structure without torsional terms (14.1). The barrier height of 2k ijkl =2 kcal/mol is only a guessed value

i

j

k

l

k ijkl

Backbone

10

1

2

3

−67.6

1.0

14

4

3

2

27.5

1.0

17

9

6

5

−147.4

1.0

4

3

2

1

160.7

1.0

Ψ

5

4

3

2

−152.3

1.0

ω

6

5

4

3

−153.1

1.0

Φ

8

6

5

4

123.7

1.0

9

6

5

4

−56.5

1.0

15

5

4

3

−32.5

1.0

16

5

4

3

86.3

1.0

7

3

2

1

−26.3

1.0

The energy can be minimized with the methods of steepest descent or conjugate gradients.

A normal mode analysis can be performed (the Hessian matrix is calculated by numerical differentiation). The r-th normal mode can be visualized by modulating the coordinates periodically according to

(14.67)

The motion of the atoms can be simulated with the Verlet method. You can stretch the O9-H17 or N4-H14 bond and observe, how the excitation spreads over the molecule.

References

3.

M.P. Allen, D.J. Tildesley, Computer Simulation of Liquids (Oxford University Press, London, 1989). ISBN 0-19-855645-4

5.

V.S. Allured, C.M. Kelly, C.R. Landis, J. Am. Chem. Soc. 113, 1 (1991) CrossRef

15.

E.J. Bautista, J.M. Seminario, Int. J. Quant. Chem. 108, 180 (2008) ADSCrossRef

19.

H. Bekker, H.J.C. Berendsen, W.F. van Gunsteren, J. Comput. Chem. 16, 527 (1995) CrossRef

33.

K.J. Bowers, E. Chow, H. Xu, R.U. Dror, M.P. Eastwood, B.A. Gregersen, J.L. Klepeis, I. Kolossvary, M.A. Moraes, F.D. Sacerdoti, J.K. Salmon, Y. Shan, D.E. Shaw, Scalable algorithms for molecular dynamics simulations on commodity clusters, in SC 2006 Conference. Proceedings of the ACM/IEEE (2006), p. 43

39.

B.R. Brooks, R.E. Bruccoleri, B.D. Olafson, D.J. States, S. Swaminathan, M. Karplus, J. Comput. Chem. 4, 187 (1983) CrossRef

40.

B.R. Brooks, C.L. Brooks III, A.D. Mackerell Jr., L. Nilsson, R.J. Petrella, B. Roux, Y. Won, G. Archontis, C. Bartels, S. Boresch, A. Caflisch, L. Caves, Q. Cui, A.R. Dinner, M. Feig, S. Fischer, J. Gao, M. Hodoscek, W. Im, K. Kuzcera, T. Lazaridis, J. Ma, V. Ovchinnikov, E. Paci, R.W. Pastor, C.B. Post, J.Z. Pu, M. Schaefer, B. Tidor, R.M. Venable, H.L. Woodcock, X. Wu, W. Yang, D.M. York, M. Karplus, J. Comput. Chem. 30, 1545 (2009) CrossRef

50.

R. Car, M. Parrinello, Phys. Rev. Lett. 55, 2471 (1985) ADSCrossRef

58.

M. Christen, P.H. Hünenberger, D. Bakowies, R. Baron, R. Bürgi, D.P. Geerke, T.N. Heinz, M.A. Kastenholz, V. Kräutler, C. Oostenbrink, C. Peter, D. Trzesniak, W.F. van Gunsteren, J. Comput. Chem. 26, 1719 (2005) CrossRef

59.

S. Chynoweth, U.C. Klomp, L.E. Scales, Comput. Phys. Commun. 62, 297 (1991) ADSCrossRef

63.

W.D. Cornell, P. Cieplak, C.I. Bayly, I.R. Gould, K.M. Merz Jr., D.M. Ferguson, D.C. Spellmeyer, T. Fox, J.W. Caldwell, P.A. Kollman, J. Am. Chem. Soc. 117, 5179 (1995) CrossRef

97.

D. Frenkel, B. Smit, Understanding Molecular Simulation: From Algorithms to Applications (Academic Press, San Diego, 2002). ISBN 0-12-267351-4

115.

J.M. Haile, Molecular Dynamics Simulation: Elementary Methods (Wiley, New York, 2001). ISBN 0-471-18439-X

136.

IUPAC-IUB Commission on Biochemical Nomenclature, Biochemistry 9, 3471 (1970) CrossRef

152.

T. Kühne, M. Krack, F. Mohamed, M. Parrinello, Phys. Rev. Lett. 98, 066401 (2007) ADSCrossRef

156.

C. Lavor, Physica D 227, 135 (2007) MathSciNetADSMATHCrossRef

157.

A. Leach, Molecular Modelling: Principles and Applications, 2nd edn. (Prentice Hall, Harlow, 2001). ISBN 978-0582382107

163.

A.D. MacKerell Jr., B. Brooks, C.L. Brooks III, L. Nilsson, B. Roux, Y. Won, M. Karplus, CHARMM: The energy function and its parameterization with an overview of the program, in The Encyclopedia of Computational Chemistry, vol. 1, ed. by P.v.R. Schleyer et al. (Wiley, Chichester, 1998), pp. 271– 277

212.

D.C. Rapaport, The Art of Molecular Dynamics Simulation (Cambridge University Press, Cambridge, 2004). ISBN 0-521-44561-2 MATHCrossRef

213.

A.K. Rappe, C.J. Casewit, K.S. Colwell, W.A. Goddar, W.M. Skiff, J. Am. Chem. Soc. 115, 10024 (1992) CrossRef

224.

T. Schlick, J. Comput. Chem. 10, 951 (1989) CrossRef

225.

T. Schlick, Molecular Modeling and Simulation (Springer, Berlin, 2002). ISBN 0-387-95404-X MATHCrossRef

259.

R.E. Tuzun, D.W. Noid, B.G. Sumpter, Macromol. Theory Simul. 5, 771 (1996) CrossRef

260.

R.E. Tuzun, D.W. Noid, B.G. Sumpter, J. Comput. Chem. 18, 1804 (1997) CrossRef

262.

W.F. van Gunsteren, S.R. Billeter, A.A. Eising, P.H. Hünenberger, P. Krüger, A.E. Mark, W.R.P. Scott, I.G. Tironi, Biomolecular Simulation: The GROMOS96 Manual and User Guide (vdf Hochschulverlag AG an der ETH Zürich and BIOMOS b.v., Zürich, Groningen, 1996)

Footnotes

1

Some force-fields like Desmond [33] or UFF [213] use a more general sum $k\sum_{m=0}^{M}c_{m}\cos(m\theta- \theta^{0})$.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_15© Springer International Publishing Switzerland 2013

15. Thermodynamic Systems

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

An important application for computer simulations is the calculation of thermodynamic averages in an equilibrium system. We discuss two different examples. In the first case the classical equations of motion are solved for a system of particles interacting pairwise by Lennard-Jones forces (Lennard-Jones fluid). The thermodynamic average is taken along the trajectory, i.e. over the calculated coordinates at different times. The inner virial is compared to the virial expansion of the Lennard-Jones system. The pair distance distribution function characterizes the order of the system. Velocity auto-correlation function and mean square displacement are compared with the Brownian model of diffusive motion.

In the second case the Metropolis method is applied to a one- or two-dimensional system of interacting spins (Ising model). The thermodynamic average is taken over a set of random configurations. The average magnetization in a magnetic field and the phase transition to the ferromagnetic state are compared with analytical expressions.

An important application for computer simulations is the calculation of thermodynamic averages in an equilibrium system. We discuss two different examples:

In the first case the classical equations of motion are solved for a system of particles interacting pairwise by Lennard-Jones forces (Lennard-Jones fluid). The thermodynamic average is taken along the trajectory, i.e. over the calculated coordinates at different times r i (t n ). We evaluate the pair distance distribution function

(15.1)

the velocity auto-correlation function

(15.2)

and the mean square displacement

(15.3)

In the second case the Metropolis method is applied to a one- or two-dimensional system of interacting spins (Ising model). The thermodynamic average is taken over a set of random configurations q (n). We study the average magnetization

(15.4)

in a magnetic field and the phase transition to the ferromagnetic state.

15.1 Simulation of a Lennard-Jones Fluid

The Lennard-Jones fluid is a simple model of a realistic atomic fluid. It has been studied by computer simulations since Verlet’s early work [118, 265] and serves as a test case for the theoretical description of liquids [142, 182] and the liquid-gas [270] and liquid-solid phase transitions [146, 176].

In the following we describe a simple computer model of 125 interacting particles¹ without internal degrees of freedom (see problems section). The force on atom i is given by the gradient of the pairwise Lennard-Jones potential (14.​24)

(15.5)

We use argon parameters m=6.69×10⁻²⁶ kg, ε=1.654×10⁻²¹ J, σ=3.405×10⁻¹⁰ m [3]. After introduction of reduced units for length $\mathbf {r}^{*}=\frac{1}{\sigma} \mathbf {r}$, energy $E^{*}=\frac{1}{\varepsilon}E$ and time $t^{*}=\sqrt{\varepsilon/m\sigma^{2}}\,t$, the potential energy

(15.6)

and the equation of motion

(15.7)

become universal expressions, i.e. there exists only one universal Lennard-Jones system. To reduce computer time, usually the 6–12 potential is modified at larger distances which can influence the simulation results [239]. In our model a simple cutoff of potential and forces at r max=10 Å is used.

15.1.1 Integration of the Equations of Motion

The equations of motion are integrated with the Verlet algorithm (Sect. 12.​11.​5)

(15.8)

(15.9)

We use a higher order expression for the velocities to improve the accuracy of the calculated kinetic energy

(15.10)

15.1.2 Boundary Conditions and Average Pressure

Molecular dynamics simulations often involve periodic boundary conditions to reduce finite size effects. Here we employ an alternative method which simulates a box with elastic walls. This allows us to calculate explicitly the pressure on the walls of the box.

The atoms are kept in the cube by reflecting walls, i.e. whenever an atom passes a face of the cube, the normal component of the velocity vector is changed in sign (Fig. 15.1). Thus the kinetic energy is conserved but a momentum of mΔv=2mv ⊥ is transferred to the wall. The average momentum change per time can be interpreted as a force acting upon the wall

(15.11)

The pressure p is given by

(15.12)

With the Verlet algorithm the reflection can be realized by exchanging the values of the corresponding coordinate at times t n and t n−1.

Fig. 15.1

Reflecting walls

15.1.3 Initial Conditions and Average Temperature

At the very beginning the N=125 atoms are distributed over equally spaced lattice points within the cube. Velocities are randomly distributed according to a Gaussian distribution for each Cartesian component v μ

(15.13)

corresponding to a Maxwell speed distribution

(15.14)

Assuming thermal equilibrium, the effective temperature is calculated from the kinetic energy

(15.15)

The desired temperature T o is established by the rescaling procedure

(15.16)

which is applied repeatedly during an equilibration run. The velocity distribution f(|v|) can be monitored. It approaches quickly a stationary Maxwell distribution (Fig. 15.2).

Fig. 15.2

(Velocity distribution)  The velocity distribution is shown for T=100 K and T=500 K (histograms) and compared to the Maxwell speed distribution (solid curves)

A smoother method to control temperature is the Berendsen thermostat algorithm [21]

(15.17)

where τ therm is a suitable relaxation time (for instance τ therm =20Δt). This method can be used also during the simulation. However, it does not generate the trajectory of a true canonical ensemble. If this is necessary, more complicated methods have to be used [128].

15.1.4 Analysis of the Results

After an initial equilibration phase the system is simulated at constant energy (NVE simulation) or at constant temperature (NVT) simulation with the Berendsen thermostat method. Several static and dynamic properties can be determined.

15.1.4.1 Deviation from the Ideal Gas Behavior

A dilute gas is approximately ideal with

(15.18)

For a real gas the interaction between the particles has to be taken into account. From the equipartition theorem it can be found that²

(15.19)

with the inner virial (Fig. 15.3)

(15.20)

which can be expanded as a power series of the number density n=N/V [231] to give

(15.21)

The virial coefficient b(T) can be calculated exactly for the Lennard-Jones gas [231]:

(15.22)

For comparison we calculate the quantity

(15.23)

which for small values of the particle density n=N/V correlates well (Fig. 15.4) with expression (15.22).

Fig. 15.3

(Inner virial)  The inner virial W (15.20, crosses and stars) is compared to pV−k B T (squares and circles) for two values of the particle density N/V=10⁻³ Å⁻¹ (a) and 1.95×10⁻³ Å⁻¹ (b), corresponding to reduced densities n ∗=σ ³ N/V of 0.040 and 0.077

Fig. 15.4

(Second virial coefficient)  The value of $\frac{V}{N} (\frac{pV}{k_{B}T}-1 )$ is shown for two values of the particle density N/V=10⁻³ Å (crosses) and 1.95×10⁻³ Å⁻¹ (circles) and compared to the exact second virial coefficient b (dashed curve) (15.22)

15.1.4.2 Structural Order

A convenient measure for structural order [78] is the radial pair distribution function (Fig. 15.5)

(15.24)

which is usually normalized with respect to an ideal gas, for which

(15.25)

Fig. 15.5

(Radial pair distribution)  The normalized radial distribution function g(R)/g ideal (R) is evaluated for kT=35 K, 100 K, 1000 K and a density of n=0.025 A ⁻³ corresponding to a reduced density n ∗=σ ³ N/V of 1.0. At this density the Lennard-Jones system shows a liquid-solid transition at a temperature of ca. 180 K [146]

For small distances g(R)/g ideal (R) vanishes due to the strong repulsive force. It peaks at the distance of nearest neighbors and approaches unity at very large distances. In the condensed phase additional maxima appear showing the degree of short (liquid) and long range (solid) order.

Equation (15.25) is not valid for our small model system without periodic boundary conditions. Therefore g ideal was calculated numerically to normalize the results shown in Fig. 15.5.

15.1.4.3 Ballistic and Diffusive Motion

The velocity auto-correlation function (Fig. 15.6)

(15.26)

decays as a function of the delay time t−t ₀ due to collisions of the particles. In a stationary state it does not depend on the initial time t ₀. Integration leads to the mean square displacement (Fig. 15.6)

(15.27)

In the absence of collisions the mean square displacement grows with (t−t ₀)², representing a ballistic type of motion. Collisions lead to a diffusive kind of motion where the mean square displacement grows only linearly with time. The transition between this two types of motion can be analyzed within the model of Brownian motion [219] where the collisions are replaced by a fluctuating random force Γ(t) and a damping constant γ.

Fig. 15.6

(Velocity auto-correlation function)  The Lennard-Jones system is simulated for k B T=200 K and different values of the density n ∗=0.12 (a), 0.18 (b), 0.32 (c), 0.62 (d). The velocity auto-correlation function (full curves) is averaged over 20 trajectories and fitted by an exponential function (dashed curves)

The equation of motion in one dimension is

(15.28)

with

(15.29)

(15.30)

The velocity correlation decays exponentially

(15.31)

with the average velocity square given by

(15.32)

and the integral of the correlation function equals

(15.33)

The average of Δx ² is

(15.34)

For small time differences t−t ₀ the motion is ballistic with the thermal velocity

(15.35)

For large time differences diffusive motion emerges with

(15.36)

with the diffusion constant given by the Einstein relation

(15.37)

For a three-dimensional simulation the Cartesian components of the position or velocity vector add up independently. The diffusion coefficient can be determined from

(15.38)

or, alternatively from (15.33) [3]

(15.39)

This equation is more generally valid also outside the Brownian limit (Green-Kubo formula). The Brownian model represents the simulation data quite well at low particle densities (Figs. 15.6, 15.7). For higher densities the velocity auto-correlation function shows a very rapid decay followed by a more or less structured tail [3, 159, 256].

Fig. 15.7

(Mean square displacement)  The Lennard-Jones system is simulated for k B T=200 K and different values of the density n ∗=0.12 (a), 0.18 (b), 0.32 (c), 0.62 (d). The mean square displacement (full curves) is averaged over 20 trajectories and fitted by a linear function (dashed lines) for t−t ₀>1.5 ps

15.2 Monte Carlo Simulation

The basic principles of Monte Carlo simulations are discussed in Chap. 8. Here we will apply the Metropolis algorithm to simulate the Ising model in one or two dimensions. The Ising model [26, 134] is primarily a model for the phase transition of a ferromagnetic system. However, it has further applications for instance for a polymer under the influence of an external force or protonation equilibria in proteins.

15.2.1 One-Dimensional Ising Model

We consider a chain consisting of N spins which can be either up (S i =1) or down (S i =−1). The total energy in a magnetic field is (Fig. 15.8)

(15.40)

and the average magnetic moment of one spin is

(15.41)

If interaction between neighboring spins is included the energy of a configuration (S ₁⋯S N ) becomes

(15.42)

The 1-dimensional model can be solved analytically [231]. In the limit N→∞ the magnetization is

(15.43)

The numerical simulation (Fig. 15.9) starts either with the ordered state S i =1 or with a random configuration. New configurations are generated with the Metropolis method as follows:

flip one randomly chosen spin S i ³ and calculate the energy change due to the change ΔS i =(−S i )−S i =−2S i

(15.44)

if ΔE<0 then accept the flip, otherwise accept it with a probability of P=e −ΔE/kT

As a simple example consider N=3 spins which have 8 possible configurations. The probabilities of the trial step T i→j are shown in Table 15.1.

Fig. 15.8

(Ising model)  N spins can be up or down. The interaction with the magnetic field is −μBS i , the interaction between nearest neighbors is −JS i S j

Fig. 15.9

(Numerical simulation of the 1-dimensional Ising model)  The average magnetization per spin is calculated from a MC simulation (circles) and compared to the exact solution (15.43). Parameters are μB=−5 and J=−2

Table 15.1

transition probabilities for a 3-spin system (p=1/3)

+++

++−

+−+

+−−

−++

−+−

−−+

−−−

+++

0

p

p

0

p

0

0

0

++−

p

0

0

p

0

p

0

0

+−+

p

0

0

p

0

0

p

0

+−−

0

p

p

0

0

0

0

p

−++

p

0

0

0

0

p

p

0

−+−

0

p

0

0

p

0

0

p

−−+

0

0

p

0

p

0

0

p

−−−

0

0

0

p

0

p

p

0

The table is symmetric and all configurations are connected.

15.2.2 Two-Dimensional Ising Model

For dimension d>1 the Ising model behaves qualitatively different as a phase transition appears. For B=0 (Fig. 15.10) the 2-dimensional Ising-model with 4 nearest neighbors can be solved analytically [171, 194]. The magnetization disappears above the critical temperature T c , which is given by

(15.45)

Below T c the average magnetization is given by

(15.46)

Fig. 15.10

(Numerical simulation of the 2-dimensional Ising model)  The average magnetization per spin is calculated for B=0 from a MC simulation (circles) and compared to (15.46)

15.3 Problems

Problem 15.1

(Lennard-Jones fluid)

In this computer experiment a Lennard-Jones fluid is simulated. The pressure is calculated from the average transfer of momentum (15.12) and compared with expression (15.19).

Equilibrate the system and observe how the distribution of squared velocities approaches a Maxwell distribution.

Equilibrate the system for different values of temperature and volume and investigate the relation between pV/N and kT.

observe the radial distribution function for different values of temperature and densities. Try to locate phase transitions.

determine the decay time of the velocity correlation function and compare with the behavior of the mean square displacement which shows a transition from ballistic to diffusive motion.

Problem 15.2

(One-dimensional Ising model)

In this computer experiment we simulate a linear chain of N=500 spins with periodic boundaries and interaction between nearest neighbors only. We go along the chain and try to flip one spin after the other according to the Metropolis method.

After trying to flip the last spin S N the total magnetization

(15.47)

is calculated. It is averaged over 500 such cycles and then compared graphically with the analytical solution for the infinite chain (15.43). Temperature and magnetic field can be varied.

Problem 15.3

(Two-state model for a polymer)

Consider a polymer (Fig. 15.11) consisting of N units which can be in two states S i =+1 or S i =−1 with corresponding lengths l ₊ and l ₋. The interaction between neighboring units takes one of the values w ₊₊,w ₊₋,w ₋₋. Under the influence of an external force κ the energy of the polymer is

(15.48)

Fig. 15.11

Two state model

This model is isomorphic to the one-dimensional Ising model,

(15.49)

(15.50)

(15.51)

Comparison with (15.42) shows the correspondence

(15.52)

(15.53)

(15.54)

In this computer experiment we simulate a linear chain of N=20 units with periodic boundaries and nearest neighbor interaction as in the previous problem.

The fluctuations of the chain conformation are shown graphically and the magnetization of the isomorphic Ising model is compared with the analytical expression for the infinite system (15.43). Temperature and magnetic field can be varied as well as the coupling J. For negative J the anti-ferromagnetic state becomes stable at low magnetic field strengths.

Problem 15.4

(Two-dimensional Ising model)

In this computer experiment a 200×200 square lattice with periodic boundaries and interaction with the 4 nearest neighbors is simulated. The fluctuations of the spins can be observed. At low temperatures ordered domains with parallel spin appear. The average magnetization is compared with the analytical expression for the infinite system (15.46).

References

3.

M.P. Allen, D.J. Tildesley, Computer Simulation of Liquids (Oxford University Press, London, 1989). ISBN 0-19-855645-4

21.

H.J.C. Berendsen, J.P.M. Postma, W.F. van Gunsteren, A. DiNola, J.R. Haak, J. Chem. Phys. 81, 3684 (1984) ADSCrossRef

26.

K. Binder, Ising model, in Encyclopedia of Mathematics, Suppl. vol. 2, ed. by R. Hoksbergen (Kluwer Academic, Dordrecht, 2000), pp. 279–281

78.

J.R. Errington, P.G. Debenedetti, S. Torquato, J. Chem. Phys. 118, 2256 (2003) ADSCrossRef

118.

J.-P. Hansen, L. Verlet, Phys. Rev. 184, 151 (1969) ADSCrossRef

128.

P.H. Huenenberger, Adv. Polym. Sci. 173, 105 (2005) CrossRef

134.

E. Ising, Beitrag zur Theorie des Ferromagnetismus. Z. Phys. 31, 253–258 (1925). doi:10.​1007/​BF02980577

142.

J.K. Johnson, J.A. Zollweg, K.E. Gubbins, Mol. Phys. 78, 591 (1993) ADSCrossRef

146.

S.A. Khrapak, M. Chaudhuri, G.E. Morfill, Phys. Rev. B 82, 052101 (2010) ADSCrossRef

159.

D. Levesque, L. Verlet, Phys. Rev. A 2, 2514 (1970) ADSCrossRef

171.

B.M. McCoy, T.T. Wu, The Two-Dimensional Ising Model (Harvard University Press, Cambridge, 1973). ISBN 0674914406 MATH

176.

C. Muguruma, Y. Okamoto, M. Mikami, Croat. Chem. Acta 80, 203 (2007)

182.

J.J. Nicolas, K.E. Gubbins, W.B. Streett, D.J. Tildesley, Mol. Phys. 37, 1429 (1979) ADSCrossRef

194.

L. Onsager, Phys. Rev. 65, 117 (1944) MathSciNetADSMATHCrossRef

219.

H. Risken, The Fokker-Planck Equation (Springer, Berlin, 1989) MATHCrossRef

231.

F. Schwabl, Statistical Mechanics (Springer, Berlin, 2003)

239.

B. Smit, J. Chem. Phys. 96, 8639 (1992) ADSCrossRef

256.

T. Tsang, H. Tang, Phys. Rev. A 15, 1696 (1977) ADSCrossRef

265.

L. Verlet, Phys. Rev. 159, 98 (1967) ADSCrossRef

270.

H. Watanabe, N. Ito, C.K. Hu, J. Chem. Phys. 136, 204102 (2012) ADSCrossRef

Footnotes

1

This small number of particles allows a graphical representation of the system during the simulation.

2

MD simulations with periodic boundary conditions use this equation to calculate the pressure.

3

Or try one spin after the other.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_16© Springer International Publishing Switzerland 2013

16. Random Walk and Brownian Motion

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

In this chapter we study Markovian discrete time random walk models. In one dimension the position of the walker after n steps approaches a Gaussian distribution. This can be checked in a computer experiment. A 3-dimensional random walk provides a simple statistical model for the configuration of a biopolymer (freely jointed chain). We generate random structures and calculate the gyration tensor, an experimentally observable quantity, which gives information on the shape of a polymer. Simulation of the dynamics is simplified if the fixed length segments of the freely jointed chain are replaced by Hookean springs. This is utilized in a computer experiment to study the dependence of the polymer extension on an applied external force (entropic elasticity). The random motion of a heavy particle in a bath of light particles (Brownian motion) can be described by Langevin dynamics, which replace the collisions with the light particles by an average friction force proportional to the velocity and a randomly fluctuating force with zero mean and infinitely short correlation time. In a computer experiment we study Brownian motion in a harmonic potential.

Random walk processes are an important class of stochastic processes. They have many applications in physics, computer science, ecology, economics and other fields. A random walk [199] is a sequence of successive random steps. In this chapter we study Markovian [166, 167]¹ discrete time² models. In one dimension the position of the walker after n steps approaches a Gaussian distribution, which does not depend on the distribution of the single steps. This follows from the central limit theorem and can be checked in a computer experiment. A 3-dimensional random walk provides a simple statistical model for the configuration of a biopolymer, the so called freely jointed chain model. In a computer experiment we generate random structures and calculate the gyration tensor, an experimentally observable quantity,which gives information on the shape of a polymer. Simulation of the dynamics is simplified if the fixed length segments of the freely jointed chain are replaced by Hookean springs. This is utilized in a computer experiment to study the dependence of the polymer extension on an applied external force (this effect is known as entropic elasticity). The random motion of a heavy particle in a bath of light particles, known as Brownian motion, can be described by Langevin dynamics, which replace the collisions with the light particles by an average friction force proportional to the velocity and a randomly fluctuating force with zero mean and infinitely short correlation time. In a computer experiment we study Brownian motion in a harmonic potential.

16.1 Markovian Discrete Time Models

The time evolution of a system is described in terms of an N-dimensional vector r(t), which can be for instance the position of a molecule in a liquid, or the price of a fluctuating stock. At discrete times t n =nΔt the position changes suddenly (Fig. 16.1)

(16.1)

where the steps are distributed according to the probability distribution³

(16.2)

The probability of reaching the position R after n+1 steps obeys the equation

(16.3)

Fig. 16.1

Discrete time random walk

16.2 Random Walk in One Dimension

Consider a random walk in one dimension. We apply the central limit theorem to calculate the probability distribution of the position r n after n steps. The first two moments and the standard deviation of the step distribution are

(16.4)

Hence the normalized quantity

(16.5)

is a random variable with zero average and unit standard deviation. The distribution function of the new random variable

(16.6)

approaches a normal distribution for large n

(16.7)

and finally from

we have

(16.8)

The position of the walker after n steps obeys approximately a Gaussian distribution centered at $\overline{r}_{n}=n\overline{b}$ with a standard deviation of

(16.9)

16.2.1 Random Walk with Constant Step Size

In the following we consider the classical example of a 1-dimensional random walk process with constant step size. At time t n the walker takes a step of length Δx to the left with probability p or to the right with probability q=1−p (Figs. 16.2, 16.3).

Fig. 16.2

(Random walk with constant step size)  The figure shows the position r n for three different 1-dimensional random walks with step size Δx=±1. The dashed curves show the width $\pm\sigma=\pm\sqrt{n}$ of the Gaussian approximation (16.8)

Fig. 16.3

Random walk with constant step size

The corresponding step size distribution function is

(16.10)

with the first two moments

(16.11)

Let the walker start at r(t ₀)=0. The probability P n (m) of reaching position mΔx after n steps obeys the recursion

(16.12)

which obviously leads to a binomial distribution. From the expansion of

(16.13)

we see that

(16.14)

or after substitution m′=n−2m=−n,−n+2⋯n−2,n:

(16.15)

Since the steps are uncorrelated we easily find the first two moments

(16.16)

and

(16.17)

16.3 The Freely Jointed Chain

We consider a simple statistical model for the conformation of a biopolymer like DNA or a protein.

The polymer is modeled by a 3-dimensional chain consisting of M units with constant bond length and arbitrary relative orientation (Fig. 16.4). The configuration can be described by a point in a 3(M+1)-dimensional space which is reached after M steps Δr i =b i of a 3-dimensional random walk with constant step size

(16.18)

Fig. 16.4

Freely jointed chain with constant bond length b

16.3.1 Basic Statistic Properties

The M bond vectors

(16.19)

have a fixed length |b i |=b and are oriented randomly. The first two moments are

(16.20)

Since different units are independent

(16.21)

Obviously the relative position of segment j

has zero mean

(16.22)

and its second moment is

(16.23)

For the end to end distance (Fig. 16.5)

(16.24)

this gives

(16.25)

Let us apply the central limit theorem for large M. For the x coordinate of the end to end vector we have

(16.26)

With the help of the averages⁴

(16.27)

(16.28)

we find that the scaled difference

(16.29)

has zero mean and unit variance and therefore the sum

(16.30)

converges to a normal distribution:

(16.31)

Hence

(16.32)

and finally in 3 dimensions

(16.33)

Fig. 16.5

(Freely jointed chain)  The figure shows a random 3-dimensional structure with 1000 segments visualized as balls (Molden graphics [223])

16.3.2 Gyration Tensor

For the center of mass

we find

and since

we have

The gyration radius [170] is generally defined by

R g can be also written as

and can be experimentally measured with the help of scattering phenomena. It is related to the gyration tensor which is defined as

Its trace is

and its eigenvalues give us information about the shape of the polymer (Fig. 16.6).

Fig. 16.6

(Gyration tensor)  The eigenvalues of the gyration tensor give information on the shape of the polymer. If the extension is larger (smaller) along one direction than in the perpendicular plane, one eigenvalue is larger (smaller) than the two other

16.3.3 Hookean Spring Model

Simulation of the dynamics of the freely jointed chain is complicated by the constraints which are implied by the constant chain length. Much simpler is the simulation of a model which treats the segments as Hookean springs (Fig. 16.7). In the limit of a large force constant the two models give equivalent results.

Fig. 16.7

Polymer model with Hookean springs

We assume that the segments are independent (self crossing is not avoided). Then for one segment the energy contribution is

If the fluctuations are small

then (Fig. 16.8)

and the freely jointed chain model (16.33) gives the entropy as a function of the end to end vector

(16.34)

If one end of the polymer is fixed at r ₀=0 and a force $\mathbf{\kappa}$ is applied to the other end, the free energy is given by

(16.35)

In thermodynamic equilibrium the free energy is minimal, hence the average extension is

(16.36)

This linear behavior is similar to a Hookean spring with an effective force constant

(16.37)

and is only valid for small forces. For large forces the freely jointed chain asymptotically reaches its maximum length of R M,max=Mb, whereas for the chain of springs R M →M(b+κ/f).

Fig. 16.8

(Distribution of bond vectors)  The bond vector distribution for a 1-dimensional chain of springs has maxima at ±b. For large force constants the width of the two peaks becomes small and the chain of springs resembles a freely jointed chain with constant bond length

16.4 Langevin Dynamics

A heavy particle moving in a bath of much smaller and lighter particles (for instance atoms and molecules of the air) shows what is known as Brownian motion [41, 76, 77]. Due to collisions with the thermally moving bath particles it experiences a fluctuating force which drives the particle into a random motion. The French physicist Paul Langevin developed a model to describe this motion without including the light particles explicitly. The fluctuating force is divided into a macroscopic friction force proportional to the velocity

(16.38)

and a randomly fluctuating force with zero mean and infinitely short correlation time

(16.39)

The equations of motion for the heavy particle are

(16.40)

with the macroscopic friction coefficient γ and the potential U(x).

The behavior of the random force can be better understood if we introduce a time grid t n+1−t n =Δt and take the limit Δt→0. We assume that the random force has a constant value during each interval

(16.41)

and that the values at different intervals are uncorrelated

(16.42)

The auto-correlation function then is given by

(16.43)

Division by Δt gives a sequence of functions which converges to a delta function in the limit Δt→0

(16.44)

Hence we find

(16.45)

Within a short time interval Δt→0 the velocity changes by

(16.46)

and taking the square gives

(16.47)

Hence for the total energy

(16.48)

we have

(16.49)

On the average the total energy $\overline{E}$ should be constant and furthermore in d dimensions

(16.50)

Therefore we conclude

(16.51)

from which we obtain finally

(16.52)

16.5 Problems

Problem 16.1

(Random walk in one dimension)

This program generates random walks with (a) fixed step length Δx=±1 or (b) step length equally distributed over the interval $-\sqrt {3}<\Delta x<\sqrt{3}$. It also shows the variance, which for large number of walks approaches $\sigma=\sqrt{n}$. See also Fig. 16.2.

Problem 16.2

(Gyration tensor)

The program calculates random walks with M steps of length b. The bond vectors are generated from M random points e i on the unit sphere as b i =b e i . End to end distance, center of gravity and gyration radius are calculated and can be averaged over numerous random structures. The gyration tensor 16.3.2 is diagonalized and the ordered eigenvalues are averaged.

Problem 16.3

(Brownian motion in a harmonic potential)

The program simulates a particle in a 1-dimensional harmonic potential

(16.53)

where κ is an external force. We use the improved Euler method (12.​36). First the coordinate and the velocity at mid time are estimated

(16.54)

(16.55)

where F n is a random number obeying (16.52). Then the values at t n+1 are calculated as

(16.56)

(16.57)

Problem 16.4

(Force extension relation)

The program simulates a chain of springs (Sect. 16.3.3) with potential energy

(16.58)

The force can be varied and the extension along the force direction is averaged over numerous time steps.

References

41.

R. Brown, Philos. Mag. 4, 161 (1828)

76.

A. Einstein, Ann. Phys. 17, 549 (1905) MATHCrossRef

77.

A. Einstein, Investigations on the Theory of Brownian Movement (Dover, New York, 1956) MATH

166.

A.A. Markov, Theory of Algorithms [Translated by Jacques J. Schorr-Kon and PST staff] Imprint Moscow, Academy of Sciences of the USSR, 1954 [Jerusalem, Israel Program for Scientific Translations, 1961; available from Office of Technical Services, United States Department of Commerce] Added t.p. in Russian Translation of Works of the Mathematical Institute, Academy of Sciences of the USSR, v. 42. Original title: Teoriya algorifmov. [QA248.M2943 Dartmouth College library. U.S. Dept. of Commerce, Office of Technical Services, number OTS 60-51085] (1954)

167.

A.A. Markov, Extension of the limit theorems of probability theory to a sum of variables connected in a chain, in Dynamic Probabilistic Systems, vol. 1: Markov Chains (Wiley, New York, 1971), reprinted in Appendix B of Howard, R.

170.

W.L. Mattice, U.W. Suter, Conformational Theory of Large Molecules (Wiley-Interscience, New York, 1994). ISBN 0-471-84338-5

199.

K. Pearson, The problem of the random walk. Nature 72, 294 (1905) ADSCrossRef

223.

G. Schaftenaar, J.H. Noordik, J. Comput.-Aided Mol. Des. 14, 123 (2000) ADSCrossRef

Footnotes

1

Different steps are independent.

2

A special case of the more general continuous time random walk with a waiting time distribution of P(τ)=δ(τ−Δt).

3

General random walk processes are characterized by a distribution function P(R,R′). Here we consider only correlated processes for which P(R,R′)=P(R′−R).

4

For a 1-dimensional polymer $\overline{\cos\theta_{i}}=0$ and $\overline{(\cos\theta_{i})^{2}}=1$. In two dimensions $\overline{\cos\theta_{i}}=\frac{1}{\pi}\int_{0}^{\pi}\cos\theta\,d\theta=0$ and $\overline{(\cos\theta_{i})^{2}}=\frac{1}{\pi}\int_{0}^{\pi}\cos^{2}\theta\, d\theta=\frac{1}{2}$. To include these cases the factor 3 in the exponent of (16.33) should be replaced by the dimension d.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_17© Springer International Publishing Switzerland 2013

17. Electrostatics

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

The electrostatic potential of a charge distribution is a solution of Poisson’s equation. If the thermal distribution of mobile charges is taken into account, like for an electrolyte or semiconductor, the Poisson-Boltzmann equation has to be solved instead.

We discretize the Poisson-Boltzmann equation and its linearized form by finite volume methods which are applicable even in case of discontinuous dielectric constant. We solve the discretized equations iteratively with the method of successive over-relaxation. The solvation energy of a charged sphere in a dielectric medium is calculated to compare the accuracy of several methods. This can be studied in a computer experiment.

Since the Green’s function is analytically available for the Poisson and Poisson-Boltzmann equations, alternatively the method of boundary elements can be applied, which can reduce the computer time, for instance for solvation models. A computer experiment simulates a point charge within a spherical cavity and calculates the solvation energy with the boundary element method.

The electrostatic potential Φ(r) of a charge distribution ρ(r) is a solution¹ of Poisson’s equation

(17.1)

which, for spatially varying dielectric constant ε(r) becomes

(17.2)

and, if mobile charges are taken into account, like for an electrolyte or semiconductor, turns into the Poisson-Boltzmann equation

(17.3)

In this chapter we discretize the Poisson and the linearized Poisson-Boltzmann equation by finite volume methods which are applicable even in case of discontinuous ε. We solve the discretized equations iteratively with the method of successive over-relaxation. The solvation energy of a charged sphere in a dielectric medium is calculated to compare the accuracy of several methods. This can be studied also in a computer experiment.

Since the Green’s function is analytically available for the Poisson and Poisson-Boltzmann equations, alternatively the method of boundary elements can be applied, which can reduce the computer time, for instance for solvation models. A computer experiment simulates a point charge within a spherical cavity and calculates the solvation energy with the boundary element method.

17.1 Poisson Equation

From a combination of the basic equations of electrostatics

(17.4)

(17.5)

(17.6)

the generalized Poisson equation is obtained

(17.7)

which can be written in integral form with the help of Gauss’s theorem

(17.8)

If ε(r) is continuously differentiable, the product rule for differentiation gives

(17.9)

which for constant ε simplifies to the Poisson equation

(17.10)

17.1.1 Homogeneous Dielectric Medium

We begin with the simplest case of a dielectric medium with constant ε and solve (17.10) numerically. We use a finite volume method (Sect. 11.​3) which corresponds to a finite element method with piecewise constant test functions. The integration volume is divided into small cubes V ijk which are centered at the grid points (Fig. 17.1)

(17.11)

Fig. 17.1

(Finite volume for the Poisson equation)  The control volume is a small cube centered at a grid point (full circle)

Integration of (17.10) over the control volume V ijk around r ijk gives

(17.12)

Q ijk is the total charge in the control volume. The flux integral is approximated by (11.​85)

(17.13)

The derivatives are approximated by symmetric differences

(17.14)

which coincides with the simplest discretization of the second derivatives (3.​40). Finally we obtain the discretized Poisson equation in the more compact form

(17.15)

which involves an average over the 6 neighboring cells

(17.16)

17.1.2 Numerical Methods for the Poisson Equation

Equation (17.15) is a system of linear equations with very large dimension (for a grid with 100×100×100 points the dimension of the matrix is 10⁶×10⁶!). Our computer experiments use the iterative method (5.​5)

(17.17)

Jacobi’s method ((5.​110) on page 74) makes all the changes in one step whereas the Gauss-Seidel method ((5.​113) on page 74) makes one change after the other. The chessboard (or black red method) divides the grid into two subgrids (with i+j+k even or odd) which are treated subsequently. The vector d r s connects points of different subgrids. Therefore it is not necessary to store intermediate values like for the Gauss-Seidel method.

Convergence can be improved with the method of successive over-relaxation (SOR, (5.​117) on page 75) using a mixture of old and new values

(17.18)

with the relaxation parameter ω. For 1<ω<2 convergence is faster than for ω=1. The optimum choice of ω for the Poisson problem in any dimension is discussed in [279].

Convergence can be further improved by multigrid methods [120, 283]. Error components with short wavelengths are strongly damped during a few iterations whereas it takes a very large number of iterations to remove the long wavelength components. But here a coarser grid is sufficient and reduces computing time. After a few iterations a first approximation Φ ₁ is obtained with the finite residual

(17.19)

Then more iterations on a coarser grid are made to find an approximate solution Φ ₂ of the equation

(17.20)

The new residual is

(17.21)

Function values of Φ ₂ on the finer grid are obtained by interpolation and finally the sum Φ ₁+Φ ₂ provides an improved approximation to the solution since

(17.22)

This method can be extended to a hierarchy of many grids.

Alternatively, the Poisson equation can be solved non-iteratively with pseudospectral methods [238, 247]. For instance, if the boundary is the surface of a cube, eigenfunctions of the Laplacian are for homogeneous boundary conditions (Φ=0) given by

(17.23)

and for no-flow boundary conditions ($\frac{\partial}{\partial n}\varPhi=0$) by

(17.24)

which can be used as expansion functions for the potential

(17.25)

Introducing collocation points r j the condition on the residual becomes

(17.26)

which can be inverted with an inverse discrete sine transformation, (respectively an inverse discrete cosine transformation for no-flux boundary conditions) to obtain the Fourier components of the potential. Another discrete sine (or cosine) transformation gives the potential in real space.

17.1.3 Charged Sphere

As a simple example we consider a sphere of radius R with a homogeneous charge density of

(17.27)

The exact potential is given by

(17.28)

The charge density (17.27) is discontinuous at the surface of the sphere. Integration over a control volume smears out this discontinuity which affects the potential values around the boundary (Fig. 17.2). Alternatively we could assign the value ρ(r ijk ) which is either ρ ₀ (17.27) or zero to each control volume which retains a sharp transition but changes the shape of the boundary surface and does not conserve the total charge. This approach was discussed in the first edition of this book in connection with a finite differences method. The most precise but also complicated method divides the control volumes at the boundary into two irregularly shaped parts [188, 189].

Fig. 17.2

(Discretization of the discontinuous charge density)  Left: the most precise method divides the control volumes at the boundary into two irregularly shaped parts. Middle: assigning either the value ρ ₀ or zero retains the discontinuity but changes the shape of the boundary. Right: averaging over a control volume smears out the discontinuous transition

Initial guess as well as boundary values are taken from

(17.29)

which provides proper boundary values but is far from the final solution inside the sphere. The interaction energy is given by (Sect. 17.5)

(17.30)

Calculated potential (Fig. 17.3) and interaction energy (Figs. 17.4, 17.5) converge rapidly. The optimum relaxation parameter is around ω≈1.9.

Fig. 17.3

(Electrostatic potential of a charged sphere)  A charged sphere is simulated with radius R=0.25 Å and a homogeneous charge density ρ=e⋅3/4πR ³. The grid consists of 200³ points with a spacing of h=0.025 Å. The calculated potential (circles) is compared to the exact solution ((17.28), solid curve), the initial guess is shown by the dashed line

Fig. 17.4

(Influence of the relaxation parameter)  The convergence of the interaction energy ((17.30), which has a value of 34.56 eV for this example) is studied as a function of the relaxation parameter ω. The optimum value is around ω≈1.9. For ω>2 there is no convergence. The dashed line shows the exact value

Fig. 17.5

(Influence of grid size)  The convergence of the interaction energy (17.30) and the central potential value are studied as a function of grid size. The dashed lines show the exact values

17.1.4 Variable ε

In the framework of the finite volume method we take the average over a control volume to discretize ε ² and Φ

(17.31)

(17.32)

Integration of (17.7) gives

(17.33)

The surface integral is

(17.34)

Applying the midpoint rule (11.​77) we find (Fig. 17.6)

(17.35)

The potential Φ as well as the product $\varepsilon(\mathbf {r})\frac{\partial\varPhi}{\partial n}$ are continuous, therefore we make the approximation [188]

(17.36)

Fig. 17.6

Face center of the control volume

From this equation the unknown potential value on the face of the control volume $\overline{\varPhi}(\mathbf {r}_{ijk}+\frac{1}{2}d\mathbf {r}_{s})$ (Fig. 17.6) can be calculated

(17.37)

which gives

(17.38)

Finally we obtain the discretized equation

(17.39)

which can be solved iteratively according to

(17.40)

17.1.5 Discontinuous ε

For practical applications models are often used with piecewise constant ε. A simple example is the solvation of a charged molecule in a dielectric medium (Fig. 17.9). Here ε=ε ₀ within the molecule and ε=ε ₀ ε ₁ within the medium. At the boundary ε is discontinuous. In (17.40) the discontinuity is replaced by a smooth transition between the two values of ε (Fig. 17.7).

Fig. 17.7

(Transition of ε)  The discontinuous ε(r) (black line) is averaged over the control volumes to obtain the discretized values ε ijk (full circles). Equation (17.40) takes the harmonic average over two neighbor cells (open circles) and replaces the discontinuity by a smooth transition over a distance of about h

If the discontinuity of ε is inside a control volume V ijk then (17.31) takes the arithmetic average

(17.41)

which corresponds to the parallel connection of two capacities (Fig. 17.8). Depending on geometry, a serial connection may be more appropriate which corresponds to the weighted harmonic average

(17.42)

Fig. 17.8

Average of ε over a control volume

17.1.6 Solvation Energy of a Charged Sphere

We consider again a charged sphere, which is now embedded in a dielectric medium (Fig. 17.9) with relative dielectric constant ε ₁.

Fig. 17.9

(Solvation of a charged sphere in a dielectric medium)  Charge density and dielectric constant are discontinuous at the surface of the sphere

For a spherically symmetrical problem (17.7) can be solved by application of Gauss’s theorem

(17.43)

(17.44)

For the charged sphere we find

(17.45)

(17.46)

(17.47)

The constant Φ(0) is chosen to give vanishing potential at infinity

(17.48)

The interaction energy is

(17.49)

Numerical results for ε ₁=4 are shown in Fig. 17.10.

Fig. 17.10

(Charged sphere in a dielectric medium)  Numerical results for ε ₁=4 outside the sphere and 200³ grid points (circles) are compared to the exact solution ((17.46), (17.47), solid curves)

17.1.7 The Shifted Grid Method

An alternative approach uses a different grid (Fig. 17.11) for ε which is shifted by h/2 in all directions [44] or, more generally, a dual grid (11.​74),

(17.50)

The value of ε has to be averaged over four neighboring cells to obtain the discretized equation

(17.51)

The shifted grid method is especially useful if ε changes at planar interfaces. Numerical results of several methods are compared in Fig. 17.12.

Fig. 17.11

(Shifted grid method)  A different grid is used for the discretization of ε which is shifted by h/2 in all directions

Fig. 17.12

(Comparison of numerical errors)  The Coulomb interaction of a charged sphere is calculated with several methods for 100³ grid points. Circles: ((17.40), ε averaged), diamonds: ((17.40), ε ⁻¹ averaged), squares: ((17.51), ε averaged), triangles: ((17.51), ε ⁻¹ averaged), solid curve: analytical solution (17.49)

17.2 Poisson-Boltzmann Equation

Electrostatic interactions are very important in molecular physics. Bio-molecules are usually embedded in an environment which is polarizable and contains mobile charges (Na⁺,K⁺,Mg⁺⁺,Cl⁻⋯).

We divide the charge density formally into a fixed and a mobile part

(17.52)

The fixed part represents, for instance, the charge distribution of a protein molecule which, neglecting polarization effects, is a given quantity and provides the inhomogeneity of the equation. The mobile part, on the other hand, represents the sum of all mobile charges (e is the elementary charge and Z i the charge number of ion species i)

(17.53)

which move around until an equilibrium is reached which is determined by the mutual interaction of the ions. The famous Debye-Hückel [69] and Gouy-Chapman models [56, 110] assume that the electrostatic interaction

(17.54)

is dominant and the density of the ions n i is given by a Boltzmann-distribution

(17.55)

The potential Φ(r) has to be calculated in a self consistent way together with the density of mobile charges. The charge density of the free ions is

(17.56)

and the Poisson equation (17.7) turns into the Poisson-Boltzmann equation [91]

(17.57)

17.2.1 Linearization of the Poisson-Boltzmann Equation

For small ion concentrations the exponential can be expanded

(17.58)

For a neutral system

(17.59)

and the linearized Poisson-Boltzmann equation is obtained:

(17.60)

With

(17.61)

and the definition

(17.62)

we have finally

(17.63)

For a charged sphere with radius a embedded in a homogeneous medium the solution of (17.63) is given by

(17.64)

The potential is shielded by the ions. Its range is of the order λ Debye =1/κ (the so-called Debye length).

17.2.2 Discretization of the Linearized Poisson-Boltzmann Equation

To solve (17.63) the discrete equation (17.39) is generalized to [181]

(17.65)

If ε is constant then we have to iterate

(17.66)

17.3 Boundary Element Method for the Poisson Equation

Often continuum models are used to describe the solvation of a subsystem which is treated with a high accuracy method. The polarization of the surrounding solvent or protein is described by its dielectric constant ε and the subsystem is placed inside a cavity with ε=ε ₀ (Fig. 17.13). Instead of solving the Poisson equation for a large solvent volume another kind of method is often used which replaces the polarization of the medium by a distribution of charges over the boundary surface.

Fig. 17.13

Cavity in a dielectric medium

In the following we consider model systems which are composed of two spatial regions:

the outer region is filled with a dielectric medium (ε ₁) and contains no free charges

the inner region (“Cavity”) contains a charge distribution ρ(r) and its dielectric constant is ε=ε ₀.

17.3.1 Integral Equations for the Potential

Starting from the Poisson equation

(17.67)

we will derive some useful integral equations in the following. First we apply Gauss’s theorem to the expression [277]

(17.68)

with the yet undetermined function G(r−r′). Integration over a volume V gives

(17.69)

Now choose G as the fundamental solution of the Poisson equation

(17.70)

which obeys

(17.71)

to obtain the following integral equation for the potential:

(17.72)

First consider as the integration volume a sphere with increasing radius. Then the surface integral vanishes for infinite radius (Φ→0 at large distances) [277].

The gradient of ε(r) is nonzero only on the boundary surface Fig. 17.14 of the cavity and with the limiting procedure (d→0)

we obtain

(17.73)

Fig. 17.14

Discontinuity at the cavity boundary

This equation allows to calculate the potential inside and outside the cavity from the given charge density and the potential at the boundary.

Next we apply (17.72) to the cavity volume (where ε=ε ₀) and obtain

(17.74)

From comparison with (17.73) we have

and the potential can be alternatively calculated from the values of its normal gradient at the boundary

(17.75)

This equation can be interpreted as the potential generated by the charge density ρ plus an additional surface charge density

(17.76)

Integration over the volume outside the cavity (where ε=ε ₁ ε ₀) gives the following expression for the potential:

(17.77)

At the boundary the potential is continuous

(17.78)

whereas the normal derivative (hence the normal component of the electric field) has a discontinuity

(17.79)

17.3.2 Calculation of the Boundary Potential

For a numerical treatment the boundary surface is approximated by a finite set of small surface elements S i , i=1…N centered at r i with an area A i and normal vector n i (Fig. 17.15). (We assume planar elements in the following, the curvature leads to higher order corrections.)

Fig. 17.15

Representation of the boundary by surface elements

The corresponding values of the potential and its normal derivative are denoted as Φ i =Φ(r i ) and $\frac{\partial\varPhi _{i}}{\partial n}=\mathbf {n}_{i}\operatorname{grad}\varPhi(\mathbf {r}_{i})$. At a point $\mathbf {r}_{j}^{\pm}$ close to the element S j we obtain the following approximate equations:

(17.80)

(17.81)

These two equations can be combined to obtain a system of equations for the potential values only. To that end we approach the boundary symmetrically with $\mathbf {r}_{i}^{\pm}=\mathbf {r}_{i}\pm d\mathbf {n}_{i}$. Under this circumstance

(17.82)

and we find

(17.83)

The integrals for i≠j can be approximated by

(17.84)

The second integral has a simple geometrical interpretation (Fig. 17.16).

Fig. 17.16

Projection of the surface element

Since $\operatorname{grad}\frac{1}{|r-r'|}=-\frac{1}{|r-r'|^{2}}\frac{r-r'}{|r-r'|}$ the area element d A is projected onto a sphere with unit radius. The integral $\oint_{S_{j}}d\mathbf {A}\operatorname{grad}_{r-}\frac{1}{|\mathbf {r_{j}}-\mathbf {r}_{j}^{-}|}$ is given by the solid angle of S j with respect to r′. For r′→r j from inside this is just minus half of the full space angle of 4π. Thus we have

(17.85)

or

(17.86)

This system of equations can be used to calculate the potential on the boundary. The potential inside the cavity is then given by (17.73). Numerical stability is improved by a related method which considers the potential gradient along the boundary. Taking the normal derivative

(17.87)

of (17.80), (17.81) gives

(17.88)

(17.89)

In addition to (17.82) we have now

(17.90)

and the sum of the two equations gives

(17.91)

or finally

(17.92)

In terms of the surface charge density this reads:

(17.93)

This system of linear equations can be solved directly or iteratively (a simple damping scheme $\sigma_{m}'\rightarrow\omega\sigma _{m}'+(1-\omega)\sigma_{m,\mathit{old}}'$ with ω≈0.6 helps to get rid of oscillations). From the surface charges σ i A i the potential is obtained with the help of (17.75).

17.4 Boundary Element Method for the Linearized Poisson-Boltzmann Equation

We consider now a cavity within an electrolyte. The fundamental solution of the linear Poisson-Boltzmann equation (17.63)

(17.94)

obeys

(17.95)

Inserting into Green’s theorem (17.69) we obtain the potential outside the cavity

(17.96)

which can be combined with (17.74), (17.79) to give the following equations [32]

(17.97)

(17.98)

For a set of discrete boundary elements the following equations determine the values of the potential and its normal derivative at the boundary:

(17.99)

(17.100)

The situation is much more involved than for the simpler Poisson equation (with κ=0) since the calculation of many integrals including such with singularities is necessary [32, 143].

17.5 Electrostatic Interaction Energy (Onsager Model)

A very important quantity in molecular physics is the electrostatic interaction of a molecule and the surrounding solvent [11, 237]. We calculate it by taking a small part of the charge distribution from infinite distance (Φ(r→∞)=0) into the cavity. The charge distribution thereby changes from λρ(r) to (λ+dλ)ρ(r) with 0≤λ≤1. The corresponding energy change is

(17.101)

Multiplication of (17.93) by a factor of λ shows that the surface charges λσ n are the solution corresponding to the charge density λρ(r). It follows that σ n (λ)=λσ n and hence

(17.102)

The second summand is the self energy of the charge distribution which does not depend on the medium. The first summand vanishes without a polarizable medium and gives the interaction energy. Hence we have the final expression

(17.103)

For the special case of a spherical cavity with radius a an analytical solution by a multipole expansion is available [148]

(17.104)

with the multipole moments

(17.105)

The first two terms of this series are:

(17.106)

(17.107)

17.5.1 Example: Point Charge in a Spherical Cavity

Consider a point charge Q in the center of a spherical cavity of radius R (Fig. 17.17). The dielectric constant is given by

(17.108)

Fig. 17.17

Surface charges

Electric field and potential are inside the cavity

(17.109)

and outside

(17.110)

which in terms of the surface charge density σ is

(17.111)

with the total surface charge

(17.112)

The solvation energy (17.103) is given by

(17.113)

which is the first term (17.106) of the multipole expansion. Figure 17.18 shows numerical results.

Fig. 17.18

(Solvation energy with the boundary element method)  A spherical cavity is simulated with radius a=1 Å which contains a point charge in its center. The solvation energy is calculated with 25×25 (circles) and 50×50 (squares) surface elements of equal size. The exact expression (17.106) is shown by the solid curve

17.6 Problems

Problem 17.1

(Linearized Poisson-Boltzmann equation)

This computer experiment simulates a homogeneously charged sphere in a dielectric medium (Fig. 17.19). The electrostatic potential is calculated from the linearized Poisson-Boltzmann equation (17.65) on a cubic grid of up to 100³ points. The potential Φ(x) is shown along a line through the center together with a log-log plot of the maximum change per iteration

(17.114)

as a measure of convergence.

Fig. 17.19

Charged sphere in a dielectric medium

Explore the dependence of convergence on

the initial values which can be chosen either Φ(r)=0 or from the analytical solution

(17.115)

the relaxation parameter ω for different combinations of ε and κ

the resolution of the grid

Problem 17.2

(Boundary element method)

In this computer experiment the solvation energy of a point charge within a spherical cavity (Fig. 17.20) is calculated with the boundary element method (17.93).

Fig. 17.20

Point charge inside a spherical cavity

The calculated solvation energy is compared to the analytical value from (17.104)

(17.116)

where R is the cavity radius and s is the distance of the charge from the center of the cavity.

Explore the dependence of accuracy and convergence on

the damping parameter ω

the number of surface elements (6×6⋯42×42) which can be chosen either as dϕ dθ or dϕ dcosθ (equal areas)

the position of the charge

References

11.

J.S. Bader et al., J. Chem. Phys. 106, 2372 (1997) ADSCrossRef

32.

A.H. Boschitsch, M.O. Fenley, H.X. Zhou, J. Phys. Chem. B 106, 2741 (2002) CrossRef

44.

R.E. Bruccoleri, J. Novotny, M.E. Davis, K.A. Sharp, J. Comput. Chem. 18, 268 (1997) CrossRef

56.

D.L. Chapman, Philos. Mag. 25, 475 (1913) MATH

69.

P. Debye, E. Hückel, Phys. Z. 24, 185 (1923) MATH

91.

F. Fogolari, A. Brigo, H. Molinari, J. Mol. Recognit. 15, 377 (2002) CrossRef

110.

G.L. Gouy, J. Phys. 9, 457 (1910) MATH

120.

M.T. Heath, Multigrid methods, in Scientific Computing: An Introductory Survey (McGraw-Hill, New York, 2002), p. 478 ff. §11.5.7, Higher Education

143.

A.H. Juffer et al., J. Phys. Chem. B 101, 7664 (1997) CrossRef

148.

J.G. Kirkwood, J. Chem. Phys. 2, 351 (1934) ADSCrossRef

181.

A. Nicholls, B. Honig, J. Comput. Chem. 12, 435 (1990) CrossRef

188.

M. Oevermann, R. Klein, J. Comp. Physiol. 219, 749 (2006) MathSciNetADSMATH

189.

M. Oevermann, C. Scharfenberg, R. Klein, J. Comput. Phys. 228(14), 5184–5206 (2009). doi:10.​1016/​j.​jcp.​2009.​04.​018 MathSciNetADSMATHCrossRef

237.

T. Simonson, Rep. Prog. Phys. 66, 737 (2003) ADSCrossRef

238.

G. Skollermo, Math. Comput. 29, 697 (1975) MathSciNet

247.

P.N. Swarztrauber, R.A. Sweet, in Handbook of Fluid Dynamics and Fluid Machinery, ed. by J.A. Schetz, A.E. Fuhs (Wiley, New York, 1996)

277.

G. Wunsch, Feldtheorie (VEB Technik, Berlin, 1973)

279.

S. Yang, M.K. Gobbert, Appl. Math. Lett. 22, 325 (2009) MathSciNetMATHCrossRef

283.

Y. Zhu, A.C. Cangellaris, Multigrid Finite Element Methods for Electromagnetic Field Modeling (Wiley, New York, 2006), p. 132 ff. ISBN 0471741108 CrossRef

Footnotes

1

The solution depends on the boundary conditions, which in the simplest case are given by lim|r|→∞ Φ(r)=0.

2

But see Sect. 17.1.5 for the case of discontinuous ε.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_18© Springer International Publishing Switzerland 2013

18. Waves

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

We simulate classical waves which are, for instance, important in acoustics and electrodynamics. We use the method of finite differences to discretize the wave equation in one spatial dimension. Numerical solutions are obtained by an eigenvector expansion using trigonometric functions or by time integration. Accuracy and stability of different methods are compared. The wave function is second order in time and can be integrated directly with a two-step method. Alternatively, it can be converted into a first order system of equations of double dimension where the velocity appears explicitly. Finally, the second order wave equation can be replaced by two coupled first order equations for two variables (like velocity and density in case of acoustic waves), which can be solved by quite general methods. We compare the leapfrog, Lax-Wendroff and Crank-Nicolson methods. In a series of computer experiments we simulate waves on a string. We study reflection at an open or fixed boundary and at the interface between two different media. We compare dispersion and damping for different methods.

Waves are oscillations that move in space and time and are able to transport energy from one point to another. Quantum mechanical wavefunctions are discussed in Chap. 21. In this chapter we simulate classical waves which are, for instance, important in acoustics and electrodynamics. We use the method of finite differences to discretize the wave equation in one spatial dimension

(18.1)

Numerical solutions are obtained by an eigenvector expansion using trigonometric functions or by time integration. Accuracy and stability of different methods are compared. The wave function is second order in time and can be integrated directly with a two-step method. Alternatively, it can be converted into a first order system of equations of double dimension. Here, the velocity appears explicitly and velocity dependent damping can be taken into account. Finally, the second order wave equation can be replaced by two coupled first order equations for two variables (like velocity and density in case of acoustic waves), which can be solved by quite general methods. We compare the leapfrog, Lax-Wendroff and Crank-Nicolson methods. Only the Crank-Nicolson method is stable for Courant numbers α>1. It is an implicit method and can be solved iteratively. In a series of computer experiments we simulate waves on a string. We study reflection at an open or fixed boundary and at the interface between two different media. We compare dispersion and damping for different methods.

18.1 Classical Waves

In classical physics there are two main types of waves:

Electromagnetic waves do not require a medium. They are oscillations of the electromagnetic field and propagate also in vacuum. As an example consider a plane wave which propagates in x-direction and is linearly polarized (Fig. 18.1). The electric and magnetic field have the form

(18.2)

Maxwell’s equations read in the absence of charges and currents

(18.3)

The fields (18.2) have zero divergence and satisfy the first two equations. Application of the third and fourth equation gives

(18.4)

which can be combined to a one-dimensional wave equation

(18.5)

with velocity c=(μ ₀ ε ₀)−1/2.

Fig. 18.1

Electromagnetic wave

Mechanical waves propagate through an elastic medium like air, water or an elastic solid. The material is subject to external forces deforming it and elastic forces which try to restore the deformation. As a result the atoms or molecules move around their equilibrium positions. As an example consider one-dimensional acoustic waves in an organ pipe (Fig. 18.2):

Fig. 18.2

(Acoustic waves in one dimension)  A mass element dm=ϱA dx at position x experiences a total force $F=F(x)+F(x+dx)=-A\frac{\partial p}{\partial x}\,dx$. Due to the conservation of mass the change of the density $\frac{\partial\varrho}{\partial t}$ is given by the net flux $J=J(x)-J(x+dx)=-\varrho_{0}A\frac{\partial v}{\partial x}\,dx$

A mass element

(18.6)

at position x experiences an external force due to the air pressure which, according to Newton’s law changes the velocity v of the element as described by Euler’s equation¹

(18.7)

The pressure is a function of the density

(18.8)

where n=1 for an isothermal ideal gas and n≈1.4 for air under adiabatic conditions (no heat exchange), therefore

(18.9)

From the conservation of mass the continuity equation (11.​10) follows

(18.10)

Combining the time derivative of (18.10) and the spatial derivative of (18.9) we obtain again the one-dimensional wave equation

(18.11)

The wave equation can be factorized as

(18.12)

which shows that solutions of the advection equation

(18.13)

are also solutions of the wave equation, which have the form

(18.14)

In fact a general solution of the wave equation is given according to d’Alembert as the sum of two waves running to the left and right side with velocity c and a constant envelope (Fig. 18.3)

(18.15)

Fig. 18.3

d’Alembert solution to the wave equation

A special solution of this kind is the plane wave solution

with the dispersion relation

(18.16)

18.2 Spatial Discretization in One Dimension

We use the simplest finite difference expression for the spatial derivative (Sects. 3.​4, 11.​2)

(18.17)

and a regular grid

(18.18)

(18.19)

This turns the wave equation into the system of ordinary differential equations (Sect. 11.​2.​3)

(18.20)

where f ₀ and f M+1 have to be specified by suitable boundary conditions (Fig. 18.4). In matrix notation we have

(18.21)

(18.22)

where for

fixed boundaries

(18.23)

periodic boundaries²

(18.24)

open boundaries

(18.25)

moving boundaries

(18.26)

A combination of different boundary conditions for both sides is possible.

Fig. 18.4

(Boundary conditions for 1-dimensional waves)  Additional boundary points x ₀,x M+1 are used to realize the boundary conditions: (a) fixed boundaries: f(x ₀)=0, $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}}(f(x_{2})-2f(x_{1}))$ or f(x M+1)=0, $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}}(f(x_{M-1})-2f(x_{M}))$, (b) periodic boundary conditions: x ₀≡x M , $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}}(f(x_{2})+f(x_{M})-2f(x_{1}))$, x M+1≡x ₁, $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}}(f(x_{M-1})+f(x_{1})-2f(x_{NM}))$, (c) open boundaries: $\frac{\partial}{\partial x}f(x_{1})=\frac{f(x_{2})-f(x_{0})}{2\Delta x}=0$, $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}}(2f(x_{2})-2f(x_{1}))$ or $\frac{\partial}{\partial x}f(x_{M})= \frac {f(x_{M+1})-f(x_{M-1})}{2\Delta x}=0$, $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}}(2f(x_{M-1})-2f(x_{M}))$, (d) moving boundaries: f(x ₀,t)=ξ ₀(t), $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}} (f(x_{2})-2f(x_{1})+\xi _{0}(t) )$ or f(x M+1,t)=ξ M+1(t), $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}} (f(x_{M-1})-2f(x_{M})+\xi _{N+1}(t) )$

Equation (18.20) corresponds to a series of mass points which are connected by harmonic springs (Fig. 18.5), a model, which is used in solid state physics to describe longitudinal acoustic waves [129].

Fig. 18.5

(Atomistic model for longitudinal waves)  A set of mass points m is connected by springs with stiffness K. The elongation of mass point number j from its equilibrium position x j =jΔx is ξ j . The equations of motion $m\ddot{\xi }_{j}=-K(\xi_{j}-\xi_{j-1})-K(\xi_{j}-\xi_{j+1})$ coincide with (18.20) with a velocity of $c=\Delta x\sqrt {\frac{k\Delta x}{m}}$

18.3 Solution by an Eigenvector Expansion

For fixed boundaries (18.20) reads in matrix form

(18.27)

with the vector of function values:

(18.28)

and the matrix

(18.29)

which can be diagonalized exactly (Sect. 9.​3). The two boundary points f(0)=0 and f((M+1)Δx)=0 can be added without any changes. The eigenvalues are

(18.30)

with the frequencies

(18.31)

This result deviates from the dispersion relation of the continuous wave equation (18.11) ω k =ck and approximates it only for kΔx≪1 (Fig. 18.6).

Fig. 18.6

Dispersion of the discrete wave equation

The general solution has the form (Sect. 11.​2.​4)

(18.32)

The initial amplitudes and velocities are

(18.33)

with F m and G m given. Different eigenfunctions of a tridiagonal matrix are mutually orthogonal

(18.34)

and the coefficients C l± follow from a discrete Fourier transformation:

(18.35)

(18.36)

(18.37)

Finally the explicit solution of the wave equation is

(18.38)

Periodic or open boundaries can be treated similarly as the matrices can be diagonalized exactly (Sect. 9.​3). For moving boundaries the expansion coefficients are time dependent (Sect. 11.​2.​4).

18.4 Discretization of Space and Time

Using the finite difference expression also for the second time derivative the fully discretized wave equation is

(18.39)

For a plane wave

(18.40)

we find

(18.41)

which can be written as

(18.42)

with the so-called Courant-number [64]

(18.43)

From (18.42) we see that the dispersion relation is linear only for α=1. For α≠1 not all values of ω and k allowed (Fig. 18.7).

Fig. 18.7

(Dispersion of the discrete wave equation)  Only for α=1 or for small values of kΔx and ωΔt is the dispersion approximately linear. For α<1 only frequencies ω<ω max=2arcsin(α)/Δt are allowed whereas for α>1 the range of k-values is bounded by k max=2arcsin(1/α)/Δx

18.5 Numerical Integration with a Two-Step Method

We solve the discrete wave equation (18.39) with fixed or open boundaries for

(18.44)

on the regular grids

(18.45)

(18.46)

(18.47)

by applying the iteration

(18.48)

This is a two-step method which can be rewritten as a one-step method of double dimension

(18.49)

with the tridiagonal matrix

(18.50)

where a ₁ and a N have the values 1 for a fixed or 2 for an open end.

The matrix M has eigenvalues (Sect. 9.​3)

(18.51)

To simulate excitation of waves by a moving boundary we add one grid point with given elongation ξ ₀(t) and change the first equation into

(18.52)

Repeated iteration gives the series of function values

(18.53)

A necessary condition for stability is that all eigenvalues of T have absolute values smaller than one. Otherwise small perturbations would be amplified. The eigenvalue equation for T is

(18.54)

We substitute the solution of the second equation

(18.55)

into the first equation and use the eigenvectors of M (Sect. 9.​3) to obtain the eigenvalue equation

(18.56)

Hence σ is one of the two roots of

(18.57)

which are given by (Fig. 18.8)

(18.58)

From

we find

(18.59)

(18.60)

and the square root in (18.58) is imaginary if

(18.61)

which is the case for

(18.62)

This holds for all k only if

(18.63)

But then

(18.64)

and the algorithm is (conditionally) stable. If on the other hand |α|>1 then for some k-values the square root is real. Here we have

(18.65)

and finally

(18.66)

which shows that instabilities are possible in this case.

Fig. 18.8

(Stability regions of the two-step method)  Instabilities appear for |α|>1. One of the two eigenvalues σ becomes unstable (|σ|>1) for waves with large k-values

18.6 Reduction to a First Order Differential Equation

A general method to reduce the order of an ordinary differential equation (or a system of such) introduces the time derivatives as additional variables (Chap. 12). The spatially discretized one-dimensional wave equation (18.22) can be transformed into a system of double dimension

(18.67)

(18.68)

We use the improved Euler method (Sect. 12.​5)

(18.69)

(18.70)

and two different time grids

(18.71)

(18.72)

(18.73)

(18.74)

We obtain a leapfrog (Fig. 18.9) like algorithm (page 231)

(18.75)

(18.76)

where the updated velocity (18.75) has to be inserted into (18.76). This can be combined into the iteration

(18.77)

Fig. 18.9

Leapfrog method

Since the velocity appears explicitly we can easily add a velocity dependent damping like

(18.78)

which we approximate by

(18.79)

under the assumption of weak damping

(18.80)

To study the stability of this algorithm we consider the homogeneous problem with fixed boundaries. With the Courant number $\alpha=\frac {c\Delta t}{\Delta x}$ (18.77) becomes

(18.81)

Using the eigenvectors and eigenvalues of M (Sect. 9.​3)

(18.82)

we find the following equation for the eigenvalues σ:

(18.83)

Solving the second equation for u and substituting into the first equation we have

(18.84)

hence

(18.85)

Instabilities are possible if the square root is real and σ<−1 (σ>1 is not possible). This is the case for

(18.86)

(18.87)

The right inequality is satisfied, hence it remains

(18.88)

This holds for all k-values if it holds for the maximum of the sine-function

(18.89)

This shows that inclusion of the damping term even favors instabilities.

18.7 Two-Variable Method

For the 1-dimensional wave equation (18.11) there exists another possibility to reduce the order of the time derivative by splitting it up into two first order equations similar to (18.9), (18.10)

(18.90)

(18.91)

Several algorithms can be applied to solve these equations [162]. We discuss only methods which are second order in space and time and are rather general methods to solve partial differential equations. The boundary conditions need some special care. For closed boundaries with f(x ₀)=0 obviously $\frac{\partial f}{\partial t}(x_{0})=0$ whereas $\frac{\partial f}{\partial x}(x_{0})$ is finite. Hence a closed boundary for f(t,x) is connected with an open boundary for g(t,x) with $\frac{\partial g}{\partial x}(x_{0})=0$ and vice versa. This is well known from acoustics (Fig. 18.10).

Fig. 18.10

(Standing waves in an organ pipe)  At the closed (left) end the amplitude of the longitudinal velocity is zero whereas the amplitudes of pressure and density changes are extremal. This is reversed at the open (right) end

18.7.1 Leapfrog Scheme

We use symmetric differences (Sect. 3.​2) for the first derivatives

(18.92)

(18.93)

to obtain the following scheme

(18.94)

(18.95)

Using different time grids for the two variables

(18.96)

this translates into the algorithm (Fig. 18.11)

(18.97)

(18.98)

To analyze the stability we insert

(18.99)

and obtain the equations

(18.100)

(18.101)

which in matrix form read

(18.102)

The maximum amplification factor G is given by the largest eigenvalue, which is one of the roots of

(18.103)

(18.104)

The eigenvalues coincide with those of the two-step method (18.58).

Fig. 18.11

(Simulation with the leapfrog method)  A rectangular pulse is simulated with the two-variable leapfrog method. While for α=1 the pulse shape has not changed after 1000 steps, for smaller values the short wavelength components are lost due to dispersion

18.7.2 Lax-Wendroff Scheme

The Lax-Wendroff scheme can be derived from the Taylor series expansion

(18.105)

(18.106)

It uses symmetric differences on regular grids (18.45), (18.46) to obtain the iteration

(18.107)

(18.108)

(18.109)

with the tridiagonal matrix

(18.110)

To analyze the stability we insert

(18.111)

and calculate the eigenvalues (compare with (18.102)) of

(18.112)

which are given by

(18.113)

The root is always imaginary and

For α<1 we find |σ|<1. The method is stable but there is wavelength dependent damping (Figs. 18.12, 18.13).

Fig. 18.12

(Stability region of the Lax-Wendroff method)  Instabilities appear for |α|>1. In the opposite case short wavelength modes are damped

Fig. 18.13

(Simulation with the Lax-Wendroff method)  A rectangular pulse is simulated with the two-variable Lax-Wendroff method. While for α=1 the pulse shape has not changed after 2000 steps, for smaller values the short wavelength components are lost due to dispersion and damping

18.7.3 Crank-Nicolson Scheme

This method takes the average of the explicit and implicit Euler methods

(18.114)

(18.115)

and uses symmetric differences on the regular grids (18.45), (18.46) to obtain

(18.116)

(18.117)

which reads in matrix notation

(18.118)

This equation can be solved formally by collecting terms at time t n+1

(18.119)

and multiplying with the inverse matrix from left

(18.120)

Now, if u is an eigenvector of D with purely imaginary eigenvalue λ (Sect. 9.​3)

(18.121)

and furthermore

(18.122)

But, since the eigenvalue of the inverse matrix is the reciprocal of the eigenvalue, the eigenvalues of

(18.123)

are given by

(18.124)

Since λ is imaginary, we find |σ|=1. The Crank-Nicolson method is stable and does not show damping like the Lax-Wendroff method. However, there is considerable dispersion. Solution of the linear system (18.119) is complicated and can be replaced by an iterative predictor-corrector method. Starting from the initial guess

(18.125)

we iterate

(18.126)

(18.127)

(18.128)

In principle this iteration could be repeated more times, but as Teukolsky showed [250], two iterations are optimal for hyperbolic equations like the advection or wave equation. The region of stability is reduced (Figs. 18.14, 18.15) compared to the implicit Crank-Nicolson method. The eigenvalues are

(18.129)

(18.130)

(18.131)

Fig. 18.14

(Simulation with the iterated Crank-Nicolson method)  A rectangular pulse is simulated with the two-variable iterated Crank-Nicolson method. Only this method is stable for values α>1

Fig. 18.15

(Simulation of a triangular pulse)  A triangular pulse is simulated with different two-variable methods (full curve: initial conditions, dashed curve: leapfrog, dash-dotted: Lax-Wendroff, dotted: iterated Crank-Nicolson). This pulse contains less short wavelength components than the square pulse and shows much less deformation even after 5000 steps

18.8 Problems

Problem 18.1

(Waves on a damped string)

In this computer experiment we simulate waves on a string with a moving boundary with the method from Sect. 18.6.

Excite the left boundary with a continuous sine function and try to generate standing waves

Increase the velocity until instabilities appear

Compare reflection at open and fixed right boundary

Observe the dispersion of pulses with different shape and duration

The velocity can be changed by a factor n (refractive index) in the region x>0. Observe reflection at the boundary x=0

Problem 18.2

(Waves with the Fourier transform method)

In this computer experiment we use the method from Sect. 18.3 to simulate waves on a string with fixed boundaries.

Different initial excitations of the string can be selected.

The dispersion can be switched off by using ω k =ck instead of the proper eigenvalues (18.31).

Problem 18.3

(Two-variable methods)

In this computer experiment we simulate waves with periodic boundary conditions. Different initial values (rectangular, triangular or Gaussian pulses of different widths) and methods (leapfrog, Lax-Wendroff, iterated Crank-Nicolson) can be compared.

References

64.

R. Courant, K. Friedrichs, H. Lewy, Math. Ann. 100, 32 (1928) MathSciNetMATHCrossRef

129.

H. Ibach, H. Lüth, Solid-State Physics: An Introduction to Principles of Materials Science. Advanced Texts in Physics (Springer, Berlin, 2003)

162.

L. Lynch, Numerical integration of linear and nonlinear wave equations. Bachelor thesis, Florida Atlantic University, Digital Commons@University of Nebraska-Lincoln, 2004

250.

S.A. Teukolsky, Phys. Rev. D 61(8), 087501 (2000). doi:10.​1103/​PhysRevD.​61.​087501 MathSciNetADSCrossRef

Footnotes

1

We consider only small deviations from the equilibrium values ϱ ₀,p ₀,v ₀=0.

2

This corresponds to the boundary condition f ₀=f ₂, $\frac{\partial}{\partial x}f(x_{1})=0$. Alternatively we could use f ₀=f ₁, $\frac{\partial}{\partial x}f(x_{1/2})=0$ which replaces the 2 s in the first and last row by 1 s.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_19© Springer International Publishing Switzerland 2013

19. Diffusion

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Diffusion is one of the simplest non-equilibrium processes. It describes the transport of heat and the time evolution of differences in substance concentrations. In this chapter, the one-dimensional diffusion equation is semi-discretized with finite differences. The time integration is performed with three different Euler methods. The explicit Euler method is conditionally stable only for small Courant number, which makes very small time steps necessary. The fully implicit method is unconditionally stable but its dispersion deviates largely from the exact expression. The Crank-Nicolson method is also unconditionally stable. However, it is more accurate and its dispersion relation is closer to the exact one. Extension to more than one dimension is easily possible, but the numerical effort increases drastically as there is no formulation involving simple tridiagonal matrices like in one dimension. The split operator approximation uses the one-dimensional method independently for each dimension. It is very efficient with almost no loss in accuracy. In a computer experiment the different schemes are compared for diffusion in two dimensions.

Diffusion is one of the simplest non-equilibrium processes. It describes the transport of heat [52, 95] and the time evolution of differences in substance concentrations [82]. In this chapter, the one-dimensional diffusion equation

(19.1)

is semi-discretized with finite differences. The time integration is performed with three different Euler methods. The explicit Euler method is conditionally stable only for small Courant number $\alpha =\frac{D\Delta t}{\Delta x^{2}}< 1/2$, which makes very small time steps necessary. The fully implicit method is unconditionally stable but its dispersion deviates largely from the exact expression. The Crank-Nicolson method is also unconditionally stable. However, it is more accurate and its dispersion relation is closer to the exact one. Extension to more than one dimension is easily possible, but the numerical effort increases drastically as there is no formulation involving simple tridiagonal matrices like in one dimension. The split operator approximation uses the one-dimensional method independently for each dimension. It is very efficient with almost no loss in accuracy. In a computer experiment the different schemes are compared for diffusion in two dimensions.

19.1 Particle Flux and Concentration Changes

Let f(x,t) denote the concentration of a particle species and J the corresponding flux of particles. Consider a small cube with volume h ³ (Fig. 19.1). The change of the number of particles within this volume is given by the integral form of the conservation law (11.​10)

(19.2)

where the source term S(r) accounts for creation or destruction of particles due to for instance chemical reactions. In Cartesian coordinates we have

(19.3)

In the limit of small h this turns into the differential form of the conservation law

(19.4)

or after division by h ³

(19.5)

Within the framework of linear response theory the flux is proportional to the gradient of f (Fig. 19.2),

(19.6)

Together we obtain the diffusion equation

(19.7)

which in the special case of constant D simplifies to

(19.8)

Fig. 19.1

Flux through a volume element

Fig. 19.2

Diffusion due to a concentration gradient

19.2 Diffusion in One Dimension

We will use the finite differences method which works well if the diffusion constant D is constant in time and space. We begin with diffusion in one dimension and use regular grids t n =nΔt, x m =mΔx, $f_{m}^{n}=f(t_{n},x_{m})$ and the discretized second derivative

(19.9)

to obtain the semi-discrete diffusion equation

(19.10)

or in matrix notation

(19.11)

with the tridiagonal matrix

(19.12)

Boundary conditions can be taken into account by introducing extra boundary points x ₀,x M+1 (Fig. 19.3).

Fig. 19.3

(Boundary conditions for 1-dimensional diffusion)  Additional boundary points x ₀,x M+1 are used to realize the boundary conditions. (a) Dirichlet boundary conditions: the function values at the boundary are given f(t,x ₀)=ξ ₀(t), $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}} (f(x_{2})-2f(x_{1})+\xi_{0}(t) )$ or f(t,x M+1)=ξ M+1(t), $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}} (f(x_{M-1})-2f(x_{M})+\xi _{M+1}(t))$. (b) Neumann boundary conditions: the flux through the boundary is given, hence the derivative $\frac{\partial f}{\partial x}$ at the boundary $f(t,x_{0})=f(t,x_{2})+2\frac{\Delta x}{D}J_{1}(t)$, $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}} (2f(x_{2})-2f(x_{1})+2\frac{\Delta x}{D}J_{1}(t) )$ or $f(t,x_{M+1})=f(t,x_{M-1})-\frac{2\Delta x}{D}J_{M}(t)$, $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}} (2f(x_{M-1})-2f(x_{M})-\frac{2\Delta x}{D}J_{M}(t))$. (c) No-flow boundary conditions: there is no flux through the boundary, hence the derivative $\frac{\partial f}{\partial x}=0$ at the boundary f(t,x ₀)=f(t,x ₂), $\frac{\partial^{2}}{\partial x^{2}}f(x_{1})=\frac{1}{\Delta x^{2}} (2f(x_{2})-2f(x_{1}))$ or f(t,x M )=f(t,x M−2), $\frac{\partial^{2}}{\partial x^{2}}f(x_{M})=\frac{1}{\Delta x^{2}} (2f(x_{M-1})-2f(x_{M}) )$

19.2.1 Explicit Euler (Forward Time Centered Space) Scheme

A simple Euler step (Sect. 12.​3) makes the approximation

(19.13)

For homogeneous boundary conditions f=0 this becomes in matrix form

(19.14)

with the tridiagonal matrix

(19.15)

where α is the Courant number for diffusion

(19.16)

The eigenvalues of M are (compare (18.​30))

(19.17)

and hence the eigenvalues of A are given by

(19.18)

The algorithm is stable if

(19.19)

which holds if

(19.20)

The maximum of the sine function is $\sin(\frac{M\pi}{2(M+1)})\approx1$. Hence the right hand inequation is satisfied and from the left one we have

(19.21)

The algorithm is stable for

(19.22)

The dispersion relation follows from inserting a plane wave ansatz

(19.23)

For α>1/4 the right hand side changes sign at

(19.24)

The imaginary part of ω has a singularity at k c and the real part has a finite value of π for k>k c (Fig. 19.4 on page 356). Deviations from the exact dispersion

(19.25)

are large, except for very small k.

Fig. 19.4

(Dispersion of the explicit Euler method)  The dispersion of the explicit method is shown for different values of the Courant number α and compared to the exact dispersion (dashed curve). The imaginary part of ω shows a singularity for α>1/4. Above the singularity ω is complex valued

19.2.2 Implicit Euler (Backward Time Centered Space) Scheme

Next we use the backward difference

(19.26)

to obtain the implicit method

(19.27)

or in matrix notation

(19.28)

which can be solved formally by

(19.29)

The eigenvalues of A are

(19.30)

and the eigenvalues of A ⁻¹

(19.31)

The implicit method is unconditionally stable since

(19.32)

The dispersion relation of the implicit scheme follows from

(19.33)

There is no singularity and ω is purely imaginary. Still, deviations from the exact expression are large (Fig. 19.5 on page 357).

Fig. 19.5

(Dispersion of the implicit Euler method)  The dispersion of the fully implicit method is shown for two different values of the Courant number α and compared to the exact dispersion (dashed curve)

Formally a matrix inversion is necessary. Numerically it is much more efficient to solve the tridiagonal system of equations (page 69).

(19.34)

19.2.3 Crank-Nicolson Method

The Crank-Nicolson method [65] which is often used for diffusion problems, combines implicit and explicit methods. It uses the Heun method (Sect. 12.​5) for the time integration

(19.35)

(19.36)

(19.37)

This approximation is second order both in time and space and becomes in matrix notation

(19.38)

which can be solved by

(19.39)

Again it is numerically much more efficient to solve the tridiagonal system of equations (19.38) than to calculate the inverse matrix.

The eigenvalues of this method are

(19.40)

Since αμ<0 it follows

(19.41)

and hence

(19.42)

On the other hand we have

(19.43)

(19.44)

(19.45)

This shows that the Crank-Nicolson method is stable [251]. The dispersion follows from

(19.46)

For α>1/2 there is a sign change of the right hand side at

(19.47)

The imaginary part of ω has a singularity at k c and ω is complex valued for k>k c (Fig. 19.6 on page 359).

Fig. 19.6

(Dispersion of the Crank-Nicolson method)  The dispersion of the Crank-Nicolson method is shown for different values of the Courant number α and compared to the exact dispersion (dashed curve). The imaginary part of ω shows a singularity for α>1/2. Above the singularity ω is complex valued. The exact dispersion is approached quite closely for α≈1/2

19.2.4 Error Order Analysis

Taylor series gives for the exact solution

(19.48)

whereas for the explicit method

(19.49)

and for the implicit method

(19.50)

The Crank-Nicolson method has higher accuracy in Δt:

(19.51)

19.2.5 Finite Element Discretization

In one dimension discretization with finite differences is very similar to discretization with finite elements, if Galerkin’s method is applied on a regular grid (Chap. 11). The only difference is the non-diagonal form of the mass-matrix which has to be applied to the time derivative [88]. Implementation of the discretization scheme (11.​170) is straightforward. The semi-discrete diffusion equation becomes

(19.52)

or in matrix form

(19.53)

This can be combined with the Crank-Nicolson scheme to obtain

(19.54)

or

(19.55)

19.3 Split-Operator Method for Multidimensions

The simplest discretization of the Laplace operator in 3 dimensions is given by

(19.56)

where

(19.57)

etc. denote the discretized second derivatives. Generalization of the Crank-Nicolson method for the 3-dimensional problem gives

(19.58)

But now the matrices representing the operators M x , M y , M z are not tridiagonal. To keep the advantages of tridiagonal matrices we use the approximations

(19.59)

(19.60)

and rearrange the factors to obtain

(19.61)

which represents successive application of the 1-dimensional scheme for the three directions separately. The last step was possible since the operators M i and M j for different directions i≠j commute. For instance

(19.62)

The Taylor series of (19.58) and (19.61) coincide up to second order with respect to αM i :

(19.63)

(19.64)

Hence we have

(19.65)

and the error order is conserved by the split operator method.

19.4 Problems

Problem 19.1

(Diffusion in 2 dimensions)

In this computer experiment we solve the diffusion equation on a two dimensional grid for

an initial distribution f(t=0,x,y)=δ x,0 δ y,0

a constant source f(t=0)=0, S(t,x,y)=δ x,0 δ y,0

Compare implicit, explicit and Crank-Nicolson method.

References

52.

Y.A. Cengel, Heat transfer—A Practical Approach, 2nd edn. (McGraw-Hill, New York, 2003), p. 26. ISBN 0072458933, 9780072458930

65.

J. Crank, P. Nicolson, Proc. Camb. Philol. Soc. 43, 50 (1947) MathSciNetADSMATHCrossRef

82.

A. Fick, Philos. Mag. 10, 30 (1855)

88.

C.A.J. Fletcher, Computational Techniques for Fluid Dynamics, vol. I, 2nd edn. (Springer, Berlin, 1991) MATH

95.

J. Fourier, The Analytical Theory of Heat (Cambridge University Press, Cambridge, 1878), reissued by Cambridge University Press, 2009. ISBN 978-1-108-00178-6

251.

J.W. Thomas, Numerical Partial Differential Equations: Finite Difference Methods. Texts in Applied Mathematics, vol. 22 (Springer, Berlin, 1995) MATH

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_20© Springer International Publishing Switzerland 2013

20. Nonlinear Systems

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

Nonlinear equations are difficult to solve and give rise to interesting phenomena. We discuss fixed points of the logistic mapping and analyze their stability. A bifurcation diagram visualizes the appearance of period doubling and chaotic behavior as a function of a control parameter. The Lyapunov exponent helps to distinguish stable fixed points and periods from chaotic regions. For continuous-time models, the iterated function is replaced by a system of differential equations. For stable equilibria all eigenvalues of the Jacobian matrix must have a negative real part. We discuss the Lotka-Volterra model, which is the simplest model of predator-prey interactions and the Holling-Tanner model, which incorporates functional response. Finally we allow for spatial inhomogeneity and include diffusive terms to obtain reaction-diffusion systems, which show the phenomena of traveling waves and pattern formation. Computer experiments study orbits and bifurcation diagram of the logistic map, periodic oscillations of the Lotka-Volterra model, oscillations and limit cycles of the Holling-Tanner model and finally pattern formation in the diffusive Lotka-Volterra model.

Nonlinear problems [123, 145] are of interest to physicists, mathematicians and also engineers. Nonlinear equations are difficult to solve and give rise to interesting phenomena like indeterministic behavior, multistability or formation of patterns in time and space. In the following we discuss recurrence relations like an iterated function [135]

(20.1)

systems of ordinary differential equations like population dynamics models [178, 214, 245]

(20.2)

or partial differential equations like the reaction-diffusion equation [83, 112, 178]

(20.3)

where f and g are nonlinear in the mathematical sense.¹ We discuss fixed points of the logistic mapping and analyze their stability. A bifurcation diagram visualizes the appearance of period doubling and chaotic behavior as a function of a control parameter. The Lyapunov exponent helps to distinguish stable fixed points and periods from chaotic regions. For continuous-time models, the iterated function is replaced by a system of differential equations. For stable equilibria all eigenvalues of the Jacobian matrix must have a negative real part. We discuss the Lotka-Volterra model, which is the simplest model of predator-prey interactions and the Holling-Tanner model, which incorporates functional response. Finally we allow for spatial inhomogeneity and include diffusive terms to obtain reaction-diffusion systems, which show the phenomena of traveling waves and pattern formation. Computer experiments study orbits and bifurcation diagram of the logistic map, periodic oscillations of the Lotka-Volterra model, oscillations and limit cycles of the Holling-Tanner model and finally pattern formation in the diffusive Lotka-Volterra model.

20.1 Iterated Functions

Starting from an initial value x ₀ a function f is iterated repeatedly

(20.4)

The sequence of function values x ₀,x ₁⋯ is called the orbit of x ₀. It can be visualized in a 2-dimensional plot by connecting the points

by straight lines (Fig. 20.1).

Fig. 20.1

(Orbit of an iterated function)  The sequence of points (x i ,x i+1),(x i+1,x i+1) is plotted together with the curves y=f(x) (dashed) and y=x (dotted)

20.1.1 Fixed Points and Stability

If the equation

(20.5)

has solutions x ∗, then these are called fixed points. Consider a point in the vicinity of a fixed point

(20.6)

and make a Taylor series expansion

(20.7)

with the notation

(20.8)

Repeated iteration gives²

(20.9)

with the sequence of deviations

The orbit moves away from the fixed point for arbitrarily small ε ₀ if |f′(x ∗)|>1 whereas the fixed point is attractive for |f′(x ∗)|<1 (Fig. 20.2).

Fig. 20.2

(Attractive fixed point)  The orbit of an attractive fixed point converges to the intersection of the curves y=x and y=f(x)

Higher order fixed points are defined by iterating f(x) several times. A fixed point of order n simultaneously solves

(20.10)

The iterated function values cycle periodically (Fig. 20.3) through

This period is attractive if

Fig. 20.3

(Periodic orbit)  The orbit of an attractive fourth order fixed point cycles through the values x ₁=f(x ₄), x ₂=f(x ₁), x ₃=f(x ₂), x ₄=f(x ₃)

20.1.2 The Lyapunov Exponent

Consider two neighboring orbits with initial values x ₀ and x ₀+ε ₀. After n iterations the distance is

(20.11)

with the so called Lyapunov exponent [161] λ which is useful to characterize the orbit. The Lyapunov exponent can be determined from

(20.12)

or numerically easier with the approximation

(20.13)

(20.14)

from

(20.15)

For a stable fixed point

(20.16)

and for an attractive period

(20.17)

Orbits with λ<0 are attractive fixed points or periods. If, on the other hand, λ>0, the orbit is irregular and very sensitive to the initial conditions, hence is chaotic.

20.1.3 The Logistic Map

A population of animals is observed yearly. The evolution of the population density N is described in terms of the reproduction rate r by the recurrence relation

(20.18)

where N n is the population density in year number n. If r is constant, an exponential increase or decrease of N results.

The simplest model for the growth of a population which takes into account that the resources are limited is the logistic model by Verhulst [264]. He assumed that the reproduction rate r depends on the population density N in a simple way (Fig. 20.4)

(20.19)

The Verhulst model (20.19) leads to the iterated nonlinear function

(20.20)

with r ₀>0, K>0. We denote the quotient of population density and carrying capacity by the new variable

(20.21)

and obtain an equation with only one parameter, the so called logistic mapping

(20.22)

Fig. 20.4

(Reproduction rate of the logistic model)  At low densities the growth rate has its maximum value r ₀. At larger densities the growth rate declines and reaches r=0 for N=K. The parameter K is called carrying capacity

20.1.4 Fixed Points of the Logistic Map

Consider an initial point in the interval

(20.23)

We want to find conditions on r to keep the orbit in this interval. The maximum value of x n+1 is found from

(20.24)

which gives x n =1/2 and max(x n+1)=r/4. If r>4 then negative x n appear after some iterations and the orbit is not bound by a finite interval since

(20.25)

The fixed point equation

(20.26)

always has the trivial solution

(20.27)

and a further solution

(20.28)

which is only physically reasonable for r>1, since x should be a positive quantity. For the logistic mapping the derivative is

(20.29)

which for the first fixed point x ∗=0 gives |f′(0)|=r. This fixed point is attractive for 0<r<1 and becomes unstable for r>1. For the second fixed point we have $|f'(1-\frac{1}{r})|=|2-r|$, which is smaller than one in the interval 1<r<3. For r<1 no such fixed point exists. Finally, for r ₁=3 the first bifurcation appears and higher order fixed points become stable.

Consider the fixed point of the double iteration

(20.30)

All roots of this fourth order equation can be found since we already know two of them. The remaining roots are

(20.31)

They are real valued if

(20.32)

For r>3 the orbit oscillates between $x_{1}^{*}$ and $x_{2}^{*}$ until the next period doubling appears for $r_{2}=1+\sqrt{6}$. With increasing r more and more bifurcations appear and finally the orbits become chaotic (Fig. 20.5).

Fig. 20.5

(Orbits of the logistic map)  Left: For 0<r<1 the logistic map has the attractive fixed point x ∗=0. Middle: In the region 1<r<3 this fixed point becomes unstable and another stable fixed point is at x ∗=1−1/r. Right: For $3<r<1+\sqrt{6}$ the second order fixed point (20.31) is stable. For larger values of r more and more bifurcations appear

20.1.5 Bifurcation Diagram

The bifurcation diagram visualizes the appearance of period doubling and chaotic behavior as a function of the control parameter r (Fig. 20.6).

Fig. 20.6

(Bifurcation diagram of the logistic map)  For different values of r the function is iterated 1100 times. The first 1000 iterations are dropped to allow the trajectory to approach stable fixed points or periods. The iterated function values x ₁₀₀₀⋯x ₁₁₀₀ are plotted in an r–x diagram together with the estimate (20.15) of the Lyapunov exponent.The first period doublings appear at r=3 and $r=1+\sqrt{6}$. For larger values chaotic behavior is observed and the estimated Lyapunov exponent becomes positive. In some regions motion is regular again with negative Lyapunov exponent

20.2 Population Dynamics

If time is treated as a continuous variable, the iterated function has to be replaced by a differential equation

(20.33)

or, more generally by a system of equations

(20.34)

20.2.1 Equilibria and Stability

The role of the fixed points is now taken over by equilibria, which are solutions of

(20.35)

which means roots of f(N). Let us investigate small deviations from equilibrium with the help of a Taylor series expansion. Inserting

(20.36)

we obtain

(20.37)

but since f(N eq )=0, we have approximately

(20.38)

with the solution

(20.39)

The equilibrium is only stable if $\mathfrak{Re}\, f'(N_{\mathit{eq}})<0$, since then small deviations disappear exponentially. For $\mathfrak{Re}\,f'(N_{\mathit{eq}})>0$ deviations will increase, but the exponential behavior holds only for not too large deviations and saturation may appear. If the derivative f′(N eq ) has a nonzero imaginary part then oscillations will be superimposed. For a system of equations the equilibrium is defined by

(20.40)

and if such an equilibrium exists, linearization gives

(20.41)

(20.42)

The equilibrium is stable if all eigenvalues λ i of the derivative matrix have a negative real part.

20.2.2 The Continuous Logistic Model

The continuous logistic model describes the evolution by the differential equation

(20.43)

To find possible equilibria we have to solve

(20.44)

which has the two roots x eq =0 and x eq =1 (Fig. 20.7).

Fig. 20.7

(Equilibria of the logistic model)  The equilibrium x eq =0 is unstable since an infinitesimal deviation grows exponentially in time. The equilibrium x eq =1 is stable since initial deviations disappear exponentially

The derivative f′ is

(20.45)

Since f′(0)=r ₀>0 and f′(1)=−r ₀<0 only the second equilibrium is stable.

20.3 Lotka-Volterra Model

The model by Lotka [160] and Volterra [266] is the simplest model of predator-prey interactions. It has two variables, the density of prey (H) and the density of predators (P). The overall reproduction rate of each species is given by the difference of the birth rate r and the mortality rate m

which both may depend on the population densities. The Lotka-Volterra model assumes that the prey mortality depends linearly on the predator density and the predator birth rate is proportional to the prey density

(20.46)

where a is the predation rate coefficient and b is the reproduction rate of predators per 1 prey eaten. Together we end up with a system of two coupled nonlinear differential equations

(20.47)

where r H is the intrinsic rate of prey population increase and m P the predator mortality rate.

20.3.1 Stability Analysis

To find equilibria we have to solve the system of equations

(20.48)

The first equation is solved by H eq =0 or by P eq =r H /a. The second equation is solved by P eq =0 or by H eq =m P /b. Hence there are two equilibria, the trivial one

(20.49)

and a nontrivial one

(20.50)

Linearization around the zero equilibrium gives

(20.51)

This equilibrium is unstable since a small prey population will increase exponentially. Now expand around the nontrivial equilibrium:

(20.52)

(20.53)

(20.54)

or in matrix notation

(20.55)

The eigenvalues are purely imaginary

(20.56)

and the corresponding eigenvectors are

(20.57)

The solution of the linearized equations is then given by

(20.58)

which describes an ellipse in the ξ–η plane (Fig. 20.8). The nonlinear equations (20.48) have a first integral

(20.59)

and therefore the motion in the H–P plane is on a closed curve around the equilibrium which approaches an ellipse for small amplitudes ξ,η.

Fig. 20.8

(Lotka-Volterra model)  The predator and prey population densities show periodic oscillations (right). In the H–P plane the system moves on a closed curve, which becomes an ellipse for small deviations from equilibrium (left)

20.4 Functional Response

Holling [124, 125] studied predation of small mammals on pine sawflies. He suggested a very popular model of functional response. Holling assumed that the predator spends its time on two kinds of activities, searching for prey and prey handling (chasing, killing, eating, digesting). The total time equals the sum of time spent on searching and time spent on handling

(20.60)

Capturing prey is assumed to be a random process. A predator examines an area α per time and captures all prey found there. After spending the time T search the predator examined an area of αT search and captured H T =HαT search prey. Hence the predation rate is

(20.61)

The handling time is assumed to be proportional to the number of prey captured

(20.62)

where T h is the handling time spent per one prey. The predation rate then is given by

(20.63)

At small densities handling time is unimportant and the predation rate is a ₀=α whereas at high prey density handling limits the number of prey captured and the predation rate approaches $a_{\infty}=\frac{1}{HT_{h}}$ (Fig. 20.9).

Fig. 20.9

Functional response of Holling’s model

20.4.1 Holling-Tanner Model

We combine the logistic model with Holling’s model for the predation rate [124, 125, 249]

(20.64)

and assume that the carrying capacity of the predator is proportional to the density of prey

(20.65)

Obviously there is a trivial equilibrium with P eq =H eq =0. Linearization gives

(20.66)

which shows that this equilibrium is unstable. There is another trivial equilibrium with P eq =0, H eq =K H . Here we find:

(20.67)

(20.68)

(20.69)

(20.70)

Let us now look for nontrivial equilibria. The nullclines (Fig. 20.10) are the curves defined by $\frac{\mathrm{d}H}{\mathrm{d}t}=0$ and $\frac{\mathrm{d}P}{\mathrm{d}t}=0$, hence by

(20.71)

(20.72)

Fig. 20.10

Nullclines of the predator-prey model

The H-nullcline is a parabola at

(20.73)

It intersects the H-axis at H=K H and H=−1/αT h and the P-axis at P=r H /α. There is one intersection of the two nullclines at positive values of H and P which corresponds to a nontrivial equilibrium. The equilibrium density H eq is the positive root of

(20.74)

It is explicitly given by

(20.75)

The prey density then follows from

(20.76)

The matrix of derivatives has the elements

(20.77)

from which the eigenvalues are calculated as

(20.78)

Oscillations appear, if the squareroot is imaginary (Fig. 20.11).

Fig. 20.11

(Holling-Tanner model)  Top: evolution from an unstable equilibrium to a limit cycle, middle: a stable equilibrium is approached with oscillations, bottom: stable equilibrium without oscillations

20.5 Reaction-Diffusion Systems

So far we considered spatially homogeneous systems where the density of a population or the concentration of a chemical agent depend only on time. If we add spatial inhomogeneity and diffusive motion, new and interesting phenomena like pattern formation or traveling excitations can be observed.

20.5.1 General Properties of Reaction-Diffusion Systems

Reaction-diffusion systems are described by a diffusion equation³ where the source term depends nonlinearly on the concentrations

(20.79)

20.5.2 Chemical Reactions

Consider a number of chemical reactions which are described by stoichiometric equations

(20.80)

The concentration of agent A i is

(20.81)

with the reaction variable

(20.82)

and the reaction rate

(20.83)

which, in general is a nonlinear function of all concentrations. The total concentration change due to diffusion and reactions is given by

(20.84)

20.5.3 Diffusive Population Dynamics

Combination of population dynamics (20.2) and diffusive motion gives a similar set of coupled equations for the population densities

(20.85)

20.5.4 Stability Analysis

Since a solution of the nonlinear equations is not generally possible we discuss small deviations from an equilibrium solution $N_{k}^{\mathit{eq}}$ ⁴ with

(20.86)

Obviously the equilibrium obeys

(20.87)

We linearize the equations by setting

(20.88)

and expand around the equilibrium

(20.89)

Plane waves are solutions of the linearized problem.⁵ Using the ansatz

(20.90)

we obtain

(20.91)

where M ₀ denotes the matrix of derivatives and D the matrix of diffusion constants. For a stable plane wave solution λ=iω is an eigenvalue of

(20.92)

with

(20.93)

If there are purely imaginary eigenvalues for some k they correspond to stable solutions which are spatially inhomogeneous and lead to formation of certain patterns. Interestingly, diffusion can lead to instabilities even for a system which is stable in the absence of diffusion [258].

20.5.5 Lotka-Volterra Model with Diffusion

As a simple example we consider again the Lotka-Volterra model. Adding diffusive terms we obtain the equations

(20.94)

There are two equilibria

(20.95)

and

(20.96)

The Jacobian matrix is

(20.97)

which gives for the trivial equilibrium

(20.98)

One eigenvalue λ ₁=−m P −D P k ² is negative whereas the second λ ₂=r H −D H k ² is positive for k ²<r H /D H . Hence this equilibrium is unstable against fluctuations with long wavelengths. For the second equilibrium we find:

(20.99)

(20.100)

For small k with $k^{2}<2\sqrt{m_{P}r_{H}}/|D_{H}-D_{P}|$ damped oscillations are expected whereas the system is stable against fluctuations with larger k (Figs. 20.12, 20.13, 20.14).

Fig. 20.12

(Lotka-Volterra model with diffusion)  The time evolution is calculated for initial random fluctuations. Colors indicate the deviation of the predator concentration P(x,y,t) from its average value (blue: ΔP<−0.1, green: −0.1<ΔP<−0.01, black: −0.01<ΔP<0.01, yellow: 0.01<ΔP<0.1, red: ΔP>0.1). Parameters as in Fig. 20.13

Fig. 20.13

(Dispersion of the diffusive Lotka-Volterra model)  Real (full curve) and imaginary part (broken line) of the eigenvalue λ (20.100) are shown as a function of k. Parameters are D H =D P =1, m P =r H =a=b=0.5

Fig. 20.14

(Traveling waves in the diffusive Lotka-Volterra model)  Initially P(x,y)=P eq and H(x,y) is peaked in the center. This leads to oscillations and a sharp wavefront moving away from the excitation. Color code and parameters as in Fig. 20.12

20.6 Problems

Problem 20.1

(Orbits of the iterated logistic map)

This computer example draws orbits (Fig. 20.5) of the logistic map

(20.101)

You can select the initial value x ₀ and the variable r.

Problem 20.2

(Bifurcation diagram of the logistic map)

This computer example generates a bifurcation diagram of the logistic map (Fig. 20.6). You can select the range of r.

Problem 20.3

(Lotka-Volterra model)

Equations (20.47) are solved with the improved Euler method (Fig. 20.8). The predictor step uses an explicit Euler step to calculate the values at t+Δt/2

(20.102)

(20.103)

and the corrector step advances time by Δt

(20.104)

(20.105)

Problem 20.4

(Holling-Tanner model)

The equations of the Holling-Tanner model (20.64), (20.65) are solved with the improved Euler method (see Fig. 20.11). The predictor step uses an explicit Euler step to calculate the values at t+Δt/2:

(20.106)

(20.107)

and the corrector step advances time by Δt:

(20.108)

(20.109)

Problem 20.5

(Diffusive Lotka-Volterra model)

The Lotka-Volterra model with diffusion (20.94) is solved in 2 dimensions with an implicit method (19.​2.​2) for the diffusive motion (Figs. 20.12, 20.14). The split operator approximation (19.​3) is used to treat diffusion in x and y direction independently. The equations

(20.110)

are equivalent to the following systems of linear equations with tridiagonal matrix (5.​3):

(20.111)

(20.112)

(20.113)

(20.114)

Periodic boundary conditions are implemented with the method described in Sect. 5.​4.

References

83.

P.C. Fife, Mathematical Aspects of Reacting and Diffusing Systems (Springer, Berlin, 1979) MATHCrossRef

112.

P. Grindrod, Patterns and Waves: The Theory and Applications of Reaction-Diffusion Equations (Clarendon Press, Oxford, 1991) MATH

123.

D. Hinrichsen, A.J. Pritchard, Mathematical Systems Theory I—Modelling, State Space Analysis, Stability and Robustness (Springer, Berlin, 2005). ISBN 0-978-3-540-441250 MATH

124.

C.S. Holling, Can. Entomol. 91, 293 (1959) CrossRef

125.

C.S. Holling, Can. Entomol. 91, 385 (1959) CrossRef

135.

V.I. Istratescu, Fixed Point Theory: An Introduction (Reidel, Dordrecht 1981). ISBN 90-277-1224-7 MATHCrossRef

145.

K. Khalil Hassan, Nonlinear Systems (Prentice Hall, Englewood Cliffs, 2001). ISBN 0-13-067389-7

160.

A.J. Lotka, Elements of Physical Biology (Williams and Wilkins, Baltimore, 1925) MATH

161.

A.M. Lyapunov, Stability of Motion (Academic Press, New York, 1966) MATH

178.

J.D. Murray, Mathematical Biology: I. An Introduction, vol. 2, 3rd edn. (Springer, Berlin, 2002). ISBN 0-387-95223-3

214.

E. Renshaw, Modelling Biological Populations in Space and Time (Cambridge University Press, Cambridge, 1991). ISBN 0-521-44855-7 MATHCrossRef

245.

S.H. Strogatz, Nonlinear Dynamics and Chaos: Applications to Physics, Biology, Chemistry, and Engineering (Perseus Books, Reading, 2001). ISBN 0-7382-0453-6

249.

J.T. Tanner, Ecology 56, 855 (1975) CrossRef

258.

A.M. Turing, Philos. Trans. R. Soc. Lond. B 237, 37 (1952) ADSCrossRef

264.

P.F. Verhulst, Mem. Acad. R. Sci. Belles Lettres Bruxelles 18, 1–42 (1845)

266.

V. Volterra, Mem. R. Accad. Naz. Lincei 2, 31 (1926)

Footnotes

1

Linear functions are additive f(x+y)=f(x)+f(y) and homogeneous f(αx)=αf(x).

2

Here and in the following f (n) denotes an iterated function, not a derivative.

3

We consider only the case, that different species diffuse independently and that the diffusion constants do not depend on direction.

4

We assume tacitly that such a solution exists.

5

Strictly this is true only for an infinite or periodic system.

Philipp O.J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3_21© Springer International Publishing Switzerland 2013

21. Simple Quantum Systems

Philipp O. J. Scherer1 

(1)

Physikdepartment T38, Technische Universität München, Garching, Germany

Abstract

We discuss several approaches to solve the one-dimensional Schrödinger equation numerically. The dispersion of simple finite differences deviates largely from the exact relation unless high order differences are used. More accurate pseudo-spectral methods evaluate the kinetic energy part in Fourier space. The time evolution can be approximated by unitary rational expressions like Cauchy’s form. Multistep differencing schemes have comparable accuracy but are explicit. The split operator approximation of the time evolution operator leads to the real-space product formula. In a computer experiment we simulate a one-dimensional wave packet.

Subsequently we study a two-state system in an oscillating field, a three-state system as a model for superexchange, the Landau-Zener model for curve-crossing and the ladder model for exponential decay. The density matrix formalism is used to describe a dissipative two-state system in analogy to Bloch’s equations for nuclear magnetic resonance. Computer experiments simulate resonance behavior, saturation and power broadening. The generation of a coherent superposition state or a spin flip are simulated and discussed in connection with the manipulation of a qubit.

In this chapter we study simple quantum systems. A particle in a one-dimensional potential V(x) is described by a wave packet which is a solution of the partial differential equation [232]

(21.1)

We discuss two approaches to discretize the second derivative. Finite differences are simple to use but their dispersion deviates largely from the exact relation, except high order differences are used. Pseudo-spectral methods evaluate the kinetic energy part in Fourier space and are much more accurate. The time evolution operator can be approximated by rational expressions like Cauchy’s form which corresponds to the Crank-Nicholson method. These schemes are unitary but involve time consuming matrix inversions. Multistep differencing schemes have comparable accuracy but are explicit methods. Best known is second order differencing. Split operator methods approximate the time evolution operator by a product. In combination with finite differences for the kinetic energy this leads to the method of real-space product formula which can applied to wavefunctions with more than one component, for instance to study transitions between states. In a computer experiment we simulate a one-dimensional wave packet in a potential with one or two minima.

Few-state systems are described with a small set of basis states. Especially the quantum mechanical two-level system is often used as a simple model for the transition between an initial and a final state due to an external perturbation.¹Its wavefunction has two components

(21.2)

which satisfy two coupled ordinary differential equations for the amplitudes C 1,2 of the two states

(21.3)

In several computer experiments we study a two-state system in an oscillating field, a three-state system as a model for superexchange, the Landau-Zener model for curve-crossing and the ladder model for exponential decay. The density matrix formalism is used to describe a dissipative two-state system in analogy to the Bloch equations for nuclear magnetic resonance. In computer experiments we study the resonance line and the effects of saturation and power broadening. Finally we simulate the generation of a coherent superposition state or a spin flip by applying pulses of suitable duration. This is also discussed in connection with the manipulation of a qubit represented by a single spin.

21.1 Pure and Mixed Quantum States

Whereas pure states of a quantum system are described by a wavefunction, mixed states are described by a density matrix. Mixed states appear if the exact quantum state is unknown, for instance for a statistical ensemble of quantum states, a system with uncertain preparation history, or if the system is entangled with another system. A mixed state is different from a superposition state. For instance, the superposition

(21.4)

of the two states |ψ ₀〉 and |ψ ₁〉 is a pure state, which can be described by the density operator

(21.5)

whereas the density operator

(21.6)

describes the mixed state of a system which is in the pure state |ψ ₀〉 with probability p ₀ and in the state |ψ ₁〉 with probability p ₁=1−p ₀. The expectation value of an operator A is in the first case

(21.7)

and in the second case

(21.8)

Both can be written in the form

(21.9)

21.1.1 Wavefunctions

The time evolution of a quantum system is governed by the time dependent Schrödinger equation [230]

(21.10)

for the wavefunction ψ. The brackets indicate that |ψ〉 is a vector in an abstract Hilbert space [122]. Vectors can be added

(21.11)

and can be multiplied with a complex number

(21.12)

Finally a complex valued scalar product of two vectors is defined²

(21.13)

which has the properties

(21.14)

(21.15)

(21.16)

(21.17)

21.1.2 Density Matrix for an Ensemble of Systems

Consider a thermal ensemble of systems. Their wave functions are expanded with respect to basis functions |ψ s 〉 as

(21.18)

The ensemble average of an operator A is given by

(21.19)

(21.20)

with the density matrix

(21.21)

The wave function of an N-state system is a linear combination

(21.22)

The diagonal elements of the density matrix are the occupation probabilities

(21.23)

and the nondiagonal elements measure the correlation of two states³

(21.24)

21.1.3 Time Evolution of the Density Matrix

The expansion coefficients of

(21.25)

can be obtained from the scalar product

(21.26)

Hence we have

(21.27)

which can be considered to be the s′, s matrix element of the projection operator |ψ〉〈ψ|

(21.28)

The thermal average of |ψ〉〈ψ| is the statistical operator

(21.29)

which is represented by the density matrix with respect to the basis functions |ψ s 〉

(21.30)

From the Schrödinger equation

(21.31)

we find

(21.32)

and hence

(21.33)

Since the Hamiltonian H is identical for all members of the ensemble we end up with the Liouville-von Neumann equation

(21.34)

With respect to a finite basis this becomes explicitly:

(21.35)

(21.36)

21.2 Wave Packet Motion in One Dimension

A quantum mechanical particle with mass m p in a one-dimensional potential V(x) is described by a complex valued wavefunction ψ(x) (Fig. 21.1). We assume that the wavefunction is negligible outside an interval [a,b]. This is the case for a particle bound in a potential well i.e. a deep enough minimum of the potential or for a wave-packet with finite width far from the boundaries. Then the calculation can be restricted to the finite interval [a,b] by applying the boundary condition

(21.37)

or, if reflections at the boundary should be suppressed, transparent boundary conditions [9].

Fig. 21.1

Potential well

All observables (quantities which can be measured) of the particle are expectation values with respect to the wavefunction, for instance its average position is

(21.38)

The probability of finding the particle at the position x ₀ is given by

(21.39)

For time independent potential V(x) the Schrödinger equation

(21.40)

can be formally solved by

(21.41)

If the potential is time dependent, the more general formal solution is

(21.42)

where $\hat{T}_{t}$ denotes the time ordering operator. The simplest approach is to divide the time interval 0…t into a sequence of smaller steps

(21.43)

and to neglect the variation of the Hamiltonian during the small interval Δt=t n+1−t n [158]

(21.44)

21.2.1 Discretization of the Kinetic Energy

The kinetic energy

(21.45)

is a nonlocal operator in real space. It is most efficiently evaluated in Fourier space where it becomes diagonal

(21.46)

21.2.1.1 Pseudo-spectral Methods

The potential energy is diagonal in real space. Therefore, pseudo-spectral (Sect. 11.​5.​1) methods [93] use a Fast Fourier Transform algorithm (Sect. 7.​3.​2) to switch between real space and Fourier space. They calculate the action of the Hamiltonian on the wavefunction according to

(21.47)

21.2.1.2 Finite Difference Methods

In real space, the kinetic energy operator can be approximated by finite differences on a grid, like the simple 3-point expression (3.​31)

(21.48)

or higher order expressions (3.​33)

(21.49)

(21.50)

etc. [94]. However, finite differences inherently lead to deviations of the dispersion relation from (21.46). Inserting ψ m =eikmΔx we find

(21.51)

for the 3-point expression (21.48),

(21.52)

for the 5-point expression (21.49) and

(21.53)

for the 7-point expression (21.50). Even the 7-point expression shows large deviations for k-values approaching k max=π/Δx (Fig. 21.2). However, it has been shown that not very high orders are necessary to achieve the numerical accuracy of the pseudo-spectral Fourier method [113] and that finite difference methods may be even more efficient in certain applications [114].

Fig. 21.2

(Dispersion of finite difference expressions)  The dispersion relation of finite difference expressions of increasing order ((21.48), (21.49), (21.50) and the symmetric 9-point approximation [94]) are compared to the exact dispersion (21.46) of a free particle (dashed curve)

21.2.2 Time Evolution

A number of methods have been proposed [7, 51, 158, 278] to approximate the short time propagator (21.44). Unitarity is a desirable property since it guaranties stability and norm conservation even for large time steps. However, depending on the application, small deviations from unitarity may be acceptable in return for higher efficiency. The Crank-Nicolson (CN) method [104, 172, 173] is one of the first methods which have been applied to the time dependent Schrödinger equation. It is a unitary but implicit method and needs the inversion of a matrix which can become cumbersome in two or more dimensions or if high precision is required. Multistep methods [131, 132], especially second order [10] differencing (SOD) are explicit but only conditionally stable and put limits to the time interval Δt. Split operator methods (SPO) approximate the propagator by a unitary product of operators [14, 67, 68]. They are explicit and easy to implement. The real-space split-operator method has been applied to more complex problems like a molecule in a laser field [61]. Polynomial approximations, especially the Chebishev expansion [53, 248], have very high accuracy and allow for large time steps, if the Hamiltonian is time independent. However, they do not provide intermediate results and need many applications of the Hamiltonian. The short time iterative Lanczos (SIL) method [57, 153, 197] is very useful also for time dependent Hamiltonians. Even more sophisticated methods using finite elements and the discrete variable representation are presented for instance in [4, 226]. In the following we discuss three methods (CN, SOD, SPO) which are easy to implement and well suited to solve the time dependent Schrödinger equation for a mass point moving in a one-dimensional potential.

21.2.2.1 Rational Approximation

Taking the first terms of the Taylor expansion

(21.54)

corresponds to a simple explicit Euler step

(21.55)

From the real eigenvalues E of the Hamiltonian we find the eigenvalues of the explicit method

(21.56)

which all have absolute values

(21.57)

Hence the explicit method is not stable.

Expansion of the inverse time evolution operator

leads to the implicit method

(21.58)

which can be rearranged as

(21.59)

Now all eigenvalues have absolute values <1. This method is stable but the norm of the wave function is not conserved. Combination of implicit and explicit method gives a method [172, 173] similar to the Crank-Nicolson method for the diffusion equation (Sect. 19.​2.​3)

(21.60)

This equation can be solved for the new value of the wavefunction

(21.61)

which corresponds to a rational approximation⁴ of the time evolution operator (Cayley’s form)

(21.62)

The eigenvalues of (21.62) all have an absolute value of

(21.63)

It is obviously a unitary operator and conserves the norm of the wavefunction since

(21.64)

as H is Hermitian H †=H and $(1+\mathrm {i}\frac{\Delta t}{2\hbar}H)$ and $(1-\mathrm {i}\frac{\Delta t}{2\hbar}H)$ are commuting operators. From the Taylor series we find the error order

(21.65)

For practical application we rewrite [149]

(21.66)

hence

(21.67)

ψ(t n+1) is obtained in two steps. First we have to solve

(21.68)

Then ψ(t n+1) is given by

(21.69)

We use the finite difference method (Sect. 11.​2) on the grid

(21.70)

and approximate the second derivative by

(21.71)

Equation (21.68) then becomes a system of linear equations

(21.72)

with a tridiagonal matrix

(21.73)

The second step (21.69) becomes

(21.74)

Inserting a plane wave

(21.75)

we obtain the dispersion relation (Fig. 21.3)

(21.76)

which we rewrite as

(21.77)

with the dimensionless parameter

(21.78)

Fig. 21.3

(Dispersion of the Crank-Nicolson method)  The dispersion relation of the Crank-Nicolson method (21.95) deviates largely from the exact dispersion (21.98), even for small values of the stability parameter α. The scaled frequency ωΔt/α is shown as a function of kΔx/π for α=0.1,1,2,5,10 (solid curves) and compared with the exact relation of a free particle ωΔt/α=(kΔx/π)² (dashed curve)

For time independent potentials the accuracy of this method can be improved systematically [261] by using higher order finite differences for the spatial derivative (Sect. 21.2.1.2) and a higher order Padé approximation (Sect. 2.​4.​1) of order [M,M] for the exponential function

(21.79)

to approximate the time evolution operator

(21.80)

However, the matrix inversion can become very time consuming in two or more dimensions.

21.2.2.2 Second Order Differencing

Explicit methods avoid the matrix inversion. The method of second order differencing [10] takes the difference of forward and backward step

(21.81)

(21.82)

to obtain the explicit two-step algorithm

(21.83)

The first terms of the Taylor series give the approximation

(21.84)

which can also be obtained from the second order approximation of the time derivative [150]

(21.85)

This two-step algorithm can be formulated as a discrete mapping

(21.86)

with eigenvalues

(21.87)

For sufficiently small time step [158]

(21.88)

the square root is real,

(21.89)

and the method is conditionally stable and has the same error order as the Crank-Nicolson method (Sect. 21.2.2.1). Its big advantage is that it is an explicit method and does not involve matrix inversions. Generalization to higher order multistep differencing schemes is straightforward [131]. The method conserves [150] the quantities ℜ〈ψ(t+Δt)|ψ(t)〉 and ℜ〈ψ(t+Δt)|H|ψ(t)〉 but is not strictly unitary [10]. Consider a pair of wavefunctions at times t ₀ and t ₁ which obey the exact time evolution

(21.90)

and apply (21.84) to obtain

(21.91)

which can be written as

(21.92)

where the time evolution operator L obeys

Expanding the sine function we find the deviation from unitarity [10]

(21.93)

which is of higher order than the error of the algorithm. Furthermore errors do not accumulate due to the stability of the algorithm (21.89). This also holds for deviations of the starting values from the condition (21.90).

The algorithm (21.84) can be combined with the finite differences method (Sect. 21.2.1.2)

(21.94)

or with the pseudo-spectral Fourier method [150]. This combination needs two Fourier transformations for each step but it avoids the distortion of the dispersion relation inherent to the finite difference method. Inserting the plane wave (21.75) into (21.84) we find the dispersion relation (Fig. 21.4) for a free particle (V=0):

(21.95)

For a maximum k-value

(21.96)

the stability condition (21.88) becomes

(21.97)

For small k the dispersion approximates the exact behavior

(21.98)

The finite difference method (21.94), on the other hand, has the dispersion relation (Fig. 21.5)

(21.99)

and the stability limit

(21.100)

The deviation from (21.98) is significant for kΔx/π>0.2 even for small values of α [150].

Fig. 21.4

(Dispersion of the Fourier method)  The dispersion relation of the SOD-Fourier method (21.95) deviates from the exact dispersion (21.98) only for very high k-values and approaches it for small values of the stability parameter α. The scaled frequency ωΔt/α is shown as a function of kΔx/π for α=0.5,0.75,1 (solid curves) and compared with the exact relation of a free particle ωΔt/α=(kΔx/π)² (dashed curve)

Fig. 21.5

(Dispersion of the finite difference method)  The dispersion relation of the SOD-FD method (21.99) deviates largely from the exact dispersion (21.98), even for small values of the stability parameter α. The scaled frequency ωΔt/α is shown as a function of kΔx/π for α=π ²/4≈2.467, 1.85, 1.23, 0.2 (solid curves) and compared with the exact relation of a free particle ωΔt/α=(kΔx/π)² (dashed curve)

21.2.2.3 Split-Operator Methods

The split-operator method approximates the exponential short time evolution operator as a product of exponential operators which are easier tractable. Starting from the Zassenhaus formula [164]

(21.101)

(21.102)

approximants of increasing order can be systematically constructed [68, 246]

(21.103)

Since these approximants do not conserve time reversibility, often the symmetric expressions

(21.104)

are preferred.

Split-Operator-Fourier Method

Dividing the Hamiltonian into its kinetic and potential parts

(21.105)

the time evolution operator can be approximated by the time-symmetric expression

(21.106)

where the exponential of the kinetic energy can be easily applied in Fourier space [80, 150]. Combining several steps (21.106) to integrate over a longer time interval, consecutive operators can be combined to simplify the algorithm

(21.107)

Real-Space Product Formulae

Using the discretization (21.48) on a regular grid the time evolution operator becomes the exponential of a matrix

(21.108)

with the abbreviations

(21.109)

The matrix can be decomposed into the sum of two overlapping tridiagonal block matrices [61, 67]⁵

(21.110)

(21.111)

The block structure simplifies the calculation of $e^{-\mathrm {i}\Delta tH_{o}}$ and $e^{-\mathrm {i}\Delta tH_{e}}$ tremendously since effectively only the exponential functions of 2×2 matrices

(21.112)

have to be calculated and the approximation to the time evolution operator

(21.113)

can be applied in real space without any Fourier transformation. To evaluate (21.112) the real symmetric matrix A m is diagonalized by an orthogonal transformation (Sect. 9.​2)

(21.114)

and the exponential calculated from

(21.115)

21.2.3 Example: Free Wave Packet Motion

We simulate the free motion (V=0) of a Gaussian wave packet along the x-axis (see Problem 21.1). To simplify the numerical calculation we set ħ=1 and m p =1 and solve the time dependent Schrödinger equation

(21.116)

for initial values given by a Gaussian wave packet with constant momentum

(21.117)

The exact solution can be easily found. Fourier transformation of (21.117) gives

(21.118)

Time evolution in k-space is rather simple

(21.119)

hence

(21.120)

and Fourier back transformation gives the solution of the time dependent Schrödinger equation in real space

(21.121)

Finally, the probability density is given by a Gaussian

(21.122)

which moves with constant velocity k ₀ and kinetic energy

(21.123)

Numerical results are shown in Figs. 21.6, 21.7 and Table 21.1.

Fig. 21.6

(Conservation of norm and energy)  The free motion of a Gaussian wave packet is simulated with the Crank-Nicolson method (CN) the second order differences method (SOD) with 3 point (21.48) 5 point (21.49) and 7-point (21.50) differences and with the real-space split-operator method (SPO). Δt=10⁻³, Δx=0.1, a=1, k ₀=3.77

Fig. 21.7

(Free wave-packet motion)  The free motion of a Gaussian wave packet is simulated. The probability density is shown for the initial Gaussian wave packet and at later times t=1,2,3,4. Results from the second order differences method with 3 point differences ((21.48), dash-dotted) and 5 point differences ((21.49), dashed) are compared with the exact solution ((21.122), thin solid line). Δt=10⁻³, Δx=0.1, a=1, k ₀=3.77

Table 21.1

(Accuracy of finite differences methods)  The relative error of the kinetic energy (21.123) is shown as calculated with different finite difference methods

Method

E kin

Crank-Nicolson (CN) with 3 point differences

7.48608

−1.6×10⁻²

Second Order Differences with 3 point differences (SOD3)

7.48646

−1.6×10⁻²

Second Order Differences with 5 point differences (SOD5)

7.60296

−4.6×10⁻⁴

Second Order Differences with 7 point differences (SOD7)

7.60638

−0.9×10⁻⁵

Split-Operator method (SOP) with 3 point differences

7.48610

−1.6×10⁻²

exact

7.60645

21.3 Few-State Systems

In the following we discuss simple models which reduce the wavefunction to the superposition of a few important states, for instance an initial and a final state which are coupled by a resonant interaction. We approximate the solution of the time dependent Schrödinger equation as a linear combination

(21.124)

of certain basis states |ϕ ₁〉⋯|ϕ M 〉⁶ which are assumed to satisfy the necessary boundary conditions and to be orthonormalized

(21.125)

Applying the method of weighted residuals (Sect. 11.​4) we minimize the residual

(21.126)

by choosing the basis functions as weight functions (Sect. 11.​4.​4) and solving the system of ordinary differential equations

(21.127)

which can be written

(21.128)

with the matrix elements of the Hamiltonian

(21.129)

In matrix form (21.128) reads

(21.130)

or more symbolically

(21.131)

If the Hamilton operator does not depend explicitly on time (H=const.) the formal solution of (21.131) is given by

(21.132)

From the solution of the eigenvalue problem

(21.133)

(eigenvalues λ and corresponding eigenvectors C λ ) we build the linear combination

(21.134)

The amplitudes a λ can be calculated from the set of linear equations

(21.135)

In the following we use the 4th order Runge-Kutta method to solve (21.131) numerically whereas the explicit solution (21.132) will be used to obtain approximate analytical results for special limiting cases.

A time dependent Hamiltonian H(t) appears in semiclassical models which treat some of the slow degrees of freedom as classical quantities, for instance an electron in the Coulomb field of (slowly) moving nuclei

(21.136)

or in a time dependent electromagnetic field

(21.137)

21.3.1 Two-State System

The two-state system (also known as two-level system or TLS; Fig. 21.8) is the simplest model of interacting states and is very often used in physics, for instance in the context of quantum optics, quantum information, spintronics and quantum dots.

Fig. 21.8

Two-state system model

Its interaction matrix is

(21.138)

and the equations of motion are

(21.139)

Equations (21.139) can be solved analytically but this involves some lengthy expressions. Let us therefore concentrate on two limiting cases:

(a)

For E ₁=E ₂ we add and subtract (21.139) to find

(21.140)

with the solution

(21.141)

For initial conditions

(21.142)

the explicit solution is given by

(21.143)

(21.144)

and the two-state system oscillates between the two states with the period

(21.145)

(b)

For V≪ΔE=E ₂−E ₁ ⁷ perturbation theory for the small quantity V/ΔE gives the following approximations:

(21.146)

(21.147)

For initial values the amplitudes a 1,2 are calculated from

(21.148)

which gives in lowest order

(21.149)

The approximate solution is

(21.150)

and the occupation probability of the initial state is

(21.151)

Numerical results are shown in Fig. 21.9.

Fig. 21.9

(Numerical simulation of a two-state system)  The equations of motion of the two-state system (21.139) are integrated with the 4th order Runge-Kutta method. For two resonant states the occupation probability of the initial state shows oscillations with the period (21.145) proportional to V ⁻¹. With increasing energy gap E ₂−E ₁ the amplitude of the oscillations decreases

21.3.2 Two-State System with Time Dependent Perturbation

Consider now a 2-state system with an oscillating perturbation (for instance an atom or molecule in a laser field; Fig. 21.10)

(21.152)

The equations of motion are

(21.153)

After the substitutions

(21.154)

(21.155)

they become

(21.156)

At larger times the system oscillates between the two states.⁸ Applying the rotating wave approximation for ω≈ω ₂₁ we neglect the fast oscillating perturbation

(21.157)

(21.158)

and substitute

(21.159)

to have

(21.160)

(21.161)

or

(21.162)

(21.163)

which shows that the system behaves approximately like a two-state system with a constant interaction V ₀/2 and an energy gap ħ(ω ₂₁−ω)=E ₂−E ₁−ħω (a comparison with a full numerical calculation is shown in Fig. 21.11).

Fig. 21.10

Two-state system in an oscillating field

Fig. 21.11

(Simulation of a two-state system in an oscillating field)  The equations of motion (21.153) are integrated with the 4th order Runge-Kutta method. At resonance the system oscillates between the two states with the frequency V/ħ. The dashed curves show the corresponding solution of a two-state system with constant coupling (Sect. 21.3.1)

21.3.3 Superexchange Model

The concept of superexchange was originally formulated for magnetic interactions [8] and later introduced to electron transfer theory [117]. It describes an indirect interaction through high energy intermediates (Fig. 21.12). In the simplest case, we have to consider two isoenergetic states i and f which do not interact directly but via coupling to an intermediate state v.

Fig. 21.12

Superexchange model

The interaction matrix is

(21.164)

For simplification we choose V ₁=V ₂.

Let us first consider the special case of a resonant intermediate state E ₂=0:

(21.165)

Obviously one eigenvalue is λ ₁=0 and the corresponding eigenvector is

(21.166)

The two remaining eigenvalues are solutions of

(21.167)

which gives

(21.168)

The eigenvectors are

(21.169)

From the initial values

(21.170)

the amplitudes are calculated as

(21.171)

and finally the solution is

(21.172)

Let us now consider the case of a distant intermediate state V≪|E ₂|. λ ₁=0 and the corresponding eigenvector still provide one solution. The two other eigenvalues are approximately given by

(21.173)

(21.174)

and the eigenvectors by

(21.175)

From the initial values

(21.176)

we calculate the amplitudes

(21.177)

and finally the solution

(21.178)

The occupation probability of the initial state is

(21.179)

which shows that the system behaves like a 2-state system with an effective interaction of

(21.180)

Numerical results are shown in Fig. 21.13.

Fig. 21.13

(Numerical simulation of the superexchange model)  The equations of motion for the model (21.164) are solved numerically with the 4th order Runge-Kutta method. The energy gap is varied to study the transition from the simple oscillation with $\omega=\sqrt{2}V/\hbar$ (21.172) to the effective two-state system with ω=V eff /ħ (21.179). Parameters are V ₁=V ₂=1, E ₁=E ₃=0, E ₂=0,1,5,20. The occupation probability of the initial (solid curves), virtual intermediate (dashed curves) and final (dash-dotted curves) state are shown

21.3.4 Ladder Model for Exponential Decay

For time independent Hamiltonian the solution (21.132) of the Schrödinger equation is a sum of oscillating terms and the quantum recurrence theorem [29] states that the system returns to the initial state arbitrarily closely after a certain time T r . However, if the initial state is coupled to a larger number of final states, the recurrence time can become very long and an exponential decay observed over a large period. The ladder model [27, 242] considers an initial state |0〉 interacting with a manifold of states |1〉⋯|n〉, which do not interact with each other and are equally spaced (Fig. 21.14)

(21.181)

The equations of motion are

(21.182)

For the special case ΔE=0 we simply have

(21.183)

with an oscillating solution

(21.184)

Here the n states act like one state with an effective coupling of $V\sqrt{n}$.

Fig. 21.14

Ladder model

For the general case ΔE≠0 we substitute

(21.185)

and have

(21.186)

Integration gives

(21.187)

and therefore

(21.188)

With the definition

(21.189)

we have

(21.190)

We replace the sum by an integral over the continuous variable

(21.191)

and extend the integration range to −∞⋯∞. Then the sum becomes approximately a delta function

(21.192)

and the final result is an exponential decay law

(21.193)

with the density of final states

(21.194)

Numerical results are shown in Fig. 21.15.

Fig. 21.15

(Numerical solution of the ladder model)  The time evolution of the ladder model (21.182) is calculated with the 4th order Runge-Kutta method for N=50 states and different values of the coupling V

21.3.5 Landau-Zener Model

This model describes crossing of two states (Fig. 21.16), for instance for colliding atoms or molecules [154, 282]. It is assumed that the interaction V is constant near the crossing point and that the nuclei move classically with constant velocity

(21.195)

For small interaction V or large velocity $\frac{\partial }{\partial t}\Delta E=\dot{Q}\frac{\partial}{\partial Q}\Delta E$ the transition probability can be calculated with perturbation theory to give

(21.196)

This expression becomes invalid for small velocities. Here the system stays on the adiabatic potential surface, i.e. P→1. Landau and Zener found the following expression which is valid in both limits:

(21.197)

In case of collisions multiple crossing of the interaction region has to be taken into account (Fig. 21.17). Numerical results are shown in Fig. 21.18.

Fig. 21.16

Slow atomic collision

Fig. 21.17

Multiple passage of the interaction region

Fig. 21.18

(Numerical solution of the Landau-Zener model)  Numerical calculations (solid curves) are compared with the Landau-Zener probability ((21.197), dashed lines) and the approximation ((21.196), dotted lines). The velocity is dΔE/dt=1

21.4 The Dissipative Two-State System

A two-state quantum system coupled to a thermal bath serves as a model for magnetic resonance phenomena, coherent optical excitations [263, 280] and, quite recently, for a qubit, the basic element of a future quantum computer [73, 183]. Its quantum state can not be described by a single wavefunction. Instead mixed quantum states have to be considered which can be conveniently described within the density matrix formalism [232].

21.4.1 Equations of Motion for a Two-State System

The equations of motion for a two-state system are

(21.198)

(21.199)

(21.200)

(21.201)

which can be arranged as a system of linear equations⁹

(21.202)

21.4.2 The Vector Model

The density matrix is Hermitian

(21.203)

its diagonal elements are real valued and due to conservation of probability

(21.204)

Therefore the four elements of the density matrix can be specified by three real parameters, which are usually chosen as

(21.205)

(21.206)

(21.207)

and satisfy the equations

(21.208)

(21.209)

(21.210)

Together they form the Bloch vector

(21.211)

which is often used to visualize the time evolution of a two-state system [81]. In terms of the Bloch vector the density matrix is given by

(21.212)

with the Pauli matrices

(21.213)

From (21.208), (21.209), (21.210) we obtain the equation of motion

(21.214)

which can be written as a cross product

(21.215)

with

(21.216)

Any normalized pure quantum state of the two-state system can be written as [75]

(21.217)

corresponding to the density matrix

(21.218)

The Bloch vector

(21.219)

represents a point on the unit sphere (the Bloch sphere, Fig. 21.19). Mixed states correspond to the interior of the Bloch sphere with the fully mixed state represented by the center of the sphere (Fig. 21.19).

Fig. 21.19

(Bloch sphere)  Left: Any pure quantum state of a two-state system can be represented by a point on the Bloch sphere. Right: The poles represent the basis states. Mixed quantum states correspond to the interior of the sphere, the center represents the fully mixed state

21.4.3 The Spin-$\frac{1}{2}$ System

An important example of a two-state system is a particle with spin $\frac{1}{2}$. Its quantum state can be described by a two-component vector

(21.220)

where the two unit vectors are eigenvectors of the spin component in z-direction corresponding to the eigenvalues $s_{z}=\pm\frac{\hbar}{2}$. The components of the spin operator are given by the Pauli matrices

(21.221)

and have expectation values

(21.222)

The ensemble average for a system of spin-$\frac{1}{2}$ particles is given by the Bloch vector

(21.223)

The Hamiltonian of a spin-$\frac{1}{2}$ particle in a magnetic field B is

(21.224)

from which the following relations are obtained

(21.225)

(21.226)

(21.227)

(21.228)

The average magnetization

(21.229)

obeys the equation of motion

(21.230)

21.4.4 Relaxation Processes—The Bloch Equations

Relaxation of the nuclear magnetization due to interaction with the environment was first described phenomenologically by Bloch in 1946 [28]. A more rigorous description was given later [201, 269] and also applied to optical transitions [179]. Recently electron spin relaxation has attracted much interest in the new field of spintronics [284] and the dissipative two-state system has been used to describe the decoherence of a qubit [46].

21.4.4.1 Phenomenological Description

In thermal equilibrium the density matrix is given by a canonical distribution

(21.231)

which for a two-state system without perturbation

(21.232)

becomes

(21.233)

where, as usually β=1/k B T. If the energy gap is very large Δ≫k B T like for an optical excitation, the equilibrium state is the state with lower energy¹⁰

(21.234)

The phenomenological model assumes that deviations of the occupation difference from its equilibrium value

(21.235)

decay exponentially with a time constant T ₁ (for NMR this is the spin-lattice relaxation time)

(21.236)

The coherence of the two states decays exponentially with a time constant T ₂ which is closely related to T ₁ in certain cases¹¹ but can be much smaller than T ₁ if there are additional dephasing mechanisms. The equation

(21.237)

describes the decay of the transversal polarization due to spatial and temporal differences of different spins (spin-spin relaxation), whereas for an optical excitation or a qubit it describes the loss of coherence of a single two-state system due to interaction with its environment.

The combination of (21.230) and the relaxation terms (21.236), (21.237) gives the Bloch equations [28] which were originally formulated to describe the time evolution of the macroscopic polarization

(21.238)

For the components of the Bloch vector they read explicitly

(21.239)

21.4.5 The Driven Two-State System

The Hamiltonian of a two-state system (for instance an atom or molecule) in an oscillating electric field $E\mathrm {e}^{-\mathrm {i}\omega_{f}t}$ with energy splitting Δ and transition dipole moment μ is

(21.240)

The corresponding magnetic field

(21.241)

(21.242)

(21.243)

is that of a typical NMR experiment with a constant component along the z-axis and a rotating component in the xy-plane.

21.4.5.1 Free Precession

Consider the special case B z =const., B x =B y =0. The corresponding Hamiltonian matrix is diagonal

(21.244)

with the Larmor-frequency

(21.245)

The equations of motion for the density matrix are

(21.246)

(21.247)

with the solution

(21.248)

(21.249)

The Bloch vector

(21.250)

is subject to damped precession around the z-axis with the Larmor frequency (Fig. 21.20).

Fig. 21.20

(Free precession)  The Bloch equations (21.239) are numerically solved with the 4th order Runge-Kutta method. After excitation with a short resonant pulse the free precession is observed. Left: The occupation difference z=ρ ₁₁−ρ ₂₂ decays exponentially to its equilibrium value. Right: In the xy-plane the Bloch vector moves on a spiral towards the equilibrium position (x=0, y=0)

21.4.5.2 Stationary Solution for Monochromatic Excitation

For the two-state system (21.240) with

(21.251)

(21.252)

the solution of the Bloch equations (21.238)

(21.253)

can be found explicitly [280]. We transform to a coordinate system which rotates around the z-axis ((13.​3) on page 243) with angular velocity ω f

(21.254)

Then

(21.255)

with

(21.256)

The matrix products are

(21.257)

and the equation of motion simplifies to

(21.258)

For times short compared to the relaxation times the solution is approximately given by harmonic oscillations. The generalized Rabi frequency Ω R follows from [96]

(21.259)

(21.260)

(21.261)

as

(21.262)

At larger times these oscillations are damped and the stationary solution is approached (Fig. 21.21) which is given by

(21.263)

The occupation difference

(21.264)

has the form of a Lorentzian. The line width increases for higher intensities (power broadening)

(21.265)

and the maximum

(21.266)

approaches zero (saturation), Figs. 21.22, 21.23.

Fig. 21.21

(Monochromatic Excitation)  The Bloch equations are solved numerically with the 4th order Runge-Kutta method for a monochromatic perturbation with ω=4, V ₀=0.5. Parameters of the two-state system are ω ₀=5, z eq =−1.0 and T ₁=T ₂=5.0. The occupation difference z=ρ ₁₁−ρ ₂₂ initially shows Rabi oscillations which disappear at larger times where the stationary value z=−0.51 is reached

Fig. 21.22

(Resonance line)  The equations of motion of the two-state system including relaxation terms are integrated with the 4th order Runge-Kutta method until a steady state is reached. Parameters are ω ₀=5, z eq =−0.8, V=0.01 and T ₁=T ₂=3.0, 6.9. The change of the occupation difference is shown as a function of frequency (circles) and compared with the steady state solution (21.263)

Fig. 21.23

(Power saturation and broadening)  The resonance line is investigated as a function of the coupling strength V and compared with the stationary solution (21.263) to observe the broadening of the line width (21.265). Parameters are ω ₀=5, z eq =−1.0, T ₁=T ₂=100 and V=0.5, 0.25, 0.125, 0.0625, 0.03125

21.4.5.3 Excitation by a Resonant Pulse

For a resonant pulse with real valued envelope V ₀(t) and initial phase angle Φ ₀

the equation of motion in the rotating system is

(21.267)

If the relaxation times are large compared to the pulse duration this describes approximately a rotation around an axis in the xy-plane (compare with (13.​24))

(21.268)

(21.269)

Since the axis is time independent, a formal solution is given by

(21.270)

with the phase angle

(21.271)

Now, since

(21.272)

(21.273)

(21.274)

the Taylor series of the exponential function in (21.270) can be summed up

(21.275)

The result is a rotation about the angle Φ around an axis in the xy-plane determined by Φ ₀ (Fig. 21.24), especially around the x-axis for Φ ₀=0 and around the y-axis for $\varPhi_{0}=\frac{\pi}{2}$.

Fig. 21.24

(Rotation of the Bloch vector by a resonant pulse)  A resonant pulse rotates the Bloch vector by the angle Φ around an axis in the x′y′-plane

After a π-pulse (Φ=π) the z-component changes its sign

(21.276)

The transition between the two basis states z=−1 and z=1 corresponds to a spin flip (Fig. 21.25). On the other hand, a π/2-pulse transforms the basis states into a coherent mixture

(21.277)

Fig. 21.25

(Spin flip by a π-pulse)  The equations of motion of the Bloch vector (21.253) are solved with the 4th order Runge-Kutta method for an interaction pulse with a Gaussian shape. The pulse is adjusted to obtain a spin flip. The influence of dephasing processes is studied. T ₁=1000, t p =1.8, V ₀=0.25. The occupation difference ρ ₁₁−ρ ₂₂=z (solid curves) and the coherence $|\rho_{12}|=\frac{1}{2}\sqrt{x^{2}+y^{2}}$ (broken curves) are shown for several values of the dephasing time T ₂=5, 10, 100, 1000

21.4.6 Elementary Qubit Manipulation

Whereas a classical bit can be only in one of two states

(21.278)

the state of a qubit is a quantum mechanical superposition

(21.279)

The time evolution of the qubit is described by a unitary transformation

(21.280)

which is represented by a complex 2×2 unitary matrix that has the general form (see also Sect. 13.​14)

(21.281)

The Bloch vector is transformed with an orthogonal matrix A, which can be found from (21.212) and the transformed density matrix UρU ⁻¹

(21.282)

Any single qubit transformation can be realized as a sequence of rotations around just two axes [126, 183, 263]. In the following we consider some simple transformations, so called quantum gates [274].

21.4.6.1 Pauli-Gates

Of special interest are the gates represented by the Pauli matrices U=σ i since any complex 2×2 matrix can be obtained as a linear combination of the Pauli matrices and the unit matrix (Sect. 13.​14). For all three of them detU=−1 and φ=π.

The X-gate

(21.283)

corresponds to rotation by π radians around the x-axis ((21.276) with Φ ₀=0)

(21.284)

It is also known as NOT-gate since it exchanges the two basis states. Similarly, the Y-gate

rotates the Bloch vector by π radians around the y-axis (21.276 with Φ ₀=π/2)

(21.285)

and the Z-gate

(21.286)

by π radians around the z-axis

(21.287)

This rotation can be replaced by two successive rotations in the xy-plane

(21.288)

The corresponding transformation of the wavefunction produces an overall phase shift of π/2 since the product of the Pauli matrices is σ x σ y =iσ z , which is not relevant for observable quantities.

21.4.6.2 Hadamard Gate

The Hadamard gate is a very important ingredient for quantum computation. It transforms the basis states into coherent superpositions and vice versa. It is described by the matrix

(21.289)

with detU H =−1 and

(21.290)

which can be obtained as the product

(21.291)

of a rotation by π radians around the x-axis and a second rotation by π/2 radians around the y-axis. The first rotation corresponds to the X-gate and the second to (21.277) with Φ ₀=π/2

(21.292)

21.5 Problems

Problem 21.1

(Wave packet motion)

In this computer experiment we solve the Schrödinger equation for a particle in the potential V(x) for an initially localized Gaussian wave packet ψ(t=0,x)∼exp(−a(x−x ₀)²). The potential is a box, a harmonic parabola or a fourth order double well. Initial width and position of the wave packet can be varied.

Try to generate the time independent ground state wave function for the harmonic oscillator.

Observe the dispersion of the wave packet for different conditions and try to generate a moving wave packet with little dispersion.

Try to observe tunneling in the double well potential.

Problem 21.2

(Two-state system)

In this computer experiment a two-state system is simulated. Amplitude and frequency of an external field can be varied as well as the energy gap between the two states (see Fig. 21.9).

Compare the time evolution at resonance and away from it.

Problem 21.3

(Three-state system)

In this computer experiment a three-state system is simulated.

Verify that the system behaves like an effective two-state system if the intermediate state is higher in energy than initial and final states (see Fig. 21.13).

Problem 21.4

(Ladder model)

In this computer experiment the ladder model is simulated. The coupling strength and the spacing of the final states can be varied.

Check the validity of the exponential decay approximation (see Fig. 21.15).

Problem 21.5

(Landau-Zener model)

This computer experiment simulates the Landau Zener model. The coupling strength and the nuclear velocity can be varied (see Fig. 21.18).

Try to find parameters for an efficient crossing of the states.

Problem 21.6

(Resonance line)

In this computer experiment a two-state system with damping is simulated. The resonance curve is calculated from the steady state occupation probabilities (see Figs. 21.22, 21.23).

Study the dependence of the line width on the intensity (power broadening).

Problem 21.7

(Spin flip)

The damped two-state system is now subject to an external pulsed field (see Figs. 21.25, 21.26, 21.27).

Try to produce a coherent superposition state (π/2 pulse) or a spin flip (π pulse).

Investigate the influence of decoherence.

Fig. 21.26

(Generation of a coherent mixture by a π/2-pulse)  The equations of motion of the Bloch vector (21.253) are solved with the 4th order Runge-Kutta method for an interaction pulse with a Gaussian shape. The pulse is adjusted to obtain a coherent mixture. The influence of dephasing processes is studied. T ₁=1000, t p =0.9, V ₀=0.25. The occupation difference ρ ₁₁−ρ ₂₂=z (solid curves) and the coherence $|\rho_{12}|=\frac{1}{2}\sqrt{x^{2}+y^{2}}$ (broken curves) are shown for several values of the dephasing time T ₂=5, 10, 100, 1000

Fig. 21.27

(Motion of the Bloch vector during $\frac{\pi}{2}$ and π pulses)  The trace of the Bloch vector is shown in the laboratory system. Left: $\frac{\pi}{2}$-pulse as in Fig. 21.26 with T ₂=1000. Right: π-pulse as in Fig. 21.25 with T ₂=1000

References

4.

I. Allonso-Mallo, N. Reguera, J. Comp. Physiol. 220, 409 (2006) ADS

7.

Z.A. Anastassi, T.E. Simos, Phys. Rep. 482–483, 1 (2009) MathSciNetCrossRef

8.

P.W. Anderson, Phys. Rev. 79, 350 (1950) ADSMATHCrossRef

9.

X. Antoine, A. Arnold, C. Besse, M. Erhardt, A. Schädle, Commun. Comput. Phys. 4, 729 (2008) MathSciNet

10.

A. Askar, A.S. Cakmak, J. Chem. Phys. 68, 2794 (1978) MathSciNetADSCrossRef

14.

A.D. Bandrauk, H. Shen, Chem. Phys. Lett. 176, 428 (1991) ADSCrossRef

27.

M. Bixon, J. Jortner, J. Chem. Phys. 48, 715 (1986) ADSCrossRef

28.

F. Bloch, Nuclear induction. Phys. Rev. 70, 460 (1946) ADSCrossRef

29.

P. Bocchieri, A. Loinger, Phys. Rev. 107, 337 (1957) MathSciNetADSMATHCrossRef

46.

G. Burkard, R.H. Koch, D.P. DiVincenzo, Phys. Rev. B 69, 64503 (2004) ADSCrossRef

51.

A. Castro, M.A.L. Marques, A. Rubio, J. Chem. Phys. 121, 3425 (2004) ADSCrossRef

53.

C. Cerjan, K.C. Kulander, Comput. Phys. Commun. 63, 529 (1991) ADSMATHCrossRef

57.

R. Chen, H. Guo, J. Chem. Phys. 111, 9944 (1999) ADSCrossRef

61.

L.A. Collins, J.D. Kress, R.B. Walker, Comput. Phys. Commun. 114, 15 (1998) ADSMATHCrossRef

67.

H. De Raedt, Comput. Phys. Rep. 7, 1 (1987) ADSCrossRef

68.

H. De Raedt, B. De Raedt, Phys. Rev. A 28, 3575 (1983) MathSciNetADSCrossRef

73.

L. Diosi, A Short Course in Quantum Information Theory (Springer, Berlin, 2007) MATH

75.

S.E. Economou, T.L. Reinecke, in Optical Generation and Control of Quantum Coherence in Semiconductor Nanostructures, ed. by G. Slavcheva, P. Roussignol (Springer, Berlin, 2010), p. 62

80.

M.D. Feit, J.A. Fleck Jr., A. Steiger, J. Comp. Physiol. 47, 412 (1982) MathSciNetADSMATH

81.

R.P. Feynman, F.L. Vernon, R.W. Hellwarth, J. Appl. Phys. 28, 49 (1957) ADSCrossRef

93.

B. Fornberg, Geophysics 52, 4 (1987) CrossRef

94.

B. Fornberg, Math. Comput. 51, 699 (1988) MathSciNetMATHCrossRef

96.

M. Fox, Quantum Optics: An Introduction (Oxford University Press, London, 2006)

104.

A. Goldberg, H.M. Schey, J.L. Schwartz, Am. J. Phys. 35, 177 (1967) ADSCrossRef

113.

R. Guantes, S.C. Farantos, J. Chem. Phys. 111, 10827 (1999) ADSCrossRef

114.

R. Guantes, A. Nezis, S.C. Farantos, J. Chem. Phys. 111, 10836 (1999) ADSCrossRef

117.

J. Halpern, L.E. Orgel, Discuss. Faraday Soc. 29, 32 (1960) CrossRef

122.

D. Hilbert, L. Nordheim, J. von Neumann, Math. Ann. 98, 1 (1927) MATHCrossRef

126.

F.-Y. Hong, S.-J. Xiong, Chin. J. Phys. 46, 379 (2008)

131.

T. Iitaka, Phys. Rev. E 49, 4684 (1994) ADSCrossRef

132.

T. Iitaka, N. Carjan, T. Strottman, Comput. Phys. Commun. 90, 251 (1995) ADSMATHCrossRef

149.

S.E. Koonin, C.M. Dawn, Computational Physics (Perseus Books, New York, 1990). ISBN 978-0201127799 MATH

150.

D. Kosloff, R. Kosloff, J. Comp. Physiol. 52, 35 (1983) ADSMATH

153.

C. Lanczos, J. Res. Natl. Bur. Stand. 45, 255 (1951) MathSciNetCrossRef

154.

L. Landau, Phys. Sov. Union 2, 46–51 (1932)

158.

C. Leforestier, R.H. Bisseling, C. Cerjan, M.D. Feit, R. Friesner, A. Guldberg, A. Hammerich, G. Jolicard, W. Karrlein, H.-D. Meyer, N. Lipkin, O. Roncero, R. Kosloff, J. Comp. Physiol. 94, 59 (1991) ADSMATH

164.

W. Magnus, Commun. Appl. Math. 7, 649 (1954) MathSciNetMATHCrossRef

172.

E.A. McCullough Jr., R.E. Wyatt, J. Chem. Phys. 51, 1253 (1969) ADSCrossRef

173.

E.A. McCullough Jr., R.E. Wyatt, J. Chem. Phys. 54, 3592 (1971) ADSCrossRef

179.

S.J. Nettel, A. Kempicki, Am. J. Phys. 47, 987 (1979) ADSCrossRef

183.

M. Nielsen, I. Chuang, Quantum Computation and Quantum Information (Cambridge University Press, Cambridge, 2000) MATH

197.

T.J. Park, J.C. Light, J. Chem. Phys. 85, 5870 (1986) ADSCrossRef

201.

D. Pines, C.P. Slichter, Phys. Rev. 100, 1014 (1955) ADSMATHCrossRef

226.

B.I. Schneider, J.-A. Collins, J. Non-Cryst. Solids 351, 1551 (2005) ADSCrossRef

230.

E. Schrödinger, Phys. Rev. 28, 1049 (1926) ADSMATHCrossRef

232.

F. Schwabl, Quantum Mechanics, 4th edn. (Springer, Berlin, 2007)

242.

B.I. Stepanov, V.P. Grobkovskii, Theory of Luminescence (Butterworth, London, 1986)

246.

M. Suzuki, Commun. Math. Phys. 51, 183 (1976) ADSMATHCrossRef

248.

H. Tal-Ezer, R. Kosloff, J. Chem. Phys. 81, 3967 (1984) ADSCrossRef

261.

W. van Dijk, F.M. Toyama, Phys. Rev. E 75, 036707 (2007) MathSciNetADSCrossRef

263.

L.M.K. Vandersypen, I.L. Chuang, Rev. Mod. Phys. 76, 1037 (2004) ADSCrossRef

269.

R.K. Wangsness, F. Bloch, Phys. Rev. 89, 728 (1953) ADSMATHCrossRef

274.

C.P. Williams, Explorations in Quantum Computing (Springer, Berlin, 2011) CrossRef

278.

K. Yabana, G.F. Bertsch, Phys. Rev. B 54, 4484 (1996) ADSCrossRef

280.

A. Yariv, Quantum Electronics (Wiley, New York, 1975)

282.

C. Zener, Proc. R. Soc. Lond. A 137(6), 696–702 (1932) ADS

284.

I. Zutic, J. Fabian, S. Das Sarma, Rev. Mod. Phys. 76, 323 (2004) ADSCrossRef

Footnotes

1

For instance collisions or the electromagnetic radiation field.

2

If, for instance the wavefunction depends on the coordinates of N particles, the scalar product is defined by $\langle \psi_{n}|\psi_{n'}\rangle = \int d^{3}r_{1}\cdots d^{3}r_{N}\psi_{n}^{*}(r_{1}\cdots r_{N})\psi _{n'}(r_{1}\cdots r_{N})$.

3

They are often called the “coherence” of the two states.

4

The Padé approximation (Sect. 2.​4.​1) of order [1,1].

5

For simplicity only the case of even M is considered.

6

This basis is usually incomplete.

7

We assume E ₂>E ₁.

8

So called Rabi oscillations.

9

The matrix of this system corresponds to the Liouville operator.

10

We assume Δ≥0, such that the equilibrium value of z=ρ ₁₁−ρ ₂₂ is negative. Eventually, the two states have to be exchanged.

11

For instance T ₂=2T ₁ for pure radiative damping.

Philipp O. J. SchererGraduate Texts in PhysicsComputational Physics2nd ed. 2013Simulation of Classical and Quantum Systems10.1007/978-3-319-00401-3© Springer International Publishing Switzerland 2013

Appendix I Performing the Computer Experiments

The computer experiments are realized as Java-applets which can be run in any browser that has the Java plug-in installed without installing anything else. They are written in a C-like fashion which improves the readability for readers who are not so familiar with object oriented programming. The source code can be studied most conveniently with the netbeans environment which is open source and allows quick generation of graphical user interfaces.

After downloading and unzipping the zipped file from extras.springer.com you have two options:

Run a program in your Browser

Open the file CP-examples.html in your browser. If the Java plug-in is installed properly you can start any one of the programs by simply clicking its number in the left hand frame.

Open a program with the netbeans environment

If you have the netbeans environment installed, you can import any of the programs as a separate project by opening the corresponding folder in the directory HTML/code/. You may have a look at the source code and compile and run it

Appendix II Methods and Algorithms

Purpose

Method

Comments

Pages

Interpolation

Lagrange polynomial

explicit form, easy to evaluate

17

Barycentric Lagrange polynomial

for evaluation at many points

17

Newton’s divided differences

new points added easily

18

Neville method

for evaluation at one point

20

Spline interpolation

smoother, less oscillatory

22

Rational interpolation

smoother, less oscillatory, often less coefficients necessary

25, 28

Padé approximation

often better than Taylor series

25

Barycentric rational interpolation

easy to evaluate

27

Rational interpolation without poles

alternative to splines, analytical

31

Multivariate interpolation

multidimensional

32

Trigonometric interpolation

periodic functions

116

Differentiation

One-sided difference quotient

low error order

37

Central difference quotient

higher error order

38

Extrapolation

high accuracy

221

Higher derivatives

finite difference methods

41

Partial derivatives

finite difference methods

42

Integration

Newton-Cotes formulae

equally spaced points

46

Trapezoidal rule

simple, closed interval

46

Midpoint rule

simple, open interval

48

Simpson’s rule

more accurate

46

Composite Newton-Cotes rules

for larger intervals

48

Extrapolation (Romberg)

high accuracy

49

Clenshaw-Curtis expressions

suitable for adaptive and multidimensional quadrature

50

Gaussian integration

high accuracy if polynomial approximation possible

52

Monte Carlo integration

high dimensional integrals

139

Linear equations

Gaussian elimination (LU reduction)

standard method for linear equations and matrix inversion

60

QR decomposition

numerically more stable

64

Iterative solution

large sparse systems

73

Jacobi relaxation

converges for diagonally dominant matrices, parallel computation possible

73

Gauss-Seidel relaxation

converges for symmetric positive definite or diagonal dominant matrices, no extra storage

74

Chessboard (black-red)

two independent subgrids, especially for Poisson equation

307

Damping and Successive over-relaxation

speeds up convergence for proper relaxation parameter

75

Multigrid

fast convergence but more complicated

307

Conjugate gradients

for symmetric positive definite matrices, preconditioning often necessary

76

Special LU decomposition

Tridiagonal linear equations

69

Sherman-Morrison formula

Cyclic tridiagonal systems

71

Root finding

Bisection

reliable but slow continuous functions

84

Regula falsi (false position)

speed and robustness between bisection and interpolation

85

Newton-Raphson

continuous derivative necessary, converges fast if starting point is close to a root

85

Interpolation (secant)

no derivative necessary, but slower than Newton

87

Inverse interpolation

mainly used by combined methods

88

Dekker’s combined method

Combination of bisection and secant method

91

Brent’s combined method

Combination of bisection, secant, and quadratic inverse interpolation methods, very popular

92

Chandrupatla’s combined method

Uses quadratic interpolation whenever possible, faster than Brent’s method, especially for higher order roots

95

Multidimensional root finding

Newton-Raphson

Needs full Hessian

97

Quasi-Newton (Broyden)

Hessian not needed, no matrix inversion

98

Function minimization

Ternary search

no gradient needed, very simple, for unimodal functions

99

Golden section search (Brent)

faster than ternary search but more complicated

101

Multidimensional minimization

Steepest descent

simple but slow

106

Conjugate gradients

faster than steepest descent

107

Newton-Raphson

fast, if starting point close to minimum, needs full Hessian

107

Quasi-Newton (BFGS,DFP)

Hessian not needed, very popular

108

Fourier transformation

Görtzel’s algorithm

efficient if only some Fourier components are needed

120

Fast Fourier transform

much faster than direct discrete Fourier transform

121

Random numbers

Linear congruent mapping

simple pseudo-random number generator

135

Marsaglia-Zamann

higher quality random numbers but more complicated

135

RN with given distribution

inverse of cumulative distribution function needed

136

Random points on unit sphere

random directions

137

Gaussian RN (Box-Muller)

Gaussian random numbers

138

Thermodynamic average

Simple sampling

inefficient

141

Importance sampling

samples preferentially important configurations

142

Metropolis algorithm

generates configurations according to a canonical distribution

142

Eigenvalue problems

Direct solution

only for very small dimension

148

Tridiagonal matrices

explicit solutions for some special tridiagonal matrices

150

Jacobi

simple but not very efficient

148

QL

efficient method for not too large matrices, especially in combination with tridiagonalization by Householder transformations

156

Lanczos

iterative method for very large matrices or if only a few eigenvalues are needed

159

Singular value decomposition (SVD)

Generalization for arbitrary matrices

167

Data fitting

Least square fit

fit a model function to a set of data

162

Linear least square fit with normal equations

simple but less accurate

163

Linear fit with orthogonalization

better numerical stability

165

Linear fit with SVD

expensive but more reliable, also for rank deficient matrices

172

Low rank matrix approximation

data compression, total linear least squares

170

Discretization

Method of lines

continuous time, discretized space

183

Eigenvector expansion

Finite differences

simplest discretization, uniform grids

180

Finite volumes

partial differential equations with a divergence term ( conservation laws), flux conservative, allows unstructured meshes and discontinuous material parameters

185

Finite elements

very flexible and general discretization method but also more complicated

196

Spectral methods

expansion with global basis functions, mostly polynomials and Fourier sums, less expensive than finite elements but not as accurate for discontinuous material parameters and complicated geometries

193

Dual grid

for finite volumes

185, 314

Weighted residuals

general method to determine the expansion coefficients

190

Point collocation

simplest criterion, often used for nonlinear problems and spectral methods

191

Sub-domains

more general than finite volumes

191

Least square

popular for computational fluid dynamics and electrodynamics

192

Galerkin

most widely used criterion, leads often to symmetric matrices

192

Fourier pseudo-spectral method

very useful whenever a Laplacian is involved, reduces dispersion

193

Boundary elements

if the Green’s function is available

204

Time evolution

Explicit forward Euler

low error order and unstable, mainly used as predictor step

210

Implicit backward Euler

low error order but stable, used for stiff problems and as corrector step

212

Improved Euler (Heun, predictor-corrector)

higher error order

213

Nordsieck predictor-corrector

implicit method, has been used for molecular dynamics

215

Gear predictor-corrector

optimized for molecular dynamics

217

Explicit Runge Kutta (2nd, 3rd, 4th)

general and robust methods, easy step size and quality control

217

Extrapolation (Gragg-Bulirsch-Stör)

very accurate and very slow

221

Explicit Adams-Bashforth

high error order but not self-starting, for smooth functions, can be used as predictor

222

Implicit Adams-Moulton

better stability than explicit method, can be used as corrector

223

Backward differentiation (Gear)

implicit, especially for stiff problems

223

Linear multistep predictor-corrector

General class, includes Adams-Bashforth-Moulton and Gear methods

224

Verlet integration

symplectic, time reversible, for molecular dynamics

225

Position Verlet

less popular

227

Velocity Verlet

often used

227

Störmer-Verlet

if velocities are not needed

228

Beeman’s method

velocities more accurate than for Störmer-Verlet

230

Leapfrog

simple but two different grids

231, 231, 343

Crank-Nicolson

implicit, stable, diffusion and Schrödinger equation

357, 347

Lax-Wendroff

hyperbolic differential equations

345

Two-step

differential equation with second order time derivative

338

Reduction to a first order equation

Derivatives treated as additional variables

340

Two-variable

transforms wave equation into a system of two first order equations

343

Split operator

approximates an operator by a product

360, 226, 399

Unitary time evolution

Rational approximation

implicit,unitary

392

Second order differencing

explicit, not exactly unitary

396

Split operator Fourier

low dispersion, needs fast Fourier transformation

399

Real space product formula

fast but less accurate, useful for wavepackets in coupled states

399

Rotation

Reorthogonalization

restore orthogonality of rotation matrix

250

Quaternions

optimum parametrization of the rotation matrix

256

Euler angles

numerical singularities

255

Explicit method

low accuracy, reorthogonalization needed

250

Implicit method

higher accuracy, orthogonal transformation

251

Molecular dynamics

Force field gradients

needed for molecular dynamics

270

Normal mode analysis

small amplitude motion around an equilibrium

274

Behrendsen thermostat

simple method to control temperature

281

Langevin dynamics

Brownian motion

301

References

1.

D.J. Acheson, Elementary Fluid Dynamics (Oxford University Press, London, 1990)

2.

N. Ahmed, T. Natarajan, K.R. Rao, IEEE Trans. Comput. 23 , 90 (1974)

3.

M.P. Allen, D.J. Tildesley, Computer Simulation of Liquids (Oxford University Press, London, 1989). ISBN 0-19-855645-4

4.

I. Allonso-Mallo, N. Reguera, J. Comp. Physiol. 220 , 409 (2006)

5.

V.S. Allured, C.M. Kelly, C.R. Landis, J. Am. Chem. Soc. 113 , 1 (1991)

6.

W. Ames, Numerical Methods for Partial Differential Equations (Academic Press, San Diego, 1992)

7.

Z.A. Anastassi, T.E. Simos, Phys. Rep. 482–483 , 1 (2009)

8.

P.W. Anderson, Phys. Rev. 79 , 350 (1950)

9.

X. Antoine, A. Arnold, C. Besse, M. Erhardt, A. Schädle, Commun. Comput. Phys. 4 , 729 (2008)

10.

A. Askar, A.S. Cakmak, J. Chem. Phys. 68 , 2794 (1978)

11.

J.S. Bader et al., J. Chem. Phys. 106 , 2372 (1997)

12.

C.T.H. Baker, Numer. Math. 15 , 315 (1970)

13.

G.A. Baker Jr., P. Graves-Morris, Padé Approximants (Cambridge University Press, New York, 1996)

14.

A.D. Bandrauk, H. Shen, Chem. Phys. Lett. 176 , 428 (1991)

15.

E.J. Bautista, J.M. Seminario, Int. J. Quant. Chem. 108 , 180 (2008)

16.

B. Beckermann, Numer. Math. 85 , 553 (2000)

17.

D. Beeman, J. Comp. Physiol. 20 , 130 (1976)

18.

G. Beer, I. Smith, C. Duenser, The Boundary Element Method with Programming: For Engineers and Scientists (Springer, Berlin, 2008)

19.

H. Bekker, H.J.C. Berendsen, W.F. van Gunsteren, J. Comput. Chem. 16 , 527 (1995)

20.

A. Ben-Israel, T.N.E. Greville, Generalized Inverses (Springer, Berlin, 2003). ISBN 0-387-00293-6

21.

H.J.C. Berendsen, J.P.M. Postma, W.F. van Gunsteren, A. DiNola, J.R. Haak, J. Chem. Phys. 81 , 3684 (1984)

22.

A. Bermann, S. Gueron, Math. Gaz. 86 , 274 (2002)

23.

J.P. Berrut, Comput. Math. Appl. 14 , 1 (1988)

24.

J.-P. Berrut, L.N. Trefethen, Barycentric Lagrange interpolation. SIAM Rev. 46 (3), 501–517 (2004)

25.

J.P. Berrut, R. Baltensperger, H.D. Mittelmann, Int. Ser. Numer. Math. 151 , 27 (2005)

26.

K. Binder, Ising model, in Encyclopedia of Mathematics , Suppl. vol. 2, ed. by R. Hoksbergen (Kluwer Academic, Dordrecht, 2000), pp. 279–281

27.

M. Bixon, J. Jortner, J. Chem. Phys. 48 , 715 (1986)

28.

F. Bloch, Nuclear induction. Phys. Rev. 70 , 460 (1946)

29.

P. Bocchieri, A. Loinger, Phys. Rev. 107 , 337 (1957)

30.

P. Bochev, M. Gunzburger, Least Squares Finite Element Methods (Springer, Berlin, 2009)

31.

E. Bonomi, J.-L. Lutton, SIAM Rev. 26 , 551 (1984)

32.

A.H. Boschitsch, M.O. Fenley, H.X. Zhou, J. Phys. Chem. B 106 , 2741 (2002)

33.

K.J. Bowers, E. Chow, H. Xu, R.U. Dror, M.P. Eastwood, B.A. Gregersen, J.L. Klepeis, I. Kolossvary, M.A. Moraes, F.D. Sacerdoti, J.K. Salmon, Y. Shan, D.E. Shaw, Scalable algorithms for molecular dynamics simulations on commodity clusters, in SC 2006 Conference . Proceedings of the ACM/IEEE (2006), p. 43

34.

G.E.P. Box, M.E. Muller, Ann. Math. Stat. 29 , 610 (1958)

35.

J.P. Boyd, Chebyshev and Fourier Spectral Methods (Dover, New York, 2001)

36.

R.P. Brent, Comput. J. 14 , 422 (1971)

37.

R.P. Brent, Algorithms for Minimization without Derivatives (Prentice Hall, Englewood Cliffs, 1973), Chap. 4

38.

I.N. Bronshtein, K.A. Semendyayev, Handbook of Mathematics , 3rd edn. (Springer, New York, 1997), p. 892

39.

B.R. Brooks, R.E. Bruccoleri, B.D. Olafson, D.J. States, S. Swaminathan, M. Karplus, J. Comput. Chem. 4 , 187 (1983)

40.

B.R. Brooks, C.L. Brooks III, A.D. Mackerell Jr., L. Nilsson, R.J. Petrella, B. Roux, Y. Won, G. Archontis, C. Bartels, S. Boresch, A. Caflisch, L. Caves, Q. Cui, A.R. Dinner, M. Feig, S. Fischer, J. Gao, M. Hodoscek, W. Im, K. Kuzcera, T. Lazaridis, J. Ma, V. Ovchinnikov, E. Paci, R.W. Pastor, C.B. Post, J.Z. Pu, M. Schaefer, B. Tidor, R.M. Venable, H.L. Woodcock, X. Wu, W. Yang, D.M. York, M. Karplus, J. Comput. Chem. 30 , 1545 (2009)

41.

R. Brown, Philos. Mag. 4 , 161 (1828)

42.

C.G. Broyden, Math. Comput. 19 , 577 (1965)

43.

C.G. Broyden, J. Inst. Math. Appl. 6 , 76 (1970)

44.

R.E. Bruccoleri, J. Novotny, M.E. Davis, K.A. Sharp, J. Comput. Chem. 18 , 268 (1997)

45.

R.L. Burden, J.D. Faires, Numerical Analysis (Brooks Cole, Boston, 2010), Chap. 2

46.

G. Burkard, R.H. Koch, D.P. DiVincenzo, Phys. Rev. B 69 , 64503 (2004)

47.

J.C.P. Bus, T.J. Dekker, ACM Trans. Math. Softw. 1 , 330 (1975)

48.

J.C. Butcher, The Numerical Analysis of Ordinary Differential Equations: Runge-Kutta and General Linear Methods (Wiley, New York, 1987)

49.

R.E. Caflisch, Monte Carlo and quasi-Monte Carlo methods. Acta Numer. 7 , 1–49 (1998)

50.

R. Car, M. Parrinello, Phys. Rev. Lett. 55 , 2471 (1985)

51.

A. Castro, M.A.L. Marques, A. Rubio, J. Chem. Phys. 121 , 3425 (2004)

52.

Y.A. Cengel, Heat transfer—A Practical Approach , 2nd edn. (McGraw-Hill, New York, 2003), p. 26. ISBN 0072458933, 9780072458930

53.

C. Cerjan, K.C. Kulander, Comput. Phys. Commun. 63 , 529 (1991)

54.

T.R. Chandrupatla, Adv. Eng. Softw. 28 , 145 (1997)

55.

H.W. Chang, S.E. Liu, R. Burridge, Linear Algebra Appl. 430 , 999 (2009)

56.

D.L. Chapman, Philos. Mag. 25 , 475 (1913)

57.

R. Chen, H. Guo, J. Chem. Phys. 111 , 9944 (1999)

58.

M. Christen, P.H. Hünenberger, D. Bakowies, R. Baron, R. Bürgi, D.P. Geerke, T.N. Heinz, M.A. Kastenholz, V. Kräutler, C. Oostenbrink, C. Peter, D. Trzesniak, W.F. van Gunsteren, J. Comput. Chem. 26 , 1719 (2005)

59.

S. Chynoweth, U.C. Klomp, L.E. Scales, Comput. Phys. Commun. 62 , 297 (1991)

60.

C.W. Clenshaw, A.R. Curtis, Numer. Math. 2 , 197 (1960)

61.

L.A. Collins, J.D. Kress, R.B. Walker, Comput. Phys. Commun. 114 , 15 (1998)

62.

J.W. Cooley, J.W. Tukey, Math. Comput. 19 , 297 (1965)

63.

W.D. Cornell, P. Cieplak, C.I. Bayly, I.R. Gould, K.M. Merz Jr., D.M. Ferguson, D.C. Spellmeyer, T. Fox, J.W. Caldwell, P.A. Kollman, J. Am. Chem. Soc. 117 , 5179 (1995)

64.

R. Courant, K. Friedrichs, H. Lewy, Math. Ann. 100 , 32 (1928)

65.

J. Crank, P. Nicolson, Proc. Camb. Philol. Soc. 43 , 50 (1947)

66.

J.W. Daniel, W.B. Gragg, L. Kaufmann, G.W. Stewart, Math. Comput. 30 , 772 (1976)

67.

H. De Raedt, Comput. Phys. Rep. 7 , 1 (1987)

68.

H. De Raedt, B. De Raedt, Phys. Rev. A 28 , 3575 (1983)

69.

P. Debye, E. Hückel, Phys. Z. 24 , 185 (1923)

70.

T.J. Dekker, Finding a zero by means of successive linear interpolation, in Constructive Aspects of the Fundamental Theorem of Algebra , ed. by B. Dejon, P. Henrici (Wiley-Interscience, London, 1969)

71.

D.P. Derrarr et al., in A Practical Approach to Microarray Data Analysis (Kluwer Academic, Norwell, 2003), p. 91

72.

DGESVD routine from the freely available software package LAPACK, http:​/​/​www.​netlib.​org/​lapack

73.

L. Diosi, A Short Course in Quantum Information Theory (Springer, Berlin, 2007)

74.

P. Duhamel, M. Vetterli, Signal Process. 19 , 259 (1990)

75.

S.E. Economou, T.L. Reinecke, in Optical Generation and Control of Quantum Coherence in Semiconductor Nanostructures , ed. by G. Slavcheva, P. Roussignol (Springer, Berlin, 2010), p. 62

76.

A. Einstein, Ann. Phys. 17 , 549 (1905)

77.

A. Einstein, Investigations on the Theory of Brownian Movement (Dover, New York, 1956)

78.

J.R. Errington, P.G. Debenedetti, S. Torquato, J. Chem. Phys. 118 , 2256 (2003)

79.

R. Eymard, T. Galloue, R. Herbin, Finite volume methods, in Handbook of Numerical Analysis , vol. 7, ed. by P.G. Ciarlet, J.L. Lions (Elsevier, Amsterdam, 2000), pp. 713–1020

80.

M.D. Feit, J.A. Fleck Jr., A. Steiger, J. Comp. Physiol. 47 , 412 (1982)

81.

R.P. Feynman, F.L. Vernon, R.W. Hellwarth, J. Appl. Phys. 28 , 49 (1957)

82.

A. Fick, Philos. Mag. 10 , 30 (1855)

83.

P.C. Fife, Mathematical Aspects of Reacting and Diffusing Systems (Springer, Berlin, 1979)

84.

J. Fish, T. Belytschko, A First Course in Finite Elements (Wiley, New York, 2007)

85.

G.S. Fishman, Monte Carlo: Concepts, Algorithms, and Applications (Springer, New York, 1995). ISBN 038794527X

86.

R. Fletcher, Comput. J. 13 , 317 (1970)

87.

C.A.J. Fletcher, Computational Galerkin Methods (Springer, Berlin, 1984)

88.

C.A.J. Fletcher, Computational Techniques for Fluid Dynamics , vol. I, 2nd edn. (Springer, Berlin, 1991)

89.

R. Fletcher, C. Reeves, Comput. J. 7 , 149 (1964)

90.

M.S. Floater, K. Hormann, Numer. Math. 107 , 315 (2007)

91.

F. Fogolari, A. Brigo, H. Molinari, J. Mol. Recognit. 15 , 377 (2002)

92.

J.A. Ford, Improved Algorithms of Illinois-Type for the Numerical Solution of Nonlinear Equations (University of Essex Press, Essex, 1995), Technical Report CSM-257

93.

B. Fornberg, Geophysics 52 , 4 (1987)

94.

B. Fornberg, Math. Comput. 51 , 699 (1988)

95.

J. Fourier, The Analytical Theory of Heat (Cambridge University Press, Cambridge, 1878), reissued by Cambridge University Press, 2009. ISBN 978-1-108-00178-6

96.

M. Fox, Quantum Optics: An Introduction (Oxford University Press, London, 2006)

97.

D. Frenkel, B. Smit, Understanding Molecular Simulation: From Algorithms to Applications (Academic Press, San Diego, 2002). ISBN 0-12-267351-4

98.

B.G. Galerkin, On electrical circuits for the approximate solution of the Laplace equation. Vestnik Inzh. 19 , 897–908 (1915)

99.

A.E. Garcia, Phys. Rev. Lett. 86 , 2696 (1992)

100.

C.W. Gear, Math. Comput. 21 , 146 (1967)

101.

C.W. Gear, Numerical Initial Value Problems in Ordinary Differential Equations (Prentice Hall, Englewood Cliffs, 1971)

102.

C.W. Gear, IEEE Trans. Circuit Theory 18 , 89–95 (1971)

103.

G. Goertzel, Am. Math. Mon. 65 , 34 (1958)

104.

A. Goldberg, H.M. Schey, J.L. Schwartz, Am. J. Phys. 35 , 177 (1967)

105.

D. Goldfarb, Math. Comput. 24 , 23 (1970)

106.

H. Goldstein, Klassische Mechanik (Akademische Verlagsgesellschaft, Frankfurt am Main, 1974)

107.

G. Golub, W. Kahan, J. Soc. Ind. Appl. Math., Ser. B Numer. Anal. 2 , 205 (1965)

108.

G.H. Golub, C.F. Van Loan, Matrix Computations , 3rd edn. (Johns Hopkins University Press, Baltimore, 1976). ISBN 978-0-8018-5414-9

109.

G.H. Golub, J.H. Welsch, Math. Comput. 23 , 221–230 (1969)

110.

G.L. Gouy, J. Phys. 9 , 457 (1910)

111.

W.B. Gragg, SIAM J. Numer. Anal. 2 , 384 (1965)

112.

P. Grindrod, Patterns and Waves: The Theory and Applications of Reaction-Diffusion Equations (Clarendon Press, Oxford, 1991)

113.

R. Guantes, S.C. Farantos, J. Chem. Phys. 111 , 10827 (1999)

114.

R. Guantes, A. Nezis, S.C. Farantos, J. Chem. Phys. 111 , 10836 (1999)

115.

J.M. Haile, Molecular Dynamics Simulation: Elementary Methods (Wiley, New York, 2001). ISBN 0-471-18439-X

116.

E. Hairer, C. Lubich, G. Wanner, Acta Numer. 12 , 399 (2003)

117.

J. Halpern, L.E. Orgel, Discuss. Faraday Soc. 29 , 32 (1960)

118.

J.-P. Hansen, L. Verlet, Phys. Rev. 184 , 151 (1969)

119.

F.j. Harris, Proc. IEEE 66 , 51 (1978)

120.

M.T. Heath, Multigrid methods, in Scientific Computing: An Introductory Survey (McGraw-Hill, New York, 2002), p. 478 ff. §11.5.7, Higher Education

121.

M.R. Hestenes, E. Stiefel, J. Res. Natl. Bur. Stand. 49 , 435 (1952)

122.

D. Hilbert, L. Nordheim, J. von Neumann, Math. Ann. 98 , 1 (1927)

123.

D. Hinrichsen, A.J. Pritchard, Mathematical Systems Theory I—Modelling, State Space Analysis, Stability and Robustness (Springer, Berlin, 2005). ISBN 0-978-3-540-441250

124.

C.S. Holling, Can. Entomol. 91 , 293 (1959)

125.

C.S. Holling, Can. Entomol. 91 , 385 (1959)

126.

F.-Y. Hong, S.-J. Xiong, Chin. J. Phys. 46 , 379 (2008)

127.

B.K.P. Horn, H.M. Hilden, S. Negahdaripour, J. Opt. Soc. Am. A 5 , 1127 (1988)

128.

P.H. Huenenberger, Adv. Polym. Sci. 173 , 105 (2005)

129.

H. Ibach, H. Lüth, Solid-State Physics: An Introduction to Principles of Materials Science . Advanced Texts in Physics (Springer, Berlin, 2003)

130.

IEEE 754-2008, Standard for Floating-Point Arithmetic, IEEE Standards Association, 2008

131.

T. Iitaka, Phys. Rev. E 49 , 4684 (1994)

132.

T. Iitaka, N. Carjan, T. Strottman, Comput. Phys. Commun. 90 , 251 (1995)

133.

R.Z. Iqbal, Master thesis, School of Mathematics, University of Birmingham, 2008

134.

E. Ising, Beitrag zur Theorie des Ferromagnetismus. Z. Phys. 31 , 253–258 (1925). doi: 10.​1007/​BF02980577

135.

V.I. Istratescu, Fixed Point Theory: An Introduction (Reidel, Dordrecht 1981). ISBN 90-277-1224-7

136.

IUPAC-IUB Commission on Biochemical Nomenclature, Biochemistry 9 , 3471 (1970)

137.

H. Jeffreys, B.S. Jeffreys, Lagrange’s interpolation formula, in Methods of Mathematical Physics , 3rd edn. (Cambridge University Press, Cambridge, 1988), p. 260, § 9.011

138.

H. Jeffreys, B.S. Jeffreys, Divided differences, in Methods of Mathematical Physics , 3rd edn. (Cambridge University Press, Cambridge, 1988), pp. 260–264, § 9.012

139.

H. Jeffreys, B.S. Jeffreys, Methods of Mathematical Physics , 3rd edn. (Cambridge University Press, Cambridge, 1988), pp. 305–306

140.

B.-n. Jiang, The Least-Squares Finite Element Method (Springer, Berlin, 1998)

141.

F. John, Duke Math. J. 4 , 300 (1938)

142.

J.K. Johnson, J.A. Zollweg, K.E. Gubbins, Mol. Phys. 78 , 591 (1993)

143.

A.H. Juffer et al., J. Phys. Chem. B 101 , 7664 (1997)

144.

E.I. Jury, Theory and Application of the Z-Transform Method (Krieger, Melbourne, 1973). ISBN 0-88275-122-0

145.

K. Khalil Hassan, Nonlinear Systems (Prentice Hall, Englewood Cliffs, 2001). ISBN 0-13-067389-7

146.

S.A. Khrapak, M. Chaudhuri, G.E. Morfill, Phys. Rev. B 82 , 052101 (2010)

147.

J. Kiefer, Proc. Am. Math. Soc. 4 , 502–506 (1953)

148.

J.G. Kirkwood, J. Chem. Phys. 2 , 351 (1934)

149.

S.E. Koonin, C.M. Dawn, Computational Physics (Perseus Books, New York, 1990). ISBN 978-0201127799

150.

D. Kosloff, R. Kosloff, J. Comp. Physiol. 52 , 35 (1983)

151.

S. Kouachi, Electron. J. Linear Algebra 15 , 115 (2006)

152.

T. Kühne, M. Krack, F. Mohamed, M. Parrinello, Phys. Rev. Lett. 98 , 066401 (2007)

153.

C. Lanczos, J. Res. Natl. Bur. Stand. 45 , 255 (1951)

154.

L. Landau, Phys. Sov. Union 2 , 46–51 (1932)

155.

H.P. Langtangen, Computational Partial Differential Equations: Numerical Methods and Diffpack Programming (Springer, Berlin, 2003)

156.

C. Lavor, Physica D 227 , 135 (2007)

157.

A. Leach, Molecular Modelling: Principles and Applications , 2nd edn. (Prentice Hall, Harlow, 2001). ISBN 978-0582382107

158.

C. Leforestier, R.H. Bisseling, C. Cerjan, M.D. Feit, R. Friesner, A. Guldberg, A. Hammerich, G. Jolicard, W. Karrlein, H.-D. Meyer, N. Lipkin, O. Roncero, R. Kosloff, J. Comp. Physiol. 94 , 59 (1991)

159.

D. Levesque, L. Verlet, Phys. Rev. A 2 , 2514 (1970)

160.

A.J. Lotka, Elements of Physical Biology (Williams and Wilkins, Baltimore, 1925)

161.

A.M. Lyapunov, Stability of Motion (Academic Press, New York, 1966)

162.

L. Lynch, Numerical integration of linear and nonlinear wave equations. Bachelor thesis, Florida Atlantic University, Digital Commons@University of Nebraska-Lincoln, 2004

163.

A.D. MacKerell Jr., B. Brooks, C.L. Brooks III, L. Nilsson, B. Roux, Y. Won, M. Karplus, CHARMM: The energy function and its parameterization with an overview of the program, in The Encyclopedia of Computational Chemistry , vol. 1, ed. by P.v.R. Schleyer et al. (Wiley, Chichester, 1998), pp. 271– 277

164.

W. Magnus, Commun. Appl. Math. 7 , 649 (1954)

165.

J. Mandel, Am. Stat. 36 , 15 (1982)

166.

A.A. Markov, Theory of Algorithms [Translated by Jacques J. Schorr-Kon and PST staff] Imprint Moscow, Academy of Sciences of the USSR, 1954 [Jerusalem, Israel Program for Scientific Translations, 1961; available from Office of Technical Services, United States Department of Commerce] Added t.p. in Russian Translation of Works of the Mathematical Institute, Academy of Sciences of the USSR, v. 42. Original title: Teoriya algorifmov. [QA248.M2943 Dartmouth College library. U.S. Dept. of Commerce, Office of Technical Services, number OTS 60-51085] (1954)

167.

A.A. Markov, Extension of the limit theorems of probability theory to a sum of variables connected in a chain, in Dynamic Probabilistic Systems, vol. 1: Markov Chains (Wiley, New York, 1971), reprinted in Appendix B of Howard, R.

168.

G. Marsaglia, A. Zaman, Ann. Appl. Probab. 1 , 462 (1991)

169.

G.J. Martyna, M.E. Tuckerman, J. Chem. Phys. 102 , 8071 (1995)

170.

W.L. Mattice, U.W. Suter, Conformational Theory of Large Molecules (Wiley-Interscience, New York, 1994). ISBN 0-471-84338-5

171.

B.M. McCoy, T.T. Wu, The Two-Dimensional Ising Model (Harvard University Press, Cambridge, 1973). ISBN 0674914406

172.

E.A. McCullough Jr., R.E. Wyatt, J. Chem. Phys. 51 , 1253 (1969)

173.

E.A. McCullough Jr., R.E. Wyatt, J. Chem. Phys. 54 , 3592 (1971)

174.

N. Metropolis, S. Ulam, J. Am. Stat. Assoc. 44 , 335 (1949)

175.

N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, E. Teller, J. Chem. Phys. 21 , 1087 (1953)

176.

C. Muguruma, Y. Okamoto, M. Mikami, Croat. Chem. Acta 80 , 203 (2007)

177.

D.E. Muller, Math. Tables Other Aids Comput. 10 , 208 (1956)

178.

J.D. Murray, Mathematical Biology: I. An Introduction , vol. 2, 3rd edn. (Springer, Berlin, 2002). ISBN 0-387-95223-3

179.

S.J. Nettel, A. Kempicki, Am. J. Phys. 47 , 987 (1979)

180.

E.H. Neville, J. Indian Math. Soc. 20 , 87 (1933)

181.

A. Nicholls, B. Honig, J. Comput. Chem. 12 , 435 (1990)

182.

J.J. Nicolas, K.E. Gubbins, W.B. Streett, D.J. Tildesley, Mol. Phys. 37 , 1429 (1979)

183.

M. Nielsen, I. Chuang, Quantum Computation and Quantum Information (Cambridge University Press, Cambridge, 2000)

184.

A. Nordsieck, Math. Comput. 16 , 22 (1962)

185.

M. Novelinkova, in: WDS’11, Proceedings of Contributed Papers, Part I (2011), p. 67

186.

G. Nürnberger, Approximation by Spline Functions (Springer, Berlin, 1989). ISBN 3-540-51618-2

187.

H.J. Nussbaumer, Fast Fourier Transform and Convolution Algorithms (Springer, Berlin, 1990)

188.

M. Oevermann, R. Klein, J. Comp. Physiol. 219 , 749 (2006)

189.

M. Oevermann, C. Scharfenberg, R. Klein, J. Comput. Phys. 228 (14), 5184–5206 (2009). doi: 10.​1016/​j.​jcp.​2009.​04.​018

190.

I.P. Omelyan, Phys. Rev. 58 , 1169 (1998)

191.

I.P. Omelyan, Comput. Phys. Commun. 109 , 171 (1998)

192.

I.P. Omelyan, Comput. Phys. 12 , 97 (1998)

193.

I.P. Omelyan, I.M. Mryglod, R. Folk, Comput. Phys. Commun. 151 , 272 (2003)

194.

L. Onsager, Phys. Rev. 65 , 117 (1944)

195.

E.L. Ortiz, SIAM J. Numer. Anal. 6 , 480 (1969)

196.

J.M. Papakonstantinou, A historical development of the ( n +1)-point secant method. M.A. thesis, Rice University Electronic Theses and Dissertations, 2007

197.

T.J. Park, J.C. Light, J. Chem. Phys. 85 , 5870 (1986)

198.

B.N. Parlett, The Symmetric Eigenvalue Problem (Society for Industrial and Applied Mathematics, Philadelphia, 1998)

199.

K. Pearson, The problem of the random walk. Nature 72 , 294 (1905)

200.

J. Peiro, S. Sherwin, in Handbook of Materials Modeling , vol. 1, ed. by S. Yip (Springer, Berlin, 2005), pp. 1–32

201.

D. Pines, C.P. Slichter, Phys. Rev. 100 , 1014 (1955)

202.

H. Pollard, Q. J. Pure Appl. Math. 49 , 1 (1920)

203.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, LU decomposition and its applications, in Numerical Recipes: The Art of Scientific Computing , 3rd edn. (Cambridge University Press, Cambridge, 2007), pp. 48–55

204.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Cholesky decomposition, in Numerical Recipes: The Art of Scientific Computing , 3rd edn. (Cambridge University Press, Cambridge, 2007), pp. 100–101

205.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Cyclic tridiagonal systems, in Numerical Recipes: The Art of Scientific Computing , 3rd edn. (Cambridge University Press, Cambridge, 2007), p. 79

206.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Relaxation mehods for boundary value problems, in Numerical Recipes: The Art of Scientific Computing , 3rd edn. (Cambridge University Press, Cambridge, 2007), pp. 1059–1065

207.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Golden section search in one dimension, in Numerical Recipes: The Art of Scientific Computing , 3rd edn. (Cambridge University Press, New York, 2007). ISBN 978-0-521-88068-8, Sect. 10.2

208.

W.H. Press, S.A. Teukolsky, W.T. Vetterling, B.P. Flannery, Second-order conservative equations, in Numerical Recipes: The Art of Scientific Computing , 3rd edn. (Cambridge University Press, Cambridge, 2007), Sect. 17.4

209.

A. Quarteroni, R. Sacco, F. Saleri, Numerical Mathematics , 2nd edn. (Springer, Berlin, 2007)

210.

S.S. Rao, The Finite Element Method in Engineering (Elsevier, Amsterdam, 2011)

211.

K.R. Rao, P. Yip, Discrete Cosine Transform: Algorithms, Advantages, Applications (Academic Press, Boston, 1990)

212.

D.C. Rapaport, The Art of Molecular Dynamics Simulation (Cambridge University Press, Cambridge, 2004). ISBN 0-521-44561-2

213.

A.K. Rappe, C.J. Casewit, K.S. Colwell, W.A. Goddar, W.M. Skiff, J. Am. Chem. Soc. 115 , 10024 (1992)

214.

E. Renshaw, Modelling Biological Populations in Space and Time (Cambridge University Press, Cambridge, 1991). ISBN 0-521-44855-7

215.

J. Rice, Mathematical Statistics and Data Analysis , 2nd edn. (Duxbury Press, N. Scituate, 1995). ISBN 0-534-20934-3

216.

J.A. Richards, Remote Sensing Digital Image Analysis (Springer, Berlin, 1993)

217.

L.F. Richardson, Philos. Trans. R. Soc. Lond. Ser. A 210 , 307–357 (1911)

218.

R.D. Richtmyer, Principles of Modern Mathematical Physics I (Springer, New York, 1978)

219.

H. Risken, The Fokker-Planck Equation (Springer, Berlin, 1989)

220.

C.P. Robert, G. Casella, Monte Carlo Statistical Methods , 2nd edn. (Springer, New York, 2004). ISBN 0387212396

221.

T.D. Romo et al., Proteins 22 , 311 (1995)

222.

I.R. Savage, E. Lukacs, in Contributions to the Solution of Systems of Linear Equations and the Determination of Eigenvalues , ed. by O. Taussky. National Bureau of Standards, Applied Mathematics Series, vol. 39 (1954)

223.

G. Schaftenaar, J.H. Noordik, J. Comput.-Aided Mol. Des. 14 , 123 (2000)

224.

T. Schlick, J. Comput. Chem. 10 , 951 (1989)

225.

T. Schlick, Molecular Modeling and Simulation (Springer, Berlin, 2002). ISBN 0-387-95404-X

226.

B.I. Schneider, J.-A. Collins, J. Non-Cryst. Solids 351 , 1551 (2005)

227.

C. Schneider, W. Werner, Math. Comput. 47 , 285 (1986)

228.

I.J. Schoenberg, Q. Appl. Math. 4 , 45–99 and 112–141 (1946)

229.

P. Schofield, Comput. Phys. Commun. 5 , 17 (1973)

230.

E. Schrödinger, Phys. Rev. 28 , 1049 (1926)

231.

F. Schwabl, Statistical Mechanics (Springer, Berlin, 2003)

232.

F. Schwabl, Quantum Mechanics , 4th edn. (Springer, Berlin, 2007)

233.

L.F. Shampine, IMA J. Numer. Anal. 3 , 383 (1983)

234.

L.F. Shampine, L.S. Baca, Numer. Math. 41 , 165 (1983)

235.

D.F. Shanno, Math. Comput. 24 , 647 (1970)

236.

J. Sherman, W.J. Morrison, Ann. Math. Stat. 20 , 621 (1949)

237.

T. Simonson, Rep. Prog. Phys. 66 , 737 (2003)

238.

G. Skollermo, Math. Comput. 29 , 697 (1975)

239.

B. Smit, J. Chem. Phys. 96 , 8639 (1992)

240.

A. Sommariva, Comput. Math. Appl. 65 (4), 682–693 (2013). doi: 10.​1016/​j.​camwa.​2012.​12.​004 [MATLAB CODES (zip file)]

241.

R. Sonnenschein, A. Laaksonen, E. Clementi, J. Comput. Chem. 7 , 645 (1986)

242.

B.I. Stepanov, V.P. Grobkovskii, Theory of Luminescence (Butterworth, London, 1986)

243.

G.W. Stewart, SIAM Rev. 35 , 551 (1993)

244.

J. Stör, R. Bulirsch, Introduction to Numerical Analysis , 3rd revised edn. (Springer, New York, 2010). ISBN 978-1441930064

245.

S.H. Strogatz, Nonlinear Dynamics and Chaos: Applications to Physics, Biology, Chemistry, and Engineering (Perseus Books, Reading, 2001). ISBN 0-7382-0453-6

246.

M. Suzuki, Commun. Math. Phys. 51 , 183 (1976)

247.

P.N. Swarztrauber, R.A. Sweet, in Handbook of Fluid Dynamics and Fluid Machinery , ed. by J.A. Schetz, A.E. Fuhs (Wiley, New York, 1996)

248.

H. Tal-Ezer, R. Kosloff, J. Chem. Phys. 81 , 3967 (1984)

249.

J.T. Tanner, Ecology 56 , 855 (1975)

250.

S.A. Teukolsky, Phys. Rev. D 61 (8), 087501 (2000). doi: 10.​1103/​PhysRevD.​61.​087501

251.

J.W. Thomas, Numerical Partial Differential Equations: Finite Difference Methods . Texts in Applied Mathematics, vol. 22 (Springer, Berlin, 1995)

252.

J.Y. Tjalling, Historical development of the Newton-Raphson method. SIAM Rev. 37 , 531 (1995)

253.

L.N. Trefethen, SIAM Rev. 50 , 67–87 (2008)

254.

L.N. Trefethen, D. Bau III, Numerical Linear Algebra (Society for Industrial and Applied Mathematics, Philadelphia, 1997), p. 1

255.

S.-H. Tsai et al., Braz. J. Phys. 34 , 384 (2004)

256.

T. Tsang, H. Tang, Phys. Rev. A 15 , 1696 (1977)

257.

M. Tuckerman, B.J. Berne, J. Chem. Phys. 97 , 1990 (1992)

258.

A.M. Turing, Philos. Trans. R. Soc. Lond. B 237 , 37 (1952)

259.

R.E. Tuzun, D.W. Noid, B.G. Sumpter, Macromol. Theory Simul. 5 , 771 (1996)

260.

R.E. Tuzun, D.W. Noid, B.G. Sumpter, J. Comput. Chem. 18 , 1804 (1997)

261.

W. van Dijk, F.M. Toyama, Phys. Rev. E 75 , 036707 (2007)

262.

W.F. van Gunsteren, S.R. Billeter, A.A. Eising, P.H. Hünenberger, P. Krüger, A.E. Mark, W.R.P. Scott, I.G. Tironi, Biomolecular Simulation: The GROMOS96 Manual and User Guide (vdf Hochschulverlag AG an der ETH Zürich and BIOMOS b.v., Zürich, Groningen, 1996)

263.

L.M.K. Vandersypen, I.L. Chuang, Rev. Mod. Phys. 76 , 1037 (2004)

264.

P.F. Verhulst, Mem. Acad. R. Sci. Belles Lettres Bruxelles 18 , 1–42 (1845)

265.

L. Verlet, Phys. Rev. 159 , 98 (1967)

266.

V. Volterra, Mem. R. Accad. Naz. Lincei 2 , 31 (1926)

267.

J. Waldvogel, BIT Numer. Math. 46 , 195 (2006)

268.

Z. Wang, B.R. Hunt, Appl. Math. Comput. 16 , 19 (1985)

269.

R.K. Wangsness, F. Bloch, Phys. Rev. 89 , 728 (1953)

270.

H. Watanabe, N. Ito, C.K. Hu, J. Chem. Phys. 136 , 204102 (2012)

271.

W. Werner, Math. Comput. 43 , 205 (1984)

272.

H. Wilf, Mathematics for the Physical Sciences (Wiley, New York, 1962)

273.

J.H. Wilkinson, J. ACM 8 , 281 (1961)

274.

C.P. Williams, Explorations in Quantum Computing (Springer, Berlin, 2011)

275.

J. Wolberg, Data Analysis Using the Method of Least Squares: Extracting the Most Information from Experiments (Springer, Berlin, 2005)

276.

L.C. Wrobel, M.H. Aliabadi, The Boundary Element Method (Wiley, New York, 2002)

277.

G. Wunsch, Feldtheorie (VEB Technik, Berlin, 1973)

278.

K. Yabana, G.F. Bertsch, Phys. Rev. B 54 , 4484 (1996)

279.

S. Yang, M.K. Gobbert, Appl. Math. Lett. 22 , 325 (2009)

280.

A. Yariv, Quantum Electronics (Wiley, New York, 1975)

281.

W.C. Yueh, Appl. Math. E-Notes 5 , 66 (2005)

282.

C. Zener, Proc. R. Soc. Lond. A 137 (6), 696–702 (1932)

283.

Y. Zhu, A.C. Cangellaris, Multigrid Finite Element Methods for Electromagnetic Field Modeling (Wiley, New York, 2006), p. 132 ff. ISBN 0471741108

284.

I. Zutic, J. Fabian, S. Das Sarma, Rev. Mod. Phys. 76 , 323 (2004)

Index

A

Adams-Bashforth

Adams-Moulton

Amplification factor

Angular momentum

Angular velocity

Attractive fixed point

Auto-correlation

Average extension

Average of measurements

B

Backward difference

Backward differentiation

Backward substitution

Ballistic motion

Beeman

BFGS

Bicubic spline interpolation

Bifurcation

Bifurcation diagram

Bilinear interpolation

Binomial distribution

Bio-molecules

Biopolymer

Birth rate

Bisection

Bloch equations

Bloch vector

Bond angle

Bond length

Boundary conditions

Boundary element

Boundary element method

Boundary potential

Boundary value problems

Box Muller

Brent

Brownian motion

Broyden

C

Calculation of π

Carrying capacity

Cartesian coordinates

Cavity

Cayley-Klein

Central difference quotient

Central limit theorem

Chain

Chandrupatla

Chaotic behavior

Characteristic polynomial

Charged sphere

Chebyshev

Chemical reactions

Chessboard method

Circular orbit

Clenshaw-Curtis

Coin

Collisions

Composite midpoint rule

Composite Newton-Cotes formulas

Composite Simpson’s rule

Composite trapezoidal rule

Computer experiments

Concentration

Condition number

Configuration integral

Conjugate gradients

Conservation laws

Continuous logistic model

Control parameter

Control volumes

Coordinate system

Correlation coefficient

Coulomb interaction

Courant number

Covariance matrix

Crank-Nicolson

Critical temperature

Crossing point

Cubic spline

Cumulative probability distribution

Cyclic tridiagonal

D

Damped string

Damping

Data fitting

Debye length

Dekker

Density matrix

Density of states

Detailed balance

Determinant

Dielectric medium

Differential equations

Differentiation

Differentiation matrix

Diffusion equation

Diffusive motion

Diffusive population dynamics

Dihedral angle

Direction set

Discontinuity

Discontinuous ε

Discrete Fourier transformation

Discretization

Disorder

Dispersion

Divided differences

Dual grid

E

Effective coupling

Effective force constant

Eigenvalue

Eigenvector expansion

Electric field

Electrolyte

Electrostatics

Elliptical differential equation

Elongation

End to end distance

Energy function

Ensemble average

Equations of motion

Equilibria

Error accumulation

Error analysis

Error function

Error of addition

Error of multiplication

Error propagation

Euler angles

Euler parameters

Euler-McLaurin expansion

Euler’s equations

Expectation value

Explicit Euler method

Exponent overflow

Exponent underflow

Exponential decay

Exponential distribution

Extrapolation

F

Fair die

Fast Fourier transformation

Few-state systems

Filter function

Finite differences

Finite elements

Finite volumes

Fixed point

Fixed point equation

Fletcher-Rieves

Floating point numbers

Floating point operations

Fluctuating force

Flux

Force

Force extension relation

Force field

Forward difference

Fourier transformation

Free energy

Free precession

Free rotor

Freely jointed chain

Friction coefficient

Friction force

Frobenius matrix

FTCS

Functional response

G

Galerkin

Gauss-Legendre

Gauss-Seidel

Gaussian distribution

Gaussian elimination

Gaussian integral rules

Gaussian integration

Gauss’s theorem

Gear

Givens

Global truncation error

Glycine dipeptide

Goertzel

Golden section search

Gradient vector

Gradients

Gram-Schmidt

Green’s theorem

Grid

Gyration radius

Gyration tensor

H

Hadamard gate

Hamilton operator

Harmonic approximation

Harmonic potential

Hessian

Heun

Higher derivatives

Hilbert matrix

Hilbert space

Histogram

Holling

Holling-Tanner model

Hookean spring

Householder

Hyperbolic differential equation

I

Implicit Euler method

Implicit method

Importance sampling

Improved Euler method

Inertia

Inevitable error

Inhomogeneity

Initial value problem

Integers

Integral equations

Integral form

Integration

Interacting states

Interaction energy

Intermediate state

Intermolecular interactions

Internal coordinates

Interpolating function

Interpolating polynomial

Interpolation

Interpolation error

Intramolecular forces

Inverse interpolation

Ising model

Iterated functions

Iterative algorithms

Iterative method

Iterative solution

J

Jacobi

Jacobi determinant

Jacobian

K

Kinetic energy

L

Ladder model

Lagrange

Lanczos

Landau-Zener model

Langevin dynamics

Laplace operator

Larmor-frequency

Laser field

Lax-Wendroff method

Leapfrog

Least square

Least square fit

Legendre polynomials

Lennard-Jones

Lennard-Jones system

Linear approximation

Linear equations

Linear fit function

Linear least square fit

Linear regression

Liouville

Local truncation error

Logistic map

Lotka-Volterra model

Low rank matrix approximation

Lower triangular matrix

LU decomposition

Lyapunov exponent

M

Machine numbers

Machine precision

Magnetization

Markov chain

Marsaglia

Matrix elements

Matrix inversion

Mean square displacement

Mesh

Method of lines

Metropolis

Midpoint rule

Milne rule

Minimization

Mixed states

Mobile charges

Modified midpoint method

Molecular collision

Molecular dynamics

Moments

Moments of inertia

Monochromatic excitation

Monte Carlo

Mortality rate

Multigrid

Multipole expansion

Multistep

Multivariate distribution

Multivariate interpolation

N

N -body system

Neumann

Neville

Newton

Newton-Cotes

Newton-Raphson

NMR

Nodes

Noise filter

Nonlinear optimization

Nonlinear systems

Nordsieck

Normal distribution

Normal equations

Normal modes

Nullclines

Numerical errors

Numerical extinction

Numerical integration

O

Observables

Occupation probability

Omelyan

One-sided difference

Onsager

Open interval

Optimized sample points

Orbit

Orthogonality

Orthogonalization

Oscillating perturbation

P

Pair distance distribution

Parabolic differential equations

Partition function

Pattern formation

Pauli matrices

Pauli-gates

Period

Period doubling

Periodic orbit

Phase angle

Phase space

Phase transition

Pivoting

Plane wave

Point collocation method

Poisson equation

Poisson-Boltzmann equation

Polarization

Polymer

Polynomial

Polynomial extrapolation

Polynomial interpolation

Population

Population dynamics

Potential energy

Predation

Predator

Predictor-corrector

Pressure

Prey

Principal axes

Probability density

Pseudo random numbers

Pseudo-spectral

Pseudo-spectral method

Pseudoinverse

Pure states

Q

QL algorithm

QR decomposition

Quality control

Quantum systems

Quasi-Newton condition

Quasi-Newton methods

Quaternion

Qubit

Qubit manipulation

R

Rabi oscillations

Random motion

Random numbers

Random points

Random walk

Rational approximation

Reaction-diffusion systems

Real space product formulae

Rectangular elements

Recurrence

Reflecting walls

Regula falsi method

Relaxation

Relaxation parameter

Reproduction rate

Residual

Resonance curve

Resonant pulse

Rigid body

Romberg

Romberg integration

Root finding

Roots

Rosenbrock

Rotation in the complex plane

Rotation matrix

Rotational motion

Rotor

Rotor in a field

Rounding errors

Runge-Kutta

S

Sampling theorem

Schrödinger equation

Secant method

Second order differencing

Self energy

Semi-discretized

Sherman-Morrison formula

Shifted grid

Simple sampling

Simpson’s rule

Simulated annealing

Singular values

Solvation

Solvation energy

Solvent

Specific heat

Spectral methods

Spin

Spin flip

Spin vector

Spline interpolation

Split operator

Stability analysis

Standard deviation

Statistical operator

Steepest descent

Step size control

Störmer-Verlet method

Sub-domain method

Successive over-relaxation

Superexchange

Superposition

Surface charge

Surface element

Symmetric difference quotient

T

Taylor series method

Ternary search

Tetrahedrons

Thermal average

Thermal equilibrium

Thermodynamic averages

Thermodynamic systems

Three-state system

Tight-binding model

Time derivatives

Time evolution

Transmission function

Trapezoidal rule

Trial step

Triangulation

Tridiagonal

Trigonometric interpolation

Truncation error

Two-state system

Two-step method

Two-variable method

U

Ultra-hyperbolic differential equation

Unimodal

Unitary transformation

Update matrix

Upper triangular matrix

V

Van der Waals

Variable ε

Variance

Vector model

Verhulst

Verlet

Vertex

Virial

Virial coefficient

W

W -matrix

Wave

Wave equation

Wave function

Wave packet

Weak form

Weddle rule

Weight function

Weighted residuals

Windowing function

Z

Z-matrix

Z-transform

Zamann