A survey of numerical methods for stochastic differential equations

The development of numerical methods for stochastic differential equations has intensified over the past decade. The earliest methods were usually heuristic adaptations of deterministic methods, but were found to have limited accuracy regardless of the order of the original scheme. A stochastic counterpart of the Taylor formula now provides a framework for the systematic investigation of numerical methods for stochastic differential equations. It suggests numerical schemes, which involve multiple stochastic integrals, of higher order of convergence. We shall survey the literature on these and on the earlier schemes in this paper. Our discussion will focus on diffusion processes, but we shall also indicate the extensions needed to handle processes with jump components. In particular, we shall classify the schemes according to strong or weak convergence criteria, depending on whether the approximation of the sample paths or of the probability distribution is of main interest.     

Why the <Emphasis Type="Italic">Euler</Emphasis> scheme in particle tracking is not enough: the shallow-sea pycnocline test case

During the last decades, the Euler scheme was the common “workhorse” in particle tracking, although it is the lowest-order approximation of the underlying stochastic differential equation. To convince the modelling community of the need for better methods, we have constructed a new test case that will show the shortcomings of the Euler scheme. We use an idealised shallow-water diffusivity profile that mimics the presence of a sharp pycnocline and thus a quasi-impermeable barrier to vertical diffusion. In this context, we study the transport of passive particles with or without negative buoyancy. A semi-analytic solutions is used to assess the performance of various numerical particle-tracking schemes (first- and second-order accuracy), to treat the variations in the diffusivity profile properly. We show that the commonly used Euler scheme exhibits a poor performance and that widely used particle-tracking codes shall be updated to either the Milstein scheme or second-order schemes. It is further seen that the order of convergence is not the only relevant factor, the absolute value of the error also is.     

The development of stochastic space transformation and diagrammatic perturbation techniques in subsurface hydrology

This paper develops concepts and methods to study stochastic hydrologic models. Problems regarding the application of the existing stochastic approaches in the study of groundwater flow are acknowledged, and an attempt is made to develop efficient means for their solution. These problems include: the spatial multi-dimensionality of the differential equation models governing transport-type phenomena; physically unrealistic assumptions and approximations and the inadequacy of the ordinary perturbation techniques. Multi-dimensionality creates serious mathematical and technical difficulties in the stochastic analysis of groundwater flow, due to the need for large mesh sizes and the poorly conditioned matrices arising from numerical approximations. An alternative to the purely computational approach is to simplify the complex partial differential equations analytically. This can be achieved efficiently by means of a space transformation approach, which transforms the original multi-dimensional problem to a much simpler unidimensional space. The space transformation method is applied to stochastic partial differential equations whose coefficients are random functions of space and/or time. Such equations constitute an integral part of groundwater flow and solute transport. Ordinary perturbation methods for studying stochastic flow equations are in many cases physically inadequate and may lead to questionable approximations of the actual flow. To address these problems, a perturbation analysis based on Feynman-diagram expansions is proposed in this paper. This approach incorporates important information on spatial variability and fulfills essential physical requirements, both important advantages over ordinary hydrologic perturbation techniques. Moreover, the diagram-expansion approach reduces the original stochastic flow problem to a closed set of equations for the mean and the covariance function.     

The development of stochastic space transformation and diagrammatic perturbation techniques in subsurface hydrology

This paper develops concepts and methods to study stochastic hydrologic models. Problems regarding the application of the existing stochastic approaches in the study of groundwater flow are acknowledged, and an attempt is made to develop efficient means for their solution. These problems include: the spatial multi-dimensionality of the differential equation models governing transport-type phenomena; physically unrealistic assumptions and approximations and the inadequacy of the ordinary perturbation techniques. Multi-dimensionality creates serious mathematical and technical difficulties in the stochastic analysis of groundwater flow, due to the need for large mesh sizes and the poorly conditioned matrices arising from numerical approximations. An alternative to the purely computational approach is to simplify the complex partial differential equations analytically. This can be achieved efficiently by means of a space transformation approach, which transforms the original multi-dimensional problem to a much simpler unidimensional space. The space transformation method is applied to stochastic partial differential equations whose coefficients are random functions of space and/or time. Such equations constitute an integral part of groundwater flow and solute transport. Ordinary perturbation methods for studying stochastic flow equations are in many cases physically inadequate and may lead to questionable approximations of the actual flow. To address these problems, a perturbation analysis based on Feynman-diagram expansions is proposed in this paper. This approach incorporates important information on spatial variability and fulfills essential physical requirements, both important advantages over ordinary hydrologic perturbation techniques. Moreover, the diagram-expansion approach reduces the original stochastic flow problem to a closed set of equations for the mean and the covariance function.     

Automated first and second order moment equations for a set of stochastic differential equations of type AŻ+BZ=C(t)

The generation of the second and higher order moment equations for a set of stochastic differential equations based on Ito's differential lemma is difficult, even for small system of equations. From the knowledge of the statistical properties of the Gaussian white noises associated with the parameters and input coefficients of a set of stochastic differential equations of typeA.+B.Z=C(t), a way to automatically generate the second order moment equations in a computer is presented in this paper. The resulting set of first and second order moment equations is also presented in the same state-space form of the original set of stochastic differential equations through a vectorization of the correlation matrix, which takes advantage of its symmetry. The procedure involved here avoids the inversion of matrixA to apply Ito's differential lemma. Therefore, the presented numerical implementation reduces the computational effort required in the formulation and solution of the moment equations. Moreover, other robust and efficient numerical deterministic integration schemes can be equally applied to the solution of the moment equations.     

Efficient simulation of geothermal processes in heterogeneous porous media based on the exponential Rosenbrock–Euler and Rosenbrock-type methods

Simulation of geothermal systems is challenging due to coupled physical processes in highly heterogeneous media. Combining the exponential Rosenbrock–Euler method and Rosenbrock-type methods with control-volume (two-point flux approximation) space discretizations leads to efficient numerical techniques for simulating geothermal systems. In terms of efficiency and accuracy, the exponential Rosenbrock–Euler time integrator has advantages over standard time-discretization schemes, which suffer from time-step restrictions or excessive numerical diffusion when advection processes are dominating. Based on linearization of the equation at each time step, we make use of matrix exponentials of the Jacobian from the spatial discretization, which provide the exact solution in time for the linearized equations. This is at the expense of computing the matrix exponentials of the stiff Jacobian matrix, together with propagating a linearized system. However, using a Krylov subspace or Léja points techniques make these computations efficient.The Rosenbrock-type methods use the appropriate rational functions of the Jacobian of the ODEs resulting from the spatial discretization. The parameters in these schemes are found in consistency with the required order of convergence in time. As a result, these schemes are A-stable and only a few linear systems are solved at each time step. The efficiency of the methods compared to standard time-discretization techniques are demonstrated in numerical examples.     

Stochastic space transforms in subsurface hydrology — Part 2: Generalized spectral decompositions and plancherel representations

In earlier publications, certain applications of space transformation operators in subsurface hydrology were considered. These operators reduce the original multi-dimensional problem to the one-dimensional space, and can be used to study stochastic partial differential equations governing groundwater flow and solute transport processes. In the present work we discuss developments in the theoretical formulation of flow models with space-dependent coefficients in terms of space transformations. The formulation is based on stochastic Radon operator representations of generalized functions. A generalized spectral decomposition of the flow parameters is introduced, which leads to analytically tractable expressions of the space transformed flow equation. A Plancherel representation of the space transformation product of the head potential and the log-conductivity is also obtained. A test problem is first considered in detail and the solutions obtained by means of the proposed approach are compared with the exact solutions obtained by standard partial differential equation methods. Then, solutions of three-dimensional groundwater flow are derived starting from solutions of a one-dimensional model along various directions in space. A step-by-step numerical formulation of the approach to the flow problem is also discussed, which is useful for practical applications. Finally, the space transformation solutions are compared with local solutions obtained by means of series expansions of the log-conductivity gradient.     

THE EFFECTS OF INACCURATE INPUT PARAMETERS ON DEPOSITION OF SUSPENDED SEDIMENT

I.~OntjcnONEngineeringPredictionsofsedimenttransPOrtinopen-Channelsystems,especiallyifunsteadyflowconditionmustbeconsidered,isfraughtwithambigUityduetospatialandtemporalvariability,measurementerrors,lumtedsamplingoftheParameters,bounceandscallconditionS,andsink/sourcetenns.Duringrecentyearsmanysophisticatedmathematicalmodelshavebecomeavailabletocalculateunsteadyflowinone,tWoorthreedimensionS.ThesemodelswillbecoupledwithtranSportequationSforbedloadorsuSpendedloadtocalculateforexamplelongt…     

Accuracy of numerical simulations of contaminant transport in heterogeneous aquifers: A comparative study

This work deals with a comparison of different numerical schemes for the simulation of contaminant transport in heterogeneous porous media. The numerical methods under consideration are Galerkin finite element (GFE), finite volume (FV), and mixed hybrid finite element (MHFE). Concerning the GFE we use linear and quadratic finite elements with and without upwind stabilization. Besides the classical MHFE a new and an upwind scheme are tested. We consider higher order finite volume schemes as well as two time discretization methods: backward Euler (BE) and the second order backward differentiation formula BDF (2). It is well known that numerical (or artificial) diffusion may cause large errors. Moreover, when the Péclet number is large, a numerical code without some stabilising techniques produces oscillating solutions. Upwind schemes increase the stability but show more numerical diffusion. In this paper we quantify the numerical diffusion for the different discretization schemes and its dependency on the Péclet number. We consider an academic example and a realistic simulation of solute transport in heterogeneous aquifer. In the latter case, the stochastic estimates used as reference were obtained with global random walk (GRW) simulations, free of numerical diffusion. The results presented can be used by researchers to test their numerical schemes and stabilization techniques for simulation of contaminant transport in groundwater.     

Stochastic analysis of multiphase flow in porous media: II. Numerical simulations

The first paper (Chang et al., 1995b) of this two-part series described the stochastic analysis using spectral/perturbation approach to analyze steady state two-phase (water and oil) flow in a, liquid-unsaturated, three fluid-phase porous medium. In this paper, the results between the numerical simulations and closed-form expressions obtained using the perturbation approach are compared. We present the solution to the one-dimensional, steady-state oil and water flow equations. The stochastic input processes are the spatially correlated logk where k is the intrinsic permeability and the soil retention parameter, . These solutions are subsequently used in the numerical simulations to estimate the statistical properties of the key output processes. The comparison between the results of the perturbation analysis and numerical simulations showed a good agreement between the two methods over a wide range of logk variability with three different combinations of input stochastic processes of logk and soil parameter . The results clearly demonstrated the importance of considering the spatial variability of key subsurface properties under a variety of physical scenarios. The variability of both capillary pressure and saturation is affected by the type of input stochastic process used to represent the spatial variability. The results also demonstrated the applicability of perturbation theory in predicting the system variability and defining effective fluid properties through the ergodic assumption.     

A review on stochastic differential equations for applications in hydrology

Fundamentals of the theory of stochastic calculus and stochastic differential equations (SDE's) which are finding increasing application in water resources engineering are reviewed. The basics of probability theory, mean square calculus and the Wiener, white Gaussian and compound Poisson processes are given in preparation for a discussion of the general Itô SDE with drift, diffusion and jump discontinuity terms driven by Gaussian white noise and compound Poissionian impulses. Also discussed are stochastic integration and the derivation of moment equations via the Itô differential rule. The lierature of SDE's is reviewed with an emphasis on the more accessible sources.     

Divergence of solutions to perturbation-based advection-dispersion moment equations

Moment equation methods are popular and powerful tools for modeling transport processes in randomly heterogeneous porous media, but the application of these methods to advection-dispersion equations often leads to erroneous oscillations. Perturbative methods, required to close systems of moment equations, become inaccurate for large perturbations; however, little quantitative theory exists for determining when this occurs for advection-dispersion equations. We consider three different methods (asymptotic approximation, Eulerian truncation, and iterative solution) for closing and solving advection-dispersion moment equations describing transport in stratified porous media with random permeability. We obtain approximate analytical expressions for time above which the asymptotic approximation to the mean diverges, in particular quantifying the impact that dispersion has on delaying—but not eliminating—divergence. We demonstrate that Eulerian truncation and iterative solution methods do not eliminate divergent behavior either. Our divergence criteria provide a priori estimates that signal a warning to the practitioner of stochastic advection-dispersion equations to carefully consider whether to apply perturbative approaches.     

Analysis of a numerical solution to the one-dimensional stochastic convection equation

Under certain conditions the concentration and flux of a substance moving in a stochastic flow field are described by the stochastic convection equation. A numerical method for solving the one-dimensional problem is studied here. The differential operator is replaced by a discrete linear operator based on finite differences. The resulting system of stochastic equations is then replaced by a system of equations whose solution is the mean concentration or mean flux. This final system is analysed and conditions for a stable numerical solution are obtained. Finally, numerical examples are given and are compared to an approximate analytical solution to the stochastic convection equation.     

Exact transverse macro dispersion coefficients for transport in heterogeneous porous media

We study transport through heterogeneous media. We derive the exact large scale transport equation. The macro dispersion coefficients are determined by additional partial differential equations. In the case of infinite Peclet numbers, we present explicit results for the transverse macro dispersion coefficients. In two spatial dimensions, we demonstrate that the transverse macro dispersion coefficient is zero. The result is not limited on lowest order perturbation theory approximations but is an exact result. However, the situation in three spatial dimensions is very different: The transverse macro dispersion coefficients are finite – a result which is confirmed by numerical simulations we performed.     

Wavelet-vaguelette decomposition of spatiotemporal random fields

A wavelet-based orthogonal decomposition of the solution to stochastic differential/pseudodifferential equations of parabolic type is derived in the cases of random initial conditions and random forcing. The family of spatiotemporal models considered can represent anomalous diffusion processes when the spatial operator involved is a fractional or multifractional pseudodifferential operator. The results obtained are applied to the generation of the sample paths of Gaussian spatiotemporal random fields in the family studied.     

Diagrammatic solutions for hydraulic head moments in 1-D and 2-D bounded domains

We present a diagrammatic method for solving stochastic 1-D and 2-D steady-state flow equations in bounded domains. The diagrammatic method results in explicit solutions for the moments of the hydraulic head. This avoids certain numerical constraints encountered in realization-based methods. The diagrammatic technique also allows for the consideration of finite domains or large fluctuations, and is not restricted by distributional assumptions. The results of the method for 1-D and 2-D finite domains are compared with those obtained through a realization-based approach. Mean and variance of head are well reproduced for all log-conductivity variances inputted, including those larger than one. The diagrammatic results also compare favorably to hydraulic head moments derived by standard analytic methods requiring a linearized form of the flow equation.     

Stochastic perturbation analysis of groundwater flow. Spatially variable soils,semi-infinite domains and large fluctuations

As is well known, a complete stochastic solution of the stochastic differential equation governing saturated groundwater flow leads to an infinite hierarchy of equations in terms of higher-order moments. Perturbation techniques are commonly used to close this hierarchy, using power-series expansions. These methods are applied by truncating the series after a finite number of terms, and products of random gradients of conductivity and head potential are neglected. Uncertainty regarding the number or terms required to yield a sufficiently accurate result is a significant drawback with the application of power series-based perturbation methods for such problems. Low-order series truncation may be incapable of representing fundamental characteristics of flow and can lead to physically unreasonable and inaccurate solutions of the stochastic flow equation. To support this argument, one-dimensional, steady-state, saturated groundwater flow is examined, for the case of a spatially distributed hydraulic conductivity field. An ordinary power-series perturbation method is used to approximate the mean head, using second-order statistics to characterize the conductivity field. Then an interactive perturbation approach is introduced, which yields improved results compared to low-order, power-series perturbation methods for situations where strong interactions exist between terms in such approximations. The interactive perturbation concept is further developed using Feynman-type diagrams and graph theory, which reduce the original stochastic flow problem to a closed set of equations for the mean and the covariance functions. Both theoretical and practical advantages of diagrammatic solutions are discussed; these include the study of bounded domains and large fluctuations.     

Diagrammatic solutions for hydraulic head moments in 1-D and 2-D bounded domains

We present a diagrammatic method for solving stochastic 1-D and 2-D steady-state flow equations in bounded domains. The diagrammatic method results in explicit solutions for the moments of the hydraulic head. This avoids certain numerical constraints encountered in realization-based methods. The diagrammatic technique also allows for the consideration of finite domains or large fluctuations, and is not restricted by distributional assumptions. The results of the method for 1-D and 2-D finite domains are compared with those obtained through a realization-based approach. Mean and variance of head are well reproduced for all log-conductivity variances inputted, including those larger than one. The diagrammatic results also compare favorably to hydraulic head moments derived by standard analytic methods requiring a linearized form of the flow equation.     

A faster simulation method for the stochastic response of hysteretic structures subject to earthquakes

A semi-analytical forward-difference Monte Carlo simulation procedure is proposed for the determination of the lower order statistical moments and the joint probability density function of the stochastic response of hysteretic non-linear multi-degree-of-freedom structural systems subject to nonstationary gaussian white noise excitation, as an alternative to conventional direct simulation methods. The method generalizes the so-called Ermak-Allen algorithm developed for simulation applications in molecular dynamics to structural hysteretic systems. The proposed simulation procedure rely on an assumption of local gaussianity during each time step. This assumption is tantamount to various linearizations of the equations of motion. The procedure then applies an analytical convolution of the excitation process, hereby reducing the generation of stochastic processes and numerical integration to the generation of random vectors only. Such a treatment offers higher rates of convergence, faster speed and higher accuracy. The procedure has been compared to the direct Monte Carlo simulation procedure, which uses a fourth-order Runge-Kutta scheme with the white noise process approximated by a broad band Ruiz-Penzien broken line process. The considered system was a multi-dimenensional hysteretic shear frame, where the constitutive equation of the hysteretic shear forces are described by a bilinear hysteretic model. The comparisons show that significant savings in computer time and accuracy can be achieved.