In detail

Adjoint divergence in long-time-averaged chaotic problems and Least Squares Shadowing

Despite the advantages of turbulence-resolving simulations (DNS, LES, WMLES, DES) and the adjoint method for sensitivity gradients, these two methods do not naturally fit together. Computing sensitivity derivatives of chaotic simulations is a fundamental challenge. Chaotic models are sensitive to initial conditions, a phenomenon popularly known as the "butterfly effect." A small perturbation to the equation can make an exponentially growing difference in its solution, until the differences are saturated by nonlinearity in the model. Solutions to the adjoint equation suffer from the same exponential growth, only that to faithfully capture the sensitivity to infinitesimal perturbations, the adjoint equation is linear, making the growth of its solution unbounded. When the adjoint equation is integrated over a long time period, the divergence of the solution can lead to a computed gradient that is orders of magnitude too large. These types of erroneous and useless gradients are computed from the adjoint solution even when the objective function consists of well-defined deterministic quantities where the chaotic fluctuations are averaged out.

Infinitely long time averages, equivalent to ensemble averages for ergodic systems, can be proven to be differentiable under certain circumstances by Ruelle's linear response theory. Finitely long time averages obtained using computational simulations, however, contain small amounts of sampling error, namely the difference between the finite average and the ensemble average. Such sampling error is trajectory-dependent and can be extremely sensitivity to small perturbations.

Estimating local errors in chaotic, broadband problems

When it comes to error estimation, the most important yet subtle difference between turbulence-resolving simulations and non-chaotic ones (e.g., laminar, RANS) is that the grid controls both the numerical and the modeling errors in coarse-grained problems with broadband spectra (i.e., large eddy simulations and similar). While the numerical errors can be estimated quite readily based on numerical analysis, e.g. from Taylor expansions or through interpolation onto a finer grid followed by residual-evaluation, this is not true for the estimation of the modeling error. A highly simplistic example is the following: imagine an LES of a flow that has a laminar shear layer in one part of the domain. If this LES uses the Smagorinsky model, this model (erroneously) produces a finite eddy viscosity in this laminar shear layer, and hence the local error is affected by the grid. In contrast, if the LES instead uses any of the models that behave correctly in this situation (e.g., Vreman, WALE, ...), the model produces the correct zero eddy viscosity regardless of the grid spacing, and thus the error is not affected by the grid. In other words, physics insights are required to estimate the local errors in LES; numerical analysis alone is not sufficient.