PySINDy Examples

This directory showcases the following examples of PySINDy in action.

Feature overview

This notebook gives an almost exhaustive overview of the different features available in PySINDy. It’s a good reference for how to set various options and work with different types of datasets.

Introduction to SINDy

We recommend that people new to SINDy start here. We give a gentle introduction to the SINDy method and how different steps in the algorithm are represented in PySINDy. We also show how to use PySINDy to learn a model for a simple linear differential equation.

Original paper

This notebook uses PySINDy to reproduce the examples in the original SINDy paper. Namely, it applies PySINDy to the following problems:

• Linear 2D ODE

• Cubic 2D ODE

• Linear 3D ODE

• Lorenz system

• Fluid wake behind a cylinder

• Logistic map

• Hopf system

Scikit-learn compatibility

Shows how PySINDy interfaces with various Scikit-learn objects.

• Cross-validation

• Sparse regressors

Differentiation

Explore the differentiation methods available in PySINDy on pure differentiation problems and as components in the SINDy algorithm.

Deeptime compatibility

See a demonstration of PySINDy objects designed to conform to the Deeptime API.

Plasma physics

Use the ConstrainedSR3 optimizer to build a constrained model for the temporal POD modes of a plasma simulation.

Trapping SINDy

This notebook applies the TrappingSR3 optimizer to various canonical fluid systems., proposed in this paper: Kaptanoglu, Alan A., et al. “Promoting global stability in data-driven models of quadratic nonlinear dynamics.” Physical Review Fluids 6.9 (2021): 094401. A preprint is found here https://arxiv.org/abs/2105.01843.

SINDyPI

This notebook applies the SINDyPI optimizer to a simple implicit ODE and was originally proposed in this paper: Kaheman, Kadierdan, J. Nathan Kutz, and Steven L. Brunton. “SINDy-PI: a robust algorithm for parallel implicit sparse identification of nonlinear dynamics.” Proceedings of the Royal Society A 476.2242 (2020): 20200279.

PDEFIND

This notebook applies the PDEFIND algorithm (SINDy for PDE identification) to a number of PDEs, and was originally proposed in this paper: Rudy, Samuel H., et al. “Data-driven discovery of partial differential equations.” Science Advances 3.4 (2017): e1602614.

Greedy Algorithms

This notebook uses the step-wise sparse regression (SSR) and forward-regression orthogonal least-squares (FROLS) algorithms, which are greedy algorithms that iteratively truncate (or add) one nonzero coefficient at each algorithm iteration.

Weak formulation SINDy

This notebook uses SINDy to identify the weak-formulation of a system of ODEs or PDEs, adding significant robustness against noise in the data.

Model ensembles

This notebook uses sub-sampling of the data and sub-sampling of the SINDy library to generate many models, and the user can choose how to average or otherwise combine these models together. This tends to make SINDy more robust against noisy data.

Cavity flow

Demonstrates the use of SINDy to learn a model for the quasiperiodic dynamics in a shear-driven cavity at Re=7500, following Callaham, Brunton, and Loiseau (2021), preprint available here https://arxiv.org/pdf/2106.02409.

Full table of contents

• Differentiators in PySINDy
  • Compare differentiation methods directly
    • Sine
    • Absolute value
  • Compare differentiators when used in PySINDy
    • Linear oscillator
    • Lorenz system
• SSR and FROLs (the greedy algorithms!) examples
• Define some functions for plotting performance as the greedy algorithms progress
• Note that the usage is a bit different because we save all the sparse models and we choose our favorite one afterwards. Below we show that we can track the MSE between the predicted and true derivative on a testing trajectory as the algorithm iterates, and then choose the model with the minimum MSE.
  • Repeat the SSR fitting with the lowest model residual method
  • FROLS greedy algorithm
• Let’s compare all three methods as the noise steadily increases, cross-validated over 10 noise instantiations
• Summary
• Quasiperiodic cavity flow
  • 1. POD analysis
  • 2. Galerkin projection
  • 3. Naive SINDy model
  • 4. Dynamic mode decomposition
  • 5. Nonlinear correlations
• 6. Manifold SINDy
• SINDy-PI Feature Overview
• Example to show library functionality
  • Solve 1D Michelson-Menten enzyme model as first demonstration
  • Okay, so how to choose which model to use?
  • Now we need to reformat all the feature strings for conversion into symbolic variables
  • Check all the model fits produce sensible models
  • Now we will:
  • We will now try a 2D example, the bacterial competence system (Mangan et al. 2016)
    • SINDy-PI model is not good here, but we proceed anyways to illustrate how to do the symbolic solve + integration in more than 1 dimensions.
    • We can simulate the system we identified but tends to be unstable – need more memory to train on additional trajectories.
• Working with Scikit-learn
  • Setup
  • Cross-validation
    • Cross-validation with TimeSeriesSplit
    • Cross-validation without TimeSeriesSplit
  • Sparse optimizers
• Summary of PySINDy YouTube tutorial videos
  • Load in dependencies and define a number of plotting functions for use later
    • Lorenz system
• Part 1: How to choose algorithm hyperparameters such as \(\lambda\)?
• Part 2: How to make SINDy more robust for system identification?
• Part 3a: Implicit ODEs
• Part 3b: Identifying partial differential equations (PDEs)
• Part 4: How to choose a regularizer and a sparse regression algorithm?
• Part 5: How to build complex candidate libraries
• An introduction to Sparse Identification of Nonlinear Dynamical systems (SINDy)
  • Main ideas
  • Approximation problem
  • Structure of PySINDy
  • Example
• Trapping SINDy
  • The trapping algorithm only applies to fluid and plasma flows with energy-preserving, quadratic nonlinear structure, so we need to explicitly constrain the coefficients to conform to this structure.
• Mean field model
• Atmospheric oscillator model
• Lorenz model
• Visualizing the trapping region for Lorenz
• MHD model
• Forced Burger’s Equation
• Von Karman shedding behind circular cylinder, Re = 100
• Comparing 3D trajectories with the POD-Galerkin model
• Energy plot illustrating the transient and long-time energy conservation
• Plot \(u(x,y,t)\) reconstructions and modes
• Original Paper: Sparse Identification of Nonlinear Dynamical systems (SINDy)
  • Simple illustrative systems
    • Linear 2D ODE
    • Cubic 2D ODE
    • Linear 3D ODE
  • Lorenz system (nonlinear ODE)
    • Measure states \(x\) and derivatives \(\dot{x}\)
    • Measure only the states \(x\), compute the derivatives
    • Compare the errors in each case
  • Fluid wake behind a cylinder (nonlinear PDE)
  • Bifurcations and normal forms
    • Logistic map
    • Hopf normal form
• PDEFIND Feature Overview
• Define Algorithm 2 from Rudy et al. (2017)
  • Using the new PDE library functionality is straightforward
• Test PDE functionality on the 1D kdV equation
• Test spatial derivative computations
• Test PDE functionality on the Kuramoto-Sivashinsky equation
• Test PDE functionality on Burgers’ equation
• Test PDE functionality on 2D Reaction-Diffusion system
  • Below, we show that ensemble methods can generate excellent model identifications on 1/3 the temporal data.
• Test PDE functionality on 3D Reaction-Diffusion system
  • Despite the very low resolution, can do quite a decent job with the system identification!
• Ensembling Feature Overview
  • Ensemble methods
    • The default number of models to generate is 20, and the default sampling strategy is to sample the entire time base with replacement, so that on average 60% or so of the data is seen.
    • Default is bragging - the median coefficients over all the models is printed by model.print(). However, model.simulate and model.predict use the coefficients of the last model that was fitted! To avoid confusion, we recommend using model.coef_list everywhere and doing all the post-processing separately.
  • Library ensemble methods (V3)
  • Now do both types of ensembling with multiple candidate drops (V4)!
  • Alternatives to averaging (bagging, P1) or taking the median (bragging, P2) of the models:
  • Repeat ensembling but now make ~1000 models and plot the distributions of coefficients
  • Plot 95% percentile bounds with mean and median test trajectories
  • Variant of library ensembling (P5):
  • Generate achievability plots from Maddu et al. 2019
  • Test ensemble functionality with weak-PDE functionality, using the Kuramoto-Sivashinsky equation. For understanding weak form use, please see the Example 14 Jupyter Notebook.
• Feature Overview
  • A note on solve_ivp vs odeint before we continue
    • Basic usage
  • Train the model
  • Assess results on a test trajectory
  • Predict derivatives with learned model
  • Simulate forward in time
    • Different forms of input data
  • Single trajectory, pass in collection times
  • Single trajectory, set default timestep
  • Single trajectory, pass in pre-computed derivatives
  • Multiple trajectories
  • Multiple trajectories, different lengths of time
  • Discrete time dynamical system (map)
  • Pandas DataFrame
    • Optimization options
  • STLSQ - change parameters
  • STLSQ - verbose (print out optimization terms at each iteration)
  • SR3
  • SR3 - with trimming
  • SR3 regularizers
  • SR3 - with variable thresholding
  • ConstrainedSR3
  • SSR (greedy algorithm)
  • FROLS (greedy algorithm)
  • LASSO
  • Ensemble methods
    • Differentiation options
  • Pass in pre-computed derivatives
  • Drop end points from finite difference computation
  • Differentiation along specific array axis
  • Smoothed finite difference
  • More differentiation options
    • Feature libraries
  • Custom feature names
  • Custom left-hand side when printing the model
  • Customize polynomial library
  • Fourier library
  • Fully custom library
  • Fully custom library, default function names
  • Identity library
  • Concatenate two libraries
  • Tensor two libraries together
  • Generalized library
    • SINDy with control (SINDYc)
  • Train the model
  • Assess results on a test trajectory
  • Predict derivatives with learned model
  • Simulate forward in time (control input function known)
  • Simulate forward in time (unknown control input function)
    • Implicit ODEs
• PDEFIND Feature Overview
  • Weak formulation system identification improves robustness to noise.
  • GeneralizedLibrary
• Plasma physics
  • Setup
  • Learning SINDy models for POD mode dynamics
    • Unconstrained SR3 algorithm
    • Constrained SR3 algorithm
    • Constrained SR3 algorithm with term-dependent thresholding
      • Encourage linear terms
      • Allow some quadratic terms
      • Show off quadratic constraints
      • Mapping out the (r,:math:lambda) model landscape
• Weak/Integral SINDy Feature Overview
  • Test Weak PDE Library setup time
• Test weak form ODE functionality on Lorenz equation
• Test weak form PDE functionality on Burgers’ equation with 20% noise
• Test weak form PDE functionality on nonlinear diffusion with mixed function-derivative terms
• Test weak-formulation PDE functionality on the Kuramoto-Sivashinsky equation
• Test weak form PDE functionality on Reaction-Diffusion system
• Test weak form PDE functionality on 3D Reaction-Diffusion system
• Deeptime objects

Differentiators in PySINDy

SSR and FROLs (the greedy algorithms!) examples

gallery_examples_14_cavity_flow.ipynb

SINDy-PI Feature Overview

Working with Scikit-learn

Summary of PySINDy YouTube tutorial videos

An introduction to Sparse Identification of Nonlinear Dynamical systems (SINDy)

Trapping SINDy

Original Paper: Sparse Identification of Nonlinear Dynamical systems (SINDy)

PDEFIND Feature Overview

Ensembling Feature Overview

Feature Overview

Plasma physics

Weak/Integral SINDy Feature Overview

Deeptime objects