Online Probabilistic Model Identification Using Adaptive Recursive MCMC
ARMCMC solves the problem of online Bayesian system identification, enabling control systems to continuously update their world models as operating conditions shift without requiring expensive batch retraining. The recursive posterior update provides a principled alternative to periodic retraining - relevant to AI systems deployed in dynamically changing regulated environments where stale models carry real operational risk. Published at IJCNN 2023.
Classical system identification assumes you collect data first, then fit a model. In practice, the environment changes - robot dynamics shift as components wear, physical conditions vary, and the system encounters new operating regimes. Online system identification solves this by continuously updating the model as data arrives.
ARMCMC (Adaptive Recursive Markov Chain Monte Carlo) provides a principled Bayesian approach to this problem: rather than maintaining a point estimate of system parameters, it maintains a probability distribution over them, updated recursively as each new observation arrives.
The Identification Problem
When a robot or control system interacts with the world, it does so according to some underlying model of dynamics - masses, friction coefficients, spring constants, or whatever parameters are appropriate for the domain. Online identification means inferring those parameters from observation, continuously.
The challenge: MCMC-based Bayesian inference is traditionally expensive (many samples required for accurate posterior approximation) and batch-oriented (requires revisiting historical data). Neither property is compatible with real-time online operation.
Approach
ARMCMC addresses this through two mechanisms:
Adaptive proposal distribution: Rather than using a fixed proposal, ARMCMC tunes the MCMC proposal covariance online using the history of accepted samples. This dramatically improves mixing speed as the chain adapts to the local posterior geometry.
Recursive importance weighting: Historical data is summarized as a weighted particle set rather than re-evaluated at each step. New observations update particle weights; periodic resampling prevents weight collapse.
Results
Evaluated on a simulated 2-DOF robot arm with deliberately introduced parameter drift. ARMCMC tracks the true parameter trajectory with bounded error throughout the drift episode, while batch MCMC (rerun periodically) lags by the batch interval and point-estimate methods diverge under sustained drift.
The uncertainty quantification property is practical: the maintained posterior allows the controller to detect when model confidence has degraded to a level that warrants conservative operation or human review.