Reusable experiment scaffolding for time-series ML: standardized configs, runs, metrics, and plots so model comparisons are fair and repeatable.
Across projects, the same problem kept repeating: experiments become hard to reproduce once there are many datasets, models, and hyperparameters. This library is a practical solution: a consistent way to launch experiments, capture configuration, compute metrics, and export results.
The least glamorous project I have built, and the one that makes every other one reproducible.
Why build experiment infrastructure?
The same failure kept happening across projects: three months in, with a dozen datasets, several model variants, and a wall of hyperparameters, nobody can say exactly how a given plot was produced. This library exists to make that situation impossible.
It is a deliberately boring piece of software, and that is the point — reproducibility is not a feature you add at the end, it is a constraint you build in from the first run.
What a run looks like
Every experiment is one command that produces a self-contained run folder: the exact config that generated it, logs, computed metrics, and plots, all in a standardized layout. Two runs are always comparable because they were produced the same way, not because someone remembered to keep them consistent.
Figure 1 — One config, one command, one auditable run folder — which is what makes comparing run A against run B honest rather than approximate.
A library of models behind one harness
The infrastructure grew out of a concrete need: comparing very different model families on the same time-series problem without the comparison quietly cheating. So alongside the run harness there is a library of adaptive and kernel filters, plus a configurable neural-network toolkit, all driven through one experiment runner.
Because every model is invoked the same way, a kernel adaptive filter and a neural network are held to identical signals and identical metrics — which is the only way a comparison between them means anything.
Kernel adaptive filters
QKLMS, KRLS, KRR, and Gaussian-process regression for nonlinear prediction.
Functional / Wiener
Functional Wiener (FWF / FWFE), the Functional Kalman Filter (FKF), ASLM, and a Wiener-filter baseline.
Configurable NNs
A hyperparameter-search toolkit including an information-theoretic stacked auto-encoder, run on GPU / HiPerGator.
Figure 2 — Different model families, one harness: every model is benchmarked on the same signals with the same metric, so cross-family comparisons are honest.
Benchmarked on standard chaotic signals
Models are exercised on the classic nonlinear time-series benchmarks, where the right answer is known and difficulty is well understood:
Mackey–Glass — the standard delay-differential benchmark for nonlinear prediction.
Lorenz and Hénon — chaotic dynamical systems with known structure.
Sunspot — a real-world series for a sanity check against messy data.
Reported as train / test MSE, the same way for every model.
Design principles
Capture configuration; never trust memory. Standardize metrics and plotting so results line up across projects. And keep iteration fast — GPU acceleration where it helps — because infrastructure nobody wants to use does not get used.
FAQ
Is this a framework like MLflow or Weights & Biases?
No — it is lighter and research-oriented. It is a model library plus an experiment runner for time-series work, not a general tracking service, though nothing stops you pairing it with one.
Why benchmark on chaotic signals like Mackey–Glass?
They are the standard proving ground for nonlinear prediction: the dynamics are known and the difficulty is well characterized, so a model's error is interpretable rather than a mystery.
Is it open source?
The code is currently private research code rather than a released package.
What I built
A consistent ‘run’ layout (configs, outputs, plots) so results are auditable.
Utilities for sweeping model variants and hyperparameters in a controlled way.
Standardized metrics and plotting helpers for time-series tasks.
Performance-focused utilities (including GPU acceleration where appropriate) to keep iteration cycles short.
Deliverables
One-command experiment runs that produce comparable outputs.
Reusable plotting and reporting utilities for papers and internal reviews.