Learned Systems

Benchmarking Learned Systems

Evaluate learned components fairly

Completed

Overview

Fixed workloads let learned systems overfit, so standard benchmarks overstate their benefit. We proposed benchmarks that vary workloads and data, report distributions and outliers in addition to averages, and account for the cost of training learned database components.

Related Publications

Towards a Benchmark for Learned Systems (SMDB 2021)