// the find
splitrb/split
:chart_with_upwards_trend: The Rack Based A/B testing framework
Split is a Redis-backed A/B testing library for Rack/Rails apps that handles variant assignment, conversion tracking, and significance testing, with a bundled Sinatra dashboard for managing experiments. It's aimed at a Rails team running a handful of in-house experiments who's fine adding Redis as a dependency rather than paying for Optimizely or LaunchDarkly.
Persistence is genuinely well thought out — separate adapters for cookie, Redis, and session storage, plus a DualAdapter that switches between them based on login state, which solves the anonymous-to-authenticated handoff most homegrown A/B setups get wrong. Assignment isn't locked to one algorithm: WeightedSample, a Whiplash multi-armed bandit, and BlockRandomization are all swappable per-experiment. The experiment lifecycle has real hooks (on_trial, on_trial_complete, on_experiment_reset) and supports goals/metrics so multiple experiments can roll up to one funnel outcome. CI, rubocop, and Code Climate coverage are all wired up, and the project is still getting pushes, not abandoned.
The README itself flags the v4.0+ caching feature as broken across multiple processes (Puma) and multi-server deployments behind a load balancer — exactly the topology most production Rails apps run — and tells you to use it 'with caution.' That's a load-bearing feature shipping with a known data-consistency bug. Redis is a hard dependency with no first-class alternative backend; db_failover just forces everyone into the control variant during an outage, silently corrupting your experiment data for that window. The stats are basic: fixed-threshold z-test buckets (90/95/99%) or beta-distribution simulation, with no protection against peeking or multiple-comparison inflation, despite the README linking Evan Miller's piece on exactly that failure mode. The dashboard ships with zero auth by default — you have to bolt on Rack::Auth::Basic or Warden yourself, so it's easy to accidentally expose a page that can flip live experiments to anyone who finds the route.