★ 48,979PythonBSD-3-Clauseupdated Jun 2026
best for anyone who needs the library every other Python data tool (numpy, scikit-learn, matplotlib) already assumes
Pandas is the default because it has to be — numpy, scikit-learn, and matplotlib all assume you're handing them a pandas DataFrame, not a polars one. The groupby/agg/transform pipeline and the time series handling (rolling windows, tz-aware indexes, resampling) are mature in ways that only show up in the edge cases, not the demo.
That maturity is also fifteen years of accumulated decisions. There are three ways to select rows, two index alignment behaviors, and Stack Overflow answers from 2019 that quietly do the wrong thing on current versions. None of it matters until your data outgrows one process — then it matters a lot. Object-dtype string columns eat RAM fast, there's no built-in parallelism, and a groupby on 100 million rows is exactly where polars or DuckDB start looking much better.
Use it because everything else already expects it. Replace it once memory or speed becomes the actual bottleneck, not before.
Our full take →GitHub ↗
★ 38,725RustMITupdated Jun 2026
best for the default pick today if you want pandas-like ergonomics with real lazy-query performance and don't need the old ecosystem
Polars is what you reach for once pandas gets slow and rewriting the pipeline in SQL isn't appealing. The expression API is genuinely well-designed — chains like `col('a').str.split(',').explode().over('group')` read naturally instead of fighting index alignment — and the lazy optimizer plus multi-threading make the speed claims real rather than benchmark theater.
The rough edges show up at the margins. Streaming mode for out-of-core data exists, but coverage has gaps, and an unsupported operation falls back to loading everything into memory silently — the exact failure you were using streaming to avoid. Minor version bumps have broken real pipelines before; teams that pinned 0.19 and jumped to 0.20+ rewrote code over renamed APIs and changed behavior.
The other cost is the ecosystem: libraries that accept DataFrames often still mean pandas specifically, so you'll sometimes `.to_pandas()` your way out of the speed you just gained. If that part of the stack doesn't touch you, polars is the better default today.
Our full take →GitHub ↗
★ 13,931PythonBSD-3-Clauseupdated Sep 2026
best for pandas-style code that no longer fits on one machine and needs to run across a cluster
Dask is for pandas code that has outgrown one machine and needs an actual cluster, not just more cores on the same box — the thing modin promises locally, dask does properly distributed, with array, dataframe, bag, and delayed collections sharing one task graph.
The catch is that there are effectively two dask dataframes living in the same repo right now: the legacy engine and the newer dask_expr planner, with no clear signal which one is the path forward. Parquet — the format most pipelines touch first — has two separate code paths for the same reason, so behavior differences tend to surface in production rather than in tests. The distributed scheduler itself isn't even in this repo; it's a separate package pinned and upgraded in lockstep.
Next to modin, you get a more capable and battle-tested distributed story; next to polars, you're trading single-machine speed for real horizontal scale. Worth it once you genuinely need the cluster, not before.
Our full take →GitHub ↗
★ 10,393PythonApache-2.0updated Feb 2026
best for existing pandas codebases that just need more cores, with no rewrite
Modin's pitch is that you change one import line and existing pandas code runs across cores or a cluster. Next to dask, that promise mostly holds — it's the same pandas API, not a separate DataFrame abstraction with its own semantics to learn.
The honesty problem is in the gaps: API coverage sits around 90% for DataFrame and 88% for Series, and unsupported operations fall back to plain pandas silently. That's the opposite of reassuring — you adopted modin specifically to dodge the slow path, and now you're on it with no warning. The docs also tell you not to switch backends (Ray, Dask, or MPI) after the first operation, because behavior there is undefined, which is an odd thing to admit for something sold as frictionless.
It earns its place when the workload is genuinely CPU-bound across many cores and the code is already pandas. For row-wise apply on small-to-medium data — most people's actual workload — the speedup the README leans on doesn't really show up.
Our full take →GitHub ↗
★ 8,504PythonMITupdated Apr 2026
best for exploring billion-row datasets out-of-core on a single laptop, no cluster required
Vaex solves a narrower problem than the rest of this list: billion-row datasets explored on one laptop, no cluster, via memory-mapped HDF5 or Arrow files. When that's genuinely the situation, the lazy expression system and the C++ aggregation engine deliver — groupby on categorical columns really does hit the billion-rows-per-second range it advertises.
The honest problem is that the project is in maintenance mode. Activity is sporadic, the community channels have gone quiet, and API coverage against pandas has real holes, window functions and multi-index operations especially. There's also an upfront tax converting existing CSV or Parquet files into vaex's native format, and the tooling for that conversion is underdocumented.
Next to dask or modin, vaex needs no cluster at all, which is the whole appeal — but you're adopting a library that isn't being actively maintained to get there, and that's a real tradeoff, not a footnote.
Our full take →GitHub ↗
★ 6,572PythonApache-2.0updated Jun 2026
best for teams who want one dataframe API that compiles down to DuckDB, Snowflake, BigQuery, or a dozen other backends
Ibis solves a different problem than speed: it's for teams tired of rewriting the same transformation logic in pandas, then SQL, then PySpark depending on where the data lives that quarter. Write it once against DuckDB locally, point it at Snowflake or BigQuery in production — in theory.
In practice the abstraction leaks the moment you go past basic filter/group-by/join. Operations that exist on one backend don't exist on another, the compatibility matrix is a maze of footnotes, and an unsupported operation gets you a cryptic exception deep in the compilation layer instead of a clear "this backend doesn't support window functions with RANGE BETWEEN." The DuckDB backend is clearly first-class; several of the twenty-plus others feel contributed once and barely touched since.
If you're permanently on one backend, this buys you very little over writing against that backend directly. It earns its keep only when the backend genuinely changes under you.
Our full take →GitHub ↗
★ 5,562RustApache-2.0updated Jun 2026
best for pipelines mixing images, audio, or video with ordinary tabular columns
Daft fills a gap none of the others here do: pipelines where images, audio, or video sit next to ordinary tabular columns, with typed, lazily decoded columns instead of opaque Python objects. Catalog support — Iceberg, Delta Lake, Unity Catalog, and more — is unusually broad for a project this size, and the AI provider layer gives OpenAI, vLLM, and others a consistent interface.
The Rust core is a real advantage, but it stops mattering the moment you load a GPU model — inference still runs through Python UDFs, so the performance story gets murkier exactly where ML pipelines spend most of their time. Distributed mode needs Ray, a substantial operational dependency on its own, and the project hasn't declared a stable v1 surface yet, so the catalog and AI modules can still change under you.
If your data is purely structured — no images, no audio — polars is faster, far more established, and doesn't ask you to run Ray. Daft only earns its place when the multimodal part is real.
Our full take →GitHub ↗
★ 5,062TypeScriptMITupdated Apr 2026
best for JavaScript/Node projects that want a pandas-like API without bringing Python into the stack
Danfo.js exists for one reason: you're in a Node or browser codebase and don't want to bring Python into it just for tabular work. The method names — loc, iloc, groupby, merge, the encoders and scalers — map closely enough to pandas that a notebook translates almost line for line, and handing a DataFrame to TensorFlow.js via `.tensor` is a nice touch if a model is the next step.
It's the least mature project on this list by a wide margin next to pandas or polars. The codebase is split across three separate source trees — base, browser, and node — each with its own package.json, so a fix in one can quietly miss the others. The README claims speed without a single benchmark number, despite a performance-test.js sitting right there in the repo. The last push was about six months ago, which isn't fatal, but check the open issues before depending on specific pandas-equivalent behavior staying put.
If your stack is already Python, nothing here beats pandas or polars directly. Danfo.js only makes sense when leaving JavaScript isn't an option.
Our full take →GitHub ↗
// questions people ask
Before you choose
Is polars actually faster than pandas, or is that marketing?
It's real. The lazy query optimizer and multi-threaded execution show up in benchmarks and in daily use — import time alone is 70ms versus pandas' 520ms — and the gap widens further on large groupbys. The catch is the ecosystem: plenty of libraries still expect a pandas DataFrame specifically, so you'll sometimes convert back and lose the speedup.
What's the actual difference between dask and modin for scaling pandas code?
Modin keeps the exact pandas API and parallelizes it underneath via Ray, Dask, or MPI — it's the closer thing to a one-line swap. Dask gives you its own chunked dataframe and array abstractions with a real task graph and planner, which is more powerful but means learning a slightly different API. Start with modin for minimal code changes; move to dask when you need genuine distributed scale.
Do I need any of these if my data fits comfortably in memory?
No. Pandas is still the right call for anything that fits in RAM on one machine without a fight. Reach for polars once pandas gets slow, and only look at dask, modin, or vaex once the data itself stops fitting.
Is ibis worth adopting if I only use one database backend?
Probably not. Ibis earns its complexity when the same transformation logic needs to run against DuckDB locally and Snowflake or BigQuery in production. If you're permanently on one backend, write against that backend's own interface directly — you'll hit fewer abstraction leaks and clearer error messages.
Pick based on where you're actually stuck today, not where you think you'll be stuck in two years — pandas is still the right default until it specifically isn't. We cover picks like these every week in the finds.dev email if you want more before you commit to one.