// the find
apache/arrow-rs
Official Rust implementation of Apache Arrow
The official Rust implementation of Apache Arrow and Apache Parquet, covering in-memory columnar arrays, Parquet read/write, Arrow Flight IPC, CSV/Avro/JSON formats, and casting/compute kernels. This is the foundational data layer underneath DataFusion and most Rust-based analytics tooling. If you're building anything data-intensive in Rust that touches columnar formats, this is what you use.
- Well-structured multi-crate workspace with clear separation of concerns — arrow-buffer, arrow-array, arrow-cast, arrow-arith etc. are independently versioned and composable, so you don't drag in the whole stack if you only need low-level buffers.
- Parquet support is production-grade: async readers, predicate pushdown, column projection, row group filtering, and encryption are all present with active maintenance.
- Release cadence is transparent and disciplined — monthly releases, quarterly breaking changes, explicit deprecation policy with at least two major versions before removal. Downstream crates can actually plan upgrades.
- Miri CI job and integration tests against the Arrow cross-language test suite give real confidence in memory safety and cross-implementation correctness, not just unit test coverage.
- The API surface is enormous and the learning curve is steep. The type system for arrays (typed generics, phantoms, multiple builder patterns) is correct but intimidating — expect to spend real time in docs before writing anything non-trivial.
- Arrow Avro crate (arrow-avro) is relatively new and not fully documented; schema evolution support and some complex union types may surprise you in production.
- The major-version-every-quarter policy means projects that can't update frequently will accumulate breaking changes. The ecosystem (DataFusion, connectors, etc.) tends to track latest closely, so staying a few versions behind can leave you stranded with incompatible crates.
- Flight SQL server implementation is fairly thin scaffolding — you implement the trait methods yourself with minimal guidance on session management, auth patterns, or state handling, so building a real server still requires significant design work.