// the find
jbogard/bulk-writer
Provides guidance for fast ETL jobs, an IDataReader implementation for SqlBulkCopy (or the MySql or Oracle equivalents) that wraps an IEnumerable, and libraries for mapping entites to table columns.
BulkWriter wraps SqlBulkCopy behind an IEnumerable/IAsyncEnumerable interface so you can stream LINQ queries straight into SQL Server without materializing the whole result set in memory. It's from Jimmy Bogard (AutoMapper, MediatR), which explains the clean pipeline abstraction. Aimed at .NET shops doing data migrations or nightly ETL loads where EF Core's change tracker overhead is the bottleneck.
The IDataReader adapter over IEnumerable is the right abstraction — SqlBulkCopy demands IDataReader but nobody wants to implement it manually. The pipeline DSL (StartWith → Project → Transform → WriteTo) handles the common ETL shape without pulling in a heavy framework. IAsyncEnumerable support means you can stream from an upstream source and write downstream without ever blowing up memory on large datasets. Tests run against a real database container, not mocks, which is the correct call for a library whose entire job is database writes.
SQL Server only for meaningful use — the MySql and Oracle equivalents mentioned in the description are not in this repo, so the multi-DB claim is aspirational at best. No built-in error handling for partial failures: if SqlBulkCopy throws mid-stream, you get an exception with no row-level diagnosis and no retry. Column mapping is reflection-based at startup, which works fine, but there's no hook for handling schema drift at runtime (column added to DB, not in your DTO — silent data loss). 242 stars for a six-year-old library from a well-known author suggests the audience is narrow; most people reach for EF Core bulk extension packages instead and this never broke through.