// the find
callstackincubator/ai
On-device LLM execution in React Native with Vercel AI SDK compatibility
A set of React Native packages for running models on the device, one per runtime: Apple's built-in Foundation Models on iOS, llama.rn for GGUF files, and MLC LLM for prebuilt models. Each provider plugs into the Vercel AI SDK, so generateText, streamText, and embed look the same whether inference runs locally or on a server. It is aimed at React Native teams that want offline or privacy-sensitive features and can handle model downloads and native build setup.
The Apple provider needs no model download and no manual linking. It calls the system model through an autolinked native module, which is the lowest-friction path on iOS 26 devices with Apple Intelligence. The README pins AI SDK versions explicitly (0.11 and below track v5, 0.12 and above track v6), which is more useful than most wrappers that leave you guessing when an SDK bump breaks them. The Llama provider exposes download with progress, prepare, and unload as separate steps, so you can free model memory when a screen closes. The DevTools package routes AI SDK telemetry spans into Rozenite, which makes on-device latency debuggable instead of guesswork.
The MLC provider ships a prebuilt runtime for four models. Any other model means recompiling the MLC runtime from source, which puts a native toolchain build in front of anyone who wants something different. Platform coverage is uneven: Apple text generation needs iOS 26 and an Apple Intelligence device, and there is no system-model option on Android, so Android features depend on downloading a model file first. The Llama path does not say how large its GGUF downloads are or what happens on an interrupted download or when storage is short. The README calls this a drop-in replacement but never lists which AI SDK features each on-device runtime supports, such as tool calling or structured output, so you find out at runtime.