ANEMLL launches new open-source AI machine learning library

ANEMLL represents a significant open-source initiative to make Large Language Models (LLMs) run efficiently on Apple devices by leveraging the Apple Neural Engine (ANE). This project addresses the growing demand for on-device AI that can operate without cloud connections, providing privacy benefits while enabling AI capabilities on edge devices like iPhones and Macs.

The big picture: ANEMLL provides a complete open-source pipeline for converting and running LLMs on Apple’s specialized AI hardware, enabling private, secure, and efficient on-device inference.

• The project aims to democratize access to on-device AI by simplifying the process of porting Hugging Face models to Apple’s Neural Engine.
• By running models directly on the device instead of in the cloud, ANEMLL prioritizes user privacy and enables AI applications that work without an internet connection.

Key components: The 0.3.0 Alpha release consists of five main tools designed to facilitate Apple Neural Engine development.

• LLM Conversion Tools allow developers to port models from standard formats to ANE-compatible versions.
• Swift Reference Implementation and Python Sample Code provide programming interfaces for different development needs.
• iOS/macOS Sample Applications demonstrate practical implementations for mobile and desktop platforms.
• ANEMLL-BENCH offers a way to evaluate and benchmark model performance on Apple hardware.

Available models: ANEMLL has already converted several popular LLMs that are ready for immediate use on Apple devices.

• The lineup includes LLAMA 3.1 variants (1B and 8B parameter models) which represent Meta’s latest general-purpose LLM architecture.
• DeepSeek and DeepHermes distilled models are also available, offering smaller, more efficient versions of larger models optimized for mobile devices.

Why this matters: On-device AI processing represents a critical evolution for applications requiring privacy, security, and offline functionality.

• By eliminating the need to send data to cloud servers, ANEMLL enables AI applications that can function in environments without reliable connectivity.
• The MIT License ensures the technology remains open and accessible for both commercial and non-commercial development.

Next steps: Interested developers can access the project’s resources through multiple channels including its website, Hugging Face repository, and GitHub page.

• Pre-converted models can be downloaded directly from the Hugging Face repository (huggingface.co/anemll).
• The project appears to have a TestFlight app available for those wanting to test the technology on their Apple devices.