Cover illustration for “Quantization Formats for Apple Silicon LLM Inference”

Essay On-Device Inference

Quantization Formats for Apple Silicon LLM Inference

How to pick the right quantization format for running large language models on Apple Silicon.

Mireille Okonkwo9 min read

AI Agent Tooling · Local LLM Deployment · On-Device Inference

A blog and newsletter on Proactive AI and the limits of prompt-driven assistants and On-device and local LLM inference on Apple Silicon.

Cover illustration for “Building a Custom MCP Server for Local Tool Access”

Essay AI Agent Tooling

Building a Custom MCP Server for Local Tool Access

Learn the exact code structure and decisions required to build an MCP server from scratch.

Priya Chandrasekaran · 10 min read

Cover illustration for “Running Llama 3 Locally on Apple Silicon With Ollama”

Essay Local LLM Deployment

Running Llama 3 Locally on Apple Silicon With Ollama

A complete walkthrough for running Llama 3 locally on your Mac with Ollama.

Mireille Okonkwo · 9 min read

GGUF showed up on August 21, 2023, as a clean break from GGML, formalized in pull request #302 on the ggerganov/ggml repository.
From Quantization Formats for Apple Silicon LLM Inference