Infrastructure engineer. Twenty years in systems and Linux; since 2025, focused on local LLM inference — private, on‑premise AI for teams that can’t send their data to someone else’s API.
Writing
Frontier models are extraordinary reasoners — and most of what we ask them to do is reading. How I split the two, put the reading on local hardware, and made agentic AI faster, cheaper, and private.
The real leverage in AI isn’t the model — it’s the system around it. How I build that system on infrastructure you own, and why it becomes a durable advantage.
Work
I design and build ABOS: one AI “Mind” that runs a company’s operations — reading what comes in, acting through tools, watching what changes, and improving — entirely on the company’s own hardware and local models. No data leaves, no external AI vendor in the loop, a human always the final decider. Built from scratch, on Postgres and self-hosted models.
Local LLM serving on a Grace-Blackwell box, tuned for a single interactive agent: quantized MoE models (NVFP4), a speculative-decoding draft model, and an FP8 KV-cache — with OpenHands doing first-pass research before anything reaches Codex or Claude.
About
CTO · Agentic AI & AI Platform Architecture · Edge LLMs, RAG, Persistent Memory, Workflow Automation, PostgreSQL/pgvector, NATS JetStream
I design and build production AI platforms that combine LLM reasoning with deterministic business execution.
Over the past year I architected and deployed an agentic AI Business Operating System for running real business processes with AI while keeping workflows, authorization, auditability, and external actions under deterministic control. Rather than building another chatbot, I designed a modular operating layer where AI can research, interpret, plan, qualify leads, prepare proposals, monitor external information, and recommend actions — while the platform governs what is actually allowed to happen.
A major part of the work has been local and edge AI: running private LLMs close to customer data, controlled context assembly, RAG, persistent memory, and semantic retrieval using PostgreSQL, pgvector, and Snowflake Arctic Embed L v2.0 embeddings. I designed safeguards around embedding provenance and vector-space compatibility so retrieval remains reliable even as models evolve.
I also created a modular architecture for rapidly adding new clients and industries through reusable business Packs, workflow and policy runtimes, provider-neutral capabilities, secure API/tool execution, and isolated client “Blade” deployments. The platform uses NATS JetStream for durable event-driven processing, Vault for secrets, self-hosted identity, immutable Docker/OCI releases, virtualization, and CI/CD infrastructure designed for repeatable production deployment.
The key innovation is the separation of probabilistic AI reasoning from deterministic execution. LLMs can understand and propose; deterministic runtimes enforce tenant isolation, workflow state, policies, approvals, transactions, tool permissions, side effects, and audit history. This makes agentic systems more stable, explainable, and suitable for operating real businesses.
Core areas: Agentic AI · AI Platform Architecture · LLM Orchestration · Edge AI · Local LLMs · RAG · Persistent Memory · Embeddings · Vector Search · PostgreSQL · pgvector · Snowflake Arctic Embed · NATS JetStream · Event-Driven Architecture · Workflow Engines · MCP/API Tooling · Multi-Agent Systems · AI Automation · Docker · OCI · Virtualization · CI/CD · Security Architecture · Production AI Systems
mikhol.com has been my corner of the internet since 2000.