Writing

Blog

Insights and practical guidance on AI infrastructure, GPU optimization, Kubernetes, and platform engineering.

Featured

The Cost-Efficient AI Stack: Ship AI Features Without the Runaway Bill

Most teams overpay for AI by routing every request to a frontier model. This is the architecture we build instead — hybrid cloud+local routing, self-hosted inference, agent orchestration, and cost-per-request observability — and the single principle that ties it together: send each unit of work to the cheapest model that can do it well.

aillmcost-optimizationhybridinfrastructurefinops

Building a Hybrid LLM Platform on EKS, Part 2: The Control Plane, IAM, and IRSA

Part 2 of our hands-on EKS series. We provision the EKS cluster into the VPC from Part 1, wire up OIDC federation and IRSA so pods authenticate without static credentials, and end with a working kubectl connection to a real cluster.

ekskubernetesaws-cdkiamirsaoidcai-infrastructuretypescript

Securing Self-Hosted LLMs and AI Agents on Kubernetes

Harden self-hosted vLLM and AI agents on Kubernetes: an auth/rate-limit gateway, gVisor tool sandboxing, prompt-injection guardrails, scoped secrets, and signed model weights — mapped to the OWASP LLM Top 10.

securityaiagentskubernetesllmprompt-injectionsupply-chain

Building a Hybrid LLM Platform on EKS, Part 1: Architecture and the Network Foundation

Part 1 of a hands-on series building the EKS-based hybrid LLM platform referenced throughout this blog. We map out the full architecture, then provision the VPC, subnets, NAT, and VPC endpoints with AWS CDK — the network foundation every later part builds on.

ekskubernetesaws-cdkllmai-infrastructurehybrid-aivpctypescript

Build a Personal AI Dev Environment: Hybrid Models, Local Inference, and a Workflow That Costs Almost Nothing

The production patterns we deploy for teams — hybrid cloud/local routing, self-hosted models, agent orchestration — scaled down to a single developer's workstation. A practical guide to building a personal AI dev environment with Ollama, Claude Code, and a local router that keeps your token bill near zero.

aillmlocal-modelsollamaclaude-codedeveloper-tools

The Agent Control Plane: Frontier Models Plan, Your Kubernetes Fleet Executes

How to orchestrate a fleet of AI agents using a shared task queue — frontier models like Claude handle planning and decomposition, while a local Kubernetes worker pool runs the high-volume execution tasks. Covers the task ledger, dynamic task creation, lane-based routing, and KEDA autoscaling.

aiagentsorchestrationkubernetesllmhybrid

Observability for LLM Applications on Kubernetes: Tokens, Traces, and Cost per Request

How to instrument self-hosted and hybrid LLM workloads with OpenTelemetry, Prometheus, and Langfuse — tracking time-to-first-token, tokens per second, GPU utilization, and unit economics down to the individual request.

kubernetesllmobservabilityopentelemetryfinopsai-infrastructure

The Hybrid AI Playbook: Cloud Models for Thinking, Local Models for Doing

How to cut your AI costs by 60-80% using a hybrid approach — Claude or GPT for planning and complex reasoning, local models like Llama and Qwen for execution tasks like code generation, summarization, and data extraction.

aillmcost-optimizationlocal-modelsollama

Self-Hosting LLMs on Kubernetes: A Practical Guide

How to deploy, serve, and autoscale open-source large language models on Kubernetes with vLLM — from GPU node pools and deployment manifests to KEDA-based autoscaling and production guardrails.

kubernetesllmgpuai-infrastructureself-hosting

Container Security on Kubernetes: A Practical Guide with Trivy, Falco, and Kyverno

Most Kubernetes clusters are running containers with known vulnerabilities, no runtime monitoring, and no policy enforcement. Here is how to fix that with three open-source tools.

kubernetessecuritytrivyfalcokyvernocontainers

How to Cut Your AWS Bill in Half Without Changing Your Architecture

Most growing teams are overpaying on AWS by 30-50%. Here is the exact checklist we use in every infrastructure audit to find and eliminate wasted spend — no migrations, no rearchitecting.

awscost-optimizationinfrastructurecloud

Using AI to Monitor Kubernetes Clusters and Make Dynamic Scaling Decisions

How to move beyond static thresholds and use AI-driven observability to detect anomalies, predict traffic patterns, and automate scaling decisions across your Kubernetes infrastructure.

kubernetesaimonitoringautoscalingobservability

A Practical Guide to AI for Small and Mid-Size Businesses

No hype, no jargon — a straightforward guide for business owners evaluating where AI actually makes sense and how to adopt it without wasting money.

aismall-businessstrategyautomation

Building a CI/CD Pipeline with Dagger That Deploys to Kubernetes

A practical guide to building a containerized CI/CD pipeline using Dagger's TypeScript SDK — from local Kind clusters to production EKS with GitHub Actions, AWS CDK, and multi-environment promotion.

daggerkubernetescicdhelmeksgithub-actionsaws-cdk

Building a Production Feature Flag Service with Claude Code

How we built FlagSignals, a full-stack feature flag platform with A/B testing and billing, using AI-assisted development.

feature-flagsai-developmentnextjssupabase

GPU Cost Optimization on Kubernetes: A Practical Guide

Learn how to reduce GPU infrastructure costs by up to 60% with proper Kubernetes scheduling, time-slicing, and right-sizing strategies.

kubernetesgpucost-optimizationinfrastructure

Platform Engineering for AI/ML Teams: Building the Foundation

How platform engineering principles transform AI/ML infrastructure from artisanal setups to scalable, self-service platforms.

platform-engineeringai-mlinfrastructuredevops

FinOps for AI Infrastructure: Beyond Cloud Cost Tags

Traditional FinOps practices fall short for AI workloads. Here's how to build a cost management strategy that accounts for GPU economics.

finopscost-managementai-infrastructurecloud