AI Engineer Notebooks: Framework-Free Hands-On Learning for Applied LLM Systems
AI Engineer Notebooks is a collection of framework-free Colaboratory notebooks designed to teach the practical skills required for AI Engineers and Forward Deployed Engineers (FDEs). The notebooks focus on building production-ready systems using raw model APIs rather than high-level frameworks, ensuring engineers understand the underlying mechanics before abstracting them away. This approach emphasizes durability over transient patterns, as frameworks and libraries evolve rapidly while core API interactions remain stable.
The project was initiated in 2026 by calmrocks and is licensed under the permissive MIT license, making it freely accessible for educational and professional use. It runs primarily on the free Groq API, with optional GPU-accelerated sections for fine-tuning and self-hosted serving, verified on Colab T4 instances. The notebooks are structured to mirror real-world interview and deployment scenarios, covering everything from prompting fundamentals to serving optimization and customer-facing system design.
The repository aligns with the Plan: Transitioning to Forward Deployed Engineer / AI Engineer, a career roadmap for backend engineers moving into AI-adjacent roles. It addresses the applied LLM stack—RAG, agents, evals, and tool integration—while avoiding deep dives into model training or GPU serving internals. The focus is on measurable, deployable systems rather than theoretical demonstrations.
How It Works
The notebooks are self-contained, with each section installing its own dependencies and reading API keys from Colab secrets. This modularity allows learners to progress sequentially or focus on specific topics. The curriculum begins with foundational skills—such as model API interactions, structured output, and tool calling—and progresses to advanced topics like agent design, security, and LLMOps. Every section incorporates exercises to reinforce concepts, and evals are treated as a core discipline, reinforcing the principle of measuring outputs before tuning systems.
A shared corpus is used across RAG and evaluation sections to ensure consistency in retrieval and generation testing. This shared data layer helps learners diagnose why RAG systems fail, emphasizing retrieval quality as the primary bottleneck. The notebooks also include optional GPU-verified appendices for LoRA fine-tuning and vLLM serving, ensuring learners can explore these concepts even when constrained by free-tier limitations.
For example, Section 01 covers prompting fundamentals, structured output, and tool calling, while Section 02 introduces evaluation methodologies that are revisited throughout the curriculum.
Use Cases
AI Engineer Notebooks is tailored for backend or full-stack engineers transitioning into AI Engineer, FDE, or Applied AI roles. These roles often require building systems on top of foundation models, integrating them into real-world workflows, and ensuring they meet production-grade reliability and security standards. The notebooks provide a hands-on way to develop the skills needed for these positions, including:
- Model API Mastery: Understanding how to interact with APIs for prompting, structured output, tool calling, and streaming without relying on frameworks like LangChain or LlamaIndex. This includes budgeting context windows, caching prompts, and optimizing for batch vs. real-time pricing.
- RAG Implementation: Learning the retrieve-augment-generate loop, embedding strategies, hybrid retrieval, reranking, and chunking techniques.
- Topics include the Model Context Protocol (MCP), progressive disclosure of skills, and harness engineering—assembling context, shaping tool results, and verifying outputs.
- Evaluation and Observability: Establishing golden sets for benchmarks, using LLMs as judges (with awareness of their failure modes), and integrating regression evals into CI pipelines. Observability covers tracing calls, logging prompts safely, and tracking cost, latency, and error metrics.
- Reliability sections cover retries, fallbacks, timeouts, and graceful degradation strategies.
- Customer-Facing Craft: Scoping ambiguous requirements, conducting discovery interviews, and building systems for real users rather than hypothetical ones. This includes writing one-page scoping documents and rehearsing demos for non-technical stakeholders.
The notebooks culminate in three end-to-end case studies that combine these skills under realistic constraints:
- Case Study A: Customer-Support Assistant: A deployed RAG+agent system that is debugged in production after a corpus migration introduces a quality regression. This study threads together sections on evals, RAG, agents, and operations.
- Case Study B: Contract Extraction (Pipeline vs. Agent): A comparison of building the same extraction task as both an agent and a pipeline, with accuracy and token-cost metrics to determine the optimal approach.
This mirrors the expectations of hiring managers, who prioritize evidence of shipped systems over theoretical knowledge.
Key Features and Capabilities
The notebooks are designed with several distinguishing features that set them apart from traditional tutorials or framework-centric guides:
- Framework-Free Approach: By avoiding frameworks like LangChain or LlamaIndex, learners first understand the raw API interactions these tools abstract. This foundational knowledge helps engineers judge when to use frameworks and when to build custom solutions. The emphasis is on durable patterns rather than transient wrappers.
- Evals as the Spine: Evaluation methodologies are integrated into every section, reinforcing the habit of measuring outputs before tuning. This includes golden sets for benchmarks, LLM-as-judge techniques (with awareness of their limitations), and regression evals treated as part of CI. The goal is to instill a disciplined approach to quality assurance.
- Free and Accessible: The entire curriculum runs on the free Groq API, eliminating barriers to entry. Sections requiring GPU acceleration (LoRA fine-tuning and self-hosted serving) include optional Colab-T4-verified appendices, ensuring learners can explore these topics even without dedicated hardware.
- Real-World Case Studies: The three case studies provide context for combining skills under constraints. For example, the customer-support assistant study simulates a production debug scenario, while the contract extraction study forces trade-off decisions between pipelines and agents. The red-team benchmark demonstrates how to evaluate model robustness systematically.
- OpenAI-Compatible Patterns: The notebooks use OpenAI-compatible API patterns, allowing skills to transfer directly to OpenAI and, with minor adjustments, to Anthropic.
- This includes discovery techniques, demo disciplines, and war stories for navigating ambiguity—a critical skill for FDE roles.
- Interview and Hiring Alignment: The curriculum is structured to align with common AI Engineer and FDE interview loops, including practical build rounds, system design questions, and behavioral scenarios. Learners emerge with the ability to explain technical concepts concisely and demonstrate their skills through deployable projects.
- Modular and Self-Paced: Each notebook is self-contained, with dependencies installed in the first cell and API keys managed via Colab secrets. This allows learners to progress at their own pace, revisiting sections as needed. The exercises at the end of each notebook reinforce practical application.
- Prompt Injection and Security: Direct and indirect prompt injection risks, output handling, and adherence to OWASP LLM Top 10 guidelines.
- Prompting: When to modify model weights (fine-tuning or LoRA) versus adjusting inputs (prompting or RAG). Cost considerations and conceptual arguments for or against each approach are explored, with optional LoRA fine-tuning exercises on Colab GPUs.
- This prepares learners for system design interviews focused on applied LLM scenarios.
The repository is actively maintained, with release notes documenting updates and verified appendices for GPU-dependent sections. Feedback and run reports are encouraged via GitHub Discussions, fostering a collaborative learning community.