AI Engineering
Foundations

How to use this book

This book is structured in eight parts, mirroring an 8-week roadmap from foundations to production. Each chapter follows the same structure: concept → real-world analogy → code example → when to use → common mistakes. Use the sidebar to jump to any chapter. The glossary in Part VIII defines every term alphabetically.

• Beginners — read Parts I and II fully before touching code
• Intermediate engineers — jump to Part III (Protocols) or Part IV (Orchestration)
• Production engineers — Parts VI and VII are your reference
• Quick lookup — Part VIII is a complete glossary

Chapter 1 What is Agentic AI?

Most software follows instructions you write explicitly — every step coded in advance. An agent follows goals you give it, then figures out the steps itself. That single distinction changes everything about how you design, build, and debug software systems.

The spectrum from automation to agency

It helps to think of a spectrum rather than a binary. Intelligence and autonomy exist on a continuum:

A traditional program (like a cron job or script) runs in a fixed, pre-written way. It cannot make decisions or adapt. A chatbot answers questions but cannot take actions in the real world. An agent is different: it looks at its environment, thinks about what to do, uses tools to take action, checks the results, and repeats this loop to achieve a goal you gave it.

The four essential properties of an agent

1. Perception — Reads data from its environment (like files, databases, or messages).
2. Reasoning — Thinks about what to do next, using the LLM as its brain.
3. Action — Uses tools, writes files, or runs workflows in the real world.
4. Memory — Remembers details across steps so it does not repeat work.

Why agentic AI is fundamentally different

Real-world analogy: the smart assistant

An agent is like a smart assistant you hired. You say: "Research our top 10 competitors and write a report." You do not have to tell them: search Google first, then Wikipedia, then open Excel. They figure out the steps themselves. If they hit a blocked website, they try a different one. They search until the job is done.

Your job as the builder is to give the agent a clear goal, the right tools, correct rules (what not to do), and the context to succeed. The agent figures out the rest.

Where agents are being deployed in 2026

Chapter 2 How LLMs Actually Work

To build reliable agents you need a working mental model of what is happening inside the model. You don't need the mathematics — but you do need to understand the behaviour well enough to predict when it will succeed and when it will fail.

The single mechanism: next-token prediction

Every large language model does exactly one thing: given a sequence of tokens, compute a probability distribution over what token comes next, then sample from that distribution. That's the entire algorithm. Everything else — apparent reasoning, knowledge, creativity — emerges from doing this at massive scale.

Tokens — not words

LLMs don't process words — they process tokens. A token is roughly 3–4 characters in English. Understanding this matters for agents:

• Context window limits are in tokens — "128K context" means 128,000 tokens, roughly 96,000 English words
• API pricing is per token (input + output separately) — your system prompt, tool schemas, tool results, and conversation history all cost money
• Code is token-dense — a 500-line Python file might be 3,000+ tokens
• Token counting rule of thumb: 1 token ≈ 0.75 English words. "Hello world" = 2 tokens. A 400-page novel ≈ 130K tokens.

The Transformer architecture — just enough to know

All modern LLMs are Transformer-based. The key component is attention: a mechanism that allows every token to "look at" every other token in the context window when computing its representation. This is what makes LLMs context-aware — a word's meaning is computed in relation to all surrounding words.

Temperature and sampling — the most important dial for agents

Temperature controls how "peaked" the probability distribution is before sampling. It is the single most important parameter for agent reliability:

For deterministic agent workflows, start with temperature 0.0–0.2. Tool calls need to be consistent and predictable. An agent that sometimes calls search_web("stripe funding") and sometimes calls searchweb("stripe funding") (hallucinated tool name) based on sampling variance is broken. Use higher temperature only for creative subtasks like ideation, drafting, or brainstorming.

Top-p and Top-k

Reasoning before action

For multi-step tasks, models generally perform better when the prompt asks them to plan before acting. The useful pattern is not exposing private chain-of-thought to the user; it is forcing the system to identify constraints, choose the right tool, check assumptions, and verify the result.

In modern agent systems, prefer concise planning traces: goal, known facts, missing information, next tool call, and verification rule. This gives the agent enough structure to make better decisions without requiring hidden reasoning to be printed verbatim.

For agents: include an internal planning or tool-selection step, but show users only a short rationale or final answer unless detailed reasoning is explicitly useful and safe.

Context windows — how to think about them

Exact context limits and prices change frequently. Treat model tables as architecture guidance, then verify the current provider docs before choosing production defaults.

Hallucination — what it is and why it happens

Hallucination is when an LLM generates confident-sounding text that is factually incorrect or completely fabricated. It's not a bug — it's a direct consequence of the architecture. The model predicts statistically plausible tokens. If the training data contained many patterns like "The CEO of [company] is [name]", the model will complete that pattern — even if it has to invent a name that sounds plausible.

In agents, hallucination takes a specific dangerous form: the model may hallucinate tool results (pretend it called a tool and invent the output), hallucinate tool names (call a tool that doesn't exist), or hallucinate parameter values (make up an API key or URL). Mitigation: explicit negative instructions in system prompt, low temperature, verification patterns.

Chapter 3 The ReAct Loop — The Foundation of Every Agent

ReAct (Reasoning + Acting) is the foundational design pattern for AI agents. First proposed in the landmark paper ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 2022), it remains the core loop running beneath every agent framework — LangGraph, CrewAI, AutoGen, and everything else. Understanding it at the implementation level is what separates engineers who can build agents from those who can only configure them.

The loop — in full

A fully traced example — step by step

Goal: "What is the current Node.js LTS version and when does it reach end of life?"

━━ Step 1 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

THOUGHT: I need to find the current Node.js LTS version.
         I must NOT rely on training data — versions change frequently.
         I'll search the web for current information.

ACTION:  web_search(query="Node.js current LTS version 2026")

OBSERVATION: {
  "answer": "Node.js 22 (codename Hydrogen) is current LTS",
  "results": [
    {"url": "https://nodejs.org/en/about/previous-releases", ...}
  ]
}

━━ Step 2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

THOUGHT: I have the version (22, Hydrogen). But the user asked about
         EOL date specifically — I only have a vague answer. I should
         fetch the official Node.js releases page for exact dates.

ACTION:  fetch_url(url="https://nodejs.org/en/about/previous-releases")

OBSERVATION: "...Node.js 22 LTS: Active Oct 2024, Maintenance Apr 2026,
              End-of-Life Apr 2027..."

━━ Step 3 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

THOUGHT: I now have complete, verified information:
         - LTS version: 22 (codename "Hydrogen")
         - EOL date: April 2027 (from official source)
         The goal is fully achieved. I can give a final answer.

FINAL ANSWER: Node.js 22 (codename "Hydrogen") is the current LTS
              version. It reaches end-of-life in April 2027.
              Source: nodejs.org/en/about/previous-releases

The ReAct loop implemented in pure Python

This is the most important code in this book. Every agent framework — LangGraph, CrewAI, AutoGen — is building on top of exactly this. Understanding it means you can debug any framework:

from typing import Any

def react_agent(
    query: str,
    tools: list[dict],      # Tool schemas (sent to LLM)
    tool_fns: dict,         # Tool name → Python function
    llm,                    # Any LLM provider
    max_steps: int = 20
) -> str:
    """
    The ReAct loop. Pure Python, no framework.
    Every agent framework is an abstraction over this.
    """

    # The agent's entire "world" = this list of messages
    # It has no other state, no memory, no awareness
    messages = [
        {"role": "system", "content": build_system_prompt(tools)},
        {"role": "user",   "content": query}
    ]

    for step in range(max_steps):
        print(f"\n── Step {step + 1} ──")

        # ── 1. Ask LLM: "Given everything so far, what next?" ────
        response = llm.complete(messages=messages, tools=tools)

        # ── 2. No tool call = model decided it's done ────────────
        if not response.tool_calls:
            print("Final answer reached.")
            return response.text

        # ── 3. Log what the model decided ────────────────────────
        if response.text:
            print(f"Thought: {response.text[:200]}")
        for tc in response.tool_calls:
            print(f"Action: {tc.name}({tc.args})")

        # ── 4. Store model's decision in message history ─────────
        # CRITICAL: both Anthropic and OpenAI require this
        messages.append(format_assistant_message(response))

        # ── 5. Execute all tool calls (in parallel for speed) ────
        results = execute_tools_parallel(response.tool_calls, tool_fns)

        # ── 6. Log results ────────────────────────────────────────
        for tc, result in zip(response.tool_calls, results):
            print(f"Observation: {str(result)[:150]}...")

        # ── 7. Add tool results to history so model can see them ─
        messages.append(format_tool_results(response.tool_calls, results))

        # ── 8. Go back to step 1 — model now has new context ─────

    # Hit step limit — request a final answer with what we have
    messages.append({
        "role": "user",
        "content": "Step limit reached. Give your best answer with what you have."
    })
    return llm.complete(messages=messages, tools=tools).text

Why the step limit exists — and common failure modes

Chapter 4 Tool Calling — Deep Dive

Tool calling (also called function calling) is the mechanism by which an LLM requests execution of external code. First formalized under the neuro-symbolic framework in the paper MRKL Systems (Karpas et al., 2022), it is the bridge between the model's language capabilities and the real world of APIs, databases, files, and services. Understanding it at the API level — before any framework hides it — is essential.

How tool calling works at the raw API level

The flow is always the same regardless of provider:

1. You define tool schemas (JSON Schema) and send them to the API alongside your messages
2. The model's response either contains a tool_use block (wants to call a tool) or a text block (final answer)
3. You execute the tool, get the result, add it to message history
4. Send the updated history back to the API — model continues from there

# Complete tool calling flow — Anthropic API
import anthropic

client = anthropic.Anthropic()

# Step 1: Define tool schemas
tools = [
    {
        "name": "web_search",
        "description": """Search the web for current information.
        Use when facts may have changed: software versions, news, stock prices.
        Be specific: 'Node.js 22 LTS EOL date' not 'Node.js info'.""",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "Specific search query. Include year for recency."
                },
                "max_results": {
                    "type": "integer",
                    "description": "Results to return (1-10). Default 5.",
                    "default": 5
                }
            },
            "required": ["query"]
        }
    }
]

# Step 2: Call the API
response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=4096,
    tools=tools,
    messages=[{"role": "user", "content": "What's the latest Node.js LTS?"}]
)

# Step 3: Parse response — could be text OR tool calls
for block in response.content:
    if block.type == "tool_use":
        tool_id   = block.id       # "toolu_01XY..." needed for result matching
        tool_name = block.name     # "web_search"
        tool_args = block.input    # {"query": "Node.js LTS 2026"} — already parsed!
        
        # Execute the tool
        result = execute_tool(tool_name, tool_args)
        
        # Step 4: Return result to the model
        # NOTE: Anthropic requires BOTH the assistant message AND the tool result
        messages = [
            {"role": "user", "content": "What's the latest Node.js LTS?"},
            {"role": "assistant", "content": response.content},  # The tool call itself
            {"role": "user", "content": [{               # The result
                "type": "tool_result",
                "tool_use_id": tool_id,
                "content": str(result)
            }]}
        ]
        # Call API again with updated history → model continues
    
    elif block.type == "text":
        print("Final answer:", block.text)

Anthropic vs OpenAI tool calling — exact differences

Parallel tool calls — the latency optimisation

When an agent needs multiple pieces of information that don't depend on each other, it can request multiple tool calls in a single LLM response. Running them in parallel cuts total latency from N×T to max(T):

from concurrent.futures import ThreadPoolExecutor
import asyncio

# Model can return multiple tool calls at once
# response.content = [ToolUseBlock(search1), ToolUseBlock(search2), ToolUseBlock(fetch1)]

# Synchronous parallel execution (simple, works with blocking libs)
def execute_parallel_sync(tool_calls: list) -> list:
    with ThreadPoolExecutor(max_workers=len(tool_calls)) as pool:
        futures = [pool.submit(execute_tool, tc.name, tc.input) for tc in tool_calls]
        return [f.result() for f in futures]

# Async parallel execution (better for production)
async def execute_parallel_async(tool_calls: list) -> list:
    tasks = [execute_tool_async(tc.name, tc.input) for tc in tool_calls]
    return await asyncio.gather(*tasks)

# Impact example:
# 3 searches × 2s each = 6s sequential
# 3 searches in parallel = 2s total

Tool schema design — the most overlooked skill

The description field IS your API contract with the model. It determines when the model uses the tool and how correctly it fills the arguments. A bad description is the #1 cause of poor agent tool selection:

Tool types you will build

Chapter 5 Prompt Engineering for Agents

Agent prompting is fundamentally different from chatbot prompting. You're not writing a one-time message — you're writing an operating manual the model will follow across 20+ decision loops, without you able to intervene. Every word matters. For detailed prompt guidelines, reference the Anthropic Developer Prompt Engineering Documentation.

System prompt anatomy — the correct structure

"""
[ROLE] — who this agent is
You are an expert [domain] specialist with [specific expertise].
This establishes the probability distribution toward expert-quality outputs.

[GOAL] — what it's trying to achieve (be specific)
Your task is to [exact goal], producing [exact output format].

[TOOLS] — what it has access to (with guidance on each)
You have access to:
  - web_search: Use for current information that may have changed recently.
    Good query: "Stripe funding round 2026". Bad query: "Stripe".
  - fetch_url: Use after web_search to read full article content.
    Do not use for PDFs or non-HTML pages.
  - write_file: Use ONCE at the end to save the final report.

[RULES] — explicit constraints and guardrails
Rules (strictly follow these):
  1. Only state facts you found in search results or fetched pages
  2. If you cannot find information, say "Not found in public sources"
  3. Do NOT call write_file until you have completed at least 3 searches
  4. Do NOT call write_file more than once
  5. If a tool returns an error, try an alternative approach — do not give up

[QUALITY] — what good output looks like
Quality standards:
  - Include source URL for every major claim
  - Flag uncertain information with [UNVERIFIED]
  - Be specific with numbers — exact figures only, never estimates

[OUTPUT FORMAT] — exact structure
Report format:
  # Company Name — Intelligence Report
  ## Overview
  ## Business model
  ...

[EXAMPLE] — one complete correct trace (most powerful signal)
Example of correct behaviour:
  User: Research Adyen
  Thought: I should search for Adyen's overview first...
  Action: web_search("Adyen payment company overview 2026")
  Observation: [results showing Adyen is a payment processor...]
  Thought: I have the overview. Now I need funding details...
  Action: web_search("Adyen revenue 2025 annual results")
  ...
"""

Few-shot examples — the most powerful prompt technique for agents

Showing the model one complete correct trace of Thought → Action → Observation → Answer is far more effective than explaining the desired pattern in prose. The model "learns" the pattern from the example and applies it:

"""
Example — how to handle a tool returning no results:

THOUGHT: I should search for the company's CEO.

ACTION: web_search(query="Acme Corp CEO 2026")

OBSERVATION: {"results": [], "answer": ""}

THOUGHT: The search returned no results. I'll try a different query —
         maybe the company name is formatted differently.

ACTION: web_search(query="Acme Corporation leadership team")

OBSERVATION: {"results": [{"content": "Jane Smith is CEO of Acme..."}]}

THOUGHT: Found it. Jane Smith is CEO.
         I can now include this in the report.
"""

Negative instructions — often more effective than positive

Telling the model what NOT to do is often more reliable than telling it what to do. The brain naturally generates the prohibited action and then suppresses it, which paradoxically reinforces it — but LLMs respond differently:

• "Do NOT invent funding amounts" works better than "Only report real funding amounts"
• "Do NOT call write_file more than once" is more reliable than "Call write_file exactly once"
• "Do NOT assume a tool succeeded if it returned an error" prevents a common failure mode

Hallucination guard phrases

These specific phrases measurably reduce confabulation in long agent runs. Include in every agent system prompt:

• "Only include information you found in search results or fetched pages. If you don't know, say 'Not found in public sources' — never invent."
• "Never estimate or approximate numbers. Use exact figures from sources, or state they are unavailable."
• "If a tool returns an error, report the error message. Do not invent what the result might have been."
• "Verify every major claim with at least one URL you actually fetched and read."

Prompt length — when more is not more

Longer system prompts are not always better. Beyond a certain point (roughly 2,000 tokens for system prompts), additional instructions start getting ignored due to attention dilution. Priority order for what to include:

1. Role and goal (always — highest attention weight)
2. Hard rules and negative constraints (always)
3. Tool usage guidance (always)
4. Output format (always)
5. One concrete example (if fits — very high value)
6. Edge case handling (optional — consider separate instructions)

Part I complete · Continue to Part II — Memory & RAG →