Chapter 3: Tools and the Autonomy Dial - Velmurugan Narayanasamy

{"pick": null, "reason": "Paper 0 might use retrieval, but its abstract never
 says how answers are improved, so I cannot tell from what I was given."}

That’s where Chapter 2 stopped, and the reply is fair. The abstract of Better Answers with Less Compute really doesn’t say how the answers get better. A person would open the paper. The model can’t, because the only text it ever sees is what we pasted in before it started.

So this chapter gives it a short list of things it may ask for: show me the abstract of paper 0, show me its full text. The code carries out the request and hands the result back. That brings back the loop from Chapter 1, except that now the model picks each step.

Once a program can take actions, I also have to decide how much it may do without asking me. That’s the second half of this chapter.

Chapter 3 of 12 · Foundations

Where we left off: The model judges papers by meaning, but it can only see what we paste into its prompt, and it can’t ask for more.

What we fix now: Give the model tools it can ask for, run them in code, feed the results back into the loop, and then decide how much the scout may do on its own.

By the end, you can:

explain what a tool is, from the model’s side and from the code’s side
build a scout that chooses its own next step: read an abstract, read the full text, or make a decision
describe three levels of autonomy, and see why one good run doesn’t tell you which level is safe

The fix: tools the model can ask for¶

A tool is a Python function plus a one-line description. The two halves live in different places:

The model only sees the name and the description, for example “read_full_text: show the full text of a paper; use it when the abstract is not enough”. It can’t run anything. It can only propose “use read_full_text on paper 0”.
Your code holds the actual function. When the model proposes a tool call, your code checks it, runs the function, and pastes the result into the next prompt.

That pasted result is the observe step from Chapter 1:

Function-calling APIs can make it look as if the model runs the tool itself. It doesn’t. Some library code on your side runs it, and I’d rather that code be mine, where I can see and limit it. The Colab bonus at the end shows the built-in version, and the call still goes through code I wrote.

There’s a practical win too. The prompt now only needs titles. The model pulls in the abstract or full text of the few papers that look promising, which is what makes a day with a hundred-plus papers workable.

Try it

The cell below defines the two tools, the prompt, and the loop. Read the printed prompt first: titles only, a list of actions, and a transcript section that starts empty. Then read the trace of the run on day 5, the day the Chapter 2 scout got stuck.

As in Chapter 2, the model here is a hand-written stand-in (mock_model), so the page runs without an API key. Its replies were written for this chapter, not recorded from a real model. The Colab bonus at the end swaps in a real one.

If the code below doesn’t let you type into it, use the badge above to open this same notebook in Google Colab, where you can edit and run it freely.

The code cells on this page build on each other, so run them in order, starting from the first one. If a cell stops with a NameError, an earlier cell hasn’t been run yet: scroll up, run the cells above it, and try again.

import json
import re

interest = ("papers about retrieval-augmented generation (RAG), or about "
            "making language models answer from documents they are given")

day_5 = [
    {"title": "Better Answers with Less Compute",
     "abstract": "We introduce a simple method that improves answer quality on two question answering benchmarks while using less compute. Code is available online.",
     "full_text": "Before answering, our method retrieves the five most similar passages from a Wikipedia index and adds them to the prompt. Because the answer comes from the passages, a much smaller model is enough."},
    {"title": "Faster Training for Speech Models",
     "abstract": "We cut speech model training time in half with a new batching scheme. Word error rate stays the same.",
     "full_text": "Audio clips of similar length are grouped into the same batch so less padding is wasted."},
]

# Tools: a function the code runs, plus a description the model sees.
def read_abstract(papers, i):
    return papers[i]["abstract"]

def read_full_text(papers, i):
    return papers[i]["full_text"]

TOOLS = {
    "read_abstract": (read_abstract, "show the abstract of paper i"),
    "read_full_text": (read_full_text, "show the full text of paper i; use it when the abstract is not enough"),
}
FINAL_ACTIONS = {
    "recommend": "recommend paper i to the researcher",
    "nothing_today": "say that no paper matches (use null for paper)",
}

def build_prompt(papers, transcript):
    titles = "\n".join(f"[{i}] {p['title']}" for i, p in enumerate(papers))
    actions = "\n".join(f"- {name}: {desc}" for name, (_, desc) in TOOLS.items())
    actions += "\n" + "\n".join(f"- {name}: {desc}" for name, desc in FINAL_ACTIONS.items())
    steps = "\n".join(transcript) if transcript else "(nothing yet)"
    return f"""You help a researcher decide which new paper to read today.
Their interest: {interest}

Today's papers (titles only):
{titles}

Actions you can take:
{actions}

What you have done so far:
{steps}

Choose ONE next action. Reply with JSON only:
{{"action": "<action name>", "paper": <paper number or null>, "reason": "<one sentence>"}}"""

print(build_prompt(day_5, []))

Everything the model knows about its abilities is in that “Actions you can take” list. Now the loop that runs it:

# Hand-written stand-in replies (NOT recorded from a real model).
# One script per day; the stand-in replies with the next line each step.
scripts = {
    "Better Answers with Less Compute": [
        '{"action": "read_abstract", "paper": 0, "reason": "The title mentions answers, so I want to see what the paper does."}',
        '{"action": "read_full_text", "paper": 0, "reason": "The abstract does not say how the answers are improved."}',
        '{"action": "recommend", "paper": 0, "reason": "The method retrieves Wikipedia passages and answers from them, which is RAG."}',
    ],
}

def mock_model(prompt):
    for first_title, script in scripts.items():
        if first_title in prompt:
            step = prompt.count("Result:")          # how many steps are done so far
            return script[min(step, len(script) - 1)]
    return '{"action": "nothing_today", "paper": null, "reason": "No script for these papers."}'

ask_model = mock_model

def parse_action(reply, papers):
    match = re.search(r"\{.*\}", reply, re.DOTALL)
    if not match:
        return {"action": "error", "reason": "No JSON found in the reply."}
    try:
        d = json.loads(match.group())
    except json.JSONDecodeError:
        return {"action": "error", "reason": "The reply was not valid JSON."}
    if d.get("action") not in TOOLS and d.get("action") not in FINAL_ACTIONS:
        return {"action": "error", "reason": f"Unknown action {d.get('action')!r}."}
    if d["action"] != "nothing_today":
        i = d.get("paper")
        if not (isinstance(i, int) and 0 <= i < len(papers)):
            return {"action": "error", "reason": f"Paper {i!r} does not exist."}
    return d

def finish(papers, d):
    if d["action"] == "nothing_today":
        print("Stop -> Nothing relevant today.")
    else:
        print(f"Recommend '{papers[d['paper']]['title']}'")

def run_scout(papers, max_steps=6):
    transcript = []
    for step in range(1, max_steps + 1):
        d = parse_action(ask_model(build_prompt(papers, transcript)), papers)   # think
        if d["action"] in FINAL_ACTIONS:
            print(f"Step {step}: {d['action']} ({d['reason']})")
            finish(papers, d)
            return
        if d["action"] == "error":
            result = f"Error: {d['reason']}"                                       # tell the model what went wrong
        else:
            tool, _ = TOOLS[d["action"]]
            result = tool(papers, d["paper"])                                       # act
        print(f"Step {step}: {d['action']}({d.get('paper')})  <- {d['reason']}")
        print(f"   Result: {result}")
        transcript.append(f"Step {step}: {d['action']}({d.get('paper')})\nResult: {result}")   # observe
    print("Out of steps.")

run_scout(day_5)

Read the trace top to bottom. The model looks at the abstract, still can’t tell, asks for the full text, and only then decides. The full text says the method retrieves Wikipedia passages before answering, so this is a RAG paper, and the scout recommends it. In Chapter 2 this paper was a dead end.

Every result goes into transcript, which is pasted into the next prompt. That’s the scout’s whole memory within a run; there’s nothing cleverer behind it. parse_action is Chapter 2’s check, grown up: an unknown action or a paper that doesn’t exist becomes an error message the model reads on its next step, instead of a crash.

The autonomy dial¶

So far the scout only reads, and reading can’t hurt anything. What I actually want from it is to do something with its decision, like writing a short summary for my blog’s reading list. Other people would see that.

Before adding a publish_summary action, I need to decide how much the scout may do on its own. I think of it as a dial with three settings:

Setting	The scout...	You...	A wrong decision costs
suggest (v1)	recommends a paper	decide everything	a few seconds of your time
draft (v2)	writes a summary and asks to publish it	approve or reject each one	a moment of reading
act (v3)	publishes on its own	check later, if at all	a wrong post on your blog

The model is identical at all three settings. What moves is where I stand in the loop, and so how far a mistake gets before someone catches it.

autonomy = "suggest"     # try "draft", then "act"
human_answer = "no"      # what you'd say when the scout asks for approval

def summarize(paper):
    return paper["full_text"].split(". ")[0] + "."

def publish_summary(paper):
    print(f"   >>> PUBLISHED on your blog: '{paper['title']}': {summarize(paper)}")

def finish(papers, d):
    if d["action"] == "nothing_today":
        print("Stop -> Nothing relevant today.")
        return
    paper = papers[d["paper"]]
    if autonomy == "suggest":
        print(f"Suggestion: read '{paper['title']}'. What happens next is up to you.")
    elif autonomy == "draft":
        print(f"Draft summary: {summarize(paper)}")
        print(f"Publish this? You answered: {human_answer!r}")
        if human_answer == "yes":
            publish_summary(paper)
        else:
            print("   Not published.")
    elif autonomy == "act":
        publish_summary(paper)

run_scout(day_5)

Same run, same decision, three endings. At "suggest" the output is a note to you. At "draft" the scout does the writing, but nothing goes out without a yes. At "act" it goes straight onto the blog.

On day 5 the decision was right, so "act" looks great: a correct post and no work. One good run is exactly what makes turning the dial up so tempting.

day_6 = [
    {"title": "Retrieval Practice Boosts Exam Scores",
     "abstract": "Students who practiced retrieving facts from memory scored 14 percent higher on final exams than students who reread their notes. We studied 300 students across four courses.",
     "full_text": "Retrieval practice means recalling information from memory, for example with flashcards, instead of rereading. No computer systems are involved."},
    {"title": "Quantizing LLMs to 3 Bits",
     "abstract": "We compress a language model's weights to 3 bits with little loss in quality. Memory use falls by 80 percent.",
     "full_text": "Weights are grouped and each group gets its own scale factor before rounding."},
]

scripts["Retrieval Practice Boosts Exam Scores"] = [
    '{"action": "read_abstract", "paper": 0, "reason": "The title mentions retrieval."}',
    '{"action": "recommend", "paper": 0, "reason": "The paper is about retrieval, which matches the interest."}',
]

autonomy = "act"
run_scout(day_6)

The scout saw “retrieving” in the abstract, skipped the full text, and published a summary of an education study on an AI blog. At "suggest" this would have cost five seconds: read the title, shrug, move on. At "act" it’s public.

Day 5 went perfectly and day 6 went wrong, with the same model, the same code and the same prompt. Watching a run or two tells you nothing about how often the scout is right. I don’t want it to get more autonomy because a few examples looked good. It should earn each step on the dial through repeated measurement: build it for one setting, measure how often it’s right, keep checking it on real use, and only then move it up.

The rest of the course is that process. I use a framework called CC/CD for it, Continuous Development and Continuous Calibration, from an article by Aishwarya Reganti and Kiriti Badam. Chapter 4 starts the first half.

Bonus (Colab only): a real model with tools¶

Open this notebook in Colab to let a real model choose the actions:

Get a free key at aistudio.google.com/apikey (no credit card needed), then paste it into API_KEY below. Treat it like a password: fine in your own private Colab copy while experimenting, never in a public repository or notebook.

The first cell runs the scout from this chapter with a real model. The second shows the same job with Gemini’s built-in function calling.

# Colab only: needs a live network call and an API key.
!pip install -q -U google-genai

from google import genai

API_KEY = "YOUR_API_KEY_HERE"
client = genai.Client(api_key=API_KEY)
MODEL = "gemini-3.5-flash"

def gemini_model(prompt):
    return client.models.generate_content(model=MODEL, contents=prompt).text

ask_model = gemini_model
autonomy = "suggest"

for name, papers in [("day 5", day_5), ("day 6", day_6)]:
    print(f"=== {name} ===")
    run_scout(papers)
    print()

Look at which tools the real model asks for, and in what order. Does it read the full text on day 6, or does it fall for “retrieval” the way the stand-in did? Run it a few times and see whether it stays the same.

The cell below does the same job with native function calling. Instead of describing the actions in the prompt and parsing JSON by hand, you pass Python functions as tools, and the model returns structured tool calls. I switch off the library’s option to run those calls automatically, so my loop still decides what actually runs, and checks the paper number first.

# Colab only: the same tools with Gemini's native function calling.
from google.genai import types

def read_abstract_tool(paper: int) -> str:
    """Show the abstract of paper number `paper`."""
    return read_abstract(day_5, paper)

def read_full_text_tool(paper: int) -> str:
    """Show the full text of paper number `paper`. Use it when the abstract is not enough."""
    return read_full_text(day_5, paper)

my_tools = {"read_abstract_tool": read_abstract_tool, "read_full_text_tool": read_full_text_tool}
settings = types.GenerateContentConfig(
    tools=list(my_tools.values()),
    automatic_function_calling=types.AutomaticFunctionCallingConfig(disable=True),  # my code runs tools
)

titles = "\n".join(f"[{i}] {p['title']}" for i, p in enumerate(day_5))
contents = [f"A researcher's interest: {interest}\n\nToday's papers:\n{titles}\n\n"
            "Use the tools to check what you need, then say which paper they should read "
            "(or none), with one sentence why."]

for step in range(6):
    response = client.models.generate_content(model=MODEL, contents=contents, config=settings)
    if not response.function_calls:
        print("Final answer:", response.text)
        break
    contents.append(response.candidates[0].content)
    for call in response.function_calls:
        paper = int(call.args.get("paper", -1))
        if call.name in my_tools and 0 <= paper < len(day_5):          # check before running
            result = my_tools[call.name](paper)
        else:
            result = f"Refused: {call.name}({dict(call.args)}) is not allowed."
        print(f"Step {step + 1}: model asks for {call.name}(paper={paper})")
        contents.append(types.Content(role="user", parts=[
            types.Part.from_function_response(name=call.name, response={"result": result})]))

Quick check¶

Before moving on, answer for yourself (no peeking ahead):

What does the model actually “see” of a tool, and what does your code hold? Why does that split matter?

The Chapter 3 prompt lists only paper titles. Why is that better than pasting every abstract, as in Chapter 2?

What would go wrong if run_scout didn’t append each result to transcript?

On day 6, the scout made the same wrong decision at every dial setting. What actually differed between “suggest” and “act”?

After day 5, would you have set the dial to “act”? What evidence would you want before doing that?

You can now: give a model tools it can ask for, run them safely in code, feed results back into the loop, and explain what the autonomy dial changes and what it doesn’t.

But one problem remains: one good run told us nothing about the next one, and we have no way to know when the scout has earned more autonomy. The next chapter starts building on purpose: pick the narrowest useful version of the scout, write down exactly what it should do, and create examples with known right answers to measure it against.

Next → Chapter 4: Scope and Curate Data