Week 1 — Lesson 4: Prompting and tool calling

A language model cannot open machines.csv or count rows. It only writes tokens. When you need a real number from NorthPeak, the model must ask your code to run a function, then read the result. This lesson covers the three roles of a chat message, temperature, few-shot examples, and that three-step tool call. Exercise 3 writes the three steps by hand.

Messages, roles and temperature

A call to a chat model is a list of messages. Each message has a role and a content. There are three roles.

The system message is the most powerful one. "Answer only with JSON" or "Say I do not know when unsure" goes there. In Exercise 2 you will see how one system message changes a refusal into an invented answer.

JSON is a small text format for structured data, such as {"site": "Toronto"}. Programs read it without guessing.

Temperature controls how random the model is. At temperature 0 the model always picks the most likely token. The same question gives the same answer. At temperature 1 it takes more chances. We asked "Give me a name for a factory robot. One word only." twice at each setting:

temperature 0 : 'Zeta'   'Zeta'
temperature 1 : '"Nova"' 'Zeta'

Use temperature 0 in this course. It makes your work repeatable.

A few-shot prompt shows the model two or three examples before the real question. "Description: Pressure dropped. Category: leak. Description: Loud grinding noise. Category: bearing_wear. Description: Control panel fault code E70. Category:" The examples show the format you want. The model copies the pattern. A small model like llama3.2 still needs a clear system message with it, and it still makes mistakes. You will measure that in Week 10.

The Python call

Here is the whole Python call. The ollama package sends the messages to the local model.

import ollama

response = ollama.chat(
    model="llama3.2",
    messages=[
        {"role": "system", "content": "Answer in one short sentence."},
        {"role": "user", "content": "What does a chiller do?"},
    ],
    options={"temperature": 0},
)
print(response.message.content)

response.message.content is the text of the answer. You will type this in Exercise 2.

The model asks, your code runs

A model cannot read a file. It cannot count rows. It cannot run Python. It only writes tokens. So how can it answer "How many machines are in Toronto?" with the real number?

With a tool call. The idea has three moves:

1. You tell the model, in the system message, which functions exist. For example count_machines(site).
2. The model does not answer the question. It writes a small JSON: {"tool": "count_machines", "site": "Toronto"}.
3. Your code reads the JSON, runs the real function, and sends the result back in a new message. Now the model writes the final sentence with the real number.

The model never runs anything. It asks. Your code runs. The model reads the result. Frameworks like LangChain hide these three moves behind one line. In Exercise 3 you will write them by hand, so you know what is inside.

A common mistake

Beginners believe the model "has" the tool. It does not. If your code forgets to run the function, the model just gets no answer and may invent one. The safety of a tool call is in your code, not in the model. Check the JSON before you run anything. Only run the functions you listed.