Week 3 — Lesson 5: Key-value stores, choosing a database, secure access

3 min

The last database shape is the simplest: a key and a value, nothing else. It is the memory and the cache of the diagnostic assistant you will build in Week 12. With five shapes now on the table, the real skill is choosing one from the question you ask, and opening it safely. This lesson covers key-value stores, the choice of database, and three one-line habits that protect the NorthPeak data: read-only connections, secrets in environment variables, least privilege.

The simplest database

A key-value store holds pairs: a key, a value. Nothing else. No columns, no joins. You ask for a key, you get the value, in a fraction of a millisecond. Redis is the best-known one. A Python dict is the same idea, inside one program.

Two jobs fit this shape. A cache: store the result of a slow computation under a key, return it instantly next time. Session state: remember what a user said three messages ago, under the key of the conversation. The assistant of Week 12 will do both.

The choice of database

The question decides the tool. Four shapes, four questions.

You askUseOur file
Count, sum, join fixed columnsRelational, SQLdiagnostics.sqlite
Store whole records with nested fieldsDocument, JSONincidents.json
Find the text closest in meaningVector storeChroma, Week 9
Follow relations, many hopsGraphnodes.csv, edges.csv
Look up one value very fast, remember stateKey-valuea dict, Redis

Most real systems use two or three. Our diagnostic assistant will use SQL for numbers, a vector store for the manuals, and a key-value cache for memory.

On the NorthPeak data: secure access

Three habits protect a database. They cost one line each.

Read-only. Open SQLite with ?mode=ro. A DELETE then fails instead of deleting.

python
import sqlite3

ro = sqlite3.connect("file:data/diagnostics.sqlite?mode=ro", uri=True)
try:
    ro.execute("DELETE FROM incidents")
except sqlite3.OperationalError as e:
    print(e)          # attempt to write a readonly database

Secrets in environment variables. A password or a path never goes in the code. Read it with os.environ.get("NORTHPEAK_DB_PATH"). Set it in the shell, or in a .env file that is not committed.

Least privilege. Give a program only the rights it needs. An assistant that answers questions needs SELECT, not DROP. On a real server you create a user that can only read. When the LLM of Week 11 writes SQL for you, this is what keeps a bad query harmless.

Data quality belongs here too. A database can enforce a rule: CHECK (load_pct BETWEEN 0 AND 100) would have refused the twelve negative loads of Week 2 at write time.

A common mistake

"It is only my laptop" is how secrets end up on GitHub. A connection string pasted in a script gets committed with the script. Use the environment variable from day one. The habit costs nothing; the leak costs a lot.