Week 9 — Lesson 1: Loading documents, chunking and metadata

3 min

Week 8 gave one vector per NorthPeak incident note. A note is one idea, about 70 characters. A manual in docs/manuals/ is forty ideas, about 2,000 characters, and one vector for all of it would be a blur. So we cut each manual into chunks of at most about 600 characters, keep one paragraph of overlap, and tag each chunk with where it came from. This lesson gives the cutting recipe and the metadata that the six manuals produce: 31 chunks.

Preview — the rest of the lesson is for enrolled readers.

Already enrolled with a code?

Your access is tied to your account, not to this link. Sign in with the same email you used in class: your course is waiting, no need to enter the code again.

Sign inNo account yet? Create one
This lesson is part of the “Week 9 — Retrieval and vector databases” module

The first modules of the course are open to everyone. For the rest you have three options: buy this course once and for all, subscribe, or enter the code handed out in class.

Are you a student on this course?

The code is tied to your account: sign in or create an account and it will be applied automatically when you come back.

No account yet? Create one