The lab kit of the course: https://github.com/hrhouma2/aiopsatlas-ml-data-diagnostics-labs-en
Every exercise of the 15 weeks runs inside this kit. You install it once, today. Read this page from top to bottom. If you are on Windows, open Appendix A when a step tells you to. If you are on Linux or macOS, open Appendix B. Never open both.
NorthPeak Manufacturing gave you one year of sensor readings and incident notes. The data does not exist yet on your laptop. A Python script builds it, always the same way, from a fixed seed. At the end of this exercise you will have: the kit folder, a Python virtual environment with every package of the course, the three clean CSV files, and two local language models.
Windows, in PowerShell:
git clone https://github.com/hrhouma2/aiopsatlas-ml-data-diagnostics-labs-en.git
cd aiopsatlas-ml-data-diagnostics-labs-en
py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
python data/make_dataset.pyLinux, macOS, WSL 2 or Git Bash:
git clone https://github.com/hrhouma2/aiopsatlas-ml-data-diagnostics-labs-en.git
cd aiopsatlas-ml-data-diagnostics-labs-en
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
python data/make_dataset.pyThen, on every system:
ollama pull llama3.2
ollama pull nomic-embed-textExpected at the end: machines : 40 rows, readings : 14600 rows, incidents : 155 rows.
The script data/make_dataset.py writes three clean files in data/clean/. Here is the head of each one, as it looks on our machine.
data/clean/machines.csv, 40 rows, one per machine:
machine_id,machine_type,site,install_year,rated_power_kw
M001,pump,Toronto,2022,45.0
M002,pump,Montreal,2016,45.0
M003,pump,Toronto,2013,45.0
M004,pump,Quebec City,2021,45.0
M005,pump,Montreal,2018,45.0machine_id — the code of the machine, M001 to M040.machine_type — pump, compressor, conveyor or chiller, 10 of each.site — Montreal, Quebec City or Toronto.install_year — the year the machine was installed, 2012 to 2022.rated_power_kw — the maximum power of the machine, in kilowatts.data/clean/readings.csv, 14,600 rows, one per machine per day of 2025:
reading_id,machine_id,date,load_pct,ambient_c,temperature_c,vibration_mm_s,pressure_bar,power_kw,fault_next_7d
1,M001,2025-01-01,33.9,-3.9,26.1,3.25,6.52,18.3,0
2,M001,2025-01-02,74.3,-3.9,39.6,4.14,6.63,34.3,0
3,M001,2025-01-03,55.7,-4.8,32.8,3.48,7.19,26.0,0
4,M001,2025-01-04,24.0,-6.3,22.0,3.36,6.31,12.9,0
5,M001,2025-01-05,33.1,-6.6,23.6,3.66,5.77,19.5,0load_pct — how hard the machine worked that day, 0 to 100.ambient_c — the outside temperature, in degrees Celsius.temperature_c, vibration_mm_s, pressure_bar, power_kw — the four sensors of the machine.fault_next_7d — 1 if a fault happened in the next seven days, else 0. This is the label.data/clean/incidents.csv, 155 rows, one per incident:
incident_id,machine_id,date,category,severity,downtime_hours,repair_cost_cad,technician,description
INC-0001,M001,2025-02-20,overheating,high,29.9,3465.33,L. Fortin,Overheating alarm on M001. Cooling fan running but airflow blocked by dust.
INC-0002,M001,2025-05-05,bearing_wear,low,2.9,582.47,M. Haddad,Bearing temperature rising on M001. Operator reports a rattling sound near the drive shaft.category — bearing_wear, overheating, leak, electrical or sensor_fault.severity — low, medium or high.description — a short note written by the technician. The text you will read with an LLM later.Open a terminal. On Windows, use PowerShell. On Linux or macOS, use the normal terminal. Go to the folder where you keep your courses. Then copy the kit with Git.
git clone https://github.com/hrhouma2/aiopsatlas-ml-data-diagnostics-labs-en.git
cd aiopsatlas-ml-data-diagnostics-labs-enLook inside. On Windows type dir. On Linux or macOS type ls.
data docs week01 README.md requirements.txtYou see data, docs, the kit README.md, requirements.txt, and one weekNN folder per published week. There is no CSV file yet. The data folder only contains make_dataset.py. You will build the data in Step 5.
The kit needs Python 3.10 or newer. Type the command for your system.
Windows:
py -3.12 --versionLinux or macOS:
python3 --versionExpected, on our machine:
Python 3.12.10If the command is not found, install Python first. Appendix A (Windows) or Appendix B (Linux, macOS) shows how.
A virtual environment is a private folder with its own Python and packages. It keeps the course packages away from the rest of your computer. We call it .venv. You create it once. You activate it every time you open a new terminal.
Windows, in PowerShell:
py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1Linux or macOS:
python3 -m venv .venv
source .venv/bin/activateThe prompt now starts with (.venv). That is how you know the environment is active. If PowerShell refuses to run the script, go to Appendix A, step A.4.
From now on, python means the Python of .venv. Check it:
python --versionPython 3.12.10The file requirements.txt lists every package of the 15 weeks. Install them all now. The download is about 1 GB. It takes a few minutes.
pip install -r requirements.txtThe last line starts with Successfully installed. Then check the three packages you will use most:
python -c "import pandas, numpy, sklearn; print('pandas', pandas.__version__); print('numpy', numpy.__version__); print('scikit-learn', sklearn.__version__)"Expected, on our machine:
pandas 3.0.5
numpy 2.5.3
scikit-learn 1.9.1Newer versions are fine. If the import fails, the venv is not active. Go back to Step 3.
One script builds every file of the course. It uses a fixed seed, 42. Running it twice gives the same bytes. So the numbers in every lesson can be checked on your laptop.
python data/make_dataset.pyExpected output, about 10 seconds:
machines : 40 rows -> data/clean/machines.csv
readings : 14600 rows -> data/clean/readings.csv (fault_next_7d = 1 on 7.4%)
incidents : 155 rows -> data/clean/incidents.csv
raw copies : readings_raw has 14750 rows (150 duplicates), 295 missing temperatures
json/graph/sqlite written.Read the four lines. 40 machines. 14,600 daily readings, with a fault label on 7.4 % of them. 155 incidents. A dirty copy with 14,750 rows for Week 2. Look inside data now:
clean graph json raw diagnostics.sqlite make_dataset.pypandas is the Python library for tables. You will use it every week. Start Python by typing python in the terminal. Then type these lines one by one.
import pandas as pd
machines = pd.read_csv("data/clean/machines.csv")
print(machines.head())
print(machines.shape) machine_id machine_type site install_year rated_power_kw
0 M001 pump Toronto 2022 45.0
1 M002 pump Montreal 2016 45.0
2 M003 pump Toronto 2013 45.0
3 M004 pump Quebec City 2021 45.0
4 M005 pump Montreal 2018 45.0
(40, 5)head() shows the first five rows. shape gives (rows, columns). Now the two other files:
readings = pd.read_csv("data/clean/readings.csv")
print(readings.shape)
print(readings.columns.tolist())
incidents = pd.read_csv("data/clean/incidents.csv")
print(incidents[["incident_id", "machine_id", "date", "category", "severity"]].head(3))(14600, 10)
['reading_id', 'machine_id', 'date', 'load_pct', 'ambient_c', 'temperature_c', 'vibration_mm_s', 'pressure_bar', 'power_kw', 'fault_next_7d']
incident_id machine_id date category severity
0 INC-0001 M001 2025-02-20 overheating high
1 INC-0002 M001 2025-05-05 bearing_wear low
2 INC-0003 M001 2025-07-31 bearing_wear lowOne last command. value_counts() counts how many rows have each value.
print(machines["machine_type"].value_counts())machine_type
pump 10
compressor 10
conveyor 10
chiller 10
Name: count, dtype: int64Type exit() to leave Python.
The course uses a local language model through Ollama. Install Ollama from ollama.com/download if you have not yet. Then download the two models. The first is 2 GB, the second is 274 MB.
ollama pull llama3.2
ollama pull nomic-embed-textCheck that both are present:
ollama listNAME ID SIZE MODIFIED
nomic-embed-text:latest 0a109f422b47 274 MB 16 minutes ago
llama3.2:latest a80c4f17acd5 2.0 GB 17 minutes agoYou will use llama3.2 in Exercise 2, in one hour.
machines.csv have?make_dataset.py print for readings?readings.csv is the label?(40, 5): 40 machines, 5 columns.readings : 14600 rows -> data/clean/readings.csv (fault_next_7d = 1 on 7.4%).fault_next_7d, the last column.Run python data/make_dataset.py a second time. Then compare the two versions of machines.csv. On Windows use Get-FileHash data\clean\machines.csv before and after. On Linux or macOS use md5sum data/clean/machines.csv. Are the two hashes the same? Why?
This exercise is the setup. There is nothing to save. The full list of commands is in the "Setup" block at the top of the page. The week01/README.md file of the kit lists the scripts of Exercises 2 and 3.
Read this appendix only if you are on Windows 10 or 11. Every step uses PowerShell, not the old Command Prompt.
py -3.12 --version
git --versionPython 3.12.10
git version 2.49.0.windows.1Your version numbers may be newer. py is the Windows Python launcher. py -3.12 always picks Python 3.12, even if you have several versions.
Long paths cause errors on Windows. Keep the kit close to the root of your disk.
mkdir C:\courses
cd C:\coursesgit clone https://github.com/hrhouma2/aiopsatlas-ml-data-diagnostics-labs-en.git
cd aiopsatlas-ml-data-diagnostics-labs-en
dirIf Git says Filename too long, run this once and clone again:
git config --global core.longpaths truepy -3.12 -m venv .venvNothing is printed. A folder .venv appears.
.\.venv\Scripts\Activate.ps1The first time, PowerShell often refuses:
.\.venv\Scripts\Activate.ps1 : File C:\courses\aiopsatlas-ml-data-diagnostics-labs-en\.venv\Scripts\Activate.ps1 cannot be loaded because running scripts is disabled on this system.This is a safety setting. Allow local scripts for your user only, then activate again:
Set-ExecutionPolicy -Scope CurrentUser RemoteSigned
.\.venv\Scripts\Activate.ps1Answer Y if PowerShell asks. The prompt now starts with (.venv).
pip install -r requirements.txtIf pip prints a warning about a new pip version, ignore it. If the install stops with Filename too long or path too long, go back to A.1 and A.2.
python data/make_dataset.py
Get-Content data\clean\machines.csv -TotalCount 3machine_id,machine_type,site,install_year,rated_power_kw
M001,pump,Toronto,2022,45.0
M002,pump,Montreal,2016,45.0Download the Windows installer from ollama.com/download. Run it. Ollama starts in the background and shows an icon in the system tray. Then, in PowerShell:
ollama pull llama3.2
ollama pull nomic-embed-text
ollama listcd C:\courses\aiopsatlas-ml-data-diagnostics-labs-en
.\.venv\Scripts\Activate.ps1That is all. The packages and the data stay on disk.
Read this appendix only if you are on Linux, macOS, WSL 2 or Git Bash. Every step uses the normal terminal.
Ubuntu or Debian:
sudo apt update
sudo apt install -y python3 python3-venv python3-pip gitFedora:
sudo dnf install -y python3 gitmacOS, with Homebrew:
brew install python@3.12 gitCheck:
python3 --version
git --versionPython 3.12.10
git version 2.49.0Your version numbers will differ. Anything from Python 3.10 works.
mkdir -p ~/courses
cd ~/coursesgit clone https://github.com/hrhouma2/aiopsatlas-ml-data-diagnostics-labs-en.git
cd aiopsatlas-ml-data-diagnostics-labs-en
lsdata docs week01 README.md requirements.txtpython3 -m venv .venvOn Ubuntu, if this fails with ensurepip is not available, install python3-venv (see B.0) and run it again.
source .venv/bin/activateThe prompt now starts with (.venv). Inside the venv, python and pip point to the venv.
pip install -r requirements.txtOn a Mac with Apple Silicon, some packages compile for a minute. That is normal.
python data/make_dataset.py
head -n 3 data/clean/machines.csvmachine_id,machine_type,site,install_year,rated_power_kw
M001,pump,Toronto,2022,45.0
M002,pump,Montreal,2016,45.0Linux:
curl -fsSL https://ollama.com/install.sh | shmacOS: download the app from ollama.com/download and open it once. Then, on both systems:
ollama pull llama3.2
ollama pull nomic-embed-text
ollama listOn Linux, if ollama pull says the server is not running, start it in a second terminal with ollama serve.
cd ~/courses/aiopsatlas-ml-data-diagnostics-labs-en
source .venv/bin/activateAll systems — ModuleNotFoundError: No module named 'pandas'. The venv is not active, or you used the wrong Python. Look for (.venv) at the start of the prompt. Activate again (Step 3).
All systems — FileNotFoundError: data/clean/machines.csv. You are not in the kit folder, or you did not run make_dataset.py. Type cd aiopsatlas-ml-data-diagnostics-labs-en, then Step 5.
All systems — python: command not found inside the venv. On some Linux systems the venv only creates python3. Use python3 instead. Inside the venv, both point to the same file.
Windows only — running scripts is disabled on this system. See A.4: Set-ExecutionPolicy -Scope CurrentUser RemoteSigned.
Windows only — python opens the Microsoft Store. Windows has a fake python alias. Use py -3.12 to create the venv. Once the venv is active, python is the right one.
Windows only — Filename too long during git clone or pip install. Move to a short folder like C:\courses and run git config --global core.longpaths true.
Linux only — The virtual environment was not created successfully because ensurepip is not available. Run sudo apt install python3-venv, delete the .venv folder, and create it again.
macOS and bash — pip: command not found. Use python3 -m pip install -r requirements.txt.
All systems — ollama: command not found. Ollama is not installed, or the terminal was open before the install. Close the terminal, open a new one, try again.