Every analysis at NorthPeak starts with a file you have never opened. The wrong first move is to compute; the right one is to look. Four pandas commands tell you what a file contains: head, info, describe and value_counts. This lesson runs them on data/clean/readings.csv and shows how to read their output.
Each command answers one question about the table.
| Command | The question it answers |
|---|---|
df.head() | What do the rows look like? |
df.info() | Which columns exist, what type, how many values are filled? |
df.describe() | For each number column: count, mean, min, max, quartiles? |
df["col"].value_counts() | For a category column: how many rows per value? |
Run them in this order, every time you open a new file. It takes one minute. It saves hours.
readings.info() on data/clean/readings.csv prints one line per column:
RangeIndex: 14600 entries, 0 to 14599
Data columns (total 10 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 reading_id 14600 non-null int64
1 machine_id 14600 non-null str
2 date 14600 non-null str
3 load_pct 14600 non-null float64
...
9 fault_next_7d 14600 non-null int64Read the middle column. 14600 non-null means every row has a value. When a column shows a smaller number, values are missing. Read the last column too. date is str, not a date. pandas does not guess dates by itself.
readings.describe().round(1) gives the numbers. For temperature_c: mean 46.2, min 3.0, max 93.0. For load_pct: min 2.8, max 100.0. These ranges look right for a machine. Later, on the raw file, they will not.
readings["fault_next_7d"].value_counts() counts the label: 13,515 zeros and 1,085 ones. And machines["site"].value_counts() gives Montreal 20, Toronto 13, Quebec City 7.
describe() only shows number columns. It says nothing about machine_id, date or site. Beginners run describe(), see no problem, and miss a site written montreal in lowercase. For text columns, use value_counts(). It shows every spelling.