Week 3 — Lesson 4: Graph databases and knowledge graphs

2 min

Many NorthPeak questions are chains of relations: an incident happened on a machine, the machine is located at a site, the incident has a category. A graph stores each thing as a node and each relation as a directed edge, so such a question becomes a path. The kit ships the NorthPeak graph as two CSV files, data/graph/nodes.csv with 207 nodes and data/graph/edges.csv with 390 edges. This lesson reads them, counts them, and walks a three-hop path with pandas.

Nodes, edges, paths

A node is a thing: a machine, a site, an incident. An edge is a relation between two nodes, with a direction and a name: M001 LOCATED_AT Toronto. A knowledge graph is a set of nodes and edges that describes a domain.

Relational tables can hold the same facts. What changes is the question. In a graph you ask for a path: start at a node, follow edges, arrive somewhere. "Which sites have the most bearing incidents?" is a path of three hops: category, incident, machine, site.

Graph databases, such as Neo4j, are built for these hops. They store each edge as a direct pointer. A ten-hop question costs ten pointer jumps, not ten joins.

On the NorthPeak data

data/graph/nodes.csv lists the 207 nodes. Each has an id, a label and a name.

text
node_id,label,name
M001,Machine,M001
Montreal,Site,Montreal
pump,MachineType,pump
bearing_wear,FaultCategory,bearing_wear
INC-0001,Incident,INC-0001

Count them: 155 incidents, 40 machines, 5 categories, 4 types, 3 sites. That is 207.

data/graph/edges.csv lists the 390 edges: source, relation, target.

text
source,relation,target
M001,LOCATED_AT,Toronto
M001,IS_A,pump
INC-0001,HAPPENED_ON,M001
INC-0001,HAS_CATEGORY,overheating

Read each edge as a sentence. "M001 is located at Toronto." "INC-0001 happened on M001." Two edges per machine, two per incident: 80 plus 310 is 390.

You do not need Neo4j to walk this graph. One hop is a merge between two edge tables. HAPPENED_ON merged with HAS_CATEGORY on the incident gives machine, incident, category. Merge again with LOCATED_AT on the machine and you have the site. Three hops, two merges.

A common mistake

Direction matters. INC-0001 HAPPENED_ON M001 goes from the incident to the machine. If you look for edges where source == "M001", you get LOCATED_AT and IS_A only. The five incidents of M001 are in target. Always ask: which end of the edge am I standing on?