FB15K: Loading FB15K-237 into Neo4j

In the previous article, I explained what FB15K and FB15K-237 are, why inverse relations matter, and what the benchmark is designed to measure.

This time, I look at the actual files.

I download FB15K-237, load its knowledge-base triples into Neo4j, and explore the resulting graph with Cypher.

The import itself is simple because the source data is already expressed as triples. The more interesting result appears after the import: the graph has rich structure, but almost no human-readable context.

What we will build

For this first exploration, each Freebase MID becomes an Entity node, and each source triple becomes a RELATION relationship.

(:Entity {mid})-[:RELATION {name, split}]->(:Entity {mid})

The relationship stores:

name: the original Freebase relation
split: train, valid, or test

This keeps the source values unchanged and preserves the benchmark boundary inside Neo4j.

Downloading FB15K-237

I used the package published by Microsoft:

FB15K-237 Knowledge Base Completion Dataset

The downloaded archive has a name similar to:

FB15K-237.2.zip

After extraction, it contains files such as:

README.txt
train.txt
valid.txt
test.txt
text_cvsc.txt
text_emnlp.txt

For this Neo4j import, I use only the knowledge-base triples:

train.txt
valid.txt
test.txt

The two text_*.txt files contain textual dependency paths derived from ClueWeb12. They are useful for research combining text and knowledge bases, but they are not required for this graph import.

Inspecting the triple format

Each line contains three tab-separated values:

/m/0grwj    /people/person/profession    /m/05sxg2

The format is:

subject MID    relation    object MID

It maps directly to a property graph:

(Entity)-[Relation]->(Entity)

The two MIDs become nodes, and the Freebase relation becomes a relationship between them.

Choosing a Neo4j model

There are at least two reasonable ways to represent the 237 Freebase relation names.

Option 1: one Neo4j relationship type per relation

(:Entity)-[:PEOPLE_PERSON_PROFESSION]->(:Entity)

This makes each relation visible as a Neo4j relationship type, but it requires transforming the original path-like names into valid type names.

Option 2: one relationship type with properties

(:Entity)-[:RELATION {
  name: "/people/person/profession",
  split: "train"
}]->(:Entity)

For this article, I use the second model.

It has several advantages for an introductory import:

the original relation string is preserved
all files can use the same Cypher pattern
the model is easy to query and explain
it works without generating 237 relationship types

The trade-off is that queries filter on r.name instead of using a specific Neo4j relationship type.

Japanese note:
237種類の関係をNeo4jのリレーションタイプに変換する方法もあります。ここでは元のFreebase関係名を変えずに保持し、ロード処理を分かりやすくするため、:RELATION の name プロパティに保存します。

Preparing the import directory

Place the three files under the Neo4j import directory.

import/
└── fb15k-237/
    ├── train.txt
    ├── valid.txt
    └── test.txt

fb15k data directory

The exact path depends on the installation.

Neo4j Desktop: open the database folder and use its import directory.
Docker: mount a local directory to /var/lib/neo4j/import.

Creating a uniqueness constraint

Before importing the triples, create a uniqueness constraint for the MID property.

CREATE CONSTRAINT entity_mid_unique IF NOT EXISTS
FOR (e:Entity)
REQUIRE e.mid IS UNIQUE;

This allows MERGE to find an existing entity instead of creating the same MID more than once.

Loading the training data

The files do not contain a header row, so the fields are accessed by position.

LOAD CSV FROM 'file:///fb15k-237/train.txt' AS row
FIELDTERMINATOR '\u0009'
CALL (row) {
  MERGE (subject:Entity {mid: row[0]})
  MERGE (object:Entity {mid: row[2]})
  CREATE (subject)-[:RELATION {
    name: row[1],
    split: 'train'
  }]->(object)
} IN TRANSACTIONS OF 10000 ROWS;

The query uses MERGE for nodes and CREATE for relationships.

This is intentional:

an entity with the same MID should be reused
every source row should become one relationship

Using MERGE for the complete relationship pattern could collapse duplicate rows. For a benchmark dataset, preserving the source records is usually clearer.

Loading validation and test data

The validation file uses the same structure:

LOAD CSV FROM 'file:///fb15k-237/valid.txt' AS row
FIELDTERMINATOR '\u0009'
CALL (row) {
  MERGE (subject:Entity {mid: row[0]})
  MERGE (object:Entity {mid: row[2]})
  CREATE (subject)-[:RELATION {
    name: row[1],
    split: 'valid'
  }]->(object)
} IN TRANSACTIONS OF 10000 ROWS;

The test file is imported in the same way:

LOAD CSV FROM 'file:///fb15k-237/test.txt' AS row
FIELDTERMINATOR '\u0009'
CALL (row) {
  MERGE (subject:Entity {mid: row[0]})
  MERGE (object:Entity {mid: row[2]})
  CREATE (subject)-[:RELATION {
    name: row[1],
    split: 'test'
  }]->(object)
} IN TRANSACTIONS OF 10000 ROWS;

The examples use CALL { ... } IN TRANSACTIONS so the import does not depend on one large transaction. The batch size can be adjusted for the available memory and Neo4j configuration.

Verifying the import

Count the entities:

MATCH (e:Entity)
RETURN count(e) AS entities;

Count all imported triples:

MATCH ()-[r:RELATION]->()
RETURN count(r) AS triples;

Count the distinct Freebase relation names:

MATCH ()-[r:RELATION]->()
RETURN count(DISTINCT r.name) AS relations;

The commonly published FB15K-237 statistics are:

Item	Count
Entities	14,541
Relation names	237
Training triples	272,115
Validation triples	17,535
Test triples	20,466
Total triples	310,116

These are the counts reported for the official FB15K-237 release. If the imported counts differ, check whether the files were modified or whether rows were skipped or duplicated during import.

Checking the benchmark splits

Because each relationship stores its source split, the counts can be checked directly:

MATCH ()-[r:RELATION]->()
RETURN r.split AS split, count(*) AS triples
ORDER BY split;

This should return one row for train, valid, and test.

Keeping the split is useful for two reasons:

the complete graph can be explored in Neo4j
the original benchmark boundary remains visible

Without the split property, the imported graph would lose an important part of the dataset’s meaning.

Finding common relations

The following query shows the most frequent Freebase relations:

MATCH ()-[r:RELATION]->()
RETURN r.name AS relation, count(*) AS triples
ORDER BY triples DESC
LIMIT 20;

A list of 237 relation names is difficult to understand at once. Ranking them by frequency provides a quick view of which kinds of facts dominate the graph.

Finding highly connected entities

The MIDs are not readable, but the graph structure can still be examined.

MATCH (e:Entity)
RETURN e.mid AS mid, COUNT { (e)--() } AS degree
ORDER BY degree DESC
LIMIT 20;

A high degree means that the entity participates in many triples. It does not reveal what the entity represents, but it identifies structural hubs in the benchmark.

Looking at one entity

Choose one MID and expand its neighborhood:

MATCH path = (e:Entity {mid: '/m/0grwj'})-[r:RELATION]-(other:Entity)
RETURN path
LIMIT 50;

fb15k relations

Neo4j Browser displays a connected graph, but the first result is less readable than a typical Neo4j demonstration.

Most nodes still look like this:

/m/0grwj
/m/05sxg2
/m/02...

The graph contains connections, but the nodes do not contain readable names.

Exploring paths

Even without labels, Cypher can reveal structural patterns.

Find paths up to two hops from an entity:

MATCH path = (:Entity {mid: '/m/0grwj'})-[:RELATION]->{1,2}(:Entity)
RETURN path
LIMIT 50;

Find a shortest path between two known MIDs:

MATCH path =  ANY SHORTEST
  (a:Entity {mid: '/m/09c7w0'})
  -[:RELATION]-{1,10}
  (b:Entity {mid: '/m/08mbj5d'})
RETURN path;

fb15k any shortest

Find entity pairs connected by more than one relation name:

MATCH (a:Entity)-[r:RELATION]->(b:Entity)
WITH a, b, collect(DISTINCT r.name) AS relations
WHERE size(relations) > 1
RETURN a.mid AS subject,
       b.mid AS object,
       relations
LIMIT 20;

These queries confirm that FB15K-237 is a real multi-relational graph. What it lacks is a human-readable semantic layer.

What the imported graph reveals

At first, I wondered whether the import was incomplete.

There were thousands of nodes and hundreds of thousands of relationships, but almost every node displayed only a MID. The triple files contained no names, descriptions, dates, sources, or images.

Nothing was missing from the import. That is what the benchmark files contain.

This made the purpose of FB15K-237 much clearer.

A knowledge graph completion model can use /m/0grwj as an entity identifier and learn a vector embedding for it. It can learn from topology without knowing the entity’s human-readable name.

A person exploring the graph has different requirements. We want to understand what each node means, where a fact came from, and whether it is current.

Japanese note:
ロード後にMIDしか表示されないのは失敗ではなく、元のベンチマークが主に構造学習を目的としているためです。人が知識を理解するには、名称、型、出典、時間などの追加情報が必要になります。

Relation names contain partial meaning

The relation names are more readable than the entity MIDs.

Examples include:

/people/person/profession
/location/location/contains
/film/film/genre

These path-like names carry some domain and type information.

However, some Freebase relations represent collapsed two-hop paths. Intermediate Compound Value Type nodes from the original Freebase model were removed when the benchmark triples were produced.

That simplification is convenient for link-prediction experiments, but it means that FB15K-237 is not a complete reconstruction of the original Freebase data model.

Adding readable names

The old Freebase API is no longer available, but Google still publishes:

the final Freebase RDF dump
deleted triples
Freebase-to-Wikidata mappings

These resources are available from the Freebase data dumps page.

A practical enrichment process would be:

collect the MIDs used in FB15K-237
extract matching labels and mappings from the Freebase dump
add properties such as name and wikidataId to the Neo4j nodes

This would make the graph easier for people to explore without changing the original benchmark triples.

The complete Freebase dump is very large, so extracting only the MIDs used by FB15K-237 is more practical than importing the entire dump for this purpose.

What the import taught me

Loading FB15K-237 into Neo4j was technically straightforward. The source files were already triples, so the mapping to nodes and relationships was direct.

A simple first model is enough:

use one Entity node label
preserve each MID as a unique node property
store the original relation name on RELATION
keep the train, valid, and test split on each relationship

After the import, Cypher can answer structural questions such as:

Which relations are most frequent?
Which entities have the highest degree?
How are two MIDs connected?
Which entity pairs have multiple relations?

The more important lesson came from what the graph did not contain.

Benchmark graph	Human-oriented knowledge graph
Entity IDs	Readable labels and descriptions
Relation names	Definitions and ontology
Topology	Provenance and temporal context
Fixed splits	Update and deletion rules
Link-prediction targets	Explanation and governance

FB15K-237 is valuable because it isolates one problem: predicting missing links. It should not be mistaken for a complete model of practical knowledge management.

The MID-only graph is therefore not a disappointing result. It shows exactly what many knowledge graph completion models receive: identifiers, relation names, benchmark splits, and graph structure.

Sometimes the best way to understand a benchmark is to inspect what it leaves out.