Building an Open-Source PEP Database for All 54 African Countries
Every regulated financial institution on earth is required to check whether its customers are politically exposed persons. The commercial databases that answer that question are excellent on European and North American officials and thin exactly where I work. So I built AfricaPEP, an MIT-licensed PEP screening platform covering all 54 African Union member states, and I maintain it in production. This is the engineering story: the parts that worked, the thresholds that matter, and the bugs that taught me the most.
Production figures from the live /stats endpoint, 14 July 2026. The README badge reads the same endpoint, so the repo never overstates the count.
The gap, concretely
FATF Recommendation 12 obliges institutions to identify PEPs, apply enhanced due diligence, and extend that scrutiny to family members and close associates. Compliance teams buy this data from Dow Jones, LexisNexis, or Refinitiv, and those products are built primarily on structured Western sources: parliamentary registers, gazettes, electoral rolls. Where government data infrastructure is thin, so is the coverage, and for much of Africa it is very thin. The practical result is a Lagos or Accra compliance officer whose screening tool reliably flags a German backbencher but returns nothing for a neighboring country's deputy minister.
For African fintechs there is a second problem: price. Commercial PEP feeds are enterprise-priced for banks. A licensed remittance startup in Kampala has the same legal screening obligation as Barclays and none of the budget. That combination, a mandatory obligation plus unaffordable data, is exactly the kind of gap open source exists to fill.
One data source, chosen deliberately
AfricaPEP currently has exactly one upstream source: Wikidata, queried through its public SPARQL endpoint, one parameterized query set per country. I want to be precise about this because it is both the project's biggest strength and its most honest limitation.
The strength is provenance. Every Wikidata entity is community-maintained and referenced, and every AfricaPEP profile carries its Wikidata QID as its identifier. When a screening hit comes back, an analyst can walk from the API response to the Wikidata entry to the underlying references. The database stores 2.3 million source references for this reason. In compliance, a record you cannot trace is a record you cannot defend to an auditor, so traceability beat volume in every design decision.
The limitation is depth. Wikidata is strong on national-level figures (heads of state, ministers, parliamentarians, senior judges) and weak on the local officials, state-enterprise boards, and military ranks that national gazettes would provide. The scraper framework is built for adding national sources, and the repo tracks Ghana's Parliament and Nigeria's National Assembly as open contribution issues, but I will not claim data I do not have. The README says the same thing.
The ontology fight nobody warns you about
“Query Wikidata for African politicians” sounds like one SPARQL query. It is four, and the reasons are instructive. The naive query (people holding a position whose country is X) misses thousands of people whose positions are linked by jurisdiction rather than country. Adding a jurisdiction branch still misses people with a citizenship and a position but no country linkage at all. So a third branch takes every citizen holding any position and filters it through Wikidata's class hierarchy, keeping only positions that are subclasses of public office.
Then that filter itself turned out to have holes: for several small countries, positions like “President of Mauritius” are simply not classed under public office in Wikidata's ontology, so the class walk silently drops a head of state. The fix that shipped last week is a keyword fallback with a denylist (so “FIFA official” and “goodwill ambassador” do not sneak in), plus a fourth branch that recovers people marked occupation-politician with no position property at all. Each branch is isolated, so one timing out can never silently regress the baseline. Coverage work on community data is not glamorous, but it is the difference between “covers 54 countries” as a marketing line and as a fact.
Architecture: a graph of record, a relational index
Two databases for one dataset sounds indulgent until you look at the two jobs. PEP data is fundamentally a graph: a person held positions at organisations, is family of other people, is cited by source records. FATF explicitly extends risk to relatives and close associates, so “who is connected to this person” is not a nice-to-have query, it is the product. Neo4j holds that graph as the source of truth, with a (Person)-[:HELD_POSITION {start, end}]->(Position) shape and explicit FAMILY_OF and ASSOCIATED_WITH edges.
Screening, though, is a search problem: fuzzy name lookup over 34 thousand rows, fast, with a ranked result. PostgreSQL with a pg_trgm trigram index does that better than any graph traversal, so the graph syncs one-way into Postgres, which also keeps the screening log (every query an institution runs is auditable). The index is disposable by design: it can be rebuilt from the graph at any time, which means schema experiments on the search side never threaten the data of record.
Two record-keeping principles run through the whole system. First, nothing is deleted: when an official leaves office the HELD_POSITION relationship gets an end date, because a former minister is still a PEP under FATF guidance, with risk that decays rather than vanishes. Second, after an early bug I will describe below, every node id is derived from content (the Wikidata QID for people, a content hash for positions), never from the run that created it.
Name matching: cheap and wide, then expensive and careful
Matching African names across colonial languages, Arabic transliteration, and local orthography is the hardest technical problem in the project. The same person can legitimately appear as Mohammed, Muhammad, Mohamed, or Muhammadu depending on who transliterated. Prefixes like Al-, El-, Ben, and Ibn attach and detach across records. French and Portuguese diacritics survive in some sources and are stripped in others.
The design is a classic two-stage retrieval-and-rerank. Stage one asks Postgres for the 50 most trigram-similar names at a threshold deliberately loosened to 0.3 below the screening threshold: cheap, indexed, recall-oriented. Stage two re-scores each candidate in Python as the maximum of an orthographic score (the better of rapidfuzz token-sort and Jaro-Winkler, so word order and typos) and a phonetic score (Metaphone plus Soundex per token, so Mohammed and Muhammad collide to the same codes). Every profile also stores generated name variants (surname-first forms, initials, prefix alternates like Abdoul/Abdul/Abdel and ben/ibn/bin, diacritic-folded forms), and the scorer runs against all of them, taking the best.
Does it work? The repo carries a 23-pair adversarial evaluation fixture, small but grounded: real hard cases (transliteration pairs that must match, similar-looking names of different people that must not) each pinned to Wikidata QIDs so ground truth is checkable. The measured numbers, published in the README and reproducible with one script:
| Decision rule | Precision | Recall | F1 |
|---|---|---|---|
| Screening gate (0.75, recall-first) | 0.71 | 1.00 | 0.83 |
| High gate (0.90), orthographic only | 0.91 | 0.83 | 0.87 |
| High gate (0.90), orthographic + phonetic | 0.92 | 1.00 | 0.96 |
Read the first and third rows together and you see the design intent. The screening gate accepts precision 0.71 to hold recall at 1.00, because in PEP screening a false positive costs an analyst a minute and a false negative costs the institution a regulatory finding. Adding phonetics to the strict gate is what buys back the transliteration recall that pure string similarity loses. And one honest limitation, documented in the repo rather than hidden: since the trigram stage gates everything, a phonetic-only match with low trigram overlap can fail to surface at all. A phonetic candidate index is on the roadmap.
Entity resolution: the same scorer, three different gates
Here is the design decision I would defend in any AML architecture review: match scores are reused across the system, but the threshold depends entirely on what the decision costs.
Screening at 0.75 shows results to a human who makes the call. Merging two profiles into one is a different act entirely: get it wrong and you have corrupted the database, possibly attaching one person's political history to another's name. So auto-merge demands 0.85 orthographic similarity, or 0.90 phonetic similarity with corroborating evidence: an exact birth date match or strong position overlap. Two names that merely sound alike never merge on sound alone. Candidate pairs are found by blocking on country plus surname initial, and anything landing between 0.70 and 0.85 goes to a review queue instead of being decided silently.
On top of the incremental resolver sits an offline probabilistic pass: Splink, the Fellegi-Sunter linkage library from the UK Ministry of Justice, running over DuckDB. It is deliberately kept out of the runtime image (it is a 400MB dependency) and deliberately conservative: auto-merge only at 99 percent match probability with corroboration, human review from 90 to 99. The AML framing writes the policy for you: a duplicate profile is mildly annoying, a false merge is a data integrity incident.
Two bugs worth admitting to
The bugs that shaped the system are more instructive than the features. First, the empty-metaphone merge: Splink's blocking used phonetic surname codes, and for names in non-Latin scripts the Metaphone of the ASCII-folded form is an empty string. Empty equals empty, so unrelated Amharic and Arabic names started false-merging on a shared blank key. The fix stores blank derived fields as SQL NULL, which never equals anything. If you build matching on multilingual data, this class of bug is waiting for you wherever a derived field can be empty.
Second, non-deterministic ids. Early versions generated fresh UUIDs for Position and Organisation nodes on every scrape run, so weekly re-scrapes quietly duplicated the graph's scaffolding around stable Person nodes. The fix derives every id from content: people key on their Wikidata QID, positions on a hash of what they are. Re-running the entire pipeline is now idempotent, which is the property that makes a weekly automated re-scrape safe at all. Related, and subtler: position term dates originally lived on the Position node, which is shared across everyone who ever held the office. Two presidents, one start date. Term dates now live on the HELD_POSITION relationship where they belong. Graph modeling mistakes are quiet like that: the query returns something plausible, just not the truth.
Running it in production
The whole stack ships as one docker compose file and runs on a single VM: FastAPI with four workers, Neo4j, Postgres, and a scheduler container that re-scrapes all 54 countries every Sunday at 02:00 UTC and syncs the search index at 06:00. The live instance serves pep.patrickaiafrica.com. A real screening call looks like this:
POST https://api-pep.patrickaiafrica.com/api/v1/screen
{ "name": "Nana Akufo-Addo" }
{
"query": "Nana Akufo-Addo",
"threshold": 0.75,
"total_matches": 2,
"matches": [
{
"pep_id": "wd:Q718601",
"matched_name": "Akufo-Addo",
"match_score": 0.8,
"pep_tier": 1,
"risk_level": "high",
"nationality": "GH",
"datasets": ["africapep-wikidata"],
"explanation": {
"name_similarity": 0.8,
"method": "orthographic_phonetic_best"
}
},
...
]
}Note the shape: the id is a Wikidata QID you can look up, the tier maps to FATF Recommendation 12 (three tiers, keyword-classified from position titles: heads of state and central bank governors at tier 1, legislators and judges at tier 2, mayors and deputy ministers at tier 3), and every match carries an explanation object. The response schema deliberately mirrors the commercial screening APIs, so swapping AfricaPEP in beside an existing provider is a field-mapping exercise, not an integration project.
The production hardening is the unglamorous list you would expect, and it exists because the API is public: per-route rate limits (60 screenings a minute, 20 batches), request-size caps, constant-time API key comparison, CORS locked to the frontend origin, Swagger disabled in production, structured error responses, and an optional mode that stores only SHA-256 hashes of screening queries so an institution's customer names never sit in my logs. CI runs 175 tests on Python 3.11 and 3.12, and a separate weekly GitHub Action smoke-tests the live production endpoint: health, a floor on the profile count, and a real screening round-trip. When a Wikidata ontology change or a bad deploy degrades the dataset, I want a red workflow before a user tells me.
Open source as a compliance feature, not a distribution hack
I released AfricaPEP under MIT in March 2026, and the v1.1.0 release in July carries work from five external contributors: API documentation, pagination headers, date-of-birth extraction, local dev tooling, country flag data. Getting an open-source project to the point where strangers can contribute took specific engineering: a bundled 510-record offline sample dataset so make seed-sample gives a populated local API in about a minute with no live scraping, issue templates for country data validation so compliance professionals can contribute expertise without writing code, and a roadmap with explicit non-goals. The two that matter: the data will never be sold, and the API will never be gated.
For screening data specifically, open source is not just a distribution choice, it is an auditability argument. A compliance officer can read the exact matching thresholds, run the evaluation script, inspect the provenance of any profile, and self-host the entire stack inside their own perimeter. No commercial provider offers that, at any price.
What I would tell you if you were building one
- Put provenance in the schema on day one. QID identifiers and 2.3 million source references cost almost nothing at ingest and are unpurchasable retroactively.
- Set thresholds by the cost of the decision, not by the score distribution. Screening, auto-merge, and offline dedup use the same similarity machinery at 0.75, 0.85, and 0.99 because a miss, a wrong merge, and a silent batch merge are different failures.
- Make ingestion idempotent before you automate it. A weekly cron re-scrape is only safe once re-running the pipeline twice produces the same graph.
- Evaluate on adversarial pairs, not random ones. Twenty-three hard, QID-grounded cases caught more regressions than any large random sample would, and the fixture doubles as documentation of what the matcher promises.
- Publish your limitations. The README states that Wikidata is the sole source and that the trigram stage can gate phonetic recall. In compliance tooling, a documented limitation is a feature; a discovered one is an incident.
The project is live, MIT-licensed, and open to contributors: national data sources (Ghana's Parliament and Nigeria's National Assembly are specced as first scrapers), Arabic transliteration for North African names, and a FollowTheMoney export for OpenSanctions interoperability are all tracked as open issues. Screen a name at pep.patrickaiafrica.com, read the code at github.com/PatrickAttankurugu/AfricaPEP, and if you work in African compliance and can validate your country's data, there is an issue template with your name on it. For the graph side of this problem, I wrote a companion piece on mapping beneficial ownership networks with Neo4j.
