All-Time FIFA Players - Transfermarkt Matching

[EfekanP]

Hi guys,

I'm working on a project to build a unified, aggregated football database by combining data from the FIFA series, Transfermarkt, PES, and Football Manager. The ultimate goal is to create a next-gen mod—similar in spirit to CEP (Career Expansion Patch)—by integrating rich external data like real-life transfers, attributes, and more.

Once the base database is solid, it could potentially serve as a universal football modding resource for career mode editors, editors of other football games, or even custom games.

What I've done so far:

1. Extracted Player Data from FIFA Series (FIFA 2005 onwards)

• Downloaded @Skoczek ’s FIFA DB collection.
• Used DB Master 08 and DB Master 15 to extract all players from each version.
• Collected: firstname, surname, commonname, jerseyname, birthdate, first_appearance, and last_appearance.

Total Players: 99,974

2. Matched Players to Transfermarkt

• Built a script to search each player on Transfermarkt using "firstname + surname" query.
• Verified matches by checking birthdate accuracy.

Matched Successfully: 73,694 players (74% of total)


Issues Encountered:

1. FIFA 07 UCL Birthdate Bug:
Birthdates are off by 2 days—possibly a time zone or data bug.

2. Special Characters & Hyphenation:
Arabic/Korean names often exclude hyphens in FIFA (e.g., Min Jae Kim vs Min-Jae Kim on Transfermarkt).

3. Name Variants (Hypocorisms):
Cases like Andy vs Andrew, Don vs Donald. A reference list of nickname mappings would help improve match accuracy.

4. Common Names vs Full Names:
Searching Roberto Carlos Da Silva Junior won’t return Roberto Carlos. Need to include commonname and jerseyname fields in the search logic.

5. Women Players
Transfermarkt doesn’t list female players, so they’ll need to be filtered out.

6. Squad File Issues (RDBM):
When opening Squad files using RDBM, many player names appear blank or missing. If anyone has a working method to extract those cleanly, I’d love to try it.

CURRENT DATASET (WIP)

PES

• Found a good player dataset from EvoWeb (last updated last year), but it lacks birthdates, which complicates exact matches.
• Open to any ideas or alternate PES datasets.

FM

• FMDB is promising but I’m cautious about making 100,000+ web requests—don’t want to trigger anti-bot or DDoS protections.
• If there’s a better or more efficient way to scrape or source data (e.g., open dumps, APIs), please let me know.

Any suggestion or help would be great.

 

[Skoczek]

for squad files, you gotta look for dcplayernames table in a squad file

 

[EfekanP]

Current progress:

First, I focused on parsing every possible FIFA DB.

I am writing tools, methods that I learned, and my discoveries for the record like a guideline.

DB Structures:
1. Standalone .db files like fifa.db, euro.db, etc. Mostly used in the FIFA 05-10 era and mobile versions of FIFA 12-15.
I used Dmitri's DB packing/unpacking tool to extract the tables.
Alternatively, you can use DBM 08 but I did not try the mobile databases.

2. (Console/XBOX version) DB + XML (like fifa_ng_db.db, fifa_ng_db-meta.xml and euro_ng_db.db, euro_ng_db-meta.xml files)
I used FIFA: Xbox 360 DB Converter to convert them to open via DB Master. Although the ReadMe file says that the tool is for FIFA 12-16 games, I managed to convert newer and older DB files as well.

3. (PC version) DB + XML (like fifa_ng_db.db, fifa_ng_db-meta.xml and euro_ng_db.db, euro_ng_db-meta.xml files)
You can directly open them through DB Master and export the tables.
tokke001 Optimized DB Master

4. Squads files with specific database header (FIFA 13 - present)
Since there are too many of them, I developed a Python script by converting sammygriffiths and GioAc96 tools to extract the files.
I don't know what this header is for exactly but these files can not be opened without this.

databaseHeader = '\x44\x42\x00\x08\x00\x00\x00\x00'

I also tried to convert DB + XML files with the same script but I could not manage to convert the playernames table. The extracted playernames table looks like this:

name,nameid,commentaryid
130837782379272298518072872660666385760246669759851200026001336218960353028103454814719961051660766417736778945565507822958723657208089217264323630167650391307531291898106871807,0,900000
133558416795476448392751808199840543184685532555045339716580052022257847916047994971169133544557241541734860215641499380045884288238242186930403551020746619894404827975484703208,1,900000
136031720810207493733370241218001296916189895985601030761704251700067746380692120923709608304021155694268355035467295621081459863684148887057443611656814854227551766282619585006,2,900000
138257694423465434539928171715139281709108812074824128369684673959491391811045474596009107295368575179765331193758043172717690708498661969952689530949541365219771529906026971637,3,900000
140483668036723375346486102212274144753477412171815844380435263650603092248776951018117647980789037033357501431746498757322032281145864559565848087152633076051574116812398264830,4,900000
142709641649981316153044032709409007797846012268807560391185813939708596292029215161186088025272457397423759048906038259366332978388506382292256634961597678498685435891928793608,5,900000
144935615263239256959601963206543870842214612365799276401936364228814100335281479304254527572432641351703374510683329608312691839788316643835158319257375296712472973161137177105,6,900000
147656249679443406834280898745718028266653475160993416092515040630105398828746807181663599921715311395962185315312653229660651695870059643962205672576462122645473941917064495644,7,900000
150129553694174452174899331763878781998157838591549107137639240307915297293390933134204074183855989138709037979756201323875387170859815546741528292907009788847652542877911745062,8,900000

5. Old Squads files without that database header (FIFA 10 - 14 Xbox Squad files)
They don't have the DB header that I shared in the 4. case, so the script did not work for them. I also tried to open them through IM12 but it did not work as well.

------------------------------------------------------------------------
Player-related Tables

All the DBs are using the teamplayerlinks table to construct team-player links as the table name implies. The table structure is also the same for all the DBs.

About parsing the names of the players, we have different structures. While I am parsing all the files, first I check which tables exist in the extracted tables then check the columns inside some tables.

1. If there is a playerwrite table (FIFA 05-06)
You can retrieve the names from the playerwrite table, you can directly access them with player IDs.

2. Only the players table exists (FIFA 07-10 PC, FIFA 12-15 Mobile)
It is enough to check the players table to get the names.

3. If there is a playernames table with the "playerid" column (FIFA 07-10 Console)
You can retrieve the names from the playernames table, you can directly access them with player IDs.

4. If there is a playernames table with the "named" column (FIFA 11 - present)
Both the players and the playernames tables should be parsed, and the IDs of the names (firstnameid, lastnameid, etc.) present in the players table and you need to look up the playernames table for matching names.

5. If there is a dcplayernames table (Squads files, FIFA 13 - present)
The method is similar to Step 4 but you will only have the dcplayernames table which does not contain all the values. I used the playernames export of each game's launch DBs (DB + XML). Then merge playernames and dcplayernames tables.

------------------------------------------------------------------------

I have still some problems with some DB files but you can check the attached DB list that I managed to parse. I changed the folder system of Skoczek for easier scripting. I used the date values of Skoczek if any, if there is no date I used the release date of each game (based on the platform)

I added the nationality IDs, team IDs (first and last appearances), and gender of each player.

You can check my FIFA Universe - All Time Player ID list from here:

FIFA Player ID List

docs.google.com

Now I am going back to Transfermarkt matching. If you have any suggestions or help I would be glad to hear. Additionally, I can share both the files and my codes if anyone wants.