AidData's Tracking Underreported Financial Flows (TUFF) methodology provides a systematic, transparent and replicable approach to generate open-source, project-level data for non-OECD suppliers of international development finance, such as China, Saudi Arabia, and Qatar.
How are these TUFF-based data collected, and what steps does AidData take to ensure their accuracy and comprehensiveness? This page provides an overview of the 1.2 version of the TUFF methodology. For questions regarding the methodology, please click here or refer to our FAQs page at http://china.aiddata.org/content/frequently_asked_questions.
AidData’s TUFF methodology was designed with two goals in mind:
Targeted searching of Factiva -- a Dow Jones search engine that draws on 31,000 sources from 200 countries—with pre-selected search terms that relate to the official financing activities of non-OECD donors
Cross-checking and updating each project record based on information triangulated from additional sources, such as data and documentation from the development finance institution websites, aid and debt information management systems owned and operated by recipient/borrower countries, implementing agency websites, embassy press releases, NGO reports, and scholarly research
Synthesizing and standardizing project-level information from thousands of supplemental online searches in English, Chinese, Spanish and Arabic
Assigning projects unique identification numbers as they are discovered and entered into the database
Systematically assigning values to 57 variables, including flow type, project status, sector, and flow class. The flow class variable is a unique categorization developed by AidData that assigns the official financing activities of non-OECD donors into ODA-like, OOF-like, and Vague Official Finance categories, which in turn enables apples-to-apples comparison with the official financing activities of OECD-DAC donors.
...and Project-Level Detail
Implementing a rigorous de-duplication process to avoid duplication of project records and overestimation of financial values.
Investigating the sources of project funding to sharply distinguish between official and unofficial finance
Tracking the progression of each project with a unique “status” variable so that one may distinguish between pledges, commitments, projects in implementation, completed projects, and suspended or cancelled projects
Assessing the “health” of each project record by assigning a score based on its comprehensiveness and the quality and diversity of its sources.
To operationalize this approach, our methodology is divided into two stages:
In Stage One, researchers follow a step-by-step guide to identify potential sources of project-level information from Factiva and publicly available official data and documentation from donor/creditor and recipient/borrower governments. Projects are given unique identification numbers as they are discovered and entered into the database. All available project information is input into our china.aiddata.org database, including links to the underlying source documentation. The primary objective of Stage One is to identify and record as many potential projects as possible.
In Stage Two, each project record created in Stage One is carefully assessed and enhanced through targeted Google and Google Scholar searches, using a corpus of relevant search terms developed by AidData staff. Researchers compile and triangulate information from these diverse sources in order to accurately populate as project fields as possible. This process of stage-two refinement is designed to minimize reliance upon "sole-sourced" records.
1) Generate and test a set of keywords which you think have valuable project information.
2) Input search phrase into Factiva and select the time period of the study.
For example, to find China's official financing in Zambia, input the following search terms: (China or Chinese or Chin*) near5 (Zambia or Zambian or Zamb* or Lusaka) AND (assistance or grant or loan or concession* or donat* or donor or interest-free or interest or preferential or joint fund or finance or package or aid)
3) Open all sources with potentially valuable project-level information.
4) Extract all project information available from the new article and paste the article into the description.
5) Use the following naming convention to title the project
[Donor Country or Organization] [Status] [Activity/Flow type] [Dollar Amount] [Recipient/Location]
Aid and debt information management systems owned and operated by recipient/borrower governments can provide valuable project details; however, the comprehensiveness of the information presented in these systems varies from country to country. Such systems can supplement other data collection efforts but typically they do not capture all incoming financial flows from non-OECD sources.
For for more information about reporting discrepancies between aid information management systems (AIMS), see http://www.cn.undp.org/content/dam/china/docs/Publications/UNDP-CH-SS-Demand-Driven%20Data.pdf
To undergo this process of collecting project-level data from an aid/debt information management system, follow these steps:
1) Navigate to the public interface of the target recipient’s aid/debt information management system.
2) Search for project information for your development finance supplier of interest. It is important to note that the design and detail of aid/debt information management systems will vary across countries. Researchers must exercise discretion in how they pull information from these systems.
3) Repeat steps 3 through 5 from Factiva searches. If RAs have already identified projects through Factiva that are also captured in an aid/debt information management system, supplement the existing project entry with any new information and cite the aid/debt information management system.
The websites of development finance institutions remain key repositories of project-level information on the official financing of non-OECD donors. For example, Chinese embassy and Economic and Commercial Counselor’s Office (ECCO) websites contain rich details regarding the official finance activities of China. Below are step-by-step instructions for extracting project information from these resources, using Chinese ECCO’s as an example. Similar to AIMS, the comprehensiveness of information will vary on a country-by-country basis. For both ECCOs and embassies, it is vital that researchers perform searches on both the English and Mandarin pages for these websites.
1) For ECCO websites examine the “Bilateral Relations” page for any project information.
2) Create or edit
project records with the information provided via the “Bilateral Relations”
page and cite accordingly.
3) For embassy websites examine the “Major Events” “Bilateral Relations” or “News” page.
4) Create of
edit project records with any new information on these pages.
Field research undertaken by scholars (in particular, area studies specialists) can provide another valuable source of project-level information. Academic publications can validate existing projects records, reveal new projects not uncovered in other TUFF searches, or provide new details about a project.
Below is a step-by-step guide to searching for relevant academic articles.
1) Input the following search phrase into Google Scholar.
[donor country] OR [donor demonym] OR [donor countr*] AND [recipient country] OR [recipient demonym] OR [recipient countr*] OR [capital of country] AND assistance OR grant OR loan OR concession* OR donat* OR donor OR interest-free OR interest OR preferential OR "joint fund" OR invest* OR finance OR package OR aid
1: China OR Chinese OR
Chin* AND Angola OR Angolan OR Angol* OR Luanda AND assistance OR grant OR loan
OR concession* OR donat* OR donor OR interest-free OR interest OR preferential
OR "joint fund" OR invest* OR finance OR package OR aid
2) Review articles with potentially relevant information.
3) If an article contains useful information, create new project entries or supplement existing ones as needed.
4) After extracting all useful information from an article, review the paper’s citations for additional sources of project-level information. Also, use the “cited by” and “related articles” features in Google Scholar to find articles that discuss similar topics and that may have useful project-level information. Repeat the above outlined steps for those articles.
The objectives of Stage Two are to triangulate project information gathered in Stage One and perform final quality assurance checks to prepare data for publication. During Stage Two, researchers should prioritize writing a clear and concise project description, confirming “Transaction Amount” or finding one if the field is blank, and populating any missing fields. Researchers should remember that all fields entered during Stage One are subject to change if new information becomes available during subsequent rounds of searching. Please find the steps for Stage 2 outlined below:
1) Pull idiosyncratic search terms from the articles and press releases uncovered during Stage One searches. For example, a loan for the construction of a dam might include the following terms: Export-Import Bank, Ethiopia, Sinohydro, dam, Omo river.
2) Input search terms into Google and review the first three pages of results.
3) Refine the search phrase if more information becomes available.
4) Repeat steps 1-3 using mandarin search terms.
5) Update the project record with any new information you have uncovered and cite any new sources of information.
6) Draft a project description using the following template:
7) If there are any notable information gaps remaining, or a coder made an assumption when recording project information, note that using a “STAFF_NOTE”. For example “STAFF_NOTE: It is unclear whether this project is financed with a loan or export credit”
8) In addition to searching Google, researchers should also review
recipient/borrower ministry websites and implementing agency websites for