How to plan and manage an individual participant data meta‐analysis. An illustrative toolkit

Individual participant data meta‐analyses (IPD‐MAs) have several benefits over standard aggregate data meta‐analyses, including the consideration of additional participants, follow‐up time, and the joint consideration of study‐ and participant‐level heterogeneity for improved diagnostic and prognostic model development and evaluation. However, IPD‐MAs are resource‐intensive and require careful budgeting of time from data contributing groups, a dedicated management team, diversity of expertise, clearly documented data sharing and authorship agreements, and consistent and clear communication. We present a toolkit to facilitate the implementation and management of IPD‐MAs, from study recruitment to retrospective harmonization. The toolkit was developed and refined over our work on multiple multinational IPD‐MA projects over the last 13 years. The toolkit's budget and email templates, agreements, project management spreadsheets, and standard operating procedures are meant to facilitate routine IPD‐MA tasks to expedite implementing and managing future IPD‐MA projects.

standard operating procedures are meant to be tailored to each team's needs to facilitate routine IPD-MA management tasks.

Potential impact for Research Synthesis Methods readers outside the authors' field
• This IPD-MA toolkit is customizable for IPD-MAs in any field with any outcome.• IPD-MAs require the effective management of large multinational teams.
Following tailored standard operating procedures and using field-tested templates may support coordinated and successful execution of IPD-MAs on time and within budget.

| INTRODUCTION
2][3][4][5] IPD-MAs have several advantages over aggregate data meta-analyses, pooling point estimates and their uncertainty across related studies.Advantages of IPD-MAs include joint consideration of participant-and study-level sources of heterogeneity; accounting for effect measure modification at the participant and study levels; consistent evaluation of confounding, interaction, and non-linear effects; standardization of exposure and outcome variables as well as their definitions; and minimizing publication and reporting bias by including participant-level data from all related studies, published or not. 1,6,7IPD-MAs also allow for additional follow-up from longitudinal studies and the inclusion of additional variables not reported in primary studies. 8Despite these benefits, IPD-MAs are resource-intensive and require extensive international collaborations and sufficient resources to support project management, data and metadata collection, and retrospective harmonization and analysis of participant-level data. 1,6,9,10hile specific guidance already exists on how to conduct an IPD-MA, 1,2,5,11 readily available and adaptable templates and standard operating procedures (SOPs) that facilitate the planning, implementation and management of an IPD-MA are currently lacking.This publication presents an illustrative toolkit to support key aspects of IPD-MAs in any field and for any outcome including planning, budgeting, and conducting data harmonization.The toolkit templates are available for download on OSF and Google Drive. 12,13The toolkit was developed and refined through our work with the DEPRESSD, 14 Zika Virus Individual Participant Data, 15 and ReCoDID Consortia. 16The toolkit includes templates for common project management, data checking, and harmonizationrelated tasks, including a budget template, SOPs, and email templates to facilitate communication with data contributing groups.The templates were developed in the context of an IPD-MA of an emerging pathogen.They should be tailored and adapted to the specific research questions, context, and outcomes of interest for a particular IPD-MA.

| Identify eligible studies
The IPD-MA management team can identify eligible studies through a systematic review of published and grey literature and snowball sampling, including referrals from other investigators.Where eligibility cannot be determined from published materials, the IPD-MA management team can contact investigators to determine eligibility (see investigator contact SOP, Table 3; email template 1.0, Table 2).The searches can be refined and repeated as more studies are published.In the case of an IPD-MA of an emerging pathogen, novel drug, or new application of an existing drug, the inclusion or exclusion criteria may be refined as the understanding of the exposure or outcome of interest evolves.Both eligible and ineligible studies can be documented in the project management sheet (Table 1) to prevent a re-review of study eligibility when the search for eligible studies is updated.

| Recruit eligible studies
Once study eligibility is confirmed, the study's principal investigators (PIs) can be formally invited to participate (email template 2.0 and 2.1; Table 2).Invitation emails that include the IPD-MA protocol and FAQ document can be followed up by a phone or video call to introduce the project and answer initial questions (email templates 3.0 and 3.1; Table 2).This call provides an opportunity for the relationship-building needed to facilitate trust and promote active engagement in the IPD-MA process.
Following the recruitment call, the IPD-MA management team can get in touch with the study team via email to request the documentation necessary to support the study's inclusion in the IPD-MA, including a signed letter of agreement for data sharing, use, and publication; documentation of ethics review and approval; codebooks or data dictionaries; and if possible, a few rows of de-identified data (email template 4.0; Table 2).Obtaining a few rows of deidentified data can help the harmonization team understand the data structure and familiarize themselves with the data formats.Multiple follow-ups may be required to obtain the required study documentation; the email templates 4.1-4.4(Table 2) account for different follow-up scenarios.
Communication with study teams can be logged in the project management sheet, including failed attempts at contacting investigators.The SOP for reaching out to new investigators provides suggested practices for communicating with study investigators (Table 3).The file organization SOP provides suggestions for naming conventions and file organization followed in prior IPD-MAs to ensure consistency in where team members save study-related documentation (Table 3).Study team contact information and receipt of the study-related documentation can be recorded in the project management sheet.

| Develop the master codebook
Researchers can use the master codebook template to record the core variables to which IPD from contributing studies will be harmonized.The template master codebook includes columns for the IPD-MA variable names, types, definitions, values, and notes from the harmonization team, the IPD-MA coding team, or external experts overseeing harmonization decisions.The template also includes a sheet with formulas for tracking the number of variables in the codebook, which is especially useful if there are several different tabs for different categories of variables, for example, exposures and outcomes, as well as a sheet with directions on how to interpret the codebook, for example, how to distinguish between studyand participant-level missing data.

| Pre-harmonization data check of the participant-level data
Figure 1 shows a suggested approach to the harmonization process, beginning with an initial check of the study's dataset.After the required study documentation has been received and the study's participation in the IPD-MA has been confirmed, the study can provide its full de-identified dataset (email template 5.0; Table 2).Receipt of the deidentified dataset can be logged in the project management and dataset tracking sheets (Tables 1 and 4).As documented in the file organization SOP (Table 3), the management team can save two copies of the dataset, the original dataset with the date received and a copy of the dataset with the same date and a version number that will then be transformed and harmonized.
Shortly after receiving the data (e.g., a week later) the data management team can plan to evaluate the data for clinically implausible values and differences in variables, variable levels, or dataset structure between the data and the study protocol, codebook, and publications (harmonization SOP; Table 3).When issues arise during the preliminary dataset review, the management team can follow up with the study team to review and resolve any questions (email template 7.0; Table 2).As the IPD-MA progresses, the data management team can document all data cleaning and harmonization-related communications in the dataset tracking sheet (Table 4) and save communication records according to the file organization SOP (Table 3) to easily refer back to prior decisions and steps in the IPD-MA process.Before checking the dataset, the IPD-MA team may want to (re)confirm that the T A B L E 3 Standard operating procedures for IPD-MA-related tasks, including harmonization.

SOP Description Link
Reaching out to and recruiting new data contributing groups Steps needed before, during, and after reaching out to studies to engage them in the IPD-MA https://osf.io/t7bjr File organization Folder organization and file naming conventions for management of recruitment, collecting required documentation, and harmonization https://osf.io/qm7jr

Cleaning and data harmonization
Harmonization of study-level variables to the master codebook https://osf.io/69yf8 Abbreviations: IPD-MA, individual participant data meta-analysis; SOP, standard operating procedure.
codebook or data dictionary shared by the study team is the most up-to-date version (email template 6.0; Table 2).

| Mapping
Before beginning the harmonization of the study dataset, a meeting (call or video) can be set up with the study team to review how the study's variables map to the IPD-MA master variables (email template 8.0; Table 2).Given limited investigator time, we found that a call rather than a series of emailed requests for clarification was the most efficient approach to mapping.The names of the harmonization and study team personnel involved and the follow-up contacts required to complete this mapping can be logged in the dataset tracking sheet (Table 4).The harmonization sheet is a key template for the mapping process and records how the each study's variables need to be transformed to correspond to the IPD-MA's variables, including coding notes (Table 4).Multiple followup with the study team may be required to finalize the variable mapping (email template 9.0; Table 2).

| Harmonization
Following a review of how the study's variables map to the IPD-MA master codebook with the study team, the harmonization code for each study can be developed using the mapping instructions in the template harmonization code (harmonization SOP; Table 2).The person who writes the harmonization code, the coding language used, and notes on harmonization-related decisions can be recorded in the dataset tracking sheet (Table 4).The harmonization code and harmonized dataset can be labeled and stored per the file organization SOP (Table 3).

| Post-harmonization data check
As per the harmonization SOP (Table 3), the IPD-MA team can then set up a meeting (call or video) with the study team to compare the variables and distributions that result from applying the harmonization code to the variable distributions in the original dataset to ensure F I G U R E 1 Suggested harmonization process for an IPD-MA, adapted from Fortier et al. 17  that the data have been appropriately interpreted in the harmonization phase (email template 10.0; Table 2).In our experience, a call with the study team was the most efficient way to ensure the study data had been appropriately harmonized.Multiple follow-ups may be required to finalize the harmonized dataset (email template 11.0; Table 2).

| Project management
IPD-MAs require a dedicated project management team to coordinate communication with the research groups that provide data.Details of the IPD-MA, including the responsibilities and estimated time needed from data contributing groups; the agreed-upon timelines for data sharing, cleaning, harmonization, and publication; and agreements around authorship, can be documented through emails and with written agreements where required.We provide a suite of templates to facilitate the project management aspects of conducting an IPD-MA which are described below.

| Budget template
The budget template provides an estimate of the actual cost of the IPD-MA, including person-time from the IPD-MA management team, subject matter experts, and the groups that contribute data.The template allows inclusion of the salaries of data-contributing group members.It estimates the time and financial contributions for each component of the IPD-MA till harmonization.This includes study identification and recruitment, defining a master codebook and the collection of metadata needed to reuse the dataset for the specific research question, and retrospective harmonization.The formulas for person-weeks in each cell are modifiable.Person weeks are divided into one-time and per-study costs for each primary activity.The budget template includes a cost increase when the number of studies exceeds the expected minimum of 10, the number of IPD-MA variables is over 100 (2% increase per variable), and the harmonized dataset will be longitudinal rather than cross-sectional (50% increase).The template estimates person weeks for specific roles for the IPD-MA project management team, data contributing groups, and subject matter experts who play a central role in codebook development, metadata extraction or collection, data cleaning, and retrospective harmonization.While time from study teams that contribute data and subject matter experts is often not covered by the IPD-MA, including these groups in the budget template facilitates realistic estimates of the time needed to support data contributors' participation in the IPD-MA.The budget template includes two sheets: (1) an example with estimates of the person-weeks and financial contributions to help end users visualize how the formulas work and (2) a clean version of the template.The budget template does not cover study risk of bias assessment, study and participant-level data imputation, analysis of the harmonized dataset, or open accessrelated publication fees.

| Gantt chart template
The Gantt chart template can help guide conversations and set realistic expectations with data contributing groups and funding bodies by establishing clear timelines for the recruitment of studies, submission of required documentation by data contributing groups, primary data and metadata collection, master codebook creation, data cleaning, harmonization, and publication.

| Job announcement templates
A dedicated team with a diverse skill set is central to the success of any IPD-MA. 5While consistency is a priority, in an academic setting where students and researchers may be working on the IPD-MA as part of their degree or internship, the personnel supporting the IPD-MA may change over time.Of all the roles in an IPD-MA, having a dedicated research coordinator who can see the project through to completion is the most important.We include job announcement templates that describe the activities and training expectations for the IPD-MA research coordinator and a coder engaged in retrospective harmonization.The job announcement templates are examples that should be modified to meet the specific requirements of the IPD-MA and comply with institutional guidelines.

| Project management spreadsheet
The project management spreadsheet template's primary purpose is to document communication between the study and the IPD-MA management team.The template has a row for each study identified through a systematic review or informal sources (i.e., researchers, ministries of health).Participation in the IPD-MA, reasons for ineligibility or non-participation, and study team contact information and details can be captured in the spreadsheet.The spreadsheet has dropdowns and related formulas for tracking the receipt of required documentation (e.g., letter of agreement, study protocol, study ethics approval, and codebook/data dictionary) and the IPD.The spreadsheet also has a column where the details of important communication or links to emails can be included to ensure continuity of understanding of the onboarding, data cleaning, and harmonization processes for each of the studies.

| Frequently asked questions (FAQ) document
When contacting eligible studies to participate in the IPD-MA for the first time, including a FAQ document with crucial information regarding the IPD-MA can help clarify initial questions and generate interest in learning more about the IPD-MA.The toolkit contains an example of the FAQ document used when communicating with data contributors about the ZIKV IPD-MA.

| Authorship, project management, and capacity building preferences survey
Despite the availability of authorship-related guidelines such as the International Committee of Medical Journal Editors (ICMJE) Authorship Guidance, 18 discussions about the inclusion and order of authors for IPD-MA-related publications can be contentious.When groups cannot reach authorship-related agreements through consensus, an online survey can determine authorship agreements through a majority vote.An example of a survey used to capture authorship preferences used as part of the ZIKV IPD-MA has been included in this toolkit.The survey also captured details about the studies, preferences for participation in consortium meetings and consortium working groups, and preferences for capacity-building activities.

| Author list and contributions spreadsheet
IPD-MAs engage a large number of authors.We developed an author list and contributions spreadsheet template to ensure collaborators can input and amend their authorship information in a centralized and transparent fashion for all IPD-MA-related publications.The project management team specifies a short name for each publication in the overview tab of the spreadsheet that corresponds to the name assigned to each spreadsheet tab, which is limited to eight characters.Authors can input their first and last names, email, affiliations, and ORCID in the overview tab and select the publications for which they have met the criteria for authorship.The template includes a separate tab for each IPD-MA-related publication where authors can input their contributions in keeping with the Contributor Roles Taxonomy (CRedIT). 19uthor names and initials are automatically copied into the specific tab for each publication for which they report that they meet authorship criteria.Duplicate sets of authors' initials for that manuscript are automatically highlighted in red for easy correction.
3.1.8| Letter of agreement for data sharing and authorship IPD-MAs generally include a non-legally binding data-sharing agreement that details IPD-MA-related data sharing and authorship decisions and may specify agreedupon plans for the reuse of the harmonized dataset following the conduct of the IPD-MA if such reuse is envisaged.We provide an example agreement covering the abovementioned information, with the caveat that some data contributing or IPD-MA management groups may require a legally binding contract.

| Email templates
The email templates, described in Table 2, can be used to manage communication with studies, from checking their eligibility to finalizing the harmonization of their dataset.As similar types of communication are required with each data contributing group, using the email templates saves time and ensures the consistent application of the IPD-MA workflow across the project management team members.The email templates we have provided should be adapted to the language and style most appropriate for each IPD-MA.The IPD-MA may require additional steps or types of communications than those envisaged in the email template workflow presented here for which the IPD-MA team can then create their own templates.

| Standard operating procedures
The SOPs listed in Table 3 provide step-by-step guidance for (1) reaching out to and recruiting data-contributing groups, (2) establishing a system for maintaining studyrelated documentation and IPD, and (3) data review, cleaning, and retrospective harmonization.By providing detailed instructions for the routine processes included in the IPD-MA, the SOPs ensure the consistent application of the IPD-MA workflow across the entire team engaged in the IPD-MA.The SOPs can be adapted for the specific processes needed for individual IPD-MAs.

| Harmonization-related spreadsheets
Templates for several spreadsheets related to the harmonization process are provided in Table 4.The spreadsheet templates offer a consistent way to document the IPD-MAs variables and definitions (master codebook), how each study's variables map to the IPD-MA's variables (harmonization sheet), and the status of each study's dataset in the harmonization workflow and communication surrounding the data during the harmonization process (dataset tracking sheet).These spreadsheets are helpful when the harmonization process is conducted manually.Data management and harmonization can also be semi-automated within different software and webbased environments, 20,21 which would then generate documentation as a product of those processes.
Further guidance and resources may be required for other critical tasks, including conducting a systematic review to identify studies, assessing study eligibility, protocol development, risk of bias assessment, and data analysis, which are not covered by the toolkit and for which we provide some publicly available resources in Table 5.

| CONCLUSION
There is extensive literature on the utility of IPD-MAs and best practices for IPD-MA-related analysis.When conducting an IPD-MA, management-and harmonization-related tasks are the most time-intensive.This article describes and provides adaptable templates, SOPs, and other documentation we have developed and refined through our work managing several large IPD-MA initiatives.The illustrative toolkit supports key IPD-MA-related project management, data cleaning, and harmonization-related tasks.The budget template supports setting realistic expectations for the time and funding needed to manage the IPD-MA and to conduct the data cleaning and harmonization-related tasks.IPD-MAs are resource-intensive and require the repetition of many tasks over time which are facilitated through the provided email templates.Transparency and consistency are key to the successful management of an IPD-MA.They are encouraged by the SOPs and letter of agreement for data sharing and authorship provided in this toolkit, all of which can be tailored to the specific IPD-MA project.IPD-MAs are essential for informing research, public health, and clinical practice.We hope the IPD-MA Toolkit templates and examples help future groups navigate the project management and harmonization challenges they confront when planning and conducting IPD-MAs.

T A B L E 4
Abbreviation: IPD-MA, individual participant data meta-analysis.
[Colour figure can be viewed at wileyonlinelibrary.com]