CERTaIN
Comparative effectiveness research (CER) evaluates and compares the benefits and harms of alternative healthcare interventions.
CER uses data generation (new studies) and data synthesis (comparisons of existing studies) to provide evidence on best practices for improving health care.
The CERTaIN program (Comparative Effectiveness Training and Instruction) at the University of Texas MD Anderson Cancer Center is offering a training opportunity in cancer CER through a grant funded by the National Cancer Institute (NCI, R25 CA257883).
This is a training program for early career cancer investigators (clinical and postdoctoral fellows and junior faculty), encompassing the broad spectrum of healthcare delivery in cancer care.
The CERTaIN program at MD Anderson comprises three educational strategies:
- Online instruction, available through EdX
- Hands-on training workshops
- Methods guidance and support
We will offer 3-day interactive workshops in three different areas of CER at different times each year:
- Observational studies and secondary data analysis
- Pragmatic clinical trials
- Evidence Synthesis: Systematic Reviews and Meta-Analysis
Once accepted to a workshop, trainees will be given access to the online portion of the program, to be completed before attending the workshop.
Trainees will attend the workshop and develop a research protocol.
They will be supported throughout the protocol development process up to one year after completing the workshop.
Applicants must be:
- Junior investigators (fellows, instructors, assistant professors, research scientists) working or training at an academic institution or non-profit research organization.
- Involved in cancer CER.
- U.S. citizen or permanent resident.
Up to 15 applicants will be competitively selected and invited to participate in the workshops.
Participation and travel costs will be covered by the grant (subject to NCI guidelines).
Program Faculty & Staff
Program Director
Maria E. Suarez-Almazor, M.D., Ph.D.
Dr. Suarez-Almazor is the Barnts Family Distinguished Professor in Cancer Research in The Department of Health Services Research and General Internal Medicine at the University of Texas MD Anderson Cancer Center. Dr. Suarez-Almazor is an internist and an epidemiologist with experience in Comparative Effectiveness Research and Patient-Centered Outcomes Research.
msalmazor@mdanderson.org
Associate Director
Maria A. Lopez-Olivo, M.D., M.Sc, Ph.D.
Dr. Lopez-Olivo is an Associate Professor in the Department of Health Services Research at the University of Texas MD Anderson Cancer Center. Dr. Lopez-Olivo has expertise in evidence-based medicine and its application to clinical decision-making, and translation of evidence into practice. amlopezo@mdanderson.org
Research Data Coordinator
Hollie E. Darnell
Hollie is a Research Data Coordinator in the Department of Health Services Research at the University of Texas MD Anderson Cancer Center.
HEDarnell@mdanderson.org
Leader for the Observational Studies and Registries curriculum
Maria E. Suarez-Almazor, M.D., Ph.D.
Dr. Suarez-Almazor is the Barnts Family Distinguished Professor in Cancer Research in The Department of Health Services Research and General Internal Medicine at the University of Texas MD Anderson Cancer Center. Dr. Suarez-Almazor is an internist and an epidemiologist with experience in Comparative Effectiveness Research and Patient-Centered Outcomes Research.
msalmazor@mdanderson.org
Leader for the Pragmatic Clinical Trials curriculum
Barry R. Davis, M.D., Ph.D.
Dr. Davis is Professor Emeritus of Biostatistics and Data Science at the University of Texas School of Public Health. Dr. Davis was the Guy S. Parcel Chair in Public Health and Professor of Biostatistics and Data Science, and Director of the Coordinating Center of Clinical Trials. Dr. Davis has extensive experience in biostatistics, and broad experience in all aspects of clinical trial study design, methodology and implementation.
barry.r.davis@uth.tmc.edu
Leader for the “Evidence Synthesis: Systematic Reviews and Meta-Analysis” curriculum
Maria A. Lopez-Olivo, M.D., Ph.D.
Dr. Lopez-Olivo is an Associate Professor in the Department of Health Services Research at the University of Texas MD Anderson Cancer Center. Dr. Lopez-Olivo has expertise in evidence-based medicine and its application to clinical decision-making, and translation of evidence into practice.
amlopezo@mdanderson.org
Faculty
Loreto Carmona, M.D., Ph.D.
Dr. Carmona was trained both as a rheumatologist (Hospital de la Princesa, Madrid) and then as a clinical epidemiologist (UCSF, San Francisco, USA). She earned her Ph.D. in Preventive Medicine, Epidemiology and Public Health with a national survey on rheumatic diseases. From 2001–2010, she worked at the Spanish Medicines Agency, as a full-time rheumatologist (Hospital Clínico San Carlos, Madrid), and as an NHS researcher, and directed the Spanish Foundation of Rheumatology Research Unit. She teaches at several universities and directs the Institute for Musculoskeletal Health in Madrid. She has over 375 publications and participates in many Spanish and European projects and post-graduate courses.
She was Editor-in-Chief of Rheumatology International and Associate Editor of Arthritis Care & Research, and currently serves on the Editorial Board of Annals of Rheumatic Diseases. She is an Honorary Member of EULAR.
Her main interests include research methodology, guideline development, patient involvement in research, and mentoring.
loreto.carmona@inmusc.eu
María Fernández, Ph.D.
Dr. María Fernández is Vice President of Population Health and Implementation Science at the University of Texas Health Science Center at Houston (UTHealth Houston) and the founding Co-Director of the UTHealth Houston Institute for Implementation Science. Dr. Fernández is also the Lorne Bain Chair of Public Health and Medicine, Professor of Health Promotion and Behavioral Sciences, and Director of the UTHealth Houston Center for Health Promotion and Prevention Research (CHPPR) at the UTHealth Houston School of Public Health. Dr. Fernandez has extensive experience developing and evaluating health promotion interventions and conducts research to improve implementation of effective programs that improve health and health equity.
maria.e.fernandez@uth.tmc.edu
David Farris, MSIS, AHIP
David Farris is a research services librarians at the MD Anderson Cancer Center Research Medical Library at MD Anderson Cancer Center. He has worked at MD Anderson since 2015, providing his expertise by conducting classes on systematic reviews and evidence synthesis for both researchers and professional librarians. Additionally, he provides support on evidence synthesis projects with MD Anderson faculty and staff.
dpfarris@mdanderson.org
Sharon H. Giordano, M.D., M.P.H.
Dr. Giordano is the Colin Powell Chair for Cancer Research, Chair and Professor of Medicine in the Department of Breast Medical Oncology at UT MD Anderson. Dr. Giordano is a clinician scientist with expertise in cancer outcomes and health services research, with particular emphasis on the use of cancer registries, claims databases, and linked datasets to evaluate treatment patterns, toxicities, disparities, and survivorship outcomes. Her research has contributed to improving the quality and equity of cancer care, particularly in breast cancer. She is a Fellow of the American Society of Clinical Oncology and has received multiple national and institutional awards recognizing her contributions to clinical research, education, and mentorship. She has an extensive publication record and serves as a mentor and collaborator on numerous federally funded research projects.
sgiordan@mdanderson.org
Tianjing Li, M.D., Ph.D.
Dr. Li is an Associate Professor of Ophthalmology and Epidemiology at the University of Colorado Anschutz Medical Campus. The goal of Dr. Li’s research is to develop, evaluate, and disseminate methods for comparing healthcare interventions and to provide trust-worthy evidence for decision-making. Dr. Li is a world-renowned expert in research synthesis. She has played many leadership roles within the Cochrane Collaboration and served as the President for the Society for Research Synthesis Methodology.
tianjing.li@cuanschutz.edu
Suja S. Rajan, Ph.D.
Dr. Rajan is an Associate Professor in the Department of Management, Policy and Community Health at The University of Texas School of Public Health. Dr. Rajan is a health economist and econometrician whose research interests include cancer research and program evaluation. Dr. Rajan has extensive training and research experience in statistical modeling, economic evaluations, econometric analysis, sample survey methods, Monte Carlo simulation and Markov modeling.
suja.s.rajan@uth.tmc.edu
Sanjay Shete, Ph.D
Sanjay Shete, Ph.D., is a Statistical Geneticist, Population Health Scientist, and Behavioral and Genetic Epidemiologist interested in developing statistical methods for genetic, population, and behavioral science data. He is a Professor of Biostatistics and Epidemiology and holder of the Betty B. Marcus endowed chair in Cancer Prevention at the University of Texas MD Anderson Cancer Center, Houston, TX. sshete@mdanderson.org
Iakovos Toumazis, Ph.D.
Dr. Toumazis is an Assistant Professor in the Department of Health Services Research at MD Anderson Cancer Center. Dr. Toumazis is a mathematician and Industrial Engineer with expertise in comparative modeling analysis specifically on microsimulation modeling and medical decision making under uncertainty.
itoumazis@mdanderson.org
Andrea B. Troxel, Sc.D.
Dr. Troxel is a Professor of Population Health and Director of the Division of Biostatistics at New York University Grossman School of Medicine. Her research integrates the development of novel clinical trial design and analysis approaches and their application to a wide range of areas in medicine and health care. Dr. Troxel is an expert in the design and analysis of pragmatic and adaptive randomized clinical trials and leads several data coordinating centers.
andrea.troxel@nyulangone.org
Richard Wyss, Ph.D., MSc
Dr. Wyss is an Assistant Professor of Medicine at Harvard Medical School and a Lead Investigator in the Division of Pharmacoepidemiology and Pharmacoeconomics at Brigham and Women’s Hospital. Dr. Wyss works as a data scientist and methodological epidemiologist with training in the theory and application of statistical methods to pharmacoepidemiology. His research is focused on developing robust health analytic tools for evaluating the comparative effectiveness and safety of pharmaceutical drugs using administrative healthcare databases. More recently, his research has become tailored to advancing semi-automated tools for generating evidence from healthcare databases on the effectiveness and safety of newly marketed medical products in population subgroups that are underrepresented in randomized clinical trials.
rwyss@bwh.harvard.edu
Hui Zhao, M.D., Ph.D.
Dr. Zhao is an Associate Professor and Statistician in the Department of Health Services Research at The University of Texas MD Anderson Cancer Center. Dr. Zhao has extensive experience in the design and analysis of observational data, including real world data from claims and other administrative sources.
huizhao@mdanderson.org
Resources
Our projects
Our projects:
Study risk of bias assessment. Library of risk of bias, quality assessment, and critical appraisal tools. The following link is a redcap project containing a repository of tools that can be used to evaluate the quality of studies included in an evidence synthesis project. The comprehensive list can be filtered by type of tool, domains, and targeted study: https://redcap.link/RoBtools.
Repository of resources for studies using real-world evidence. Library of datasets used in oncology. The link is a redcap project containing a comprehensive list of datasets that can be used to determine the best resource to conduct studies using real-world evidence.
Our workshops
1. Evidence Synthesis: Systematic Reviews and Meta-Analysis
Introduction to Systematic Reviews. To support protocol development and transparent reporting, several key resources are available. PROSPERO is an international registry where you can register systematic review protocols to reduce duplication and improve transparency (https://www.crd.york.ac.uk/prospero/#loginpage). A structured PROSPERO registration template is also available to guide submissions (https://www.crd.york.ac.uk/prospero/documents/PROSPERO%20registration%20form.pdf). For scoping reviews and other evidence syntheses, the Open Science Framework (OSF) provides a flexible registry (https://osf.io/) along with a protocol template (https://osf.io/etwmf). Reporting standards are essential, and the PRISMA extension for protocols offers guidance (http://prisma-statement.org/Extensions/Protocols).
Literature Searching. Comprehensive literature searching is foundational to systematic reviews. A helpful starting point is the curated list of electronic databases provided by MD Anderson Libraries (https://mdanderson.libguides.com/systematicreviews). Searching trial registries such as ClinicalTrials.gov (https://clinicaltrials.gov/), WHO ICTRP (http://apps.who.int/trialsearch/), and other registries (https://www.hhs.gov/ohrp/international/clinical-trial-registries/index.html) ensures inclusion of ongoing and unpublished studies. Free bibliographic databases such as PubMed (https://pubmed.ncbi.nlm.nih.gov/), PubMed Central (https://pmc.ncbi.nlm.nih.gov/), Europe PMC (https://europepmc.org/), LILACS (https://lilacs.bvsalud.org/), and the Cochrane Library (https://www.cochranelibrary.com/) are also essential resources.
Managing References. Efficient reference management is critical for organizing search results and deduplicating records. Free tools such as Zotero (https://www.zotero.org/) and Mendeley (https://www.mendeley.com/) allow users to store, organize, and annotate citations, and integrate with word processors for manuscript preparation.
Study Selection. Screening studies systematically requires structured workflows and specialized tools. Covidence (https://www.covidence.org) is widely used for managing screening and review processes. Other platforms include DistillerSR (https://www.distillersr.com/products/distillersr-systematic-review-software), EPPI-Reviewer (https://eppi.ioe.ac.uk/cms/Default.aspx?tabid=2914), and JBI SUMARI (https://sumari.jbi.global/). Free screening tools such as Rayyan (https://www.rayyan.ai/), PICO Portal (https://picoportal.org/), CADIMA (https://www.cadima.info/index.php/area/evidenceSynthesisDatabase), Abstrackr (http://abstrackr.cebm.brown.edu/account/login), the SR Accelerator (https://sr-accelerator.com/#/) and ASReview (https://asreview.nl/) can also support study selection.
To document the process, the PRISMA flow diagram is recommended (https://www.prisma-statement.org/prisma-2020-flow-diagram), and a broader repository of systematic review tools is available at https://systematicreviewtools.com/.
Data Extraction. Data extraction requires careful planning and standardized forms. Methodological guidance can be found in key publications (e.g., https://onlinelibrary.wiley.com/doi/10.1002/9781119536604.ch5; https://www.ajpmonline.org/article/S0749-3797(99)00122-1/pdf; https://ebm.bmj.com/content/26/3/88). Tools such as Covidence (https://www.covidence.org) and other platforms including DistillerSR, EPPI-Reviewer, JBI SUMARI, and RevMan are widely used platforms that support structured, manual, and semi-automated data collection workflows. In addition, emerging tools such as MetaReviewer, RobotReviewer, Colandr, and ContentMine incorporate automation and machine learning approaches to assist with extracting and organizing study information, helping to improve efficiency when handling large volumes of data. The Systematic Review Data Repository (https://srdrplus.ahrq.gov/) supports data storage and sharing. Reviewers can also use calculators and RevMan tools (https://training.cochrane.org/resource/revman-calculator) and specialized software (e.g., WebPlotDigitizer, PlotDigitizer, Graph2Data, GraphreaderV2, Engauge Digitizer) to extract data from figures. For handling missing or incomplete data, publications such as https://pubmed.ncbi.nlm.nih.gov/25524443/ and https://pubmed.ncbi.nlm.nih.gov/17555582/ provide practical guidance and accompanying Excel tools to support imputation of summary statistics (e.g., deriving standard deviations from other reported measures), which can be particularly useful during meta-analysis preparation. Standardized data extraction forms are available from Cochrane (https://training.cochrane.org/sites/training.cochrane.org/files/public/uploads/resources/downloadable_resources/English/Collecting%20data%20-%20form%20for%20RCTs%20and%20non-RCTs.doc; https://epoc.cochrane.org/sites/epoc.cochrane.org/files/uploads/Resources-for-authors2017/good_practice_data_extraction_form.doc), JBI (https://bmjopen.bmj.com/content/bmjopen/6/4/e010654/DC2/embed/inline-supplementary-material-2.pdf?download=true), and AHRQ (https://www.ncbi.nlm.nih.gov/books/NBK92432/), providing structured templates for consistent data collection. Additionally, a broader repository of systematic review tools is available at https://systematicreviewtools.com/.
Risk of Bias. Assessing risk of bias is a critical component of evidence synthesis. Guidance is available through the Cochrane Handbook and other methodological resources (e.g., https://training.cochrane.org/handbook/current/chapter-08; https://www.sciencedirect.com/science/article/pii/S0022202X16323569; https://www.ncbi.nlm.nih.gov/books/NBK91433/; https://www.nhmrc.gov.au/guidelinesforguidelines/develop/assessing-risk-bias). A variety of tools are available depending on study design, including RoB 2 for randomized trials, ROBINS-I for non-randomized studies, QUADAS-2 for diagnostic accuracy, and PROBAST for prognostic studies. The JBI critical appraisal tools (https://jbi.global/critical-appraisal-tools) provide additional options. Our library of tools can be accessed at https://redcap.link/RoBtools and a repository of appraisal tools is available at https://osf.io/dmrq6. The LATITUDES network (https://www.latitudes-network.org/) also provides useful resources. Visualization of risk of bias assessments can be done using robvis (https://www.riskofbias.info/welcome/robvis-visualization-tool).
Qualitative Synthesis. When meta-analysis is not appropriate, qualitative synthesis methods are used. Guidance is available through Cochrane resources (https://training.cochrane.org/resource/reporting-guideline-synthesis-without-meta-analysis-swim; https://www.jclinepi.com/article/S0895-4356(23)00014-8/fulltext) and reporting standards such as SWiM (https://www.bmj.com/content/368/bmj.l6890). Additional frameworks include narrative synthesis guidance (https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=ed8b23836338f6fdea0cc55e161b0fc5805f9e27) and thematic synthesis approaches (https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/1471-2288-8-45), which support structured interpretation of qualitative findings.
Meta-analysis. Meta-analysis allows quantitative synthesis of study results. Guidance is provided by Cochrane (https://training.cochrane.org/handbook/current/chapter-10) and other methodological papers (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4145560/; https://journals.sagepub.com/doi/10.3102/0034654319877153). Software options include RevMan (https://revman.cochrane.org/info) and commercial tools (https://meta-analysis.com/), as well as free tools such as Meta-Essentials (https://www.erim.eur.nl/research-support/meta-essentials/), MetaXL (https://www.epigear.com/index_files/metaxl.html), Metawin (https://www.metawinsoft.com/), Meta Easy (https://www.jstatsoft.org/article/view/v030i07), OpenMeta[Analyst] (http://www.cebm.brown.edu/openmeta/; https://code.google.com/archive/p/openmeta/), and MetaLight (https://eppi.ioe.ac.uk/cms/Resources/Tools/MetaLightsoftwareforteachingmetaanalysis/tabid/3086/Default.aspx#:~:text=MetaLight%20is%20a%20software%20application%20designed%20to,available%20and%2C%20as%20it%20uses%20the%20Silverlight). Statistical analysis can also be conducted using STATA, R (metafor package: https://cran.r-project.org/web/packages/metafor/index.html), JASP (https://jasp-stats.org/2017/11/15/meta-analysis-jasp/), or SPSS. The NNT calculator and visualizations (http://www.nntonline.net/visualrx/) provide additional tools for interpreting results.
Heterogeneity. Understanding and addressing heterogeneity is critical in meta-analysis. Guidance is available in the Cochrane Handbook and methodological literature (https://training.cochrane.org/handbook/current/chapter-10; https://pmc.ncbi.nlm.nih.gov/articles/PMC1767262/; https://www.sciencedirect.com/science/article/pii/S2213422023000938), along with practical advice on common mistakes (https://meta-analysis.com/download/commonmistakes/Common%20Mistakes%20-%20Heterogeneity.pdf).
Network Meta-analysis. Network meta-analysis enables comparison of multiple interventions simultaneously. Key concepts and introductory materials are available through Cochrane (https://training.cochrane.org/resource/key-concepts-network-meta-analysis-nma), along with a dedicated toolkit (https://methods.cochrane.org/cmi/network-meta-analysis; https://methods.cochrane.org/cmi/network-meta-analysis-toolkit).
Metasynthesis. Metasynthesis integrates findings from qualitative studies. Methodological guidance is available through key academic resources (https://files.eric.ed.gov/fulltext/ED603222.pdf; https://www.proquest.com/docview/200783486?accountid=7135&sourcetype=Scholarly%20Journals), offering frameworks for synthesizing qualitative evidence across studies.
Publication Bias. Publication bias can affect the validity of systematic reviews. Methodological guidance is available through Cochrane (https://training.cochrane.org/resource/identifying-publication-bias-meta-analyses-continuous-outcomes) and key publications (https://www.mayoclinicproceedings.org/article/S0025-6196(11)63030-9/fulltext; https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5953768/). The ROB-ME tool (https://www.riskofbias.info/welcome/rob-me-tool) provides a structured approach to assessing bias due to missing evidence.
Certainty of evidence. The GRADE approach can be used to assess certainty of evidence and strength of recommendations. GRADEpro software (https://www.gradepro.org/) supports the creation of Summary of Findings tables and structured evaluation of evidence quality.
GRADE for NMA. Applying GRADE to network meta-analysis requires additional considerations. Guidance on rating certainty and preparing Summary of Findings tables is available here: https://training.cochrane.org/resource/grade-approach-rate-certainty-evidence-network-meta-analysis-and-summary-findings-tables, along with methodological details in the Cochrane Handbook (https://training.cochrane.org/handbook/current/chapter-11#section-11-5).
Translating Systematic Reviews to Consumers. Communicating findings effectively to non-specialist audiences is essential. Guidance on patient engagement and plain language communication is available (https://researchinvolvement.biomedcentral.com/articles/10.1186/s40900-022-00358-6; https://ktdrr.org/resources/plst/tool/PLST_Training.pdf). Tools such as plain language resources (e.g., https://centerforplainlanguage.org/; https://plainlanguagenetwork.org/), and the Plain Language Summary Tool (https://ktdrr.org/resources/plst/) support translation of evidence. Additional resources on infographics (https://training.cochrane.org/online-learning/knowledge-translation/how-share-cochrane-evidence/choose-right-dissemination-produ-7) and health literacy (e.g., CDC and NIH links below) can help improve accessibility:
Health Literacy-Plain Language Materials & Resources. https://www.cdc.gov/healthliteracy/developmaterials/plainlanguage.html; https://www.plainlanguage.gov/
Plain Language at NIH. https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/plain-language
Plain Language Medical Dictionary. https://apps.lib.umich.edu/medical-dictionary/
Plain Language Thesaurus for Health Communications. https://stacks.cdc.gov/view/cdc/11500/
Universal Patient Language. https://www.upl.org/about
Appraisal of Published Systematic Reviews. Critical appraisal of systematic reviews ensures methodological rigor. Tools include ROBIS (https://www.bristol.ac.uk/population-health-sciences/projects/robis/robis-tool/), AMSTAR-2 (https://amstar.ca/Amstar_Checklist.php), and CASP checklists (https://casp-uk.net/casp-tools-checklists/systematic-reviews-meta-analysis-observational-studies/; https://health.usf.edu/-/media/Files/Medicine/GME/Evidence-Based-Medicine-Course/critical-appraisal-checklist.ashx; https://www.cebm.ox.ac.uk/resources/ebm-tools/critical-appraisal-tools), which provide structured approaches to evaluating quality.
Responsible Conduct of Research. Maintaining ethical standards in research is essential. The NIH provides training resources (https://oir.nih.gov/sourcebook/ethical-conduct/responsible-conduct-research-training), and the Office of Research Integrity offers additional guidance (https://ori.hhs.gov/ori-introduction-responsible-conduct-research).