![[PDF]](/pb-assets/Icons/adobe-pdf-icon-1498649896243.png){kind=icon}
Abstract
Respondent-driven sampling (RDS) employs a variant of a link-tracing network sampling strategy to collect data from hard-to-reach populations. By tracing the links in the underlying social network, the process exploits the social structure to expand the sample and reduce its dependence on the initial (convenience) sample.
The current estimators of population averages make strong assumptions in order to treat the data as a probability sample. We evaluate three critical sensitivities of the estimators: (1) to bias induced by the initial sample, (2) to uncontrollable features of respondent behavior, and (3) to the without-replacement structure of sampling.
Our analysis indicates: (1) that the convenience sample of seeds can induce bias, and the number of sample waves typically used in RDS is likely insufficient for the type of nodal mixing required to obtain the reputed asymptotic unbiasedness; (2) that preferential referral behavior by respondents leads to bias; (3) that when a substantial fraction of the target population is sampled the current estimators can have substantial bias.
This paper sounds a cautionary note for the users of RDS. While current RDS methodology is powerful and clever, the favorable statistical properties claimed for the current estimates are shown to be heavily dependent on often unrealistic assumptions. We recommend ways to improve the methodology.
References
|
Abdul-Quader, Abu S., Heckathorn, Douglas D., McKnight, Courtney, Bramson, Heidi, Nemeth, Chris, Sabin, Keith, Gallagher, Kathleen, Des, Jarlais Don C. 2006. “Effectiveness of Respondent-Driven Sampling for Recruiting Drug Users in New York City: Findings from a Pilot Study.” Journal of Urban Health 83:459–76. Google Scholar | Crossref | Medline | ISI | |
|
Barndorff-Nielsen, O. E. 1978. Information and Exponential Families in Statistical Theory. New York: Wiley. Google Scholar | |
|
Bernhardt, Annette, Milkman, Ruth, Theodore, Nik, Heckathorn, Douglas, Auer, Mirabai, DeFilippis, James, Luz, Ana González, Narro, Victor, Perelshteyn, Jason, Poison, Diana, Spiller, Michael 2009. “Broken Laws, Unprotected Workers: Violations of Employment and Labor Laws in America's Cities.” Technical Report, Employment Law Project, New York, NY 10038 (http://www.nelp.org). Google Scholar | |
|
Centers for Disease Control. 2008. “Consultation on Respondent-Driven Sampling.” Verbal Discussion, Atlanta, GA. Google Scholar | |
|
Diaconis, Persi 2009. “The Markov Chain Monte Carlo Revolution.” Bulletin of the American Mathematical Society 46:179–205. Google Scholar | Crossref | ISI | |
|
Frost, Simon D. W., Brouwer, Kimberly C., Firestone, Cruz Michelle A., Ramos, Rebeca, Elena, Ramos Maria, Lozada, Remedios M., Magis-Rodriguez, Carols, Strathdee, Steffanie A. 2006. “Respondent-Driven Sampling of Injection Drug Users in Two U.S.-Mexico Border Cities: Recruitment Dynamics and Impact on Estimates of HIV and Syphilis Prevalence.” Journal of Urban Health 83:83–97. Google Scholar | Crossref | ISI | |
|
Gile, Krista J. 2008. “Inference from Partially-Observed Network Data.” PhD dissertation, Department of Statistics, University of Washington. Google Scholar | |
|
Gile, Krista J. 2009. “Improved Inference for Respondent-Driven Sampling Data with Application to HIV Prevalence Estimation.” Under review. Google Scholar | |
|
Gile, Krista J., Handcock, Mark S. 2009. “Network Model Assisted Inference from Respondent Driven Sampling Data.” Unpublished manuscript. Nuffield College, University of Oxford. Google Scholar | |
|
Gilks, Walter R., Richardson, Sylvia, Spiegelhalter, David J.(eds.). 1996. Markov Chain Monte Carlo in Practice. New York: Chapman and Hall. Google Scholar | |
|
Goel, Sharad, Salganik, Mathew J. 2009. “Respondent Driven Sampling as Markov Chain Monte Carlo.” Statistics in Medicine 28:2202–29. Google Scholar | Crossref | Medline | ISI | |
|
Goodman, Leo A. 1961. “Snowball Sampling.” Annals of Mathematical Statistics 32:148–70. Google Scholar | Crossref | |
|
Handcock, Mark S., Gile, Krista J. 2010. “Modeling Networks from Sampled Data.” Annals of Applied Statistics. Google Scholar | Crossref | Medline | ISI | |
|
Handcock, Mark S., Hunter, David R., Butts, Carter T., Goodreau, Steven M., Morris, Martina 2003. Statnet: Software Tools for the Statistical Modeling of Network Data. Statnet Project http://statnetproject.org/, Seattle, WA. R package version 2.0. Google Scholar | |
|
Hansen, Morris H., Hurwitz, William N 1943. “On the Theory of Sampling from Finite Populations.” Annals of Mathematical Statistics 14:333–62. Google Scholar | Crossref | |
|
Heckathorn, Douglas D. 1997. “Respondent-Driven Sampling: A New Approach to the Study of Hidden Populations.” Social Problems 44:174–99. Google Scholar | Crossref | ISI | |
|
Heckathorn, Douglas D. 2002. “Respondent-Driven Sampling II: Deriving Valid Population Estimates From Chain-Referral Samples of Hidden Populations.” Social Problems 49:11–34. Google Scholar | Crossref | ISI | |
|
Heckathorn, Douglas D. 2007. “Extensions of Respondent-Driven Sampling: Analyzing Continuous Variables and Controlling for Differential Recruitment.” Pp. 151–207 in Sociological Methodology, vol. 37, edited by Xie, Yu Boston, MA: Blackwell Publishing. Google Scholar | SAGE Journals | |
|
Heckathorn, Douglas D. 2009. “Respondent Driven Sampling.” http://www.respondentdrivensampling.org. Google Scholar | |
|
Heckathorn, Douglas D., Jeffri, Joan 2001. “Finding the Beat: Using Respondent-Driven Sampling to Study Jazz Musicians.” Poetics 28:307–29. Google Scholar | Crossref | ISI | |
|
Horvitz, Daniel G., Thompson, Donovan J. 1952. “A Generalization of Sampling Without Replacement from a Finite Universe.” Journal of the American Statistical Association 47:663–85. Google Scholar | Crossref | ISI | |
| John Jay College Symposium. 2007. “Respondent-Driven Sampling and Social Network Analysis Symposium.” Verbal Discussion, New York. August 10. Google Scholar | |
|
Johnston, Lisa G. 2009. Personal communication. Google Scholar | |
|
Johnston, Lisa G., Malekinejad, Moshen, Kendall, Carl, Iuppa, Irene M., Rutherford, George W. 2008. “Implementation Challenges to Using Respondent-Driven Sampling Methodology for HIV Biological and Behavioral Surveillance: Field Experiences in International Settings.” AIDS and Behavior 12:131–41. Google Scholar | Crossref | ISI | |
|
Malekinejad, Mohsen, Johnston, Lisa, Kendall, Carl, Kerr, Ligia, Rifkin, Marina, Rutherford, George 2008. “Using Respondent-Driven Sampling Methodology for HIV Biological and Behavioral Surveillance in International Settings: A Systematic Review.” AIDS and Behavior 12:105–30. Google Scholar | Crossref | ISI | |
|
Muhib, Farzana B., Lins, Lillian S., Stueve, Ann, Miller, Robin L., Ford, Wesley L., Johnson, Wayne D., Smith, Philip J. 2001. “A Venue-Based Method for Sampling Hard-to-Reach Populations.” Public Health Reports 2001; 116 Suppl 1: 216222, Association of Schools of Public Health, Washington, DC. Google Scholar | |
|
Neely, W. Whipple 2009. “Bayesian Methods for Data from Respondent Driven Sampling.” PhD dissertation, Department of Statistics, University of Wisconsin, Madison. Google Scholar | |
|
Peterson, James A., Reisinger, Heather S., Schwartz, Robert P., Mitchell, Shannon G., Kelly, Sharon M., Brown, Barry S., Agar, Michael H. 2008. “Targeted Sampling in Drug Abuse Research: A Review and Case Study.” Field Methods 20:155–70. Google Scholar | SAGE Journals | ISI | |
|
Saidel, Tobi, Adhikary, Rajatashuvra, Mainkar, Mandar, Dale, Jayesh, Loo, Virginia, Rahman, Motiur, Ramesh, Banadakoppa M., Paranjape, Ramesh S. 2008. “Baseline Integrated Behavioural and Biological Assessment Among Most At-risk Populations in Six High-Prevalence States of India: Design and Implementation Challenges.” AIDS 22:S12–34. Google Scholar | Crossref | ISI | |
|
Salganik, Matthew J. 2006. “Variance Estimation, Design Effects, and Sample Size Calculations for Respondent-Driven Sampling.” Journal of Urban Health: Bulletin of the New York Academy of Medicine 83:98–112. Google Scholar | Crossref | ISI | |
|
Salganik, Matthew J., Heckathorn, Douglas D. 2004. “Sampling and Estimation in Hidden Populations Using Respondent-Driven Sampling.” Pp. 193–239. in Sociological Methodology, Vol. 34. edited by Stolzenberg, Ross M. Boston, MA: Blackwell Publishing. Google Scholar | SAGE Journals | |
|
Simic, Milena, Johnston, Lisa G., Platt, Lucy, Baros, Sladjana, Andjelkovic, Violeta, Novotny, Tom, Rhodes, Tim 2006. “Exploring Barriers to ‘Respondent-Driven Sampling’ in Sex Worker and Drug-Injecting Sex Worker Populations in Eastern Europe.” Journal of Urban Health 83:83–97. Google Scholar | Crossref | ISI | |
|
Snijders, Tom A. B., Pattison, Philippa, Robins, Garry L., Handcock, Mark S. 2006. “New Specifications for Exponential Random Graph Models.” Pp. 99–153 in Sociological Methodology, Vol. 36, edited by Stolzenberg, Ross M. Boston, MA: Blackwell Publishing. Google Scholar | SAGE Journals | |
|
Thompson, Steven K. 2002. Sampling. 2nd ed. New York: Wiley. Google Scholar | |
|
Thompson, Steven K., Frank, Ove 2000. “Model-Based Estimation with Link-Tracing Sampling Designs.” Survey Methodology 26:87–98. Google Scholar | |
|
van Duijn, Marijtje A. J., Handcock, Mark S., Gile, Krista J. 2009. “A Framework for the Comparison of Maximum Pseudo Likelihood and Maximum Likelihood Estimation of Exponential Family Random Graph Models.” Social Networks 31:52–62. Google Scholar | Crossref | Medline | ISI | |
|
Volz, Erik, Heckathorn, Douglas D. 2008. “Probability Based Estimation Theory for Respondent Driven Sampling.” Journal of Official Statistics 24:79–97. Google Scholar | ISI | |
|
Volz, Erik, Wejnert, Cyprian, Degani, Ismail, Heckathorn, Douglas D. 2007. Respondent-Driven Sampling Analysis Tool (RDSAT), Version 5.6. Google Scholar | |
|
Walters, Karina, Simoni, Jane 2002. “Health Survey of Two-Spirited Native Americans,” Grant no. 1R01MH065871–01. National Institute of Mental Health. Google Scholar | |
|
Waiters, John K., Biernacki, Patrick 1989. “Targeted Sampling: Options for the Study of Hidden Populations.” Social Problems 36:416–30. Google Scholar | Crossref | ISI | |
|
Wejnert, Cyprian, Heckathorn, Douglas D. 2008. “Web-Based Network Sampling: Efficiency and Efficacy of Respondent-Driven Sampling for Online Research.” Sociological Methods and Research 37:105–34. Google Scholar | SAGE Journals | ISI |
