Interrater Reliability of Manual Pavement Distress Evaluations
Publication: Journal of Transportation Engineering
Volume 136, Issue 2
Abstract
Many government agencies are faced with the challenge of pavement evaluation and maintenance as part of their pavement management systems. These agencies perform or contract manual or automated distress surveys over the pavement network to monitor the structure performance and obtain the necessary data to calculate pavement condition indexes. Although manual distress ratings are done according to well-defined criteria, a certain amount of subjectivity and the experience of the raters have an undoubted influence on the ratings. This study proposes a new approach to estimate the interrater or intercrew reliability for manual or semiautomated distress data collection. The proposed analysis acknowledges that a certain degree of variability in the visual distress ratings is likely to occur and, thus, minimum acceptable values of complete and partial agreements of the crews or raters are suggested. The statistical approach to validate the level of agreement between the ratings of two raters or crews is based on the use of the chi-square distribution to test hypotheses about multinomial experiments.
Get full access to this article
View all available purchase options and get full access to this article.
Acknowledgments
The writers thank Robert S. Young, P.E., Pavement Preservation Engineer, and Tito T. Medina, Jr., Supervisor of the Pavement Evaluation Unit, New Mexico DOT, for their valuable collaboration during all stages of the pavement evaluation program and data collection in 2006 and 2007. The enlightening discussions with Robert S. Young regarding distress data variability and QA/QC provided useful insights. The contents of this article reflect the views and ideas of the writers who are solely responsible for the analysis, conclusions, and accuracy of the data presented herein.
References
Dowdy, S., Wearden, S., and Chilko, D. (2004). Statistics for research, 3rd Ed., Wiley, Hoboken, N.J.
Federal Highway Administration (FHWA). (2000). “Variability of pavement distress data from manual surveys.” Techbrief Rep. No. FHWA-RD-00-160, Turner-Fairbank Highway Research Center, FHWA, McLean, Va., ⟨http://www.tfhrc.gov/pavement/ltpp/pdf/00160.pdf⟩ (June 1, 2009).
Federal Highway Administration (FHWA). (2003). “Distress identification manual for the long-term pavement performance program.” Rep. No. FHWA-RD-03-031, Office of Infrastructure Research and Development, FHWA, McLean, Va.
Hogg, R. V., and Craig, A. T. (1995). Introduction to mathematical statistics, 5th Ed., Prentice-Hall, Englewood Cliffs, N.J.
Larson, C. D., Sami, N., and Luhr, D. R. (2000). “Structure approach to managing the quality of pavement distress data: Virginia Department of Transportation experience.” Transp. Res. Rec., 1699, 72–80.
Livneh, M. (1994). “Repeatability and reproducibility of manual pavement distress survey methods.” Proc., 3rd Int. Conf. on Managing Pavements, Transportation Research Board, Washington, D.C., 279–289.
Morian, D., Stoeffels, S., and Firth, D. J. (2002). “Quality management of pavement performance data.” Proc., 2002 Pavement Evaluation Conf., Roanoke, Va.
National Cooperative Highway Research Program (NCHRP). (1993). “Accreditation for the long-term pavement performance studies pavement distress raters.” Rep. No. SHRP-P-653, Strategic Highway Research Program, National Academy of Sciences, Washington, D.C.
NCHRP. (2004). “Automated pavement distress collection techniques. A synthesis of highway practice.” NCHRP synthesis 334, Transportation Research Board, Washington, D.C., ⟨http://onlinepubs.trb.org/onlinepubs/nchrp/nchrp_syn_334.pdf⟩.
New Mexico Department of Transportation (NMDOT). (2004). The New Mexico Department of Transportation’s network level pavement management program, ⟨http://nmshtd.state.nm.us/upload/images/ITIS/Pavement Management.pdf⟩ (June 1, 2009).
Pearson, K. (1900). “On the criterion that a given system of derivations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling.” Philosophical Magazine, Series 5, 50, 157–175.
Prakash, A., Sharma, B. N., and Kazmierowski, J. (1994). Investigation into observational variations in pavement condition survey, Transportation Research Board, Washington, D.C., 290–301.
Rada, G. R., Bhandari, R. K., Elkins, G. E., and Bellinger, W. Y. (1997). “Assessment of long-term pavement performance program manual distress data variability. Bias and precision.” Transp. Res. Rec., 1592, 151–168.
Rada, G. R., Wu, C. L., Bhandari, R. K., Shekharan, A. R., Elkins, G. E., and Miller, J. S. (1999). “Study of LTPP distress data variability.” Vol. 1, Rep. No. FHWA-RD-99-074, Office of Infrastructure Research and Development, FHWA, McLean, Va.
Selezneva, O., Mladenovic, G., Speir, R., Amenta, J., and Kennedy, J. (2004). “QA sampling considerations for automated distress data collection and processing for National Park Service road inventory program.” Proc., 83rd Annual Meeting of the Transportation Research Board (CD-ROM), Transportation Research Board, Washington, D.C.
Shahin, M., and Kohn, S. (1981). “Pavement maintenance management for roads and parking lots.” Final Rep. No. CERL-TR-M-294, U.S. Army Construction Engineering Research Laboratory, Champaign, Ill.
Smith, R. E., Freeman, T. J., and Pendelton, O. J. (1996). “Evaluation of automated pavement distress data collection procedures for local agency pavement management.” Rep. No. GC10470, Washington State DOT, Olympia, Wash.
Stoeffels, S., Morian, D., Firth, D. J., and Larson, C. D. (2001). “Quality analysis methods for pavement distress data.” Proc., 80th Annual Meeting of the Transportation Research Board (CD-ROM), Transportation Research Board, Washington, D.C.
Information & Authors
Information
Published In
Journal of Transportation Engineering
Volume 136 • Issue 2 • February 2010
Pages: 165 - 172
Copyright
© 2010 ASCE.
History
Received: Oct 15, 2007
Accepted: Sep 23, 2009
Published online: Jan 15, 2010
Published in print: Feb 2010
Authors
Metrics & Citations
Metrics
Citations
Download citation
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.