While the revamped safety measurement system (SMS) deployed by the Federal Motor Carrier Safety Administration (FMCSA) within its Compliance, Safety, Accountability or “CSA” program is “conceptually sound,” a long-awaited report from the National Academies of Sciences, Engineering, and Medicine (NAS) concludes that several features need improvement.
The report concluded that, over the next two years, FMCSA should develop a more “statistically principled approach” based on an item response theory (IRT) model to identify motor carriers that “need interventions” from a safety performance perspective.
The NAS also identified a number of “data quality issues” within the information gathered by the FMCSA to make safety fitness determinations.
Thus it recommended that FMCSA should continue to collaborate with states and other agencies to improve the collection of data on vehicle miles traveled (VMT) and on crashes; data which is often missing and of unsatisfactory quality.
“Including vehicle miles traveled data by state and month will enable SMS to account for varied environments where carriers travel – for example, in icy winter weather in the North,” NAS said in its report. “In addition, there is information available in police narratives not represented in the data used that could be helpful in understanding the contributing factors in a crash.”
David Heller, vice president of government affairs for the Truckload Carriers Association (TCA), stressed to Fleet Owner that the “data inconsistencies” within the CSA program is what has long concerned the industry, not the CSA program itself.
“TCA supports, in theory the notion of CSA. However, [this] recent study … reaffirms what our industry has been saying all along; that there are inconsistencies with the SMS,” he noted. “Safety minded, forward thinking carriers should be able to stand behind their safety accomplishments and being judged on those achievements, in an equitable manner. [That] should be the basis of what CSA is all about.”
Chris Spear, president and CEO of the American Trucking Associations (ATA), echoed that same sentiment.
“This report has confirmed much of what we have said about the program for some time,” he said in a statement. “The program, while a valuable enforcement tool, has significant shortcomings that must be addressed and we look forward to working with FMCSA to strengthen the program.”
Specifically, ATA noted the NAS study validated the trucking industry’s concerns about the inclusion of certain types of violations in the CSA system, that geographic enforcement disparities can have a significant impact on carriers’ scores and that the collection and use of clean inspections is critical to the accuracy of the program.
“We strongly agree with the NAS conclusion that CSA should be based less on the ‘subject-matter expertise’ of enforcement and more on the empirically validated data,” added Sean Garney, ATA’s director of safety policy. “We also see great potential in the recommendation that FMCSA overhaul the current CSA methodology in favor of a new, more adaptive, data-centric model with the potential to address serious flaws in the system.”
He noted that, to maximize CSA’s potential, there is an “urgent need” to address issues regarding data sufficiency and accuracy – specifically when looking at crash and carrier exposure data.
“Until more study – and correction – of these issues is complete, ATA strongly believes FMCSA should continue to keep CSA scores out of the public domain,” Garney pointed out.
To a degree, the NAS report backs up ATA’s position on the public sharing of such safety data.
“The committee that conducted the study and wrote the [NAS] report was unable to recommend whether SMS percentile ranks should be made public because it would require a formal evaluation to understand the consequences of public knowledge of the information,” the group said in a statement.
NAS also highlighted several other items as part of its report on SMS data protocols:
- About 100,000 fatality- or injury-causing crashes involving large trucks and buses occur in the U.S. each year.
- Approximately 3.5 million CMV roadside inspections are conducted yearly by specially trained inspectors on approximately 900 potential violations of safety regulations.
- Those violations fall into six categories: unsafe driving, hours of service compliance, vehicle maintenance, controlled substances/alcohol use, hazardous materials compliance, and driver fitness.
- In addition to that roadside inspection data, NAS urged FMCSA to research ways of collecting data on carrier characteristics – including driver turnover rates, type of cargo, and method and level of compensation.
- For example, compensation levels are relevant because it is known that drivers who are better-compensated, and those not compensated based on miles traveled, have fewer crashes.
