“A scientist must be like a child. If he sees a thing, he must say that he sees it, whether it was what he thought he was going to see or not. See first, think later, then test. But always see first. Otherwise you will only see what you were expecting.” Douglas Adams, So Long, and Thanks for All the Fish
National Motorcycle Training (NMI) Uses The Scientific Method
We start by putting our thoughts into the form “Statement because Explanation.” Our understanding and theories of motorcycling danger are developed and expanded using the scientific method. We always apply the following mantra to our theories,”The Statement must be measurable and disprovable, the explanation difficult to manipulate, and reporting of results to be completed in a truthful manner.” This mantra helps guide us when using the scientific method. If a statement is testable, that is, has measurable quantities, then we can use it in the scientific method. If the statement is not testable or does not have measurable qualities, then we do not consider it scientific. As our theories are developed and tested, and the data checked for accuracy and scientific significance, we publish the theories and data. This way others can test and offer feedback to the theories, which then will enhance and grow our theories, knowledge, and information.
An important element of scientific method is that, “A scientific hypothesis can be supported by evidence, but cannot ever be proved, only disproved.” For example, the statement, “A person cannot live forever” is not a scientific theory or hypothesis, however it is a reasonable statement. That statement is not scientific because it cannot be disproved. We will try to avoid such statements because they are not useful for scientific evaluations.
“Scientific knowledge is tested by observation, not derived from it.” Karl Popper
People hope to have their ideas backed up by scientific theories and explanations, but due to excess enthusiasm often those theories and explanations are flawed and are then adjusted to match the test results. We promote the idea that only “good” theories are worth testing. We define “good theory” or “good explanation” as one that cannot easily be varied or changed as a result of the collected measurements and observations. We would define a theory or explanation that can easily be changed as a “bad theory” or “bad explanation.” Our theories and explanations will be evaluated per these definitions, and we will be careful to state our theories and explanations in ways that make them difficult to vary.
We must address the fact that scientist are also human, and so prone to the human foibles. What concerns us here is that scientific method requires truthful reporting of the facts. The truth has no sides. Humans, including scientists who are reporting results or analysis, can make misleading statements, lies of omission, and outright lies. When scientist are doing this, they are not using the scientific method. Scientific method requires truthful and forthright reporting of the facts.
“Measurable, Disprovable, the Explanation difficult to manipulate, and the reporting of results in a truthful manner” is a way to describe the scientific method.
(Ref: Francis Bacon, 17th century, Karl Popper, 20th Century, David Deutsch, 21st Century, see Scientific Method for a little more on this method.)
We are careful to define the terms we use in our data analyses, in order to be correctly understood. The term ‘Rider’ or ‘Motorcyclist’ are not used, because they are ambiguous.
We use the following terms:
Motorcycles1: Mopeds, two-wheeled motorcycles2, three-wheeled motorcycles, off-road motorcycles, scooters, mini bikes, and pocket bikes are all motorcycles.
Our definition for “Motorcycles” follows the NHTSA definition for motorcycles.
Click here for the page of the FARS Coding Manual that explains the different Body Types for Motorcycles. Note that Vehicle Body Type (80) Motorcycle means Two-Wheeled-Street Motorcycles2.
Please see this table of the FARS Analysis Manual that explains how the different body type elements are grouped for motorcycles. Here is where NHTSA defines motorcycles as Vehicle Body Types (80-89). There is a subtlety here. On the table, motorcycles (plural with an s) is defined as body types 80-89. Then body type 80 is defined as motorcycle (no s).
1NMI definition of terms accept the realities of fatality data collection, especially as performed by investigative (law enforcement) officers at crash scenes. Fatality data collection has a long history and vehicles have evolved greatly since the first fatality reports. We carefully track Vehicle Body Type in our fatality data. We acknowledge that training and licensing may require terms and definitions that may reasonably vary from what is identified as Vehicle Body Type coded from law enforcement crash reports.
2Two-Wheeled Street Motorcycles are what has historical been called “motorcycles.” Crashed Two-Wheeled Street Motorcycles in the NHTSA Body Type Code are coded, 80 Motorcycle. Sidecars? What about motorcycles equipped with sidecars (sidecar rigs)? Motorcycles equipped with a side car are included in the definition of (two-wheeled) motorcycles (FARS Body Type 80) when discussing data. We acknowledge that training curricula and licensing may have other useful and necessary definitions for motorcycles. Motorcycles with sidecars are purposefully included in NHTSA’s and our two-wheeled motorcycle definition and trikes/spyders are purposefully included in three-wheeled motorcycle definition due to data collection realities. Historically, motorcycle crashes began with both types of crashes, motorcycles and motorcycles with sidecars attached. Crashes began to occur and data tracked. Since motorcycle and motorcycle with sidecars fatalities began simultaneously, they are forever coupled in the data. We strongly acknowledge motorcycle sidecar driver training and licensing as separate from motorcycle driver training and licensing. The dynamics and techniques for controlling a sidecar rig are different than those of controlling a motorcycle without a sidecar. Regarding licensure, states currently license/endorse sidecar rigs differently, some states use Class C, some use Class M, and some have a special class of license endorsement, possibly for pragmatic reasons.
Persons carried within or on a motor vehicle in transport: Driver, Passenger, Occupant.
We use the definitions of “Person Type” provided from NHTSA and the FARS coding manual.
Driver of vehicle in transport: the person carried within or upon a vehicle, who is operating or controlling the vehicle.
Passenger of vehicle in transport: a person carried within or upon a vehicle, who is not operating the vehicle.
Unknown Occupant of vehicle in transport: is used when it cannot be determined if the person was the driver or passenger, but it is known that the person was an occupant of a motor vehicle in-transport.
Total Occupants of vehicles in transport: This is the sum all drivers and passengers. This sum includes “Unknown Occupants,” because even if an unknown occupant is in a crash, the unknown occupant was either a driver or passenger and thus would be included in the sum of drivers and passengers.
Vehicle Body Types: Vehicle Body Types Motorcycles and Vehicle Body Types Passenger Vehicles.
Table of the FARS Analysis Manual that explains how the different body types elements are classified.
FARS Coding Manual that explains the different Vehicle Body Types for Motorcycles.
AF – All Fatalities in crashes involving one or more of a particular vehicle body type.
AFMC – All Fatalities in crashes involving one or more motorcycles.
AFPV – All Fatalities in crashes involving one or more passenger vehicles.
Crash: First we determine the Vehicle Body Types, for example “Motorcycles.” With the definition established, it can be stated that, “A motorcycle crash is a crash that involves one or more motorcycles.”
Danger: the chance of bodily harm to a person or persons.
Fatality: motorist or non-motorist involved in a crash with a motor vehicle traveling on a traffic way customarily open to the public, and must have resulted in the death of a motorist or a non-motorist within 30 days of the crash. Ref: Fatality Analysis Reporting System data base.
HDF – Helmeted Driver Fatalities
Killed: fatality of a person through unnatural causes, such as a motor vehicle crash.
LDF – Licensed Driver Fatalities
LMDF – Licensed Motorcycle Driver Fatalities
LPVDF – Licensed Passenger Vehicle Driver Fatalities
Morbidity: a measure of gruesome or grisly injury to a person, whether or not fatal.
Rate: a calculation of the relative frequency of an action, compared to a base such as population or exposure time.
Society: the general population of people in a given geographical or political area.
TDF – Total Driver Fatalities
TMDF – Total Motorcycle Driver Fatalities
TPVDF – Total Passenger Vehicle Driver Fatalities
Transfer Momentum: Momentum is a vector quantity that can be transferred from one body to another.
Transform Energy: Energy is a scaler quantity that can be transformed into different forms, such as Kinetic, Potential, Chemical, Mechanical, Electrical and other types of energies.
We use crash, slide, and collision this way: Slides, collisions, and combination of slides and collisions are all crashes. If the slide is very short in time and distance, we use the term collision. If the collision happens for a long time and over some distance, we use the term slide.
VMT – Vehicle Miles Traveled
VMT data should not be used for historical comparisons because the statistical methodology for both VMT and vehicle registrations was changed over the years. We are confident in using the VMT data starting with 2012. We believe that VMT is a superior exposure measure to vehicle registrations, so we downplay current registration data.
For more regarding the warnings for not using registrations from historical comparisons, please see this link:
NHTSA’s WARNING to not use registrations for historical comparisons
Caveat: Just because a ratio is mathematically correct does not make it scientific or meaningful. In addition to the ratios of the well defined fatality counts listed above (for example, the ratio LMDF/MDF = percent licensed), we use population (Pop) with count “All Fatalities” and Vehicle Miles Traveled (VMT) with count “Driver Fatalities.” These ratios are scientifically meaningful.
We are able to access raw data from NHTSA’s Fatal Accident Reporting System (FARS). We consider this fatality data very reliable. Although injuries may go unreported, fatalities occurring on public roadways are reliably reported. Although we typically depict fatality rates based on “fatalities per year”, there are other ways to provide additional insight into motorcycle fatalities. For example, to better compare the danger of motorcycling to other vehicles, we can ratio All Fatalities/Motorcycle (AFMC) and All Fatalities/Passenger Vehicle (AFPV), comparing the annual relative dangers of the two classes of vehicles.
Some additional meaningful fatality rates are:
Pop: Fatalities per general Population for a given area. Pop can be used to measure the danger of specific vehicles to society (Societal Danger). We have high confidence in the accuracy of the population count reported by the U.S. Census Bureau because the population count is very exhaustive, and used throughout the government and society for many purposes.
We use AFMC/Pop extensively to MODEL Societal Danger. This numerator/denominator combination is meaningful. All fatalities in the crash were members of the population. Governments should use the AFMC/Pop to track, and make adjustments to their safety programs, so as to minimize this rate. Additionally, AFMC/Pop rates gives us a way to make meaningful comparisons in time for a given geographical location as well as comparing different geographical locations during the same time period. For example, “Compare the 1990s fatality rate to the 2000s in California” and, “Compare the fatality rate of California to New York for 2005.”
VMT: Fatalities per Vehicle Miles Traveled. VMT can be used to measure the exposure to danger of a specific category of driver (Driver Danger). VMT is very important data and much improvement has been made on techniques to determine accurate numbers. This is because certain taxes will be based on VMT in the near future.
VMT is an excellent measure of exposure to danger. Whatever the danger, the more miles traveled, the more time of exposure to the danger for the driver.
We use TDF/VMT to model Driver Danger.
We use the Highway Statistics books published annually by the US DOT Federal Highway Administration to obtain VMT.
For example, here is a link to the 2013 VM-1 table.
“Counting Cars” and checking VMT: Practicing counting passenger vehicles can help you gain insight to the vast difference between Motorcycle VMT and Passenger Vehicle VMT. While observing traffic, count the number of passenger vehicles you see. You can use particular stretches of road, or simple try to count for five minutes from any observation point. It is exciting to learn how to do this!
Vehicle Registrations (Reg or VR): Fatalities per total vehicles registered (VR). This has difficulties in accuracy like VMT. VR is an exposure measure, the more registrations, the more miles will be driven. VR can be used to support VMT claims. Vehicle Registrations (VR or Reg) have been used in the past to measure fatality rates, such as Fatalities per total vehicles registered. VR is theoretically a measure of exposure, assuming that more registered vehicles indicates more miles driven. VR can provide a useful measure of danger over time for a specific type of vehicle. For instance, it is reasonable to compare the fatality rate of motorcycles from one year to another, since the use of motorcycles can be assumed to not change dramatically from year to year. Likewise, it is reasonable to compare the fatality rate of passenger cars in the same area from one year to another. The limitation of using VR for rates is attempting to compare different types of vehicles that have obvious differences in use. For example it is unreasonable to compare the fatality rate of classic cars to taxicabs, or compare the fatality rate of passenger cars to the fatality rate of motorcycles. NMI cautions that any comparison of danger between different types of vehicles based solely on registration numbers is likely to be inaccurate and misleading.
Fatalities per new vehicle units sold (NVU). Industry exerts effort for accuracy with these numbers. NVU can be used to support VR claims.
NVU and VR can be used to determine the popularity of a specific type of vehicle. Generally, NVU is provided by a manufacturers’ organization, the Motorcycle Industry Council; while VR is provided to NHTSA by state licensing departments.
Avoiding Confirmation Bias
“We are not taking a side on an issue. We share information that is factual. The truth does not have sides.” NMI
Confirmation bias is the seeking out of information that confirms the statement or belief, and discounts information that conflicts with the statement or belief. We are watchful to prevent confirmation bias from eroding our analysis of the data. Note that “bad” or “easily varied” theories are strongly susceptible to confirmation bias. This is why our explanations are not easily varied.
We have found the following terms useful for monitoring confirmation bias:
The quality of being factual.
Truthy or Truthiness:
The quality of preferring facts one wishes to be true, rather than facts known to be true.
“Just because it sounds right [has good truthiness] does not mean it is right [truth].”
We carefully use the 4-Box scientific method to check for “Truthiness” of a statement. This 4-Box method is the simplest test for checking whether a statement belongs in the “Old Paradigm – Motorcycles can be safe and can be enjoyable” or the “New Paradigm – Motorcycles are dangerous and can be enjoyable.” (Examples will be posted here in the future.)
Raw Data Comment
National Motorcycle Training Institute always posts the raw data, with reference to where the data was obtained. This is so anyone else may go to the source and verify or our analysis, or provide conflicting analysis. We encourage you to apply the scientific method to our work. For fatality counts, we use National Highway Transportation Safety Administration (NHTSA) data from the Fatal Accident Reporting System (FARS) database. We have a high degree of confidence in NHTSA’s fatality counts. We are careful to monitor the published changes in the method of counting through the years of the data base’s life. Starting in 1991 the data we look at in this database became well defined. We caution readers to understand the changes made to the data base regarding FARS data prior to 1991 before splicing any data from FARS from years prior to 1991.
Assessing the Danger of Motorcycling to Society
We use the rate and changes in rate of Total Motorcycle All Fatalities divided by Population, TMAF/Pop, to assess Societal Dangers.
We compare the fatality rates and changes in rates of Motorcycle Related Fatalities with Passenger Vehicle Related Fatalities to measure relative danger.
Assessing State Motorcycle Licensing Programs
We use the rate and changes in rate of Licensed Motorcycle Driver Fatalities per Vehicle Miles Traveled LMDF/VMT, to measure the danger for licensed drivers. We use the percent of Licensed Motorcycle Driver Fatalities/Total Motorcycle Driver Fatalities, LMDF/TMDF to measure the effectiveness of licensing programs. If effective, licensed drivers should be under represented and the unlicensed drivers should be over represented in the fatality rates.
Assessing Effectiveness of Motorcycle Safety Programs
We compare the results of all analyses above when assessing the effectiveness of motorcycle safety programs.