Stop the Trolley!
California Autonomous Driving Test Statistics 2020
The CA DMV, on Feb 9th, 2021
released a batch of reports filed by multiple companies
testing autonomous cars in the bay area. All companies testing autonomous cars on public
roads are required to report statistics and details related to miles driven and disengagements,
otherwise known as human-takeovers. The batch of data visualized here corresponds to the period
spanning Dec 1, 2019 to Nov 30, 2020.
While the statistics are useful to look at large scale trends, they cannot be taken as an
apples-to-apples comparison between companies. In addition to the fact that various companies are
targeting different markets and business models, miles per disengagement is a
faulty approximation of maturity of the system since the numbers are affected by where companies drive,
the difficulty of driving scenarios cars are tested in, which disengagements are considered valid for
reporting by each company, etc. For example, it is widely known in the industry that many companies drive
long stretches of highways to collect easy miles because highways aren't nearly as hard as urban environments,
thereby improving their miles and miles to disengagements ratios.
Many companies are also known to reproduce their on-road disengagemnets in simulation and report only if it
reoccurs despite fixing.
All of this allows for a wide room of interpretation and as such these numbers should be taken with
a self-driven truck load of salt.
What we've attempted here is a pure visualization of aggregate reported statistics.
Feel free to play around with it and let us know if you spot any inaccuracies. Original data dumps
for disengagements can be found
here.
Additionally, reports of collisions involving AVs can be found
here.
Note that here, a car is counted in fleet size only if it clocks at least 1 mile over the entire
reporting period and does not account for cars that are registered but have been inactive. Do checkout our visualization of last
year's
data (California Autonomous Driving Test Statistics 2019) here!
Miles, Fleet and Disengagements
Waymo and Cruise alone account for more than half of the total fleets operating in California and more than three quarters of all miles driven in the state. Also, due to the pandemic, most companies clocked in fewer miles in 2020 than in 2019 with the exceptions of a few like Apple and Zoox who actually improved their miles driven. Segmented by month, it can be seen that for most companies, miles driven almost halved in March 2020 and became practically zero in Apriland May 2020 while local governments advised strict shelter-in-place. Since then driving has picked up but not quite matched pre-covid levels manifesting that across the industry the pandemic has deccelerated the progress of AV testing and development.
Note : Hover over / Click various visual elements below to see more details.
Miles vs Fleet vs Disengagements
Cruise and Waymo are outliers with big fleets and high miles but reasonable number of disengagements.
On the other hand Toyota Research and Mercedes-Benz R&D are also outliers for having order of magnitude more
disengagements for order of magnitude less miles. They could also be orders of magnitude more forthcoming in
their reporting compared to others. One could also note that Pony and Nuro are clocking in a lot of miles
with low disengagements for a comparatively small fleet size.
From the second graph, we can infer that
Cruise and Waymo are way ahead in their testing game distantly followed by AutoX, Pony AI and WeRide.
Note : Hover over / Click various visual elements below to see more details.
Disengagement actors, reasons & locations
Interestingly, among the largest players, Cruise has only had 27 reportable disengagements the entire year and around 60% of them are atrributed to “precautionary takeover on account of other road user behaving poorly”. Waymo also reported only 21 disengagements across the year although the reasons they mentioned admit errors of the AV system. Most companies test their technology on roads marked "streets", ie urban areas, while the reporting of disengagement source (actor) is often arbitrary.
Disengagements Locations{: Street | : Highway | : Freeway | : Parking Facility | : Rural}
Disengagement initiated by Actors {👤: Test Driver/Passenger | 🤖: AV System}
Note : Hover over / Click the cards below to see more details on the disengagement reasons.
2020 vs 2019
While most companies have reduced their total number of disengagements across the two years,
Apple, Zoox, AIMotive and Qualcomm
have atually reported more disengagements in 2020 than in 2019. Among these Apple and Zoox have
somewhat proportationately increased their total miles as well while the other two drove fewer
miles.
Waymo, Zoox, Aurora and Apple have increased their active fleet/ cars deployed in California while Cruise, Nuro,
Mercedes-Benz
and Nvidia have decreased year-on-year.
Lyft, Toyota Research, Mercedes-Benz and Nvidia have reported noteworthy decrease in disengagements
in 2020 compared to 2019.
Conspicuously missing from the visualization is Tesla
who is synonymous with ("full" *cough cough*) self-driving in wider media.
The company claims that their system on public roads in California
is technically of L2/L3 capability, which is a lower level of autonomy than others listed here and
thereby, Tesla is not bound to report testing statistics to the State of California.
Additionally, 8 companies that drove non-zero miles in 2019 shut their testing in California in 2020, notable among
them being Baidu, and Drive AI which got acquired by Apple. Also, 10 companies that never tested in
California in 2019 now do in 2020, including Argo AI.
Note : Hover over / Click various visual elements below to see more details.