How augmented reality paves the way for fully autonomous cars

Visionary technology turns your windscreen into a billboard.

Here's a heads-up: augmented assistance is the next big thing for new vehicles.

While major brands — Tesla excepted — aren't rushing to promote fully autonomous driving in production vehicles (largely due to legislative and liability issues), they are all rolling out augmented reality.

Mini's "X-ray vision" goggles that let the driver "see through" panels and Jaguar Land Rover's "transparent" windscreen pillars may seem futuristic but they're destined to find their way into your next car well before governments work out how to deal with autonomous travel.

Augmented reality (AR) software can detect and highlight potential obstacles, from inattentive pedestrians to animals near the roadside at night.

The premise behind AR is that the improved field of vision should enhance safety by eliminating blind spots. Heads-up displays heralded the start of the shift, projecting speed and satnav directions on to the windscreen to minimise the time drivers had their eyes off the road.

The next step is to project information on to the entire windscreen; potentially with different data for the driver and passenger. For example, the driver sees a projection of which lane to be in while passengers see prices and times for a coming concert at a venue the car is approaching.

Augmented reality is to autonomous driving what plug-in hybrids are to electric vehicles

Camera and sensor-based detection of objects is mature enough for Japan to pass a law last month to allow cameras to replace side mirrors on new models. EU legislation is expected later this year.

The contention is that cameras have a wider field of view and can adapt to changes in light intensity better than a human relying on a conventional sheet of reflective glass (we can't see in infra-red).

Beyond the advantage of not having to wipe the dew off the mirror on an early-morning start, camera mirrors are touted as improving the look and (marginal) fuel efficiency of vehicles, which won't have the extra drag of wing mirrors.

Augmented reality is to autonomous driving what plug-in hybrids are to electric vehicles: an interim step before the technology is fully deployed.

As such, AR-equipped vehicles accustom drivers to accept overlays from sensors and cameras and help develop trust in the technology before the advent of autonomy.

Automotive AR isn't just confined to the in-car experience. Mercedes-Benz and Volvo have signed deals with Microsoft to use its "HoloLens" headset to create virtual showrooms, changing interior designs, wheel shapes and exterior colours with the swipe of a hand or touch of a button.

Mercedes-Benz has updated its Rescue Assist app to give emergency services a virtual view of the crashed vehicle and help determine the best way to extricate the occupants without cutting through high-voltage cables, batteries, fuel lines or airbags that haven't deployed.

Others, including Porsche, are using AR to help design vehicles and improve quality assurance during production by overlaying the "virtual" shape of a component against the real item to ensure ideal fit and placement.

Global technology analytics group ABI Research reckons more than 11 million cars with advanced heads-up displays will be sold by 2025. Managing director Dominique Bonte says that despite the potential advantages, AR needs to be restricted to relevant data to avoid information overload.

"It will be critical to use AR sparingly ... and only to display relevant, contextual information when needed to improve the driver's perception of the road environment and reducing response times," she says.

In Australia, the satnav error is about 1.5 metres

Australia's lack of roadside infrastructure and the absence of centimetre-precise mapping data mean the more advanced AR features — and autonomous driving functions — aren't available here.

Prestige brands in Europe already use information on corner radii, camber and climate to determine ideal car speed and positioning. In Australia, the satnav error is about 1.5 metres, because the tectonic plate we sit on moves north about 7cm a year and the last update took place in 1994.

It's not all bad news, though. GeoScience Australia plans to have amended data available early next year. It will initially be out by 20cm — still a big improvement but given ever-narrowing lane widths, enough to make you nervous — because the data will be based on where Australia will be in 2020. That's about the time car makers predict autonomous driving will be a reality in Europe.

To cope with the precision needed for autonomous travel, the European prestige triad of Audi, BMW and Mercedes-Benz last year bought Nokia's high-resolution mapping unit HERE to help refine co-ordinates.

It overlays GPS data with physical details (identifying a building or a landmark or a sensor-equipped intersection) and then continually cross-references that against the on-board sensors of cars on the road. No news yet on when that level of precision will apply here.

In the beginning...

General Motors launched the first commercial heads-up display in its Oldsmobile Cutlass in 1988. BMW pioneered the modern version with the launch of the 2004 5 Series.

Would you like to see more augmented reality features in new cars? Tell us what you think in the comments below.