13 challenges that come with autonomous vehicles

Hear from CIOs, CTOs and other C-level and senior executives about data and AI strategies at the Future of Work Summit on January 12, 2022. Learn more

Telecommunication operation: the technology that enables a human being to remotely monitor, assist and even drive an autonomous vehicle.

Telecom operation is a seemingly simple option, but it involves several technologies and systems in order to be implemented securely. In it first article of this series, we have established what telecommunications operation is and why it is crucial for the future of autonomous vehicles (AVs). In it second article, we showed the regulatory traction and weight gained for this technology. In it third and fourth articles, we explained two of the many technical challenges that needed to be overcome to enable remote assistance and vehicle operation. That fifth article explained how it is all achieved in the safest possible way. In this installation, we get to the most important person in the entire loop, the customer.

There is an Israeli journalist, Sarah Tuttle-Singer, who writes the stories she hears from taxi drivers. She has so many that she even wrote a book with the best of them. This makes perfect sense. Cabbies spend all day driving around with different passengers, and when people get bored, they talk. Inevitably, taxi drivers have a million anecdotes to share. Unfortunately, with the impending transition to autonomy, these stories will disappear.

The problem of autonomous vehicles

There is an existential problem with removing a driver from the vehicle. You simply can not deliver 100% service availability or a satisfactory level of customer experience, not today and not in 50 years. Human interaction is mission critical. There are many cases where an autonomous system is unable to respond to the level that the customer wants and is entitled to. The lack of a person behind the wheel not only means that there will be downtime for the vehicle due to being confused by various situations, but that seemingly simple services cannot be provided. The reason is simple, a machine does not know how to interact with a human in the same way that a human does. The customer experience is possibly the biggest problem of them all when it comes to autonomy, and they fall into four main categories: passenger concerns, emergencies, deliveries, and monitored zones.

Modern luxury car dashboard with control of phone rotary knobs, AC air conditioner, radio, media, my car and Auto buttons

Above: Image via Stockphoto.com, licensed to Amit Rosenzweig

Image credit: Stockphoto.com/licensed to Amit Rosenzweig

Passengers’ concerns

1. Discomfort of the passenger: Maybe they feel anxious or insecure. Without any human driver to respond to their needs, the passenger has no one to communicate with; no one to quell their fears, calm them down or address the source of the problem.

2. Control of A / C or infotainment system: Not all passengers have the same comfort levels or needs, and not everyone knows how to operate certain systems. Problems with temperature control or entertainment on board are inevitable. Still, there is no one to talk to.

3. Special drop-off points: At the moment, when you arrive near your destination, you just say, “can you drop me off at that door / car / tree / etc?” and typically the driver will do just that. With a self-driving system, not only will it be able to, but it can bring you to what is technically the right address, but not an ideal place for easy access. This is especially important for elderly passengers and people going for medical treatments.

4. Passenger violations: Not all problems originate from the vehicle. What if the passenger is not wearing a seat belt or there are too many people squeezing in the back seat? A human driver would fix them and the problem would be solved. Would a robotic taxi at all be able to recognize these problems let alone deal with them?

5. Forgotten things or even worse, forgotten baby: In 1999, world-famous cellist Yo-Yo Ma forgot his $ 2.5 million cello in a taxi. Passengers have even forgotten sleeping children. Right now it is easy to call the driver (whose number is on the receipt). This is no longer an option when there is no driver who can stop and check the vehicle for persons or belongings.

6. Vandalism: Unfortunately, not all people are good. In fact, some are pretty bad, while others such as teenagers are simply careless and indifferent. Passengers can damage or destroy a vehicle during a trip. Without the driver’s watchful eyes, this behavior would go unchecked. The result is higher maintenance costs, reduced profits and downtime for the vehicle.

Forgot baby

Above: Image via Stockphoto.com, licensed to Amit Rosenzweig

Image credit: Stockphoto.com/licensed to Amit Rosenzweig


7. Law enforcement: When a vehicle is driving irregularly or there is a condition to be investigated, police use a loudspeaker to warn the driver to stop. Without a driver on board, the police cannot do their job and the consequences can be unfortunate.

8. Doctor / ambulances: Similar situation as the police. Maybe a passenger has a medical problem and manages to make a 911 call. The vehicle must still know where and when to stop to allow doctors to provide passengers with the necessary treatment they need.

Police officer stops a speeding car

Above: Image via Stockphoto.com, licensed to Amit Rosenzweig

Image credit: Stockphoto.com/licensed to Amit Rosenzweig


9. Incorrect / damaged / missing package: When you receive a package from a courier, they deliver it to you and ask you to sign it. If the received package is very damaged, or just the wrong thing, you can tell them and they will handle it. If you are missing an item, they will go back and bring what they forgot. A robot would simply automatically move on to its next destination.

10. Finding the right customer: Today, there are only a few vendors with delivery bots. However, these numbers will multiply as technology improves and production costs fall. Soon there will be many robots with many receivers. A delivery bot can deliver to a place where several people are waiting for different orders. For the computer, it is impossible to determine which human it will need to deliver. This can result in a confused device and a frustrated customer.

Participated in zones

11. Content validation: When a truck arrives for pick-up or drop-off, there is a need for the gatekeeper to confirm the records, the contents of the delivery and direct the vehicle to its specific parking space or loading space. An autonomous system does not respond well to pointing in a direction or verbal commands, nor can it explain why there may be a discrepancy between the information it has and the instructions given by the companion.

12. Change of orders: Once inside a facility, someone may need to divert a vehicle from one task to another. This is especially true for construction zones where a vehicle may have a number of tasks and their order changes. There is no way for this person in the field to communicate these changing needs.

13. Injuries: Once the vehicle has arrived, there will be some degree of vehicle inspection. If there is anything wrong with the vehicle or damage has occurred, there is no one to tell.

The solution

For those who read the previous articles in this series, the answer should be obvious. For those who did not, the solution is to have a distant human in the loop. At present, none of the aforementioned problems is a problem as there are human drivers and couriers. Autonomy cannot solve these problems. There will still be a need for a human being to deal with these problems in the most efficient, effective and safe way possible. Therefore, telecom operation is the only choice. But as with most solutions, it has its own challenges.

To bring a human into the loop

When a telecommunications operation session is triggered, it does not just go to the first available telecom operator (TO). This is for the same reason that when you call customer service, there is a routing system that, based on your input, will direct you to the agent with the expertise you need. But unlike a regular call center, the customer does not necessarily have the option to “press three for traffic problems” or “press pound to repeat this message”.

The first possible solution would be that there is a telecommunications operations manager (TM) who will answer the calls for assistance first and, after determining the level of complication and the need, direct the session to a specific operator. This would be very inefficient and would mean that TM is not available to perform their actual job – to manage. Instead, there must be an automatic and intelligent way of directing the need for human intervention without … human intervention.

When a telecommunications operation session is triggered, the first thing that is automatically detected is the source of the request. Did the passenger trigger the need for a TO? Was it a first responder or law enforcement? Or was it the vehicle itself? Each of these situations requires a different type of reaction and therefore a different type of TO. Within a given telecommunications operations team, there will be some who are junior and some who are senior, some who are more customer-oriented, and others who are more technical. A senior TO may be authorized for remote assistance and remote driving, while their junior counterpart is only allowed to assist. Some TOs may be primarily for customer interaction situations, so if the session is triggered by the customer, they will be the ones looped in. On the other hand, if the vehicle triggers the session, not because of a confusing traffic situation, but because of a technical situation issue, a completely different answer would be needed.

Remote assistance is complex

We’ve just found that the seemingly simple aspect of who answers which call is complicated. For the challenges of establishing that connection yourself, see our previous articles on network connectivity and video compression.

There is another challenge. Once the telecommunications operation session has started, the TO must understand what exactly is going on. There is a serious amount of information they need to receive that needs to be put into their screen so that they can better understand the situation. This problem multiplies when you jump between several vehicles. There must be processes and tools built-in so that there is minimal delay between the start of a session and the goal of solving the problem for the vehicle and to the customer’s satisfaction.

Enables autonomy

If autonomous vehicle providers ever want to have mass-rolled robotic fleets, they need to make sure man is in the loop. It is a complicated and complex process from start to finish and a completely different technology than the autonomy itself. This is why telecommunications service providers exist and why industry leaders like Motion choose to rely on them for this mission-critical function. Everything else and they are not on the right path towards the self-driving future.

DataDecision Makers

Welcome to the VentureBeat community!

DataDecisionMakers is the place where experts, including the technical people who perform data work, can share data-related insights and innovation.

To learn about cutting-edge ideas and up-to-date information, best practices, and the future of data and data technology, join DataDecisionMakers.

You might even consider contributing your own article!

Read more from DataDecisionMakers

Leave a Comment