In-car Gaming at Mercedes-Benz

Sven-Eric Molzahn floors the gas pedal, taps on the brake, and then races around a sharp left curve. Next, he returns to a straight stretch of the track and accelerates once again — but suddenly the rear wheels skid and the car’s progress is blocked by the crash barrier. He overlooked a banana peel that was lying on the track. Fortunately, the bodywork remains unharmed. That’s because even though Molzahn has forcefully jerked the steering wheel around and slammed his foot on the brake, the CLA he’s sitting in hasn’t moved a single inch.

Don’t get it wrong: The coupe is a series-produced vehicle and is of course approved for road use. But it’s way more than that: You could really call it a game console on wheels. The people responsible for creating it are Sven-Eric Molzahn and his colleague Fabian Gajek. The two of them have taken the video game SuperTuxKart, an open-source variant of a classic Nintendo game, and integrated it into a real vehicle.

The game runs on a screen in the center console, and the user steers the virtual go-cart exactly the way he or she would steer a real car. But it gets even better: The faster the virtual vehicle goes, the more air streams into the real car through the ventilation nozzles. The ambient lighting also plays along: If the car travels underwater in the course of the game, the lighting turns blue. If it drives through the desert, it’s red. And when the traffic light signals the start of the race, the ambient lighting is green.

Showing what MBUX can do

It was a surprise hit. Bild, German’s best-selling newspaper, reported about gaming in the CLA. And the topic created quite a buzz in automotive and tech journals. But how on earth did these two programmers come up with the idea of turning a series-produced vehicle into a gaming console? “We needed a use case that would enable us to show what the technology in our vehicles is already capable of doing today,” says Sven-Eric Molzahn.

Gaming is actually not the team’s main focus. The two colleagues are working on a platform that will enable other specialist units at Daimler to integrate their software as smoothly as possible into the telematics system MBUX (Mercedes-Benz User Experience). “We’re basically working on a kind of plug-and-play system,” Molzahn explains. “Our aim is to enable everyone who knows even a little bit about software to program an application for his or her particular use case, which will then also run on MBUX.

In order for that to happen, the hardware in the vehicles doesn’t have to be significantly adapted. This means that the game could run just as well in every series-produced vehicle. I think it’s realistic to expect that in a few years we’ll be able to customize MBUX by adding individual apps.” And it doesn’t require much thought to come up with good ideas about which apps could be useful inside a car. What about in-car office applications that enable the driver to dictate to-do lists or shopping lists while on the road? Or payment apps that make it unnecessary to stand in line to pay for filling up at the gas station — or to pay for recharging an electric car’s battery at a charging station, which has tended to be a complicated process so far.

This CLA will not be returning to Kecskemét anytime soon

The creators’ basic idea was to present a video game as a possible use case. Their idea became a showcase presentation that took only four weeks to develop. First, the programmers borrowed a CLA from the Mercedes-Benz plant in Kecskemét. “We told our colleagues in Hungary that we were creating a showcase presentation for the Mobile World Congress (MWC) in Barcelona and that they’d have the car back in two or three weeks,” Molzahn recalls. “That was when we wanted to return the CLA. At least that was the plan,” he says with a grin.

That was a few months ago — but the CLA has still not been returned to Kecskemét. The enthusiastic public reception of their playful presentation convinced the team that the CLA’s use case as a video game console was just too spectacular to ignore. That’s why he’s touring further: his most recent appearances were at the Tech Open Air in Berlin and at the gamescom in Cologne. And at the International Motor Show (IAA) in Frankfurt, visitors could also marvel at a gaming mobile – this time a GLC.

The team had not expected in-car gaming to be such a hit. “We knew that some people would think it was really cool, but we couldn’t have imagined that it would make the rounds inside Daimler and even in the media to this extent,” says Molzahn.

In-car gaming — a business model?

But will we really play games during car trips in the future? After all, we have to concentrate on the road traffic while driving, so how would that work? The engineer Alexander Satanowsky is only too familiar with the bemused looks he gets from people when he tells them that at the moment his main occupation is the development of in-car gaming. As the head of the NeXt Scouting team, his job is to identify up-and-coming technological and regional trends in their early stages.

Previously, he closely monitored new technologies related to autonomous driving. Today he’s looking for ways to make the ride exciting for the passengers through interactive entertainment — for example, in a future when highly automated driving is a reality. That’s how he ended up working on in-car gaming.

Since the summer of 2017, Satanowsky and his team have been addressing the question of whether this would be a possible business area for us. Thanks to his job profile, he’s fairly accustomed to presenting people with ideas that seem strange at first glance. “Of course there are quite a few people who ask us if we’re out of our minds,” he says with a laugh. “But I really like to deal with technologies that get people talking.”

Well then, let’s be frank: Does in-car gaming have the potential to become a genuine business model for Mercedes-Benz in the next few years? Alexander Satanowsky believes the answer is yes — and he’s got good arguments to back up his opinion. “Today the gaming market is already bigger than the music industry and bigger than the TV market,” he says. “And in technology scouting an early warning signal regarding trends is always the behavior of investors.

At the moment, gaming and eSports are attracting tremendous interest.” He’s thinking of a world in which self-driving vehicles are no longer a vision of the future but a daily reality. “One day customers will be able to simply use an app to order a fully automatic vehicle for getting from A to B. When that happens, we’ll need good reasons to persuade them that it’s highly desirable to be chauffeured in a Mercedes,” he says.

But why is an automobile an ideal game console? Sven Domroes, the co-leader of the car2play project, points out all the features that cars already have today: “Inside a car, we already have lots of physical systems that appeal to the senses: the steering wheel and the pedals, of course — but also the seat heating, the ventilation system, and the seat belt tensioners.”

All of these features enable interactions between what’s happening on the screen and what the gamer experiences. For example, warm or cold air could stream out from the ventilation nozzles, depending on the game environment. An explosion on the screen could be intensified by the sudden activation of the seat heating. And vehicle scenting, which is already available as optional equipment in selected Mercedes-Benz models, could be used to infuse the vehicle’s interior with appropriate aromas. The gamer will smell fresh grass when he or she crosses a meadow, but be hit by the stench of burning rubber on the racetrack.

“Our goal is to make the in-car gaming experience much more intense than anything that would be possible when you’re playing on a mobile phone or at your desk,” explains Domroes. “That’s what we mean by immersive gaming — diving into an all-encompassing virtual environment.”

Today many people are already gaming inside cars — without knowing it

But does autonomous driving have to triumph before in-car gaming can become a reality? Not necessarily. “My son already enjoys the pleasure of autonomous driving — and he never tips me as his chauffeur. Sometimes he even runs off without saying goodbye,” says Alexander Satanowsky with a laugh.

Sven Domroes adds, “Or just think about a touring coach. Bus passengers have always enjoyed autonomous driving. And in a bus, we can address more than 50 people at once with our gaming applications.” Above and beyond such considerations, in-car gaming could of course also be an interesting way of passing time while the driver is standing at the charging station with his or her battery-electric vehicle.

Of course Satanowsky and Domroes can’t yet predict whether, when, and exactly how their idea of immersive gaming in vehicles will catch on. But both of them are convinced of one thing: that every person has an inherent play instinct. “Imitate what I’m doing,” Satanowsky demands.

He taps the table rhythmically with his right hand, and also taps with his left hand on every third beat. We’re perplexed, but we join in. “Does this seem familiar to you?” he asks. We nod. “Have you ever done this inside your car? When you’re stuck in a traffic jam, for example?” We nod again. “In that case, you’ve already played a game in your car,” says Satanowsky with a mischievous smile. “

And now, imagine this: You’re stuck in traffic, and out of sheer boredom you tap a rhythm on your steering wheel. With your tapping you’re playing a virtual set of drums. Through his or her tapping, your passengers are playing another instrument. All of this is combined with a band or a huge orchestra by your onboard computer — and the music composition that is being created in this way is coming out of the loudspeakers.”

A look at the lab: The simulator of the in-car gamers

An autonomously driving vehicle is not a prerequisite for this use case. All that’s needed is a normal traffic situation during rush hour in a big city. The same goes for quiz formats, for example, which the driver could play via voice control without taking his or her hands off the steering wheel. “That could also be a way to keep the kids happy in the back seat. Games for the whole family would be another option,” says Satanowsky.

A look at the lab: The simulator of the in-car gamers

What other possibilities will open up when the world of video games enters our cars? Rose Sturm is aware of the limitations the driver will be subject to until we have autonomously driving cars. “We can always get the technology into the car somehow, that’s not a problem,” she says.

“But of course safety is the top priority. Even in the case of highly automated vehicles, we have to ensure that if a risky situation crops up, the driver can immediately intervene. And here a certain question naturally occurs: In this kind of situation, how fast can a person switch from the game world to the real world?

That is a significant legal question, among other things — and how we answer it definitely depends on how fast the technology progresses. But because Satanowsky and his team explicitly focus on future trends, in their lab they have already hazarded a look at the in-car gaming of the distant future.

In their simulator they’ve got many components that we’re familiar with from series-produced vehicles. There’s a driver’s seat that reacts to what’s happening in the game — for example, by moving forwards, backwards, and sideways. There’s a seat belt with a belt buckle tensioner. And of course there are large-format ventilation nozzles. The video game runs on a 49-inch curved monitor that is located where the windshield would be in a normal vehicle.

While the virtual go-cart drive in the CLA was still a comparatively comfortable leisure activity, driving a lap on the virtual Nürburgring in the immersive simulator of Alexander Satanowsky’s team is a tough challenge for would-be race car drivers.

Especially because every mistake is punished: If the gamer makes contact with the wall, the seat shakes violently and the seat belt tensioner tightens up. And if the player forgets to shift to a higher gear at the right time, the engine overheats, and the ventilation nozzles mercilessly transmit this heat to the person sitting in the driver’s seat. After just one lap, you have really been shaken up and you are really sweating. And you doubt that all of this has just been a game.