Mercedes-Benz MBUX Hyperscreen: Sajjad Khan on the EQS debut

Mercedes-Benz has unveiled the MBUX Hyperscreen, the first fully glass digital dashboard - a launch many assumed would eventually come from a Silicon Valley firm rather than from Stuttgart.

That announcement is notable coming from a car maker long associated with caution and tradition, yet one that has reshaped its image over the past decade - not only through the styling of its models, but also through the way the company is structured and operates.

Sajjad Khan, Daimler’s Chief Technology Officer, explains the new MBUX Hyperscreen, which will make its debut as an option on the Mercedes-Benz EQS.

As well as being, without question, the largest screen ever fitted to a car, Mercedes-Benz also says it is the smartest. What is that claim based on?

Sajjad Khan: The Hyperscreen presents only the information you need, precisely when you need it. Rather than forcing you to hunt through sub-menus, it works the other way round: the data comes to you. In our view, it sets a new benchmark in the industry, because it blends the analogue world with the digital one, and does so at a very high standard.

So it was never an ego project - the “mine is bigger than yours” type of thing?

SK: Not at all. Building the world’s biggest display was never the aim in itself. What drove us was the ambition to deliver something genuinely valuable - innovative and intelligent - for our customers, with an uncompromising focus on the user and on digital thinking.

“Zero layer”

What is the Hyperscreen made of, and how is information arranged?

SK: Beneath a gently curved surface we have three independent displays that, to the user, look like a single interface. The central screen appears particularly vivid thanks to the latest OLED technology and, because simplicity matters so much to us, we introduced a new interaction level we call the “zero layer”.

The necessary, personalised content is positioned directly in the driver’s line of sight, so there is no need to dig through sub-menus - something that could take attention away from the driver’s primary task in the car. We use different colours depending on the driving mode, and in the instrument area we created a new flying-saucer-shaped icon that helps visualise energy recuperation status, acceleration and g-forces.

Can you be more specific?

SK: Absolutely - especially because this really does need explanation, given that it represents a reinvention of how humans interact with the car. Work on the “zero layer” concept began three years ago. It is built around three very different modules, where everything is displayed without the need to navigate sub-menus. And because it runs on artificial intelligence (AI), the system quickly learns the user’s preferences and habits, then delivers what they need.

It looks striking on the outside and is extremely intelligent underneath, linking to the outside world and to everyone in the vehicle. All functions and features - battery charging, entertainment, phone, navigation, social media, climate functions, connectivity, massage and so on - remain visible and available at all times, fully connected and tailored to the user, combining considerable computing power with AI.

In your testing with “human guinea pigs”, was the “zero layer” principle found to be easy to use?

SK: Yes, because it places apps and functions into a proactive, smooth user interface, right at the user’s fingertips. The key navigation map remains permanently visible in the centre and, beneath it, we position the communication and entertainment controls.

Since the first generation of our new operating system (MBUX) launched in 2018, we studied customer feedback and found that four out of five actions (while the car is moving) relate to navigation, media and phone calls. That is why we prioritised access to those areas. Through AI, the system learns what the user does most often inside the vehicle and brings those shortcuts to the screen, while less-used items sit further back.

Can you give an example of the kind of learning the Hyperscreen can do?

SK: Of course. The EQS chassis can raise itself to increase ground clearance - helpful for steep garage entrances or for getting over speed humps. The system stores the GPS location where the driver used the lift function, and the next time the car approaches that same GPS point, the MBUX suggests, “by its own ‘decision’”, raising the EQS. The driver can then accept or reject the prompt.

We are used to interface systems focusing almost entirely on the driver, but as autonomous vehicles (AVs) move closer to becoming real, it makes sense that passengers will be given more consideration too. Do you agree?

SK: Completely - and that is why we offer something for every passenger, especially the front passenger, who has their own display within the Hyperscreen. That screen does not interfere with driving-related functions, but the person in the front passenger seat can help the driver by searching for a destination, adjusting the sound or selecting music, and they can also view general vehicle information.

That passenger can share information with anyone else in the car and, in some countries (depending on local legislation), they can even watch videos while travelling, because we use a technology (based on an intelligent camera) that prevents the driver from seeing those images - reducing the risk of distraction from the road and traffic.

But there is no need to feel sorry for the driver: they also benefit from the same impressive MBUX interaction we recently introduced on the new S-Class, including a large Augmented Reality (AR) projection.

Interaction

Some manufacturers seem to be working towards a solution, pioneered by Panasonic, where large buttons float on the screen and let users make quick, instinctive adjustments. Mercedes-Benz has opted for an approach where every function is controlled directly via the screen surface itself. Why?

SK: There are many technologies and different conceptual approaches for how users interact with their cars in this digital era - touch, gesture control, voice commands, vision and so on. Our solution, the Hyperscreen, enables full control by combining different technologies and AI, to the point that the user barely notices when they are using the senses nature gave us (speech, hearing, sight and touch), because everything is fully integrated.

In 2012 your company showed, at CES in Las Vegas, the CUBE gesture-operation concept, where the user raised their arms to ‘touch’ virtual controls. The direction taken now is technically and conceptually different. What prompted that shift in thinking?

SK: The idea of using the windscreen as a projection surface has always been on our minds, but right now the head-up display in the new S-Class is as large as it is possible to make. The main drawback is that only the driver can see it, and ergonomically it may not be wise to have drivers waving their arms and hands up and down, and side to side, while driving.

It could become an option in the medium to long term, but at present the technology does not allow it. In any case, we have managed to incorporate many of those ideas and types of content into our large central display.

How do you ensure the driver is not overly distracted by such a large touch-sensitive surface?

SK: Preventing driver distraction has been an obsession for us from day one. That is why information brings itself forward or fades into the background automatically, based on what the system learns from the user’s behaviour over time. In addition, voice and gesture inputs add to a control experience that is simple, intuitive and effortless.

One example: advanced pixel technology increases the brightness of the information that has just been requested and then, with the help of a camera, it can darken the front passenger’s screen from the driver’s perspective, so that when the driver looks towards that screen they cannot see the image (while the front passenger still can). To give a sense of how far we went to reduce distractions, more than 90% of information is available on the first level of the Hyperscreen and/or via voice command.

How to stand out from (future) competition?

A Samsung tablet and an iPad look far more alike than the dashboards of a Mercedes-Benz and a BMW. In other words, how do you inject brand DNA into a dashboard that is “just” a huge glass surface - one that will probably look much like the glass surfaces your rivals will present in the future?

SK: It is true that when the dashboard becomes entirely screen-based, some of the differentiation we always achieved through hardware - shapes, detailing and so on - is reduced. Here, we kept the jet-turbine-style air vents at the outer ends of the dashboard, but because the enormous display dominates, we have to express that DNA more through how the system is used and less through its physical form. The same is also achieved through specific features, colours and functionality.

Only a few months ago you presented the new MBUX in the new S-Class. Now this even more revolutionary Hyperscreen arrives. Do you worry it could irritate early S-Class buyers who may feel they now have only the second-best system on the market…

SK: The Hyperscreen is closely tied to the EQ brand - more progressive and fully electric. The S-Class display is outstanding, but it is aimed at a slightly more traditional user, with its balanced blend of analogue and digital technology.

The Hyperscreen is not better than the S-Class MBUX; it is simply different, because both concepts share the same underlying technology platform (we have eye-tracking and 3D technology in both systems, for example). For that reason, I do not believe there is any cause for “jealousy” or disappointment among those who have already chosen the new S-Class.

Is there more than one central computer behind this system? And what size are the screens?

SK: We managed to package the full computing power into a single unit serving all three displays, even though we use both LCD and OLED screens with different resolutions. The driver and front passenger side displays are 12.3 inches, and the centre screen measures 17.5 inches diagonally.

What type of glass is used?

SK: It is a very sophisticated, slightly curved glass, produced by bonding together different types of display. In areas with tighter curvature we use dry bonding, and in the less curved areas we use wet bonding. We also had to deal with different PPI (pixels per inch), so you can imagine how complex the industrialisation process for this dashboard is…

Is the plan to bring the Hyperscreen concept to smaller models in the EQ sub-brand after it debuts on the EQS at the end of this year?

SK: We are working to offer similar technology solutions for upcoming EQ models and, as always, that will take time - particularly in significantly lower market segments. But we believe touch control in different forms is the future, alongside AI and voice control.