Hriday Checker

Home

About

Projects

Contact

Roadlens

AR Based Passenger Window

Overview

Problem

Modern vehicles focus heavily on driver-centric systems, leaving passenger needs overlooked.

Challenges include:

Lack of interaction with external environments.
Repetitive and passive content.
Over-reliance on personal devices.

My Process

Research

BrainStorming

Mind Mapping

Concept Development

Wireframing

Prototyping

Testing and Iterations

Final Implementation

Future Scope

Introduction

Modern car travel experiences have predominantly focused on driver needs, leaving passengers with limited engaging options. This project aims to complete this gap by changing passive back-seat journeys into interactive, educational, and entertaining adventures through advanced technologies.

Cars are no longer just a means of transportation, they have evolved into high-tech interactive spaces designed to offer more than safety and efficiency.

Research

Background

Research shows that 56% of all car rides globally include at least one passenger, with an average of 1.5 people per ride. Passengers may include families, friends, colleagues, or even chauffeur-driven clients, each with diverse expectations and requirements.

Passengers often rely on personal devices, leading to isolation and screen fatigue.
Users value immersive interactions with their environment over passive entertainment.

Research shows that 56% of all car rides globally include at least one passenger, with an average of 1.5 people per ride. Passengers may include families, friends, colleagues, or even chauffeur-driven clients, each with diverse expectations and requirements.

Passengers often rely on personal devices, leading to isolation and screen fatigue.
Users value immersive interactions with their environment over passive entertainment.

Insight

Despite these insights, the concept of a "good passenger experience" remains under-explored.

56%

So our biggest question

What factors contribute to a convenient and engaging passenger experience in cars?

How can advanced technologies facilitate people's interaction with the external environment?

So what do passengers look for ?

Scenery and Landmarks

Natural landscapes such as mountains, forests, rivers, or open fields.

Cityscapes, historical landmarks, or culturally significant structures.
Unique roadside attractions like sculptures, quirky signs, or art installations.

Natural landscapes such as mountains, forests, rivers, or open fields.

Cityscapes, historical landmarks, or culturally significant structures.
Unique roadside attractions like sculptures, quirky signs, or art installations.

RoadSide Activity

Local markets, street performances, or daily life in villages and towns.
Wildlife sightings in rural or natural areas.

Local markets, street performances, or daily life in villages and towns.
Wildlife sightings in rural or natural areas.

Travel related information

Route Markers, Highway signs, and distance boards
Information about nearby facilities like restaurants, gas stations or tourist spots

Route Markers, Highway signs, and distance boards
Information about nearby facilities like restaurants, gas stations or tourist spots

So what are passengers preference for entertainment ?

Casual observers

Passengers who enjoy the natural and cultural scenery around them.
Prefer minimal distractions and value peaceful observation.

Passengers who enjoy the natural and cultural scenery around them.
Prefer minimal distractions and value peaceful observation.

Engagement Seekers

Passengers who look for active entertainment like games, music, or movies.
Prefer multitasking options combining observation with virtual activities.

Passengers who look for active entertainment like games, music, or movies.
Prefer multitasking options combining observation with virtual activities.

Interactive explorers

Those who actively seek information about what they see outside, such as landmark details or historical insights.

Our current entertainment modes

Traditional Options

Listening to songs, podcasts, or local radio stations.
Watching pre-downloaded or streaming content on personal devices.

Listening to songs, podcasts, or local radio stations.
Watching pre-downloaded or streaming content on personal devices.

Offline engagement

Travel conversations, group games like “I spy,” or observing passing vehicles and signs.

Emerging technology

Integrated screens offering on-demand entertainment and games.
Portable VR systems for fully immersive experiences (still niche).

Integrated screens offering on-demand entertainment and games.
Portable VR systems for fully immersive experiences (still niche).

Interactive options

Basic in-car screens offering maps, music, or limited apps.

Basic in-car screens offering maps, music, or limited apps.

Challenges

Limited Interaction with the Outside Environment

Current systems are primarily focused on content like movies, music, or games, which are disconnected from the real world outside the car.
Passengers miss the opportunity to engage with their environment or learn about the landmarks they pass by.

Current systems are primarily focused on content like movies, music, or games, which are disconnected from the real world outside the car.
Passengers miss the opportunity to engage with their environment or learn about the landmarks they pass by.

Repetitive and Passive Content

Most in-car entertainment is passive, like watching movies or listening to music, which can become repetitive on long journeys.
The lack of interactive engagement leaves passengers bored or disinterested, particularly on long trips.

Most in-car entertainment is passive, like watching movies or listening to music, which can become repetitive on long journeys.
The lack of interactive engagement leaves passengers bored or disinterested, particularly on long trips.

Over-Reliance on Personal Devices

In many cases, passengers rely on their own smartphones or tablets for entertainment, which can lead to screen fatigue.
The use of personal devices also isolates passengers, diminishing the shared experience of the journey

In many cases, passengers rely on their own smartphones or tablets for entertainment, which can lead to screen fatigue.
The use of personal devices also isolates passengers, diminishing the shared experience of the journey

Safety concerns

Overly immersive or distracting entertainment systems may pose safety risks. Passengers could become overly engaged with the content, affecting their ability to assist the driver or maintain awareness of surroundings.
Eyes-off-road engagement can lead to heightened distractions, especially if AR content is too immersive or requires too much interaction.

Overly immersive or distracting entertainment systems may pose safety risks. Passengers could become overly engaged with the content, affecting their ability to assist the driver or maintain awareness of surroundings.
Eyes-off-road engagement can lead to heightened distractions, especially if AR content is too immersive or requires too much interaction.

Lack of Personalisation and Educational Value

Entertainment is generally uniform and doesn’t reflect personal preferences or educational needs.
The opportunity for passengers to learn about their surroundings, explore new places virtually, or gain context for landmarks is often missing.

Entertainment is generally uniform and doesn’t reflect personal preferences or educational needs.
The opportunity for passengers to learn about their surroundings, explore new places virtually, or gain context for landmarks is often missing.

Inflexible Systems

Current infotainment systems tend to offer limited personalisation or customisation for passengers.
Content is not always adapted to the journey’s context (e.g., location-based information, real-time engagement with the scenery).
They are often designed for short-term use, not long trips, and can’t accommodate diverse passenger needs (e.g., kids vs. adults).

Current infotainment systems tend to offer limited personalisation or customisation for passengers.
Content is not always adapted to the journey’s context (e.g., location-based information, real-time engagement with the scenery).
They are often designed for short-term use, not long trips, and can’t accommodate diverse passenger needs (e.g., kids vs. adults).

Literature Study

Passenger Comfort and Convenience

Passenger experience is a critical factor in travel, encompassing emotions, preferences, perceptions, and physical comfort, as defined by the ISO standard 9241-210.
Studies emphasise the growing demand for systems that enhance physical comfort, well-being, and provide access to information beyond traditional infotainment.

Passenger experience is a critical factor in travel, encompassing emotions, preferences, perceptions, and physical comfort, as defined by the ISO standard 9241-210.
Studies emphasise the growing demand for systems that enhance physical comfort, well-being, and provide access to information beyond traditional infotainment.

Research on rear-seat infotainment highlights passengers’ need for

Entertainment Content: Movies, music, and interactive media.
Trip-Related Information: Access to routes, locations, and real-time updates.
Contextual Information: Insights into the surrounding environment, such as landscape views and local landmarks.

Entertainment Content: Movies, music, and interactive media.
Trip-Related Information: Access to routes, locations, and real-time updates.
Contextual Information: Insights into the surrounding environment, such as landscape views and local landmarks.

Passenger interaction

Studies highlight the value of enabling passengers to interact not only with the driver but also with their external environment, enhancing their overall travel experience.
Personalised interaction creates a sense of involvement, turning passive travel into a more engaging and dynamic journey.

Studies highlight the value of enabling passengers to interact not only with the driver but also with their external environment, enhancing their overall travel experience.
Personalised interaction creates a sense of involvement, turning passive travel into a more engaging and dynamic journey.

Current concepts in the market

Toyota’s “Window to the World”

A concept that uses AR to overlay interactive content onto car windows, allowing passengers to explore and learn about their surroundings.

A concept that uses AR to overlay interactive content onto car windows, allowing passengers to explore and learn about their surroundings.

General Motors’ “Window of Opportunity”:

Focused on integrating real-time interactions and entertainment into the passenger experience.

Focused on integrating real-time interactions and entertainment into the passenger experience.

Features and Purpose

These systems provide contextual information, such as taking photos or working or connectivity.
They offer real-time interactions, enabling passengers to engage with virtual content projected onto the external environment.
Designed to combine functional utility with playful engagement, these AR-enabled windows transform passive travel into an interactive journey.

These systems provide contextual information, such as taking photos or working or connectivity.
They offer real-time interactions, enabling passengers to engage with virtual content projected onto the external environment.
Designed to combine functional utility with playful engagement, these AR-enabled windows transform passive travel into an interactive journey.

Gesture Control

Technologies such as Time-of-Flight (ToF) cameras and infrared sensors enable passengers to interact with virtual objects through simple hand movements, eliminating the need for physical touch.
Examples of interactions include swiping to navigate, tapping to select, or zooming into objects in the augmented environment.

Technologies such as Time-of-Flight (ToF) cameras and infrared sensors enable passengers to interact with virtual objects through simple hand movements, eliminating the need for physical touch.
Examples of interactions include swiping to navigate, tapping to select, or zooming into objects in the augmented environment.

Voice Commands

A hands-free interface allows passengers to seamlessly control the AR system using natural voice commands.
Passengers can switch between virtual environments, activate specific features, or initiate storytelling modes with ease.
This intuitive method of interaction enhances accessibility and convenience, making the system user-friendly for passengers of all ages.

A hands-free interface allows passengers to seamlessly control the AR system using natural voice commands.
Passengers can switch between virtual environments, activate specific features, or initiate storytelling modes with ease.
This intuitive method of interaction enhances accessibility and convenience, making the system user-friendly for passengers of all ages.

The role of these tech

Augmented Reality (AR) and personalised infotainment systems have the potential to redefine back-seat experiences by offering immersive, interactive features.
These innovations can transform ordinary trips into memorable and meaningful journeys, catering to both entertainment and educational needs.

Augmented Reality (AR) and personalised infotainment systems have the potential to redefine back-seat experiences by offering immersive, interactive features.
These innovations can transform ordinary trips into memorable and meaningful journeys, catering to both entertainment and educational needs.

THE DESIGN PHASE

Brainstorming

We started by brainstorming ideas for improving the driver’s experience but shifted our attention to back-seat passengers due to regulatory limits.Noting a lack of engaging entertainment, we addressed challenges and utilized technology to craft a unique experience.After analyzing market offerings and refining ideas, we landed on a concept that met our vision.

Define

Target Audience

Families

Special focus on mothers

Our target age for children is between 6-18. At this stage in life your brain starts to develop and curious.

Age 6-12

Children in this age group are curious, imaginative, and love exploring through storytelling, play, and interactive experiences.

Age 12-18

Teens value freedom, enjoy entertainment, and gravitate towards social and tech-rich environments,

During trips, mothers often choose to sit in the backseat with their children to attend to their needs.

Realisations

The concept began with the realisation that family road trips often lack engaging options for passengers aged 6–18.

Younger children become restless, teens lose interest, and families miss out on quality bonding moments.

Questions we asked

How can we make travel engaging for children aged 6–18?

How can technology enhance real-world experiences?

How do we ensure all family members, regardless of age, can interact and bond?

Conceptualisation

Environments

Key scenarios identified include historical reconstructions, fantasy worlds, and educational overlays, which are detailed in the Proposed Solution section.

Interactivity

Gesture-based interactions, real-time object manipulation, and voice-triggered commands were prioritized for an engaging and user-friendly experience.

Alignment with reality

Accurate integration of virtual elements with real-world objects was deemed essential, achieved through the use of camera feeds and GPS data.

Prototyping

Design tools

We are using Figma, a powerful design tool, to create high-fidelity wireframes and interactive prototypes.

Prototypes

These prototypes form the foundation for visualizing and refining the user experience before implementation

CONCEPT

Our Proposed

The proposed Concept introduces a system designed using the advanced technologies to enhance the back-seat passenger experience.

Vision

This project is a design-based approach to conceptualise, develop, and assess a system for back-seat passengers in automobiles.

The goal is to transform passive travel into immersive journeys by overlaying augmented content onto real-world views, creating an engaging and interactive experience.

Introduction

Cameras and sensors capture the surrounding landscape, while a projector-based or transparent OLED display seamlessly aligns virtual content with real-world objects.

The AR system acts as a virtual storyteller, allowing passengers to uncover the cultural and historical significance of their surroundings.

By merging technology with storytelling, the system transforms ordinary journeys into engaging adventures, redefining how passengers interact with their environment.

Features

Providing real-time information about geographical features, landmarks, and wildlife encountered during the journey.

The system uses real-time footage captured by external cameras and GPS data to synchronise AR content with the external environment. This ensures that AR overlays align seamlessly with real-world landmarks and terrain.

Passengers can interact with AR elements using hand gestures, such as swiping to navigate, tapping to select objects, or zooming into virtual reconstructions.

A voice-activated interface enables passengers to switch themes, activate storytelling modes, or trigger specific AR interactions without needing to touch the display.

the Concept

ROADLENS

Prototype

Testing

Participants

Age 6-12 = 3

Age 12-18 = 4

Age 6-12 = 3

Age 12-18 = 4

Environment

Conducted in a simulated car environment mimicking real-world conditions.

Metrics

Immersion: How engaging and realistic the AR content feels.
Ease of Use: Intuitiveness of gesture and voice controls.
Accuracy: Alignment of AR overlays with real-world elements.

Results

Immersion

Ease of use

Accuracy

TECHNOLOGY

CURRENT TESTING SETUP

For the current phase of testing, we are utilising a projector-based system to display augmented content onto car windows.

This setup allows us to:
1. Visualise and evaluate the passenger experience in real-world driving conditions.
2. Iterate quickly during testing, refining the system before transitioning to advanced display technologies.

FUTURE VISION

While projectors are ideal for prototyping, transparent OLED displays are our long-term goal.

These displays offer a superior blend of transparency and visibility, allowing passengers to view both real-world and augmented content seamlessly.

TESTING FOCUS

Key objectives during testing include:

1. Ensuring AR projections align accurately with passing scenery using camera and GPS integration.
2. Evaluating how effectively the system delivers an immersive, engaging experience for passengers.

GESTURE CONTROL

Embedded sensors integrated into the vehicle detect passenger hand movements, enabling intuitive interactions like swiping, tapping, and zooming.
This feature ensures passengers can seamlessly explore the AR environment without the need for physical devices.

VOICE COMMAND SYSTEM

The AR system includes a voice-controlled interface for a hands-free experience.
Passengers can use natural language commands to navigate, switch modes, or trigger specific interactions, ensuring accessibility for users of all ages.

The AR system includes a voice-controlled interface for a hands-free experience.
Passengers can use natural language commands to navigate, switch modes, or trigger specific interactions, ensuring accessibility for users of all ages.

CAMERA AND GPS INTEGRATION

Real-time camera feeds and GPS tracking capture the external environment and pinpoint the vehicle’s exact position.
Data is synced with a cloud-based server for low-latency processing, ensuring that AR overlays remain aligned with real-world visuals and landmarks.

Real-time camera feeds and GPS tracking capture the external environment and pinpoint the vehicle’s exact position.
Data is synced with a cloud-based server for low-latency processing, ensuring that AR overlays remain aligned with real-world visuals and landmarks.

Advanced Sensor technology

The system includes depth sensors, gesture-detection cameras, and microphones to track inputs and environmental conditions accurately.
These sensors ensure:
- Precise recognition of gestures and voice commands.
- Seamless synchronization of virtual content with real-world objects."

The system includes depth sensors, gesture-detection cameras, and microphones to track inputs and environmental conditions accurately.
These sensors ensure:
- Precise recognition of gestures and voice commands.
- Seamless synchronization of virtual content with real-world objects."

FUTURE SCOPE

Transition from projector-based prototypes to transparent OLED and micro-LED displays, ensuring superior visual fidelity and seamless AR integration.

Testing the prototype using AR software in the real world Scenario.

Optimise hardware for lower power consumption and leverage cloud computing to reduce on-device processing, making the system more eco-friendly and efficient.

THANK YOU

If there’s any question please kindly to ask me. Let’s Discuss

waffledesigns12@gmail.com or +91 96991 82986

If there’s any question please kindly to ask me. Let’s Discuss

waffledesigns12@gmail.com or +91 96991 82986

DATA REFERENCES

1. Melanie Berger, Bastian Pfleging, Regina Bernhaupt. Designing for a Convenient In-Car Passenger Experience: A Repertory Grid Study. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.117-139, 10.1007/978-3-030-85616-8_9. hal-04196871
2. Ashratuz Zavin Asha, Fahim Anzum, Patrick Finn, Ehud Sharlin, and Mario Costa Sousa. 2020. Designing External Automotive Displays: VR Prototypes and Analysis. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’20), September 21–22, 2020, Virtual Event, DC, USA. ACM, New York, NY, USA, 9 pages. https: [//doi.org/10.1145/3409120.3410658](https:doi.org/10.1145/3409120.3410658)
3. Melanie Berger, Aditya Dandekar, Regina Bernhaupt, and Bastian Pfleging. 2021. An AR-Enabled Interactive Car Door to Extend In-Car Infotainment Systems for Rear Seat Passengers. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 6 pages. https: [//doi.org/10.1145/3411763.3451589](https://doi.org/10.1145/3411763.3451589)
4. Bengler, K.: Driver and Driving Experience in Cars. In: Meixner, G., Müller, C. (Eds.) Automotive User Interfaces, pp. 79–94, Springer, Cham (2017).
5. (37)Matsumura, K., Kirk, D.: 2018. On Active Passengering: Supporting In-Car Experiences. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1 (4), 154 (2018).
6. (41)Meschtscherjakov, A., Wilfinger, D., Gridling, N., Neureiter, N., Tscheligi, M.: Capture the Car! Qualitative in-Situ Methods to Grasp the Automotive Context. In: Proceedings of the 3rd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ACM, Salzburg (2011).
7. BMW Driver Profiles, https://www.bmw.ca/en/topics/experience/connected-drive/a-to-z.html, last accessed 2021/01/26.
8. (12)BTS 2001 national household travel survey, [https://www.bts.gov/archive/publica](https://www.bts.gov/archive/publica-)tions/highlights_of_the_2001_national_household_travel_survey/section_02, last accessed: 2020/09/15.
9. (42) NHTS Homepage, https://nhts.ornl.gov/, last accessed 2020/09/12
10. (30) Inbar, O., Tractinsky, N.: Make a Trip an Experience: Sharing in-Car Information with Passengers. In: Extended Abstracts on Human Factors in Computing Systems, ACM, Van-couver (2011).
11. (46) Pfleging, B., Rang, M., Broy, N.: Investigating User Needs for Non-Driving-Related Activities during Automated Driving. In: Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia, ACM, Rovaniemi (2016)
12. Perterer, N., Sundström, P., Meschtscherjakov, A., Wilfinger, D., Tscheligi, M.: Come Drive with Me: An EthnographicStudy of Driver-Passenger Pairs to Inform Future in-Car Assistance. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work, ACM, San Antonio (2013)
13. (26) Gridling, N., Meschtscherjakov, A., Tscheligi, M.: I Need Help! Exploring Collaboration in the Car. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work Companion, pp. 87-90. ACM, Seattle (2012).
14. (51) Shove, E.: Converging Conventions of Comfort, Cleanliness and Convenience. Journal of Consumer Policy 26 (12), 395–418 (2003).
15. https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience
16. https://www.instagram.com/zuck/reel/CsWCv1OuEg9/
17. Kohei Matsumura and David S. Kirk. 2017. On Active Passengering: Supporting In-Car Experiences. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 154 (December 2017), 23 pages. https://doi.org/10.1145/3161176
18. [https://www.energy.gov/eere/vehicles/articles/fotw-1333-march-11-2024-2022-average-number-occupants-trip-household#:~:text=According to the 2022 National,household vehicles was 1.5 persons](https://www.energy.gov/eere/vehicles/articles/fotw-1333-march-11-2024-2022-average-number-occupants-trip household#:~:text=According%20to%20the%202022%20National,household%20vehicles%20was%201.5%20persons).
19. Ashratuz Zavin Asha, Fahim Anzum, Patrick Finn, Ehud Sharlin, and Mario Costa Sousa. 2020. Designing External Automotive Displays: VR Prototypes and Analysis. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’20), September 21–22, 2020, Virtual Event, DC, USA. ACM, New York, NY, USA, 9 pages. https:[//doi.org/10.1145/3409120.3410658](https://doi.org/10.1145/3409120.3410658)
20. Melanie Berger, Aditya Dandekar, Regina Bernhaupt, and Bastian Pfleging. 2021. An AR-Enabled Interactive Car Door to Extend In-Car Infotainment Systems for Rear Seat Passengers. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 6 pages. https:[//doi.org/10.1145/3411763.3451589](https://doi.org/10.1145/3411763.3451589)
21. Melanie Berger, Bastian Pfleging, Regina Bernhaupt. Designing for a Convenient In-Car Passenger Experience: A Repertory Grid Study. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.117-139, 10.1007/978-3-030-85616-8_9. hal-04196871

HRIDAY CHECKER

HRIDAY CHECKER