Roadlens
AR Based Passenger Window
Overview
Problem
Modern vehicles focus heavily on driver-centric systems, leaving passenger needs overlooked.
Challenges include:
Lack of interaction with external environments.
Repetitive and passive content.
Over-reliance on personal devices.
My Process
Research
BrainStorming
Mind Mapping
Concept Development
Wireframing
Prototyping
Testing and Iterations
Final Implementation
Future Scope
Introduction
Modern car travel experiences have predominantly focused on driver needs, leaving passengers with limited engaging options. This project aims to complete this gap by changing passive back-seat journeys into interactive, educational, and entertaining adventures through advanced technologies.
Cars are no longer just a means of transportation, they have evolved into high-tech interactive spaces designed to offer more than safety and efficiency.
Research
Background
Research shows that 56% of all car rides globally include at least one passenger, with an average of 1.5 people per ride. Passengers may include families, friends, colleagues, or even chauffeur-driven clients, each with diverse expectations and requirements.
Passengers often rely on personal devices, leading to isolation and screen fatigue.
Users value immersive interactions with their environment over passive entertainment.
Insight
Despite these insights, the concept of a "good passenger experience" remains under-explored.
56%
So our biggest question
What factors contribute to a convenient and engaging passenger experience in cars?
How can advanced technologies facilitate people's interaction with the external environment?
So what do passengers look for ?
Scenery and Landmarks
Natural landscapes such as mountains, forests, rivers, or open fields.
Cityscapes, historical landmarks, or culturally significant structures.
Unique roadside attractions like sculptures, quirky signs, or art installations.
RoadSide Activity
Local markets, street performances, or daily life in villages and towns.
Wildlife sightings in rural or natural areas.
Travel related information
Route Markers, Highway signs, and distance boards
Information about nearby facilities like restaurants, gas stations or tourist spots
So what are passengers preference for entertainment ?
Casual observers
Passengers who enjoy the natural and cultural scenery around them.
Prefer minimal distractions and value peaceful observation.
Engagement Seekers
Passengers who look for active entertainment like games, music, or movies.
Prefer multitasking options combining observation with virtual activities.
Interactive explorers
Those who actively seek information about what they see outside, such as landmark details or historical insights.
Our current entertainment modes
Traditional Options
Listening to songs, podcasts, or local radio stations.
Watching pre-downloaded or streaming content on personal devices.
Offline engagement
Travel conversations, group games like “I spy,” or observing passing vehicles and signs.
Emerging technology
Integrated screens offering on-demand entertainment and games.
Portable VR systems for fully immersive experiences (still niche).
Interactive options
Basic in-car screens offering maps, music, or limited apps.
Challenges
Limited Interaction with the Outside Environment
Current systems are primarily focused on content like movies, music, or games, which are disconnected from the real world outside the car.
Passengers miss the opportunity to engage with their environment or learn about the landmarks they pass by.
Repetitive and Passive Content
Most in-car entertainment is passive, like watching movies or listening to music, which can become repetitive on long journeys.
The lack of interactive engagement leaves passengers bored or disinterested, particularly on long trips.
Over-Reliance on Personal Devices
In many cases, passengers rely on their own smartphones or tablets for entertainment, which can lead to screen fatigue.
The use of personal devices also isolates passengers, diminishing the shared experience of the journey
Safety concerns
Overly immersive or distracting entertainment systems may pose safety risks. Passengers could become overly engaged with the content, affecting their ability to assist the driver or maintain awareness of surroundings.
Eyes-off-road engagement can lead to heightened distractions, especially if AR content is too immersive or requires too much interaction.
Lack of Personalisation and Educational Value
Entertainment is generally uniform and doesn’t reflect personal preferences or educational needs.
The opportunity for passengers to learn about their surroundings, explore new places virtually, or gain context for landmarks is often missing.
Inflexible Systems
Current infotainment systems tend to offer limited personalisation or customisation for passengers.
Content is not always adapted to the journey’s context (e.g., location-based information, real-time engagement with the scenery).
They are often designed for short-term use, not long trips, and can’t accommodate diverse passenger needs (e.g., kids vs. adults).
Literature Study
Passenger Comfort and Convenience
Passenger experience is a critical factor in travel, encompassing emotions, preferences, perceptions, and physical comfort, as defined by the ISO standard 9241-210.
Studies emphasise the growing demand for systems that enhance physical comfort, well-being, and provide access to information beyond traditional infotainment.
Research on rear-seat infotainment highlights passengers’ need for
Entertainment Content: Movies, music, and interactive media.
Trip-Related Information: Access to routes, locations, and real-time updates.
Contextual Information: Insights into the surrounding environment, such as landscape views and local landmarks.
Passenger interaction
Studies highlight the value of enabling passengers to interact not only with the driver but also with their external environment, enhancing their overall travel experience.
Personalised interaction creates a sense of involvement, turning passive travel into a more engaging and dynamic journey.
Current concepts in the market
Toyota’s “Window to the World”
A concept that uses AR to overlay interactive content onto car windows, allowing passengers to explore and learn about their surroundings.
General Motors’ “Window of Opportunity”:
Focused on integrating real-time interactions and entertainment into the passenger experience.
Features and Purpose
These systems provide contextual information, such as taking photos or working or connectivity.
They offer real-time interactions, enabling passengers to engage with virtual content projected onto the external environment.
Designed to combine functional utility with playful engagement, these AR-enabled windows transform passive travel into an interactive journey.
Gesture Control
Technologies such as Time-of-Flight (ToF) cameras and infrared sensors enable passengers to interact with virtual objects through simple hand movements, eliminating the need for physical touch.
Examples of interactions include swiping to navigate, tapping to select, or zooming into objects in the augmented environment.
Voice Commands
A hands-free interface allows passengers to seamlessly control the AR system using natural voice commands.
Passengers can switch between virtual environments, activate specific features, or initiate storytelling modes with ease.
This intuitive method of interaction enhances accessibility and convenience, making the system user-friendly for passengers of all ages.
The role of these tech
Augmented Reality (AR) and personalised infotainment systems have the potential to redefine back-seat experiences by offering immersive, interactive features.
These innovations can transform ordinary trips into memorable and meaningful journeys, catering to both entertainment and educational needs.
THE DESIGN PHASE
Brainstorming
We started by brainstorming ideas for improving the driver’s experience but shifted our attention to back-seat passengers due to regulatory limits.Noting a lack of engaging entertainment, we addressed challenges and utilized technology to craft a unique experience.After analyzing market offerings and refining ideas, we landed on a concept that met our vision.
Define
Target Audience
Families
Special focus on mothers
Our target age for children is between 6-18. At this stage in life your brain starts to develop and curious.
Age 6-12
Children in this age group are curious, imaginative, and love exploring through storytelling, play, and interactive experiences.
Age 12-18
Teens value freedom, enjoy entertainment, and gravitate towards social and tech-rich environments,
During trips, mothers often choose to sit in the backseat with their children to attend to their needs.
Realisations
The concept began with the realisation that family road trips often lack engaging options for passengers aged 6–18.
Younger children become restless, teens lose interest, and families miss out on quality bonding moments.
Questions we asked
How can we make travel engaging for children aged 6–18?
How can technology enhance real-world experiences?
How do we ensure all family members, regardless of age, can interact and bond?
Conceptualisation
Environments
Key scenarios identified include historical reconstructions, fantasy worlds, and educational overlays, which are detailed in the Proposed Solution section.
Interactivity
Gesture-based interactions, real-time object manipulation, and voice-triggered commands were prioritized for an engaging and user-friendly experience.
Alignment with reality
Accurate integration of virtual elements with real-world objects was deemed essential, achieved through the use of camera feeds and GPS data.
Prototyping
Design tools
We are using Figma, a powerful design tool, to create high-fidelity wireframes and interactive prototypes.
Prototypes
These prototypes form the foundation for visualizing and refining the user experience before implementation
CONCEPT
Our Proposed
The proposed Concept introduces a system designed using the advanced technologies to enhance the back-seat passenger experience.
Vision
This project is a design-based approach to conceptualise, develop, and assess a system for back-seat passengers in automobiles.
The goal is to transform passive travel into immersive journeys by overlaying augmented content onto real-world views, creating an engaging and interactive experience.
Introduction
Cameras and sensors capture the surrounding landscape, while a projector-based or transparent OLED display seamlessly aligns virtual content with real-world objects.
The AR system acts as a virtual storyteller, allowing passengers to uncover the cultural and historical significance of their surroundings.
By merging technology with storytelling, the system transforms ordinary journeys into engaging adventures, redefining how passengers interact with their environment.
Features
Providing real-time information about geographical features, landmarks, and wildlife encountered during the journey.
The system uses real-time footage captured by external cameras and GPS data to synchronise AR content with the external environment. This ensures that AR overlays align seamlessly with real-world landmarks and terrain.
Passengers can interact with AR elements using hand gestures, such as swiping to navigate, tapping to select objects, or zooming into virtual reconstructions.
A voice-activated interface enables passengers to switch themes, activate storytelling modes, or trigger specific AR interactions without needing to touch the display.
the Concept
ROADLENS
Prototype
Testing
Participants
Age 6-12 = 3
Age 12-18 = 4
Environment
Conducted in a simulated car environment mimicking real-world conditions.
Metrics
Immersion: How engaging and realistic the AR content feels.
Ease of Use: Intuitiveness of gesture and voice controls.
Accuracy: Alignment of AR overlays with real-world elements.
Results
Immersion
Ease of use
Accuracy
TECHNOLOGY
CURRENT TESTING SETUP
For the current phase of testing, we are utilising a projector-based system to display augmented content onto car windows.
This setup allows us to:
1. Visualise and evaluate the passenger experience in real-world driving conditions.
2. Iterate quickly during testing, refining the system before transitioning to advanced display technologies.
FUTURE VISION
While projectors are ideal for prototyping, transparent OLED displays are our long-term goal.
These displays offer a superior blend of transparency and visibility, allowing passengers to view both real-world and augmented content seamlessly.
TESTING FOCUS
Key objectives during testing include:
1. Ensuring AR projections align accurately with passing scenery using camera and GPS integration.
2. Evaluating how effectively the system delivers an immersive, engaging experience for passengers.
GESTURE CONTROL
Embedded sensors integrated into the vehicle detect passenger hand movements, enabling intuitive interactions like swiping, tapping, and zooming.
This feature ensures passengers can seamlessly explore the AR environment without the need for physical devices.
VOICE COMMAND SYSTEM
The AR system includes a voice-controlled interface for a hands-free experience.
Passengers can use natural language commands to navigate, switch modes, or trigger specific interactions, ensuring accessibility for users of all ages.
CAMERA AND GPS INTEGRATION
Real-time camera feeds and GPS tracking capture the external environment and pinpoint the vehicle’s exact position.
Data is synced with a cloud-based server for low-latency processing, ensuring that AR overlays remain aligned with real-world visuals and landmarks.
Advanced Sensor technology
The system includes depth sensors, gesture-detection cameras, and microphones to track inputs and environmental conditions accurately.
These sensors ensure:
- Precise recognition of gestures and voice commands.
- Seamless synchronization of virtual content with real-world objects."
FUTURE SCOPE
Transition from projector-based prototypes to transparent OLED and micro-LED displays, ensuring superior visual fidelity and seamless AR integration.
Testing the prototype using AR software in the real world Scenario.
Optimise hardware for lower power consumption and leverage cloud computing to reduce on-device processing, making the system more eco-friendly and efficient.
THANK YOU
If there’s any question please kindly to ask me. Let’s Discuss
waffledesigns12@gmail.com or +91 96991 82986
DATA REFERENCES
1. Melanie Berger, Bastian Pfleging, Regina Bernhaupt. Designing for a Convenient In-Car Passenger Experience: A Repertory Grid Study. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.117-139, 10.1007/978-3-030-85616-8_9. hal-04196871
2. Ashratuz Zavin Asha, Fahim Anzum, Patrick Finn, Ehud Sharlin, and Mario Costa Sousa. 2020. Designing External Automotive Displays: VR Prototypes and Analysis. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’20), September 21–22, 2020, Virtual Event, DC, USA. ACM, New York, NY, USA, 9 pages. https: [//doi.org/10.1145/3409120.3410658](https:doi.org/10.1145/3409120.3410658)
3. Melanie Berger, Aditya Dandekar, Regina Bernhaupt, and Bastian Pfleging. 2021. An AR-Enabled Interactive Car Door to Extend In-Car Infotainment Systems for Rear Seat Passengers. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 6 pages. https: [//doi.org/10.1145/3411763.3451589](https://doi.org/10.1145/3411763.3451589)
4. Bengler, K.: Driver and Driving Experience in Cars. In: Meixner, G., Müller, C. (Eds.) Automotive User Interfaces, pp. 79–94, Springer, Cham (2017).
5. (37)Matsumura, K., Kirk, D.: 2018. On Active Passengering: Supporting In-Car Experiences. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1 (4), 154 (2018).
6. (41)Meschtscherjakov, A., Wilfinger, D., Gridling, N., Neureiter, N., Tscheligi, M.: Capture the Car! Qualitative in-Situ Methods to Grasp the Automotive Context. In: Proceedings of the 3rd International Conference on Automotive User Interfaces and Interactive Vehicular Applications, ACM, Salzburg (2011).
7. BMW Driver Profiles, https://www.bmw.ca/en/topics/experience/connected-drive/a-to-z.html, last accessed 2021/01/26.
8. (12)BTS 2001 national household travel survey, [https://www.bts.gov/archive/publica](https://www.bts.gov/archive/publica-)tions/highlights_of_the_2001_national_household_travel_survey/section_02, last accessed: 2020/09/15.
9. (42) NHTS Homepage, https://nhts.ornl.gov/, last accessed 2020/09/12
10. (30) Inbar, O., Tractinsky, N.: Make a Trip an Experience: Sharing in-Car Information with Passengers. In: Extended Abstracts on Human Factors in Computing Systems, ACM, Van-couver (2011).
11. (46) Pfleging, B., Rang, M., Broy, N.: Investigating User Needs for Non-Driving-Related Activities during Automated Driving. In: Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia, ACM, Rovaniemi (2016)
12. Perterer, N., Sundström, P., Meschtscherjakov, A., Wilfinger, D., Tscheligi, M.: Come Drive with Me: An EthnographicStudy of Driver-Passenger Pairs to Inform Future in-Car Assistance. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work, ACM, San Antonio (2013)
13. (26) Gridling, N., Meschtscherjakov, A., Tscheligi, M.: I Need Help! Exploring Collaboration in the Car. In: Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work Companion, pp. 87-90. ACM, Seattle (2012).
14. (51) Shove, E.: Converging Conventions of Comfort, Cleanliness and Convenience. Journal of Consumer Policy 26 (12), 395–418 (2003).
15. https://www.ericsson.com/en/blog/2020/1/self-driving-car-passenger-experience
16. https://www.instagram.com/zuck/reel/CsWCv1OuEg9/
17. Kohei Matsumura and David S. Kirk. 2017. On Active Passengering: Supporting In-Car Experiences. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 154 (December 2017), 23 pages. https://doi.org/10.1145/3161176
18. [https://www.energy.gov/eere/vehicles/articles/fotw-1333-march-11-2024-2022-average-number-occupants-trip-household#:~:text=According to the 2022 National,household vehicles was 1.5 persons](https://www.energy.gov/eere/vehicles/articles/fotw-1333-march-11-2024-2022-average-number-occupants-trip household#:~:text=According%20to%20the%202022%20National,household%20vehicles%20was%201.5%20persons).
19. Ashratuz Zavin Asha, Fahim Anzum, Patrick Finn, Ehud Sharlin, and Mario Costa Sousa. 2020. Designing External Automotive Displays: VR Prototypes and Analysis. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’20), September 21–22, 2020, Virtual Event, DC, USA. ACM, New York, NY, USA, 9 pages. https:[//doi.org/10.1145/3409120.3410658](https://doi.org/10.1145/3409120.3410658)
20. Melanie Berger, Aditya Dandekar, Regina Bernhaupt, and Bastian Pfleging. 2021. An AR-Enabled Interactive Car Door to Extend In-Car Infotainment Systems for Rear Seat Passengers. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 8–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 6 pages. https:[//doi.org/10.1145/3411763.3451589](https://doi.org/10.1145/3411763.3451589)
21. Melanie Berger, Bastian Pfleging, Regina Bernhaupt. Designing for a Convenient In-Car Passenger Experience: A Repertory Grid Study. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.117-139, 10.1007/978-3-030-85616-8_9. hal-04196871