Speech-2-Context: We can create new experiences with what is being said
TL;DR
OR
To do:
- Discover how easily you could browse through all related topics/fact checked for each episode
- Click on the audioplayer to see an interactive timeline with pinpointed contextual highlights that helps the user
The Context Revolution: AI-Powered Information Layers
A new generation of applications is emerging that leverages AI's ability to add rich contextual information to speech, text, and video content. We're already seeing this technology in practice. On X, Grok allows users to instantly clarify or expand on any tweet with the click of a button—a perfect example of text-to-context functionality.
For video-to-context, Apple TV and Prime Video's X-Ray feature provides scene-by-scene breakdowns showing cast information, music tracks, and production details, augmenting the viewing experience. (great for shopping experience as well of course!).
The next major wave will likely focus on speech-based applications:
Podcasts (as demonstrated in our prototype)
Personal voice notes that automatically expand thoughts into detailed contexts (similar to Wispr Flow's approach)
Live AMAs and Q&A sessions
Corporate presentations and meetings
Real-time AR experiences
Real-Time AR Integration
Perhaps most exciting is the potential for real-time implementation through AR glasses. We're already seeing early versions with live translation services from Meta AI, Google Live Translate, and Apple Live Translation. Soon, conversations and thoughts could be instantly transformed into contextual overlays, helping users understand far more about their immediate environment.
Information Surfacing Possibilities
This technology can surface information in two key ways:
Within-format context: Deep dives into single sources (podcast episodes, presentations, etc.)
Cross-temporal context: Connections over time (yearly report comparisons, recurring themes across multiple podcast episodes, etc.)
The Framework for Innovation
Endless applications can be build in a matter of days when done right:
The development pattern is straightforward and scalable:
—> Source (meeting, presentation, podcast, face-2-face discussion, concert, theater, …)
—> Transformer (Recall.io, OpenAI Whisper, Fireflies, … )
—> Contextual UI focussed on what you want to add as context
What would you build?