MH
Open Source Library/Demo

Sync Audio With Lexical

A specialized open-source project showcasing how to map audio timestamps to specific text nodes within the Lexical rich text editor. This allows for 'karaoke-style' reading experiences where text is highlighted in real-time as the audio plays.

Timeline: 2024
Role: Lead Developer

Project Links

My Role & Responsibilities

Role

Lead Developer

Key Responsibilities

  • Developed the logic for timestamp mapping between audio files and Lexical JSON
  • Created a high-performance highlighting system for large text bodies
  • Documented the implementation for other developers to integrate into LMS platforms

Technical Details

Technology Stack

frontend
  • React
  • Next.js
  • Lexical
backend
    database
      devops
      • GitHub Actions
      tools
      • TypeScript

      Architecture

      Client-side mapping engine for real-time DOM manipulation based on audio state

      Screenshots & Media

      Real-time text highlighting synchronized with audio playback

      Real-time text highlighting synchronized with audio playback

      Challenges & Solutions

      Challenge:

      Highlighting specific words within a deeply nested JSON structure without performance lag

      Solution:

      Implemented an index-based lookup system that avoids re-scanning the entire document on every time update

      Achievements & Impact

      Open Source Release

      Provided a foundational tool for developers building accessible reading applications

      Lessons Learned

      • Deep dive into Lexical's internal node structure
      • Efficient state management for high-frequency updates

      Explore more projects?

      Go To Projects