如何在音频讲述时根据网站上的音频实时突出显示文本答案

【问题标题】：how to highlight text as per audio on a website in realtime as the audio narrates it如何在音频讲述时根据网站上的音频实时突出显示文本
【发布时间】：2022-09-25 21:58:03
【问题描述】：

我试图弄清楚根据音频使用哪种技术来突出显示文本。很像https://speechify.com/ 正在做的事情。

这是假设我能够运行 TTS 算法并且能够将文本转换为语音。我尝试了多个来源，但我无法确定在音频说话时突出显示文本的确切技术或方法。

任何帮助将非常感激。我已经在互联网上浪费了 2 天时间来解决这个问题，但没有运气:(

标签： reactjs annotations text-to-speech

【解决方案1】：

一个简单的方法是使用SpeechSynthesisUtterance boundary event 提供的事件侦听器来使用vanilla JS 突出显示单词。发出的事件为我们提供了 char 索引，因此无需为正则表达式或超级 AI 的东西发疯 :)

首先，确保 API 可用

const synth = window.speechSynthesis
if (!synth) {
  console.error('no tts for you!')
  return
}

tts 话语发出一个“边界”事件，我们可以用它来突出显示文本。

let text = document.getElementById('text')
let originalText = text.innerText
let utterance = new SpeechSynthesisUtterance(originalText)
utterance.addEventListener('boundary', event => {
  const { charIndex, charLength } = event
  text.innerHTML = highlight(originalText, charIndex, charIndex + charLength)
})
synth.speak(utterance)

完整示例：

const btn = document.getElementById("btn")

const highlight = (text, from, to) => {
  let replacement = highlightBackground(text.slice(from, to))
  return text.substring(0, from) + replacement + text.substring(to)
}
const highlightBackground = sample => `<span style="background-color:yellow;">${sample}</span>`

btn && btn.addEventListener('click', () => {
  const synth = window.speechSynthesis
  if (!synth) {
    console.error('no tts')
    return
  }
  let text = document.getElementById('text')
  let originalText = text.innerText
  let utterance = new SpeechSynthesisUtterance(originalText)
  utterance.addEventListener('boundary', event => {
    const { charIndex, charLength } = event
    text.innerHTML = highlight(originalText, charIndex, charIndex + charLength)
   })
  synth.speak(utterance)
})

CodeSandbox link

这是非常基本的，您可以（并且应该）改进它。

编辑

糟糕，我忘记了这被标记为 ReactJs。这是与 React 相同的示例（codesandbox 链接在 cmets 中）：

import React from "react";

const ORIGINAL_TEXT =
  "Call me Ishmael. Some years ago—never mind how long precisely—having little or no money in my purse, and nothing particular to interest me on shore, I thought I would sail about a little and see the watery part of the world.";

const splitText = (text, from, to) => [
  text.slice(0, from),
  text.slice(from, to),
  text.slice(to)
];

const HighlightedText = ({ text, from, to }) => {
  const [start, highlight, finish] = splitText(text, from, to);
  return (
    <p>
      {start}
      <span style={{ backgroundColor: "yellow" }}>{highlight}</span>
      {finish}
    </p>
  );
};

export default function App() {
  const [highlightSection, setHighlightSection] = React.useState({
    from: 0,
    to: 0
  });
  const handleClick = () => {
    const synth = window.speechSynthesis;
    if (!synth) {
      console.error("no tts");
      return;
    }

    let utterance = new SpeechSynthesisUtterance(ORIGINAL_TEXT);
    utterance.addEventListener("boundary", (event) => {
      const { charIndex, charLength } = event;
      setHighlightSection({ from: charIndex, to: charIndex + charLength });
    });
    synth.speak(utterance);
  };

  return (
    <div className="App">
      <HighlightedText text={ORIGINAL_TEXT} {...highlightSection} />
      <button onClick={handleClick}>klik me</button>
    </div>
  );
}

【讨论】：

糟糕，对不起，我忘了这被标记为“reactjs”。这是另一个与反应相同的代码框：codesandbox.io/s/tts-with-word-highlight-react-fz15tu?file=/src/…

【解决方案2】：

tts-react 提供了一个钩子 useTts 接受 markTextAsSpoken 参数，该参数将突出显示正在说出的单词。

这是一个例子：

import { useTts } from 'tts-react'

const TTS = ({ children }) => {
  const { ttsChildren, play } = useTts({ children, markTextAsSpoken: true })

  return (
    <div>
      <button onClick={play}>
        Click to hear the text spoken
      </button>
      {ttsChildren}
    </div>

  )
}

const App = () => {
  return <TTS>Some text to be spoken.</TTS>
}

您也可以从 CDN 加载它：

<!DOCTYPE html>
<html lang="en-US">
  <head>
    <title>tts-react UMD example</title>
    <script src="https://unpkg.com/react@18/umd/react.development.js"></script>
    <script src="https://unpkg.com/react-dom@18/umd/react-dom.development.js"></script>
    <script src="https://unpkg.com/@babel/standalone/babel.min.js"></script>
    <script src="https://unpkg.com/tts-react@1.2.0/dist/umd/tts-react.min.js"></script>
  </head>
  <body>
    <div id="root"></div>
    <script type="text/babel">
      const root = ReactDOM.createRoot(document.getElementById('root'))
      const { TextToSpeech, useTts } = TTSReact
      const CustomTTS = ({ children }) => {
        const { play, ttsChildren } = useTts({ children, markTextAsSpoken: true })

        return (
          <>
            <button onClick={() => play()}>Play</button>
            <div>{ttsChildren}</div>
          </>
        )
      }

      root.render(
        <>
          <CustomTTS>
            <p>Highlight words as they are spoken.</p>
          </CustomTTS>
          <TextToSpeech markTextAsSpoken>
            <p>Highlight words as they are spoken.</p>
          </TextToSpeech>
        </>
      )
    </script>
  </body>
</html>

【讨论】：