音视频的同步
音视频同步可检测到的阈值在 +45ms 到 -125ms 之间, 可接受的阈值平均约为 +90ms 到 -185 ms, 正值表示声音相对于图像超前多少
在 T-REC-G.114-200305 中总结到:
- ~280ms 有些用户不满意
- ~380ms 许多用户不满意
- ~500ms 几乎所有都不满意
延迟来自哪里, 可以看看音视频各自的流程
- 音频处理流程
延迟可能来自如下步骤
- 从设备捕获音频的延迟
- 输入音频处理的延迟 3A
- 音频编码的延迟
- 网络传输的延迟
- 抖动缓冲的延迟
- 音频解码的延迟
- 音频输出处理的延迟
- 从设备输出到耳朵的延迟
- 视频处理流程
WebRTC 中有相关的度量
- estimatedPlayoutTimestamp of type DOMHighResTimeStamp
This is the estimated playout time of this receiver's track. 这就是接收的音轨估计的回放时间
The playout time is the NTP timestamp of the last playable audio sample or video frame that has a known timestamp (from an RTCP SR packet mapping RTP timestamps to NTP timestamps), extrapolated with the time elapsed since it was ready to be played out. This is the "current time" of the track in NTP clock time of the sender and can be present even if there is no audio currently playing.
这个回放时间是最近的可播放的音频采样或视频帧的 NTP timestamp
这个 NTP timestamp 通过 RTCP SR 包的 RTP timestamp 与 NTP timestamp 的映射算出来的,
This can be useful for estimating how much audio and video is out of sync for two tracks from the same source, audioInboundRtpStats.estimatedPlayoutTimestamp - videoInboundRtpStats.estimatedPlayoutTimestamp.