HLS is broken for B-frames for many encoders (SOLUTION ATTACHED)

[adamroach]

Which version are you using?

801edf7

Which operating system are you using?

macOS Arm64 standard

Describe how to replicate the issue

Steps to Reproduce

This is easily reproducible on macOS Arm64, and the exact circumstances may rely on the hardware encoder present on an M3 Max processor, but the root cause is general, and the issue should reproduce in many other circumstances as well.

1. start the server
2. publish using RTMP with OBS on a Mac, using Apple VT HEVC Hardware Encoder, with Use B-Frames checked
3. read with any HLS client

The viewing stream will abort with an error stating the DTS is not monotonic.

Root Cause

For this specific encoder

In this specific case, the issue is that the HLS library infers DTS from the received PTS and the contents of the H.265 NALs:

https://github.com/bluenviron/gohlslib/blob/2ca9f8774e88c4d4c9eec560dfa71b41eeead5eb/muxer_segmenter.go#L278

	dts, err := track.h265DTSExtractor.Extract(au, pts)

This Extract() code in turn relies on the presence of the H.265 parameters vui_num_units_in_tick and vui_num_units_in_tick, which are optional:

https://github.com/bluenviron/mediacommon/blob/42c77ce0d0d1cf764f74b790d02be8914702f10f/pkg/codecs/h265/dts_extractor.go#L205-L207

	if d.spsp.VUI == nil || d.spsp.VUI.TimingInfo == nil {
		return pts, nil
	}

For this particular encoder, the TimingInfo field for the VUI is nil (or, in spec language: vui_timing_info_present_flag is 0):

&{
  AspectRatioInfoPresentFlag:false
  AspectRatioIdc:0
  SarWidth:0
  SarHeight:0
  OverscanInfoPresentFlag:false
  OverscanAppropriateFlag:false
  VideoSignalTypePresentFlag:true
  VideoFormat:5
  VideoFullRangeFlag:false
  ColourDescriptionPresentFlag:true
  ColourPrimaries:1
  TransferCharacteristics:1
  MatrixCoefficients:1
  ChromaLocInfoPresentFlag:false
  ChromaSampleLocTypeTopField:0
  ChromaSampleLocTypeBottomField:0
  NeutralChromaIndicationFlag:false
  FieldSeqFlag:false
  FrameFieldInfoPresentFlag:false
  DefaultDisplayWindow:<nil>
  TimingInfo:<nil>
}

https://github.com/bluenviron/mediacommon/blob/42c77ce0d0d1cf764f74b790d02be8914702f10f/pkg/codecs/h265/dts_extractor.go#L226-L228

	timeDiff := int64(samplesDiff) * 90000 *
		int64(d.spsp.VUI.TimingInfo.NumUnitsInTick) / int64(d.spsp.VUI.TimingInfo.TimeScale)
	dts := pts - timeDiff

In the general case

In the more general case, the problem is that mediamtx only conveys PTS through the system, discarding the DTS information received via (e.g.) RTMP:

mediamtx/internal/protocols/rtmp/reader.go

Line 554 in da1a00f

cb(msg.DTS+msg.PTSDelta, tu)

mediamtx/internal/unit/base.go

Lines 9 to 14 in da1a00f

	// Base contains fields shared across all units.
	type Base struct {
	RTPPackets []*rtp.Packet
	NTP time.Time
	PTS int64
	}

The handling of H.264 and H.265 in the HLS library does some clever calculation to try to recover DTS in the presence of B-frames, but they rely on assumptions that don't always hold (VUI missing timing information is one, but there are others in the DTS extraction code).

Solution

Fundamentally, the issue arises from the system not tracking DTS from input to output. Fixing this in a way that doesn't fail when the DTS extraction code's assumptions don't hold requires small changes to many parts of the code -- not just in mediamtx, but in mediacommon and gohlslib as well. Here is a complete solution:

0001-mediacommon.patch
0002-gohlslib.patch
0003-mediamtx.patch

These API changes will require a /v3 release for incorporation into the library projects, as they require breaking changes to several signatures.

I'm not submitting these as PRs yet because the solution spans multiple repositories and has project-level implications (due to the version change).

Related Issues

Although I haven't diagnosed it, there is a very high probability that #4768 is related to this DTS discarding and recalculation behavior.

Server logs

No response

Network dump

No response

[aler9]

Hello, propagating DTS is something we want to avoid since it's not supported by many protocols and therefore it wouldn't solve much.

Instead, please post a sample video encoded with the Apple VT HEVC Hardware Encoder and B-frames, since it can be used to fix the DTS extractor.

[adamroach]

Here are two examples -- one for the H.264 issue and another for the H.265 issue:

• H.264
• H.265

[MatisOne]

Thank you @adamroach
My issue was with NVENC, but looking at the code, it's for every encoder.
I'll take a look at your patches, but the best would be to fix it for everyone (all encoders). ;)