Skip to content

Conversation

tikuma-lsuhsc
Copy link
Contributor

This is my attempt no 2 at fixing Issue #1987, which is caused by FFmpeg's av_frame_seek or avcodec_flush_buffers function to discard the pal8 palette from the codec context. (I think this is very much intentional behavior.) To mitigate this behavior, we need to keep track of the palette side data, which is attached to a packet of a pal8 video stream (most certainly on the first packet but additional ones are also possible anywhere else in the video). Since pal8 pixel format is seldom used in modern video encoding, PyAV to address this issue natively is a disservice to 99%+ of the user base (I think). Instead, this PR adds ability to extract (copy) a side data from one packet and apply it to another packet.

I have a rudimentary test class added to tests/test_packet.py but in essence we have

with av.open(avifile) as container:

    iterpackets = container.demux()
    pkt = next(iterpackets)  # has palette

    assert pkt.has_side_data("palette") # check if packet has palette side data

    sdata = pkt.get_side_data("palette") # copy the palette side data for later use

    nxt = next(iterpackets)  # next packet has no palette
    sdata1 = nxt.get_side_data("palette") # returns a sidedata object with empty data 
    assert not sdata1 # can check if object holds data or not via __bool__()

    nxt.set_side_data(sdata, move=True) # apply the copied palette to this packet by moving the data array to the packet
    assert not sdata # data moved, no longer holds information

Other notables:

  • PacketSideDataTypeLiteral to specify the side data type (rather than Enum as done for the frame side data). This is my personal preference but if the Enum approach is preferable, I can modify the PR to conform to the standard.
  • Added a test video to tests/assets/pyav_curated/ffmpeg folder as the video was created via FFmpeg and its testsrc input filter.
  • "Light" docstrings have been added to packet.py. If they needs to reside in the stub file and must be done so manually, I'd be happy to accommodate.

@tikuma-lsuhsc
Copy link
Contributor Author

@WyattBlue - I failed to realize that the test video file in tests/assets folder would be ignored by git. Do you have any recommendation on how to add the video to the repo?

@WyattBlue
Copy link
Member

Ideally, you get the video from fate with fate("your-path-here") in the tests

@tikuma-lsuhsc tikuma-lsuhsc force-pushed the add_packet_side_data_handler branch from d3e4d14 to a965bda Compare September 2, 2025 00:53
@tikuma-lsuhsc
Copy link
Contributor Author

OK, I found a mov file with a side data packet in FATE repo, and modified the tests.

Also, I added Packet.iter_side_data method although I'm not sure of its utility. (I primarily did so to scan packets for a side data for the sake of test video lol)

@WyattBlue WyattBlue merged commit 8105052 into PyAV-Org:main Sep 2, 2025
5 checks passed
@tikuma-lsuhsc
Copy link
Contributor Author

@WyattBlue - Thanks for the quick merge! Also, this was my first stub at Cython and learned a good bit from this PR. Pretty neat how it works (though extremely frustrating at times).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants