How to free buffer required by av_frame_new_side_data_from_buf()?

[FrostKiwi]

Hey there :]
I am injecting an H.264 Stream with a custom SEI message to insert per-frame custom data. Work great. However, I am causing a per-frame memory leak, which I am not quite sure how to solve.

I modified H264VideoStreamEncoder.cs
The relevant passage is:

fixed (byte* pMessageData = message)
{
    AVBufferRef* MetaDataBuffer = ffmpeg.av_buffer_alloc((ulong)message.Length);
    MetaDataBuffer->data = pMessageData;
    AVFrameSideData* sideData = ffmpeg.av_frame_new_side_data_from_buf(&frame, AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, MetaDataBuffer);
}

I assume av_buffer_alloc() is the culprit, so I tried to av_buffer_unref() it, but it either causes the encoder to exit or doesn't do anything at all, depending on where I insert it. Tried to make that pointer global and then unref it, tried to av_frame_remove_side_data() in different orders. I'm having a hard time understanding this jump to the .dll and what happens there, especially since the VisualStudio 2022 profiler keeps telling me, that the heap is small and just fine.

Full modified H264VideoStreamEncoder.cs for reference

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using System.Text;
using FFmpeg.AutoGen.Abstractions;

public sealed unsafe class H264VideoStreamEncoder : IDisposable
{
    private readonly Size _frameSize;
    private readonly int _width;
    private readonly int _height;
    private readonly int _linesizeY;
    private readonly AVCodec* _pCodec;
    private readonly AVCodecContext* _pCodecContext;
    private readonly Stream _stream;
    private readonly int _uSize;
    private readonly int _ySize;

    public H264VideoStreamEncoder(Stream stream, int fps, int width, int height)
    {
        _stream = stream;
        _width = width;
        _height = height;

        var codecId = AVCodecID.AV_CODEC_ID_H264;
        _pCodec = ffmpeg.avcodec_find_encoder(codecId);
        if (_pCodec == null) throw new InvalidOperationException("Codec not found.");

        _pCodecContext = ffmpeg.avcodec_alloc_context3(_pCodec);
        _pCodecContext->width = width;
        _pCodecContext->height = height;
        _pCodecContext->time_base = new AVRational { num = 1, den = fps };
        _pCodecContext->pix_fmt = AVPixelFormat.AV_PIX_FMT_GRAY8;
        _pCodecContext->max_b_frames = 0;
        ffmpeg.av_opt_set(_pCodecContext->priv_data, "udu_sei", "1", 0);
        ffmpeg.av_opt_set(_pCodecContext->priv_data, "preset", "veryfast", 0);
        ffmpeg.av_opt_set(_pCodecContext->priv_data, "crf", "25", 0);

        ffmpeg.avcodec_open2(_pCodecContext, _pCodec, null).ThrowExceptionIfError();


        _linesizeY = width;
    }

    public void Dispose()
    {
        ffmpeg.avcodec_close(_pCodecContext);
        ffmpeg.av_free(_pCodecContext);
    }

    public void Encode(AVFrame frame, MachineData machineData)
    {
        if (frame.format != (int)_pCodecContext->pix_fmt)
            throw new ArgumentException("Invalid pixel format.", nameof(frame));
        
        // Some Sanity checks
        if (frame.width != _width) throw new ArgumentException("Invalid width.", nameof(frame));
        if (frame.height != _height) throw new ArgumentException("Invalid height.", nameof(frame));
        if (frame.linesize[0] < _linesizeY) throw new ArgumentException("Invalid Y linesize.", nameof(frame));

        // The required `uuid_iso_iec_11578` as required by the H.264 spec, to
        // be recognized as a `User data unregistered` SEI message.
        string UUID = "139FB1A9446A4DEC8CBF65B1E12D2CFD";

        string custom_datapacket = string.Format(UUID +
            "{{" +
            "\"timestamp\":\" 
            ...
            <redacted>
            ...
            }}",
            <redacted>);
        byte[] message = Encoding.ASCII.GetBytes(custom_datapacket);

        fixed (byte* pMessageData = message)
        {
            AVBufferRef* MetaDataBuffer = ffmpeg.av_buffer_alloc((ulong)message.Length);
            MetaDataBuffer->data = pMessageData;
            AVFrameSideData* sideData = ffmpeg.av_frame_new_side_data_from_buf(&frame, AVFrameSideDataType.AV_FRAME_DATA_SEI_UNREGISTERED, MetaDataBuffer);
        }
        var pPacket = ffmpeg.av_packet_alloc();
        try
        {
            // Basic encoding loop explained: 
            // https://ffmpeg.org/doxygen/4.1/group__lavc__encdec.html

            // Give the encoder a frame to encode
            ffmpeg.avcodec_send_frame(_pCodecContext, &frame).ThrowExceptionIfError();

            // From https://ffmpeg.org/doxygen/4.1/group__lavc__encdec.html:
            // For encoding, call avcodec_receive_packet().  On success, it will return an AVPacket with a compressed frame.
            // Repeat this call until it returns AVERROR(EAGAIN) or an error.
            // The AVERROR(EAGAIN) return value means that new input data is required to return new output.
            // In this case, continue with sending input.
            // For each input frame/packet, the codec will typically return 1 output frame/packet, but it can also be 0 or more than 1.
            bool hasFinishedWithThisFrame;

            do
            {
                // Clear/wipe the receiving packet
                // (not sure if this is needed, since docs for avcoded_receive_packet say that it will call that first-thing
                ffmpeg.av_packet_unref(pPacket);

                // Receive back a packet; there might be 0, 1 or many packets to receive for an input frame.
                var response = ffmpeg.avcodec_receive_packet(_pCodecContext, pPacket);

                bool isPacketValid;

                if (response == 0)
                {
                    // 0 on success; as in, successfully retrieved a packet, and expecting us to retrieve another one.
                    isPacketValid = true;
                    hasFinishedWithThisFrame = false;
                }
                else if (response == ffmpeg.AVERROR(ffmpeg.EAGAIN))
                {
                    // EAGAIN: there's no more output is available in the current state - user must try to send more input
                    isPacketValid = false;
                    hasFinishedWithThisFrame = true;
                }
                else if (response == ffmpeg.AVERROR(ffmpeg.AVERROR_EOF))
                {
                    // EOF: the encoder has been fully flushed, and there will be no more output packets
                    isPacketValid = false;
                    hasFinishedWithThisFrame = true;
                }
                else
                {
                    // AVERROR(EINVAL): codec not opened, or it is a decoder other errors: legitimate encoding errors
                    // , otherwise negative error code:
                    throw new InvalidOperationException($"error from avcodec_receive_packet: {response}");
                }

                if (isPacketValid)
                {
                    var packetStream = new UnmanagedMemoryStream(pPacket->data, pPacket->size);
                    packetStream.CopyTo(_stream);
                }
            } while (!hasFinishedWithThisFrame);
        }
        finally
        {
            ffmpeg.av_packet_free(&pPacket);
        }
    }

    public void Drain()
    {
        // From https://ffmpeg.org/doxygen/4.1/group__lavc__encdec.html:
        // End of stream situations. These require "flushing" (aka draining) the codec, as the codec might buffer multiple frames or packets internally for performance or out of necessity (consider B-frames). This is handled as follows:
        // Instead of valid input, send NULL to the avcodec_send_packet() (decoding) or avcodec_send_frame() (encoding) functions. This will enter draining mode.
        // 	Call avcodec_receive_frame() (decoding) or avcodec_receive_packet() (encoding) in a loop until AVERROR_EOF is returned. The functions will not return AVERROR(EAGAIN), unless you forgot to enter draining mode.

        var pPacket = ffmpeg.av_packet_alloc();

        try
        {
            // Send a null frame to enter draining mode
            ffmpeg.avcodec_send_frame(_pCodecContext, null).ThrowExceptionIfError();

            bool hasFinishedDraining;

            do
            {
                // Clear/wipe the receiving packet
                // (not sure if this is needed, since docs for avcoded_receive_packet say that it will call that first-thing
                ffmpeg.av_packet_unref(pPacket);

                var response = ffmpeg.avcodec_receive_packet(_pCodecContext, pPacket);

                bool isPacketValid;

                if (response == 0)
                {
                    // 0 on success; as in, successfully retrieved a packet, and expecting us to retrieve another one.
                    isPacketValid = true;
                    hasFinishedDraining = false;
                }
                else if (response == ffmpeg.AVERROR(ffmpeg.AVERROR_EOF))
                {
                    // EOF: the encoder has been fully flushed, and there will be no more output packets
                    isPacketValid = false;
                    hasFinishedDraining = true;
                }
                else
                {
                    // Some other error.
                    // Should probably throw here, but in testing we get error -541478725
                    isPacketValid = false;
                    hasFinishedDraining = true;
                }

                if (isPacketValid)
                {
                    var packetStream = new UnmanagedMemoryStream(pPacket->data, pPacket->size);
                    packetStream.CopyTo(_stream);
                }
            } while (!hasFinishedDraining);
        }
        finally
        {
            ffmpeg.av_packet_free(&pPacket);
        }
    }
}

How do I properly add that SEI message and free the associated buffer?

[FrostKiwi]

I insert a at ffmpeg.av_free(&MetaDataBuffer); at this line after ffmpeg.av_packet_free(&pPacket); and it has no effect.

So I am a bit confused as to how to do memory management from this opaque call to a DLL.

[Ruslan-B]

It is managed memory monitoring :)
https://stackoverflow.com/questions/1345377/unmanaged-memory-and-managed-memory#:~:text=The%20Microsoft%20definition%20is%20that,program%20or%20the%20operating%20system.

[FrostKiwi]

It is managed memory monitoring :) https://stackoverflow.com/questions/1345377/unmanaged-memory-and-managed-memory#:~:text=The%20Microsoft%20definition%20is%20that,program%20or%20the%20operating%20system.

Yeah, I understand that part, that this is the reason the memory monitoring does not pick up on it. However, I still don't understand how I am supposed to free() that AvBuffer from C# code via FFmpeg.AutoGen. Do you have an idea?

[Ruslan-B]

I'll suggest to look outside in C or C++ usages - even ask this question on stackoveflow. I do generation of bindings and keep eye on most used use cases - that it. Eventually all boils down to C API usage - C# is is just second class here but with all powers of C.

[FrostKiwi]

I'll suggest to look outside in C or C++ usages - even ask this question on stackoveflow. I do generation of bindings and keep eye on most used use cases - that it. Eventually all boils down to C API usage - C# is is just second class here but with all powers of C.

Thx. I suspected as such. I'm just a bit confused as to when that free is supposed. After then frame is sent? After the packet is freed? I read through the docs and how I think its supposed to work doesn't line up with whats happening. Created a Stack overflow question here:
https://stackoverflow.com/questions/75576858/how-to-free-an-avbuffer-and-prevent-a-memory-leak-when-created-with-ffmpeg-auto
If I figure it out, I'll post it here and close the issue.