Memory leak while reading frames from preloaded video buffer

Created by: igor-kondratiev

This leak can easily be reproduced with this script

import io

import av


ITERATIONS = 1000
INPUT_SAMPLE_PATH = "/path/to/any/mp4"


def main():
    with open(INPUT_SAMPLE_PATH, "rb") as f:
        input_data = f.read()

    for i in range(ITERATIONS):
        print(f"Iteration {i + 1} started")

        buffer = io.BytesIO(input_data)
        container = av.open(buffer)

        frame = next(container.decode(video=0))

    input("Press enter to exit...")


if __name__ == '__main__':
    main()

Increasing number of iterations will cause memory usage constant rising. I'm using version 0.3.3 but 0.4.0 is affected as well. Python version: 3.6.5

I've made some investigations and found actual leaking place. The problem is ContainerProxy::__dealloc__ method. In case of python file-like buffer there will be two extra objects allocated - self.iocontext and self.buffer (spoiler: they are both leaking). Next code - self.ptr.pb = self.iocontext will force ffmpeg to set AVFMT_FLAG_CUSTOM_IO (this happens here). Another important thing is that avformat_close_input will not free buffer and iocontext if AVFMT_FLAG_CUSTOM_IO flag is set - avformat_close_input code. As a result, both self.iocontext and self.iocontext.buffer will not be deallocated.

You could also use valgrind with provided script to verify described behavior. Important note is that valgrind will not point to the buffer that is being allocated in ContainerProxy::__init__. libAV can reallocate provided buffer for some reasons, so valgrind will show only the last reallocation call-stack.

I've made a fix for version 0.3.3 (here is link to commit) but there is no release branch for this version and I can't create a PR. Are there any chances to release a patch for version 0.3.3?

The code in ContainerProxy::__dealloc__ was changed in current develop version, but it will not be difficult to adapt this fix for current version. Do you have any plans about dates for the next PyAV release where this fix could be included?

Thank you in advance!