Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • B buck
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 201
    • Issues 201
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 22
    • Merge requests 22
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Meta
  • buck
  • Merge requests
  • !1659

Avoid leaking file descriptors when serving cache artifacts

  • Review changes

  • Download
  • Email patches
  • Plain diff
Closed Administrator requested to merge github/fork/FuegoFro/server_fd_leak into master Dec 02, 2017
  • Overview 5
  • Commits 1
  • Pipelines 0
  • Changes 1

Created by: FuegoFro

Summary:

The current implementation calls ByteSource#openStream, and none of the objects involved there close the stream in their finalize method, so the process keeps the file descriptors for the temp files open forever. This a) prevents the actual content from being removed and b) can cause the process to hit its maximum number of file descriptors.

The change here is to use ByteSource#copyTo(OutputStream), which uses Guava's Closer class to ensure that the stream will be closed, then calls the exact ByteStreams.copy method that was being used before. Source for the newly used method can be found here.

Test Plan:

I ran the Buck daemon's HTTP server locally before the change and saw that cat /proc/sys/fs/file-nr reported a monotonically increasing number of file descriptors held, and that lsof reported the process holding onto a bunch of deleted files like buck-out/bin/outgoing_rulekey7730674455440792418.tmp.

After this change, I ran the server locally and saw that the number of used file descriptors stayed stable over time and verified with lsof that the process wasn't holding onto an tmp files.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: github/fork/FuegoFro/server_fd_leak