Regularly updating dependencies for SQuaRE services is both a high-impact security intervention and a benefit for long-term maintenance. It reduces the operational burden of a service by making frequent, controlled, incremental updates rather than large and hard-to-debug flag days. This document explores options for dependency management and proposes a hybrid system using Dependabot and WhiteSource Renovate, supplemented with locally-written software.
2 Problem statement¶
Security reviews of both SQuaRE internal services (SQR-037) and the Science Platform (SQR-041) identified security patching and upgrades of third-party software as one of the highest security risks and most impactful mitigations. Determining whether a given update of third-party software fixes a security issue can be challenging; where possible, keeping all dependencies current is the ideal approach. Regularly updating all dependencies spreads out upgrade work more smoothly, results in fewer flag days, and avoids getting stuck on an outdated, unsupported version when a security vulnerability is discovered. For more discussion of this approach, see Software dependencies and keeping them up-to-date, among other articles.
Continuous dependency updates would be challenging or infeasible for the stack code underlying astronomical data processing for Vera C. Rubin Observatory. Updating that code requires extensive validation and therefore must be carefully planned. Thankfully, however, most of that software is not part of the security attack surface of SQuaRE or Science Platform services, and many services run by SQuaRE do not have the same challenges. We therefore intend to regularly update all third-party software dependencies for all SQuaRE-managed services by default, making exceptions only where this would interfere with reproducible astronomical data processing or cause instability problems.
Anything done regularly should be automated.
That automation should come in the form of GitHub pull requests, which can then trigger normal test suites for individual repositories to catch incompatibilities.
To begin with, all pull requests will be manually reviewed.
In the longer term, some pull requests should be merged automatically if tests pass (updates of test-only dependencies such as
mypy, for example).
2.1 Types of dependencies¶
The following types of external dependencies are present in SQuaRE services:
- Python PyPI packages
- Ruby gems
- Helm charts (both third-party and SQuaRE-internal charts)
- Kustomize external resources
- Docker images referenced in Helm charts and Kustomize resources
- Docker images used as a base in
- Docker images referenced in
- GitHub Actions referenced in GitHub Workflows
- pre-commit plugins for Python packages
2.2 Other desired features¶
- Free for public repositories on GitHub.
- Bundle multiple low-risk updates into a single pull request.
- Support frozen requirements files for Python applications, which requires updating the frozen transitive dependencies as well.
- Cloud service preferred over self-hosting.
3 Survey of available software¶
A short survey via Google uncovered the following services for automated dependency management:
Dependabot has been acquired by GitHub and is free for public repositories.
GitHub makes its security updates easily available through GitHub repository settings, but it can also send PRs for every out-of-date dependency.
Dockerfile files, and GitHub Actions, but not the other types of dependencies of interest to SQuaRE.
Every dependency update is sent as a separate PR, so its flood of PRs can feel a bit overwhelming.
Dependabot can handle the PIP frozen requirements files, although it updates each dependency (including transitive dependencies) separately. Since frozen dependencies require updating the entire dependency chain, this could potentially result in more than a PR per day per repository.
If packages use the new
.github/dependabot.yml configuration mechanism, Dependabot results are available directly in the GitHub repository UI.
Dependabot is the least likely to disappear in the future, since it is a GitHub first-party service now. For the same reason, it’s likely to get continued development. Dependabot is hosted by GitHub as a cloud service.
Snyk is primarily a security scanner rather than a dependency update automation. It attempts to find security vulnerabilities in a package, including via dependencies, and reports on them. It can generate pull requests to fix dependencies with security vulnerabilities, but at least in the free version this process seems fairly manual. Multiple dependency updates are batched into a single pull request. However, the generated PR only pins dependencies to greater than the last security vulnerability, rather than to a specific version. It does support frozen PIP dependency files, but does not detect out of date dependencies with no security implications.
Synk can analyze Kubernetes resources and Helm charts, but appears to only look for potential security issues, not for out-of-date dependencies.
The security scanning capabilities of Synk look useful, but it doesn’t have the right set of features for automated dependency management.
3.3 WhiteSource Renovate¶
Renovate is a freemium service also available as an open source version. It can be run as a command-line tool or as a Docker image (with Kubernetes support), or is available as a GitHub App managed by WhiteSource.
Compared to the other alternatives, it has excellent Kubernetes support.
It checks Helm chart dependencies and Docker image dependencies in
It also has excellent Docker support, checking for dependencies in
It advertises Kustomize support as well, but unfortunately the current release as of this writing appears not to properly support Kustomize resources where the
kustomize.yaml file is not in the root directory of the repository.
Renovate appears to not support
It has experimental support for
Dependabot supports all three.
Unfortunately, Renovate also does not support PIP
requirements.txt files with pinned hashes, which is our standard for Python applications.
Unlike Dependabot, Renovate can be configured to batch dependency updates into a single PR. This is highly configurable and can be done by language, path, package name, or other criteria. Renovate can also be configured to automatically merge some updates based on criteria such as package name or major versus minor version update.
Renovate supports almost everything that Dependabot supports except for GitHub Actions. Neither supports pre-commit hook dependencies. I did not test Renovate support for NPM or Ruby gem dependencies.
Since Renovate is a freemium product from a startup, there is some risk that WhiteSource will remove or limit their free tier in the future or cease open source development, particularly given that they are now competing against a first-party GitHub service. However, the open source version could potentially be maintained by the community.
We have two main options:
- Use a single system and live with the features that it’s missing.
- Use some mix of multiple systems.
We can also supplement either approach with code we write and maintain ourselves.
Dependabot is attractive because of the strong GitHub integration and first-party support, and because it handles frozen PIP requirements files. It’s also very simple to set up and configure. Renovate isn’t too bad, but it’s a bit more complex. However, it doesn’t support Kubernetes, which is a significant enough gap that we would need to supplement it with our own code. Its inability to group updates into a single PR for packages with frozen PIP dependencies is also likely to be overwhelming.
The combination of Dependabot and Renovate covers all of the types of dependencies except for Kustomize resources, pre-commit plugins. However, that would mean using Dependabot for Python dependencies for applications with frozen PIP requirements files and dealing with large numbers of PRs.
The best approach therefore seems to be to use Dependabot for Python library and GitHub Actions dependencies, Renovate for Kubernetes and Docker dependencies, and custom code for frozen PIP requirements, Kustomize resources, and pre-commit hooks. This requires configuring two systems, but we can template the configuration files. They shouldn’t require much maintenance work. We can choose which of the two systems to use for the dependencies that both systems support based on how well they handle them. Given the current implementations, Renovate is the best default choice because of the ability to configure automatic merging and to group multiple updates into a single PR.
Therefore, the recommendation is:
- Use Dependabot for Python library dependencies, GitHub Actions, and other language dependencies Renovate doesn’t support.
- Use Renovate for all other dependencies except Python applications with frozen PIP requirements files, Kustomize resources, and pre-commit hooks.
- Write our own code to handle Python applications, Kustomize resources, and pre-commit hooks, with an eye to phasing it out if either of the other systems improve.