Which Rebar Plugins Should We Test First? 👀
Working on Rebar v4 has been a great experience. We have had the chance to dig into how the tool works and make changes we are proud of: replacing vendored functions with OTP equivalents, moving command-line parsing from getopt to OTP's argparse, and working on plugin dependency locking... Now we need to make sure those changes still work well with the plugins you use in real projects.
We know there are a lot of rebar plugins out there, and testing all of them would not be very useful. Some cover a busy, everyday workflow. Some are older, niche, or now part of rebar itself.
So we have put together an initial ranking. It is not a list of plugins we plan to take over. It is a suggestion for the tests that will help us most as we work on Rebar v4.
How Do We Rank Them?
We gave each plugin a score out of 100. The score is based on five simple questions:
| What to look at | Weight |
|---|---|
| How widely the plugin is used | 25 |
| How much of Rebar it exercises | 25 |
| Whether it is maintained and current | 20 |
| How useful the workflow is to cover | 20 |
| Whether it can run reliably in CI | 10 |
Then we figured the recommended plugins on the Rebar3 website would be a good starting point. It is a helpful list, but it is not enough on its own... A plugin can be listed there and still be a poor fit for regular testing. What matters most is how much it is used by the community, which parts of rebar it relies on, and whether we can test it reliably.
These are the plugins we would put at the front of the queue:
| Rank | Plugin | Score | Why it is useful |
|---|---|---|---|
| 1 | rebar3_hex | 93 | Package build and publishing workflows |
| 2 | pc | 91 | Native-code builds, hooks, and clean behaviour |
| 3 | rebar3_ex_doc | 87 | Documentation, project metadata, and dependencies |
| 4 | rebar3_lint | 84 | A familiar CI and provider workflow |
| 5 | covertool | 82 | Test output and coverage reports |
| 6 | rebar3_gpb_plugin | 80 | Code generation and pre-compile hooks |
| 7 | rebar3_format | 74 | Formatting in CI |
| 8 | rebar3_proper | 73 | Property-testing workflows |
| 9 | rebar3_appup_plugin | 70 | Release upgrades and appup generation |
| 10 | rebar3_path_deps | 68 | Local and path-based dependencies |
The next group is still worth checking, especially before a release:
| Rank | Plugin | Score | Test priority |
|---|---|---|---|
| 11 | rebar3_auto | 65 | Nightly, if watcher tests are reliable in CI |
| 12 | rebar3_run | 59 | Nightly testing |
| 13 | rebar_mix | 54 | Targeted testing for Mix dependencies |
| 14 | rebar3_sbom | 53 | Nightly compliance testing |
| 15 | rebar3_neotoma_plugin | 48 | Legacy code-generation smoke test |
| 16 | rebar3_efmt | 46 | Optional modern formatter track |
| 17 | rebar3_diameter_compiler | 45 | Optional niche code generation |
| 18 | rebar3_depup | 43 | Optional dependency-management fixture |
| 19 | rebar3_eqc | 37 | Only where QuickCheck is available in CI |
| 20 | rebar3_erlydtl_plugin | 35 | Low-priority legacy smoke test |
Btw, aliases belong in Rebar's own tests because alias support is built in, and rebar3_vendor has been superseded by rebar3_path_deps, so testing both would add little value.
Get Involved
We promised our Kickstarter backers that we would keep them involved in this work, we will send out a separate poll tomorrow 😉. When you rank the plugins, think about the things you really use rebar for: publishing packages, compiling native code, generating code, testing, making releases... or something else entirely.
If a plugin you rely on is nowhere near the top of our list, we definitely want to know. We are not trying to defend a score. We want to build a test suite that catches the kind of regressions that would get in your way.
Oh, and you can already try our current release candidate: Rebar v4.0.0-rc.1. Even if you weren't part of our Kickstarter, you can still try it out and play around with it.
Thank you to everyone supporting the Rebar Kickstarter. This is the kind of careful, unglamorous work that helps Rebar v4 arrive in good shape.