[Servercert-wg] Fixing lag between requirements changes and linter updates

[Aaron Gable]

In the last six months, by our count there have been at least:
- 7 bugzilla incident reports due to not marking the basicConstraints
extension critical (1 <https://bugzilla.mozilla.org/show_bug.cgi?id=1888060>,
2 <https://bugzilla.mozilla.org/show_bug.cgi?id=1887008>, 3
<https://bugzilla.mozilla.org/show_bug.cgi?id=1883416>, 4
<https://bugzilla.mozilla.org/show_bug.cgi?id=1888104>, 5
<https://bugzilla.mozilla.org/show_bug.cgi?id=1885132>, 6
<https://bugzilla.mozilla.org/show_bug.cgi?id=1886135>, 7
<https://bugzilla.mozilla.org/show_bug.cgi?id=1875820>)
- 5 bugzilla incident reports due to encoding Subject attributes in an
incorrect order (1 <https://bugzilla.mozilla.org/show_bug.cgi?id=1864204>, 2
<https://bugzilla.mozilla.org/show_bug.cgi?id=1886624>, 3
<https://bugzilla.mozilla.org/show_bug.cgi?id=1883731>, 4
<https://bugzilla.mozilla.org/show_bug.cgi?id=1883620>, 5
<https://bugzilla.mozilla.org/show_bug.cgi?id=1883779>)
- 3 bugzilla incident reports due to not including the CPS URI in an EV
certificate (1 <https://bugzilla.mozilla.org/show_bug.cgi?id=1883843>, 2
<https://bugzilla.mozilla.org/show_bug.cgi?id=1886257>, 3
<https://bugzilla.mozilla.org/show_bug.cgi?id=1888016>)
- and 7 other incidents due to missing various other requirements from the
profiles ballot (1 <https://bugzilla.mozilla.org/show_bug.cgi?id=1861069>, 2
<https://bugzilla.mozilla.org/show_bug.cgi?id=1876565>, 3
<https://bugzilla.mozilla.org/show_bug.cgi?id=1884532>, 4
<https://bugzilla.mozilla.org/show_bug.cgi?id=1884714>, 5
<https://bugzilla.mozilla.org/show_bug.cgi?id=1886406>, 6
<https://bugzilla.mozilla.org/show_bug.cgi?id=1887096>, 7
<https://bugzilla.mozilla.org/show_bug.cgi?id=1875942>).

Many of these incidents cite reliance on linting systems (such as zlint
<https://github.com/zmap/zlint>, pkilint
<https://github.com/digicert/pkilint>, cablint
<https://github.com/amazon-archives/certlint>, and x509lint
<https://github.com/kroeckx/x509lint>) to report whether actual issuance
practices are in line with the required profiles. And many of these
incidents cite the fact that ballot SC-062 was not enforced by zlint
immediately on 2023-09-15 as a reason that the non-compliance was not
caught.

Obviously there are many potential improvements that can be made here,
including both process and technical improvements within each CA, and we're
sure that they will be. But the scale of these incidents suggests to me
that there may be systemic changes *we* can make to enable easier
compliance with certificate profile changes.

We think that it would make sense for any proposed ballot which touches
Section 7 of the BRs (or equivalent sections in the EVGs) to be accompanied
by a PR against zlint which adds or modifies checks to enforce the proposed
ballot text.

Such a ballot would not necessarily have to be written by the ballot author
(this is what endorsers are for!), and zlint already has capabilities to
not start enforcing a lint until a specified Effective Date in the future,
so incorporating upcoming ballot requirements into zlint ahead of time
should be fairly easy and straightforward.

We know that we certainly plan to do this for any future ballots we
propose. What we don't know is how we would go about actually encouraging
this behavior. Just setting new community norms about asking for such PRs
during the discussion period? Adding something to our bylaws to require
such a PR in the official ballot proposal? Do others have ideas?

Thanks,
Samantha Frank & Aaron Gable
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cabforum.org/pipermail/servercert-wg/attachments/20240401/981c4552/attachment-0001.html>