[Servercert-wg] CANCEL Notice of Review Period – Ballot SC35

[Dimitris Zacharopoulos (HARICA)]

I think you have a point about A + B, so we should have independent 
review periods for each ballot as we did with version 1.6.4 (we had 
independent ballots to be reviewed and then a combined/aggregated new 
version of the BRs).

I also received some personal messages supporting the "aggregated" 
practice as it seems to be much more efficient for a CA's compliance 
review to check against one new BR rather than 2 or 3 separate ones 
within a very short timeframe.

More answers inline.

On 2020-09-15 5:34 μ.μ., Ryan Sleevi wrote:
>
>
> On Tue, Sep 15, 2020 at 4:33 AM Dimitris Zacharopoulos (HARICA) 
> <dzacharo at harica.gr <mailto:dzacharo at harica.gr>> wrote:
>
>     On 14/9/2020 8:08 μ.μ., Ryan Sleevi wrote:
>>     Yes, I'm aware of the "what", but it's not clear the "why".
>>
>>     The act of combining ballots is relatively new, as you can see
>>     from https://cabforum.org/baseline-requirements-documents/ .
>>     Producing multiple versions of the Guidelines, linear based on
>>     when the Ballot concluded, was something our GitHub flow
>>     intentionally was to make easy. While that page stopped listing
>>     dates of adoption around Ballot 189, you can see previous ballot
>>     pairs (e.g. 171+164, 125+118+134+135) that did that.
>>
>>     It seems worth figuring out the challenges you're facing, since
>>     it was meant to be very easy to create a new version of the
>>     document for each ballot, even ballots that conclude closely, and
>>     to have IP reviews as such.
>
>     The administrative overhead of updating public web sites, sending
>     additional emails, and the fact that we would have versions of the
>     Guidelines that would be valid for a few seconds (which seems
>     unreasonable for a public standards document), are some of the
>     reasons behind aggregated final guidelines. Version 1.6.4
>     aggregated 3 ballots, 1.6.7 and 1.7.1. had 2 and now 1.7.2.
>
>
>     This process was discussed with Dean and Wayne back in February
>     2019, and we all considered it compliant with our Bylaws. The
>     results of each IPR review period were sent to the public lists
>     without receiving any objections or concerns.
>
>     Although we have documented a GitHub workflow that supports the
>     most common case (one ballot, one IPR review, one Final
>     Maintenance Guideline), it should not prohibit aggregated ballots
>     to minimize administrative overhead or the production of
>     Guidelines that have some reasonable validity time.
>
>     If there are strong objections to this process, we can revise it
>     going forward.
>
>
> I think we should understand the concerns, so we can make sure we have 
> solutions that work.
>
> The concerns for aggregated ballots is that, from an IP perspective, 
> it makes multiple ballots share the same fate and can easily delay 
> adoption. For example, imagine there are Ballots A and B. A makes a 
> change to one section, B makes a change to another, and only through 
> their combined implementation does a Member raise an IP concern.
>
> In the serialized approach, A then B, A can be introduced to the IP 
> review and not introduce any IP obligations. It can become effective. 
> When B is introduced and undergoes IP review, and an exclusion is 
> filed (on the B+A), then B is basically sent to the PAG to work 
> through what to do, while A remains effective.
>
> In the combined approach, A and B, when A+B trigger the IP review and 
> result in the IP obligation. An exclusion is filed, and now A+B are 
> sent to a PAG. The act of understanding whether it is A introducing 
> the IP obligation or B introducing the IP obligation is left for the 
> PAG to sort, with neither A nor B being effective until the PAG 
> reaches recommendations.
>
> This was a concern debated at length throughout our governance 
> process, and was indeed one of the core motivations for our IPR policy 
> update (which, you may recall, was motivated by the removal of the 
> "any other method" introducing obligations on the other enumerated 
> methods). Our IP policy was revised to try to make it easier to 
> distinguish whether A or B was introducing the obligation, and allow 
> modifications like A (and subsequent modifications) to continue, even 
> while B was addressed within the PAG.
>
> Beyond that explicit reason, the distinct versioning also helps better 
> review changes and tie back to ballots. It treats Ballots as they are 
> - logical units of change - rather than aggregations of change. The 
> Forum first started with an aggregation-of-change approach (through 
> submitting amendments and errata) and quickly confounded the ability 
> to understand the reconciled state. The same issue applies to unified 
> ballots. Given that we have CAs asserting that it's difficult to find 
> changes in Ballots they explicitly voted for, after lengthy reviews, 
> and with policies incorporated in at least two root programs, I'm 
> incredibly sensitive to wanting to make sure CAs are set up to succeed 
> in following the Requirements, by making changes discrete and digestible.
>
> So that's the context about "why" serialized was intended. I'm also 
> totally sympathetic to the difficulty involved in producing ballots, 
> especially as we haven't quite got automation up to speed, and so I 
> can understand why wanting to avoid that work is undesirable. What I 
> take from your reply is that there are several tasks that, as a 
> consequence of our current tooling, represent undesirable time 
> commitments. It's unclear if that's "5 minutes more" or "5 hours 
> more", and so your help in breaking down how much time it takes for 
> the following would be greatly useful, as it will help prioritize what 
> is most important to resolve. I suspect these can become priorities 
> for the Infrastructure WG.
>
> 1) Merge a single Ballot into GitHub

This greatly depends on whether the ballot proposer has used the 
documented and recommended (not yet normative/required) process, using 
GitHub to create a redline. Even with GitHub, I've seen different 
"flavors" of redlines, for example with and without pull requests, 
making it somewhat challenging to find the right steps to create a 
cabforum/documents branch that will also create the artifacts (docx 
document) from which I can create the redline.

If we are talking about the Baseline Requirements. this process takes 
between 15 minutes to 1,5 hours in the more weird cases. For the other 
Guidelines, it takes between 30 minutes to 1 hour.

> 2) Produce a Final Guideline and a Redlined Version
Once I have the review period redline ready, it usually takes between 30 
minutes to 1 hour to create the final documents, upload them to the wiki 
(the word versions), produce the final PDF versions and upload them to 
the public web site.

> 3) Draft and send the e-mail for the IP review period announcement
This usually takes between 10 and 20 minutes because I am using 
templates on Thunderbird and just replacing the links and attachments.

> 4) Upload the Final Guideline and Redlined Version to the website
> 5) Update the Website to link to these

10 to 20 minutes for both 4 and 5.

> 6) Draft and send the e-mail concluding the IP review period

Again 10 to 20 minutes.

>
> As a number of these tasks are very automatable, figuring out what we 
> can do so that it represents <30 minutes of additional work seems... a 
> reasonable target, right? This only comes up when we have multiple 
> ballots concluding within a short period of eachother, which although 
> historically rare, certainly was more common over the summer as stuff 
> got batched to accommodate Covid disruptions.

The most challenging and time consuming tasks come with ballots that 
update more than one Guideline, for which we don't have the same tools 
ready to produce docx versions. This requires me to do the GitHub 
redlines for the BRs and docx versions (manually) for EV Guidelines or NSRs.

Even for the BRs, I usually have to remove text after the ToC because it 
repeats information that is supposed to be in the cover page. Some of 
these minor issues need to be fixed, and I would really appreciate the 
help of the Infrastructure Subcommittee.

Would your concerns be addressed if we had a separate/distinct IPR 
review for each ballot and once each IPR review is cleared with no 
essential claims filed, produce an aggregate document, or you still 
prefer separate Final documents with distinct versioning to be created? 
What do others think?

Best regards,
Dimitris.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cabforum.org/pipermail/servercert-wg/attachments/20200915/25fbd56d/attachment-0001.html>