<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
I think you have a point about A + B, so we should have independent
review periods for each ballot as we did with version 1.6.4 (we had
independent ballots to be reviewed and then a combined/aggregated
new version of the BRs). <br>
<br>
I also received some personal messages supporting the "aggregated"
practice as it seems to be much more efficient for a CA's compliance
review to check against one new BR rather than 2 or 3 separate ones
within a very short timeframe.<br>
<br>
More answers inline.<br>
<br>
<div class="moz-cite-prefix">On 2020-09-15 5:34 μ.μ., Ryan Sleevi
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr"><br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Sep 15, 2020 at 4:33
AM Dimitris Zacharopoulos (HARICA) <<a
href="mailto:dzacharo@harica.gr" moz-do-not-send="true">dzacharo@harica.gr</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div>
<div>On 14/9/2020 8:08 μ.μ., Ryan Sleevi wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Yes, I'm aware of the "what", but it's
not clear the "why".
<div><br>
</div>
<div>The act of combining ballots is relatively new,
as you can see from <a
href="https://cabforum.org/baseline-requirements-documents/"
target="_blank" moz-do-not-send="true">https://cabforum.org/baseline-requirements-documents/</a>
. Producing multiple versions of the Guidelines,
linear based on when the Ballot concluded, was
something our GitHub flow intentionally was to make
easy. While that page stopped listing dates of
adoption around Ballot 189, you can see previous
ballot pairs (e.g. 171+164, 125+118+134+135) that
did that.</div>
<div><br>
</div>
<div>It seems worth figuring out the challenges you're
facing, since it was meant to be very easy to create
a new version of the document for each ballot, even
ballots that conclude closely, and to have IP
reviews as such.</div>
</div>
</blockquote>
<br>
The administrative overhead of updating public web sites,
sending additional emails, and the fact that we would have
versions of the Guidelines that would be valid for a few
seconds (which seems unreasonable for a public standards
document), are some of the reasons behind aggregated final
guidelines. Version 1.6.4 aggregated 3 ballots, 1.6.7 and
1.7.1. had 2 and now 1.7.2. </div>
</blockquote>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div> <br>
This process was discussed with Dean and Wayne back in
February 2019, and we all considered it compliant with our
Bylaws. The results of each IPR review period were sent to
the public lists without receiving any objections or
concerns.<br>
<br>
Although we have documented a GitHub workflow that
supports the most common case (one ballot, one IPR review,
one Final Maintenance Guideline), it should not prohibit
aggregated ballots to minimize administrative overhead or
the production of Guidelines that have some reasonable
validity time.<br>
<br>
If there are strong objections to this process, we can
revise it going forward.<br>
</div>
</blockquote>
<div><br>
</div>
<div>I think we should understand the concerns, so we can make
sure we have solutions that work.</div>
<div><br>
</div>
<div>The concerns for aggregated ballots is that, from an IP
perspective, it makes multiple ballots share the same fate
and can easily delay adoption. For example, imagine there
are Ballots A and B. A makes a change to one section, B
makes a change to another, and only through their combined
implementation does a Member raise an IP concern.</div>
<div><br>
</div>
<div>In the serialized approach, A then B, A can be introduced
to the IP review and not introduce any IP obligations. It
can become effective. When B is introduced and undergoes IP
review, and an exclusion is filed (on the B+A), then B is
basically sent to the PAG to work through what to do, while
A remains effective.</div>
<div><br>
</div>
<div>In the combined approach, A and B, when A+B trigger the
IP review and result in the IP obligation. An exclusion is
filed, and now A+B are sent to a PAG. The act of
understanding whether it is A introducing the IP obligation
or B introducing the IP obligation is left for the PAG to
sort, with neither A nor B being effective until the PAG
reaches recommendations.</div>
<div><br>
</div>
<div>This was a concern debated at length throughout our
governance process, and was indeed one of the core
motivations for our IPR policy update (which, you may
recall, was motivated by the removal of the "any other
method" introducing obligations on the other enumerated
methods). Our IP policy was revised to try to make it easier
to distinguish whether A or B was introducing the
obligation, and allow modifications like A (and subsequent
modifications) to continue, even while B was addressed
within the PAG.</div>
<div><br>
</div>
<div>Beyond that explicit reason, the distinct versioning also
helps better review changes and tie back to ballots. It
treats Ballots as they are - logical units of change -
rather than aggregations of change. The Forum first started
with an aggregation-of-change approach (through submitting
amendments and errata) and quickly confounded the ability to
understand the reconciled state. The same issue applies to
unified ballots. Given that we have CAs asserting that it's
difficult to find changes in Ballots they explicitly voted
for, after lengthy reviews, and with policies incorporated
in at least two root programs, I'm incredibly sensitive to
wanting to make sure CAs are set up to succeed in following
the Requirements, by making changes discrete and
digestible. </div>
<div><br>
</div>
<div>So that's the context about "why" serialized was
intended. I'm also totally sympathetic to the difficulty
involved in producing ballots, especially as we haven't
quite got automation up to speed, and so I can understand
why wanting to avoid that work is undesirable. What I take
from your reply is that there are several tasks that, as a
consequence of our current tooling, represent undesirable
time commitments. It's unclear if that's "5 minutes more" or
"5 hours more", and so your help in breaking down how much
time it takes for the following would be greatly useful, as
it will help prioritize what is most important to resolve. I
suspect these can become priorities for the Infrastructure
WG.</div>
<div><br>
</div>
<div>1) Merge a single Ballot into GitHub</div>
</div>
</div>
</blockquote>
<br>
This greatly depends on whether the ballot proposer has used the
documented and recommended (not yet normative/required) process,
using GitHub to create a redline. Even with GitHub, I've seen
different "flavors" of redlines, for example with and without pull
requests, making it somewhat challenging to find the right steps to
create a cabforum/documents branch that will also create the
artifacts (docx document) from which I can create the redline.<br>
<br>
If we are talking about the Baseline Requirements. this process
takes between 15 minutes to 1,5 hours in the more weird cases. For
the other Guidelines, it takes between 30 minutes to 1 hour.<br>
<br>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>2) Produce a Final Guideline and a Redlined Version</div>
</div>
</div>
</blockquote>
Once I have the review period redline ready, it usually takes
between 30 minutes to 1 hour to create the final documents, upload
them to the wiki (the word versions), produce the final PDF versions
and upload them to the public web site.<br>
<br>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>3) Draft and send the e-mail for the IP review period
announcement</div>
</div>
</div>
</blockquote>
This usually takes between 10 and 20 minutes because I am using
templates on Thunderbird and just replacing the links and
attachments.<br>
<br>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>4) Upload the Final Guideline and Redlined Version to the
website</div>
</div>
</div>
</blockquote>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>5) Update the Website to link to these</div>
</div>
</div>
</blockquote>
<br>
10 to 20 minutes for both 4 and 5.<br>
<br>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div>6) Draft and send the e-mail concluding the IP review
period</div>
</div>
</div>
</blockquote>
<br>
Again 10 to 20 minutes.<br>
<br>
<blockquote type="cite"
cite="mid:CACvaWvbYetssSf3RQyTFB2ikMEyU23f=MPUsMe3JswZL5GMYxg@mail.gmail.com">
<div dir="ltr">
<div class="gmail_quote">
<div><br>
</div>
<div>As a number of these tasks are very automatable, figuring
out what we can do so that it represents <30 minutes of
additional work seems... a reasonable target, right? This
only comes up when we have multiple ballots concluding
within a short period of eachother, which although
historically rare, certainly was more common over the summer
as stuff got batched to accommodate Covid disruptions.</div>
</div>
</div>
</blockquote>
<br>
The most challenging and time consuming tasks come with ballots that
update more than one Guideline, for which we don't have the same
tools ready to produce docx versions. This requires me to do the
GitHub redlines for the BRs and docx versions (manually) for EV
Guidelines or NSRs.<br>
<br>
Even for the BRs, I usually have to remove text after the ToC
because it repeats information that is supposed to be in the cover
page. Some of these minor issues need to be fixed, and I would
really appreciate the help of the Infrastructure Subcommittee.<br>
<br>
Would your concerns be addressed if we had a separate/distinct IPR
review for each ballot and once each IPR review is cleared with no
essential claims filed, produce an aggregate document, or you still
prefer separate Final documents with distinct versioning to be
created? What do others think?<br>
<br>
Best regards,<br>
Dimitris.<br>
</body>
</html>