[Servercert-wg] Ballots SC20 and SC21

[Tobias S. Josefowitz]

On Fri, 31 May 2019, Ryan Sleevi wrote:

> On Fri, May 31, 2019 at 1:17 PM Tobias S. Josefowitz <tobij at opera.com>
> wrote:
>
>> But is it not the case that all a CA would have to say currently is "Hi
>> Ryan, hi $auditor, meet Hans. Hans reviews our configurations weekly. He
>> pinky-swears."? Sure, saying this would not technically make them
>> compliant, but how do you even go about that distinction?
>
> I think we're in agreement that the pinky-swear wouldn't be compliant. Your
> question about how it would be detected is apt - I very much want to
> improve the detectability, which is why I've focused and emphasized so much
> on the importance of transparency (at least with respect to the BRs), which
> we've seen has been tremendous. As our WebTrust and ETSI friends can
> attest, I've similarly pushed for transparency in what they're assessing -
> for example, having the auditor report that they found Hans' pinky-swear to
> be an appropriate demonstration of compliance (which then allows me to look
> closer at both the CA and the auditor).

This is indeed a subject that I severly lack experience in. Just putting 
this out there, I appreciate your experience and your insights in this 
matter.

> However, in this case, I'm considering the 'when things go wrong' scenario.
>
> When Hans doesn't do this, and the Web server gets popped and an XSS is
> added to slurp the RA's/CA's credentials, and then use that to manually
> approve a certificate for 'google.com' (for example), I go ask the CA for
> an incident report and ask them to tell me what they were doing and why
> their systems should have fixed this. When they reveal Hans wasn't actually
> doing this, it's clear that the CA was violating the NetSec requirements,
> and calls into question what the auditor was doing to assess that Hans was
> and had been, since they clearly weren't. We've got an unambiguous
> violation of the requirements.

And here I am unsure if you do not give the current 1h of the NSRs more 
credit than it deserves. First of all, "[Human] Review" certainly includes 
a bit of "best effort"; if not in the scope of items to be reviewed (even 
that is unclear to me), then certainly in what will be the result of the 
review. We do, after all, incorporate the human element, and humans make 
mistakes (or so they say).

Imagine the scenario of a CA based on Windows-family operating systems. An 
adversary makes a change to the registry that cripples the security of the 
CA. Now, is the registry configuration? Are only some parts of it 
configuration? Can we blame Hans if he misses a change (I am assuming the 
registry may change wildly, this may not be the case in the real world), 
and can we blame Hans for deeming a change in it - or any configuration 
actually - to be benign when it is not?

And last but not least, what if a CA says "yes sure, Hans complained about 
this issue since 2016, but we are not in a habit of listening to him 
anyway"; the current language in the NSRs does not require any resulting 
action whatsoever. That is, unless you consider at least some 
configuration issues found under review to be a "Critical Vulnerability" 
(do you?). Or am I missing another mechanism?

That said, what would we need to do language-wise to put you in a position 
to act on a "less-than-stellar" assessment of security-relevant 
configurations in the same way that you say you could act on review 
failures?