[cabfpub] [Trans] What's the load on a CT log?

[Ben Laurie]

On 13 March 2014 20:27, Rob Stradling <rob.stradling at comodo.com> wrote:
> I'm not sure average load tells the whole story.

Agreed.

> Won't there be a surge in audit traffic in the aftermath of a busy site
> installing a new cert?

Yes.

>
>
> On 13/03/14 16:06, Ben Laurie wrote:
>>
>> Several people have asked me this recently. Here's a nice way to estimate
>> load.
>>
>> Let's assume a single log that takes all the load.
>>
>> Firstly, we see about 5,000 new certificates a day, so that's around
>> 0.06 new certificates per second. Clearly a trivial load.
>>
>> Next is load from audit (i.e. from browsers that wish to validate SCTs
>> accompanying certificates they see). Given some assumptions, we can
>> calculate the load from audit.
>>
>> * Clients cache audit results.
>>
>> * There are approximately b = 2.5B browsers in the world
>> (http://www.internetworldstats.com/stats.htm).
>>
>> * The average user visits w = 89 websites a month
>> (http://www.creditloan.com/blog/how-the-world-spends-its-time-online/
>> quoting a Nielsen report). Assume these are all TLS sites.
>>
>> * Assume a certificate lifetime of l = 12 months.
>>
>> So, each user sees w / l new certificates a month. Each new
>> certificate needs to be audited, which means in practice, three web
>> operations (fetch STH, fetch STH consistency proof, fetch SCT
>> inclusion proof) - it might be a good idea to create a new API to do
>> all three in one go.
>>
>> So, total average load is 3 * b * w / l ~ 20,000 web fetches per
>> second. If we optimise the API we can get that down to 7,000 qps. Each
>> query (in the optimised case) would be around 3 kB, which gives a
>> bandwidth of around 150 kb/s.
>>
>> Monitors add extra load, but should only be at around the new
>> certificate rate - i.e. ~ .06 * number of monitors fetches per second.
>>
>> IMO, this is achievable on a single machine (modulo reliability), with
>> some care. Clearly not a vast farm, however its done.
>>
>> In practice, no one log would have to take this full load, this is a
>> worst case analysis.
>>
>> _______________________________________________
>> Trans mailing list
>> Trans at ietf.org
>> https://www.ietf.org/mailman/listinfo/trans
>>
>
> --
> Rob Stradling
> Senior Research & Development Scientist
> COMODO - Creating Trust Online
> Office Tel: +44.(0)1274.730505
> Office Fax: +44.(0)1274.730909
> www.comodo.com
>
> COMODO CA Limited, Registered in England No. 04058690
> Registered Office:
>   3rd Floor, 26 Office Village, Exchange Quay,
>   Trafford Road, Salford, Manchester M5 3EQ
>
> This e-mail and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the sender by
> replying to the e-mail containing this attachment. Replies to this email may
> be monitored by COMODO for operational or business reasons. Whilst every
> endeavour is taken to ensure that e-mails are free from viruses, no
> liability can be accepted and the recipient is requested to use their own
> virus checking software.