[cabfpub] What's the load on a CT log?
benl at google.com
Thu Mar 13 16:06:06 UTC 2014
Several people have asked me this recently. Here's a nice way to estimate load.
Let's assume a single log that takes all the load.
Firstly, we see about 5,000 new certificates a day, so that's around
0.06 new certificates per second. Clearly a trivial load.
Next is load from audit (i.e. from browsers that wish to validate SCTs
accompanying certificates they see). Given some assumptions, we can
calculate the load from audit.
* Clients cache audit results.
* There are approximately b = 2.5B browsers in the world
* The average user visits w = 89 websites a month
quoting a Nielsen report). Assume these are all TLS sites.
* Assume a certificate lifetime of l = 12 months.
So, each user sees w / l new certificates a month. Each new
certificate needs to be audited, which means in practice, three web
operations (fetch STH, fetch STH consistency proof, fetch SCT
inclusion proof) - it might be a good idea to create a new API to do
all three in one go.
So, total average load is 3 * b * w / l ~ 20,000 web fetches per
second. If we optimise the API we can get that down to 7,000 qps. Each
query (in the optimised case) would be around 3 kB, which gives a
bandwidth of around 150 kb/s.
Monitors add extra load, but should only be at around the new
certificate rate - i.e. ~ .06 * number of monitors fetches per second.
IMO, this is achievable on a single machine (modulo reliability), with
some care. Clearly not a vast farm, however its done.
In practice, no one log would have to take this full load, this is a
worst case analysis.
More information about the Public