<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1540390442;
mso-list-template-ids:-1451836410;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1
{mso-list-id:1725835313;
mso-list-template-ids:896801884;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Unfortunately, exclusively focusing on research by Googlers introduces a huge selection bias into this list, making it completely useless as a research overview. A lot of really good research in this area happens at CMU, for example.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>We should all remember that at the same meeting, two Googlers explicitly stated based on no evidence at all that they were confident that there was a difference between 90 day certificates and two year certificates for phishing sites, despite the fact that the typical lifetime of a phishing certificate is best measured in hours. Starting with the conclusion you want, and then working backwards to find the arguments and data that matches them is the wrong way to think about hard problems. <o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>An excellent paper that I happened to read on the plane to London is “Instrumenting Simple Risk Communication for Safer Browsing”, by Camp et al from the recent security & human behavior workshop at CMU:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><a href="https://www.heinz.cmu.edu/~acquisti/SHB2018/participants.htm">https://www.heinz.cmu.edu/~acquisti/SHB2018/participants.htm</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><a href="http://ljean.com/files/Toolbar_Extension.pdf">http://ljean.com/files/Toolbar_Extension.pdf</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I highly recommend the paper, it’s very relevant and up to date. I wish I had time to do a proper survey of all the existing research; I’m sure there’s lots of other good stuff out there.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>-Tim<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b>From:</b> Public [mailto:public-bounces@cabforum.org] <b>On Behalf Of </b>Ryan Sleevi via Public<br><b>Sent:</b> Monday, June 18, 2018 10:16 AM<br><b>To:</b> CABFPub <public@cabforum.org><br><b>Subject:</b> [cabfpub] Research references for CAs<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p style='margin:0in;margin-bottom:.0001pt'><span style='font-family:"Arial",sans-serif;color:#222222;background:white'>During our recent F2F, there were some questions from CAs and other browsers about research that has informed some of the decisions on how the Chrome UI, particularly the security UI, has evolved. </span><span style='font-family:"Arial",sans-serif'><o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></p><p style='margin:0in;margin-bottom:.0001pt'><span style='font-family:"Arial",sans-serif;color:#222222;background:white'>Google has participated in, as well as authored, several research studies that pertain to these topics. In order to ensure the quality of methodology, scale, and analysis, each of these papers underwent review by Conference committee or a group of peers as defined by the publication venue.</span><span style='font-family:"Arial",sans-serif'><o:p></o:p></span></p><p class=MsoNormal><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></p><p style='margin:0in;margin-bottom:.0001pt'><span style='font-family:"Arial",sans-serif;color:#222222'>A list of some of the peer-reviewed research published by Googlers in widely well-respected journals and conferences:</span><span style='font-family:"Arial",sans-serif'><o:p></o:p></span></p><ul style='margin-top:0in' type=disc><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub41323"><span style='color:#1155CC'>Alice in Warningland: A Large-Scale Field Study of Browser Security Warning Effectiveness</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub42546"><span style='color:#1155CC'>Your Reputation Precedes You: History, Reputation, and the Chrome Malware Warning</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub41927"><span style='color:#1155CC'>Experimenting At Scale With Google Chrome's SSL Warning</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub43265"><span style='color:#1155CC'>Improving SSL Warnings: Comprehension and Adherence</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub45366"><span style='color:#1155CC'>Rethinking Connection Security Indicators</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub45374"><span style='color:#1155CC'>A Week to Remember: The Impact of Browser Warning Storage Policies</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub46359"><span style='color:#1155CC'>Where the Wild Warnings Are: Root Causes of Chrome Certificate Errors</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub46197"><span style='color:#1155CC'>Measuring HTTPS adoption on the web</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://blues.cs.berkeley.edu/wp-content/uploads/2018/01/chi18-warnings.pdf"><span style='color:#1155CC'>An Experience Sampling Study of User Reactions to Browser Warnings in the Field</span></a><o:p></o:p></span></li><li style='color:#222222;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l0 level1 lfo1;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://ai.google/research/pubs/pub46306"><span style='color:#1155CC'>152 Simple Steps to Stay Safe Online: Security Advice for Non-Tech-Savvy Users</span></a><o:p></o:p></span></li></ul><p class=MsoNormal><span style='font-family:"Arial",sans-serif'><o:p> </o:p></span></p><p style='margin:0in;margin-bottom:.0001pt'><span style='font-family:"Arial",sans-serif;color:black'>Additionally, in hallway conversations, there were discussions about other research into the PKI ecosystem. A few resources that CAs may not have been aware of, also appearing in top-tier conferences and publications:</span><span style='font-family:"Arial",sans-serif'><o:p></o:p></span></p><ul style='margin-top:0in' type=disc><li style='color:black;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://zakird.com/papers/https_interception.pdf"><span style='color:#1155CC'>The Security Impact of HTTPS Interception</span></a><o:p></o:p></span></li><li style='color:black;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://censys.io/static/censys.pdf"><span style='color:#1155CC'>A Search Engine Backed by Internet-Wide Scanning</span></a><o:p></o:p></span></li><li style='color:black;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://zakird.com/papers/zlint.pdf"><span style='color:#1155CC'>Tracking Certificate Misissuance in the Wild</span></a><o:p></o:p></span></li><li style='color:black;margin-top:0in;margin-bottom:0in;margin-bottom:.0001pt;mso-list:l1 level1 lfo2;vertical-align:baseline'><span style='font-family:"Arial",sans-serif'><a href="https://jhalderm.com/pub/papers/https-perspectives-imc16.pdf"><span style='color:#1155CC'>Towards a Complete View of the Certificate Ecosystem</span></a><o:p></o:p></span></li></ul><p class=MsoNormal style='margin-bottom:12.0pt'><o:p> </o:p></p></div></div></div></body></html>