Security slider usability testing results
Hi all: The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9Z87hg4XWW8O... In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction. Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.” Cheers, Linda P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecuritySlider -- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2
Interesting, apologies if this is trivial/already-considered-and-bad/etc How about settings with names something like Mostly Harmless Basic Minimal This avoids the direct statement of comparison in the name, so might preclude people avoiding a safer setting they might otherwise choose 'cause it sounds too paranoid. but still shold be clear what order they're in. (I was going to suggest "Safe" for the highest one, but cringe at ever actually saying that simpliciter. Plus I'm a big Douglas Adams fan. Actually I was also going to suggest "Undici" because, like Starbucks, we could name our largest size with the same big number regardless of whether that still corresponds to any units---except we've got security that goes to _eleven_. OK tired. Need to go home.) aloha, Paul On Thu, Mar 09, 2017 at 04:57:54PM -0600, Linda Naeun Lee wrote:
Hi all:
The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9Z87hg4XWW8O...
In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction.
Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.”
Cheers, Linda
P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecuritySlider
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2 _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
On 2017-03-09 17:26, Paul Syverson wrote:
Interesting, apologies if this is trivial/already-considered-and-bad/etc
No apologies! Thank you for your feedback.
How about settings with names something like Mostly Harmless Basic Minimal
We did iterate through the copy, but this is appreciated since the feedback says we should probably look into things more. I like your suggestions because they don't associate safety with the settings (which isn't false, but it's not something that we can guarantee people). The more correct thing might be to tell them about the reduced functionality, with a hint to the fact that these measures might protect you. Avoiding negative things (like things stop working and users don't know why) are much much much more important than including positive thing (like making them feel proactive about their security). The former loses users, the latter is a temporary high at best.
This avoids the direct statement of comparison in the name, so might preclude people avoiding a safer setting they might otherwise choose 'cause it sounds too paranoid. but still shold be clear what order they're in.
I agree. I actually like the progression of standard > something > basic. But that's only my opinion; don't know how users would feel.
(I was going to suggest "Safe" for the highest one, but cringe at ever actually saying that simpliciter. Plus I'm a big Douglas Adams fan. Actually I was also going to suggest "Undici" because, like Starbucks, we could name our largest size with the same big number regardless of whether that still corresponds to any units---except we've got security that goes to _eleven_. OK tired. Need to go home.)
Hmm! This inspires me to work on the copy again. Thanks! Cheers, Linda
aloha, Paul
On Thu, Mar 09, 2017 at 04:57:54PM -0600, Linda Naeun Lee wrote:
Hi all:
The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9Z87hg4XWW8O...
In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction.
Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.”
Cheers, Linda
P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecuritySlider
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2 _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2
Hi, I also want to add some points :). 1. "Safer", "Safest" sounds really strange form me. Already this words suggest "There is no much difference between us". For me, if I read this, I have not really an idea what I can expect. But that's me :) You need names which sounds more differently. 2. I can understand that it's not clear what is the different between "Safer" and "Safest" in the explanations. If people don't know what HTTPS is (and a lot of people don't understand it, sadly), they don't understand the difference between the two options. And if people have no idea "how the internet works" or better "how the content of a website works", they, of course, also don't understand the difference of the other items of the explanations. => They have no idea what they are doing if they change between "Safer" and "Safest". But "Safest" sounds "super secure" so it has to be something for paranoid people :) I do not really know how you can make it better. Perhaps with a very simply and small example between the two options. Something which can be explained in a short sentence or whatever (e.g. a small icon/image/symbol/animation/.gif etc., people like visual illustrations :). Like: If you choose option A and you visit a site which uses B (e.g. java script) this C could happened 3. What means "Standard"? The first impression, only after reading the word "standard": TorBrowser=Firefox? The second, after reading the explanation: What kind of features are enabled? What does that mean? Is it secure, now? Yes or no? The word "features" is very nebulous. In particular if I read the item: Orfox + features enabled = sounds secure website + features enable = sounds insecure => Sounds inconsistent. "I'm confused!" Bye, Carolin Am Freitag, den 10.03.2017, 10:40 -0600 schrieb Linda Naeun Lee:
On 2017-03-09 17:26, Paul Syverson wrote:
Interesting, apologies if this is trivial/already-considered-and-bad/etc
No apologies! Thank you for your feedback.
How about settings with names something like Mostly Harmless Basic Minimal
We did iterate through the copy, but this is appreciated since the feedback says we should probably look into things more.
I like your suggestions because they don't associate safety with the settings (which isn't false, but it's not something that we can guarantee people). The more correct thing might be to tell them about the reduced functionality, with a hint to the fact that these measures might protect you.
Avoiding negative things (like things stop working and users don't know why) are much much much more important than including positive thing (like making them feel proactive about their security). The former loses users, the latter is a temporary high at best.
This avoids the direct statement of comparison in the name, so might preclude people avoiding a safer setting they might otherwise choose 'cause it sounds too paranoid. but still shold be clear what order they're in.
I agree. I actually like the progression of standard > something > basic. But that's only my opinion; don't know how users would feel.
(I was going to suggest "Safe" for the highest one, but cringe at ever actually saying that simpliciter. Plus I'm a big Douglas Adams fan. Actually I was also going to suggest "Undici" because, like Starbucks, we could name our largest size with the same big number regardless of whether that still corresponds to any units---except we've got security that goes to _eleven_. OK tired. Need to go home.)
Hmm! This inspires me to work on the copy again. Thanks!
Cheers, Linda
aloha, Paul
On Thu, Mar 09, 2017 at 04:57:54PM -0600, Linda Naeun Lee wrote:
Hi all:
The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9 Z87hg4XWW8O4uk/edit?usp=sharing
In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction.
Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.”
Cheers, Linda
P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecurit ySlider
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2 _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
-- Carolin Zöbelein / Nick: Samdney PGP: D4A7 35E8 D47F 801F 2CF6 2BA7 927A FD3C DE47 E13B -----------------------------------------------------------------------
On 2017-03-13 16:13, Carolin Zöbelein wrote:
Hi,
I also want to add some points :).
1. "Safer", "Safest" sounds really strange form me. Already this words suggest "There is no much difference between us". For me, if I read this, I have not really an idea what I can expect. But that's me :) You need names which sounds more differently.
I agree with you, but may argue that it's more descriptive than "low, medium, and high."
2. I can understand that it's not clear what is the different between "Safer" and "Safest" in the explanations. If people don't know what HTTPS is (and a lot of people don't understand it, sadly), they don't understand the difference between the two options. And if people have no idea "how the internet works" or better "how the content of a website works", they, of course, also don't understand the difference of the other items of the explanations. => They have no idea what they are doing if they change between "Safer" and "Safest". But "Safest" sounds "super secure" so it has to be something for paranoid people :)
Nice to hear one person's opinion. We'll definitely look into making the copy better. But we think it's an improvement from the old one and will go ahead with it with the intent to redo it later.
I do not really know how you can make it better. Perhaps with a very simply and small example between the two options. Something which can be explained in a short sentence or whatever (e.g. a small icon/image/symbol/animation/.gif etc., people like visual illustrations :). Like: If you choose option A and you visit a site which uses B (e.g. java script) this C could happened
I think an explanation and a visual are great ideas!
3. What means "Standard"? The first impression, only after reading the word "standard": TorBrowser=Firefox?
People were worried that "low" was the default setting, and asking why the default security level for a browser was "low." So we decided to switch it to standard, something, something else.
The second, after reading the explanation: What kind of features are enabled? What does that mean? Is it secure, now? Yes or no?
The word "features" is very nebulous. In particular if I read the item: Orfox + features enabled = sounds secure website + features enable = sounds insecure => Sounds inconsistent. "I'm confused!"
Agreed, but we were writing to a general audience that wouldn't understand more details than that. The point of the text is to communicate just enough information to make a setting decision, not to educate people. I'll make sure this isn't confusing or distracting, though. Thanks for the feedback!
Bye, Carolin
Am Freitag, den 10.03.2017, 10:40 -0600 schrieb Linda Naeun Lee:
On 2017-03-09 17:26, Paul Syverson wrote:
Interesting, apologies if this is trivial/already-considered-and-bad/etc
No apologies! Thank you for your feedback.
How about settings with names something like Mostly Harmless Basic Minimal
We did iterate through the copy, but this is appreciated since the feedback says we should probably look into things more.
I like your suggestions because they don't associate safety with the settings (which isn't false, but it's not something that we can guarantee people). The more correct thing might be to tell them about the reduced functionality, with a hint to the fact that these measures might protect you.
Avoiding negative things (like things stop working and users don't know why) are much much much more important than including positive thing (like making them feel proactive about their security). The former loses users, the latter is a temporary high at best.
This avoids the direct statement of comparison in the name, so might preclude people avoiding a safer setting they might otherwise choose 'cause it sounds too paranoid. but still shold be clear what order they're in.
I agree. I actually like the progression of standard > something > basic. But that's only my opinion; don't know how users would feel.
(I was going to suggest "Safe" for the highest one, but cringe at ever actually saying that simpliciter. Plus I'm a big Douglas Adams fan. Actually I was also going to suggest "Undici" because, like Starbucks, we could name our largest size with the same big number regardless of whether that still corresponds to any units---except we've got security that goes to _eleven_. OK tired. Need to go home.)
Hmm! This inspires me to work on the copy again. Thanks!
Cheers, Linda
aloha, Paul
On Thu, Mar 09, 2017 at 04:57:54PM -0600, Linda Naeun Lee wrote:
Hi all:
The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9 Z87hg4XWW8O4uk/edit?usp=sharing
In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction.
Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.”
Cheers, Linda
P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecurit ySlider
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2 _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2
Part of the problem I think is it's really a safety/usability tradeoff slider. So high safety is low usability and vice versa. Not sure how to best express the tradeoff succinctly. Maybe there's a metaphor -- motorcycle, car, tank? On Tue, Mar 14, 2017 at 12:10 PM Linda Naeun Lee <linda@torproject.org> wrote:
On 2017-03-13 16:13, Carolin Zöbelein wrote:
Hi,
I also want to add some points :).
1. "Safer", "Safest" sounds really strange form me. Already this words suggest "There is no much difference between us". For me, if I read this, I have not really an idea what I can expect. But that's me :) You need names which sounds more differently.
I agree with you, but may argue that it's more descriptive than "low, medium, and high."
2. I can understand that it's not clear what is the different between "Safer" and "Safest" in the explanations. If people don't know what HTTPS is (and a lot of people don't understand it, sadly), they don't understand the difference between the two options. And if people have no idea "how the internet works" or better "how the content of a website works", they, of course, also don't understand the difference of the other items of the explanations. => They have no idea what they are doing if they change between "Safer" and "Safest". But "Safest" sounds "super secure" so it has to be something for paranoid people :)
Nice to hear one person's opinion. We'll definitely look into making the copy better. But we think it's an improvement from the old one and will go ahead with it with the intent to redo it later.
I do not really know how you can make it better. Perhaps with a very simply and small example between the two options. Something which can be explained in a short sentence or whatever (e.g. a small icon/image/symbol/animation/.gif etc., people like visual illustrations :). Like: If you choose option A and you visit a site which uses B (e.g. java script) this C could happened
I think an explanation and a visual are great ideas!
3. What means "Standard"? The first impression, only after reading the word "standard": TorBrowser=Firefox?
People were worried that "low" was the default setting, and asking why the default security level for a browser was "low." So we decided to switch it to standard, something, something else.
The second, after reading the explanation: What kind of features are enabled? What does that mean? Is it secure, now? Yes or no?
The word "features" is very nebulous. In particular if I read the item: Orfox + features enabled = sounds secure website + features enable = sounds insecure => Sounds inconsistent. "I'm confused!"
Agreed, but we were writing to a general audience that wouldn't understand more details than that. The point of the text is to communicate just enough information to make a setting decision, not to educate people. I'll make sure this isn't confusing or distracting, though.
Thanks for the feedback!
Bye, Carolin
Am Freitag, den 10.03.2017, 10:40 -0600 schrieb Linda Naeun Lee:
On 2017-03-09 17:26, Paul Syverson wrote:
Interesting, apologies if this is trivial/already-considered-and-bad/etc
No apologies! Thank you for your feedback.
How about settings with names something like Mostly Harmless Basic Minimal
We did iterate through the copy, but this is appreciated since the feedback says we should probably look into things more.
I like your suggestions because they don't associate safety with the settings (which isn't false, but it's not something that we can guarantee people). The more correct thing might be to tell them about the reduced functionality, with a hint to the fact that these measures might protect you.
Avoiding negative things (like things stop working and users don't know why) are much much much more important than including positive thing (like making them feel proactive about their security). The former loses users, the latter is a temporary high at best.
This avoids the direct statement of comparison in the name, so might preclude people avoiding a safer setting they might otherwise choose 'cause it sounds too paranoid. but still shold be clear what order they're in.
I agree. I actually like the progression of standard > something > basic. But that's only my opinion; don't know how users would feel.
(I was going to suggest "Safe" for the highest one, but cringe at ever actually saying that simpliciter. Plus I'm a big Douglas Adams fan. Actually I was also going to suggest "Undici" because, like Starbucks, we could name our largest size with the same big number regardless of whether that still corresponds to any units---except we've got security that goes to _eleven_. OK tired. Need to go home.)
Hmm! This inspires me to work on the copy again. Thanks!
Cheers, Linda
aloha, Paul
On Thu, Mar 09, 2017 at 04:57:54PM -0600, Linda Naeun Lee wrote:
Hi all:
The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9 Z87hg4XWW8O4uk/edit?usp=sharing
In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction.
Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.”
Cheers, Linda
P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecurit ySlider
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2 _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
-- Current Key: https://pgp.mit.edu/pks/lookup?search=lindanaeunlee GPG Fingerprint: FA0A C9BE 2881 B347 9F4F C0D7 BE70 F826 5ED2 8FA2 _______________________________________________ tor-project mailing list tor-project@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-project
+ Thomas :) On 3/9/17 17:57, Linda Naeun Lee wrote:
Hi all:
The results of the security slider usability testing is here: https://docs.google.com/document/d/1Wr4e9OftQaIyvU-p2pN9JcdLsOAl9Z87hg4XWW8O...
In short, users seemed to choose the setting that would be right for them, functionality wise, even if they didn’t have good security understanding or mild misconceptions. UI should account for multiple ways of interaction.
Some people said interesting things. Highlights include: -(the "safest" setting has bad connotations) P12: “I’m not sure, I don’t think I’ll be doing anything that would require that amount of safety. *giggles*” -(people making emotional decisions)P13: “I would probably choose the “safe” setting, there's the potential for more content being blocked on the safest setting, and I'm the kind of dum-dum who's willing to take my chances.” -(not understanding on-the-wire vs machine security defenses) P14: “I would choose the standard setting- I’m just going off of the experience I’ve had on the website I currently visit. I have Norton and feel like that keeps my computer pretty safe.”
Cheers, Linda
P.S.: I've been working on a more understandable security slider for a couple months now; documentation here: https://trac.torproject.org/projects/tor/wiki/doc/UX/OrfoxSecuritySlider
participants (5)
-
Arthur D. Edelstein -
Carolin Zöbelein -
isabela -
Linda Naeun Lee -
Paul Syverson