Canvas Breakage Ideas
Sanketh is tackling some fingerprinting patches; he's doing great. We ran across two questions we wanted input on. Background: Frequently sites break from the canvas permission prompt in the following way: User uploads image Website tries to display image back to user Image is white because it rendered the image to a canvas then tries to read the canvas data Examples: https://bugzilla.mozilla.org/show_bug.cgi?id=1631673 https://bugzilla.mozilla.org/show_bug.cgi?id=1456378 https://bugzilla.mozilla.org/show_bug.cgi?id=1573834 1) Sanketh had the idea of after granting permission, we show an additional prompt suggesting the user reload the page; since websites are not built to handle our 'well just try it again, it'l work this time' change to canvas APIs. Thoughts? 2) In https://bugzilla.mozilla.org/show_bug.cgi?id=1631673 Gijs had the idea of changing our behavior if the user has uploaded a file, and using this as a queue to automatically allow canvas extraction. Specifically he focused on allowing the website to read out the file the user has just uploaded; and that only. That would be ideal, but -with no testing and just hypothesizing - I doubt it would work because some as simple as e.g resizing the image would cause the match to fail and be dis-allowed. But we could test this. My idea was much simpler: if the user has uploaded a file, we take that as a queue they trust the service; and then grant the canvas permission prompt. (In as tightly as scoped a manner as possible, but the scoping is really just a bandaid over the problem....) Sanketh and Simon pointed out that this is dangerous: just because a user uploaded a file doesn't mean they consented to be fingerprinted. And they're right; if a user is trying to have an anonymous account or something similar, uploading a file is not a trusted relationship permitting fingerprinting. So the question then is, it seems like given Tor's strict stance, the only way this could be implemented was if the data read from the canvas was an exact match on the uploaded data. Is that accurate? If so, the next step would be to test these websites, because if they don't behave that way it's probably not worth implementing this at all. -tom
Hello, --[ With regard to (1) Created <https://bugzilla.mozilla.org/show_bug.cgi?id=1633813>. --[ With regard to (2) My beef was with allowing canvas extraction without prompt if the user had uploaded a file (as suggested here <https://bugzilla.mozilla.org/show_bug.cgi?id=1631673#c10>.) I am totally cool with showing a prompt if the user had uploaded a file to the website. tl;dr: I agree with Tom that uploading a file indicates trust but we seem to disagree on the specifics. I think canvas prompting is compatible with Tor. At worst, the user is shown a prompt for every website they upload a file to, which I think is a small number. At best, this makes a lot of websites more usable (with the user expressly consenting to be fingerprinted.) If we want to go down the route of only allowing extraction of user input, it might be, as Tom already hinted, technically challenging. We cannot just store a hash of the input, we would probably need to do something fancy like uncompress it and store a fuzzy hash of that. Best, Sanketh ?On 2020-04-28, 12:51 AM, "Tom Ritter" <tom@ritter.vg> wrote: Sanketh is tackling some fingerprinting patches; he's doing great. We ran across two questions we wanted input on. Background: Frequently sites break from the canvas permission prompt in the following way: User uploads image Website tries to display image back to user Image is white because it rendered the image to a canvas then tries to read the canvas data Examples: https://bugzilla.mozilla.org/show_bug.cgi?id=1631673 https://bugzilla.mozilla.org/show_bug.cgi?id=1456378 https://bugzilla.mozilla.org/show_bug.cgi?id=1573834 1) Sanketh had the idea of after granting permission, we show an additional prompt suggesting the user reload the page; since websites are not built to handle our 'well just try it again, it'l work this time' change to canvas APIs. Thoughts? 2) In https://bugzilla.mozilla.org/show_bug.cgi?id=1631673 Gijs had the idea of changing our behavior if the user has uploaded a file, and using this as a queue to automatically allow canvas extraction. Specifically he focused on allowing the website to read out the file the user has just uploaded; and that only. That would be ideal, but -with no testing and just hypothesizing - I doubt it would work because some as simple as e.g resizing the image would cause the match to fail and be dis-allowed. But we could test this. My idea was much simpler: if the user has uploaded a file, we take that as a queue they trust the service; and then grant the canvas permission prompt. (In as tightly as scoped a manner as possible, but the scoping is really just a bandaid over the problem....) Sanketh and Simon pointed out that this is dangerous: just because a user uploaded a file doesn't mean they consented to be fingerprinted. And they're right; if a user is trying to have an anonymous account or something similar, uploading a file is not a trusted relationship permitting fingerprinting. So the question then is, it seems like given Tor's strict stance, the only way this could be implemented was if the data read from the canvas was an exact match on the uploaded data. Is that accurate? If so, the next step would be to test these websites, because if they don't behave that way it's probably not worth implementing this at all. -tom
I'd also agree that uploading an image/file should not be taken as an implicit permission for reading the canvas. With respect to 1), I guess there are two parts: 1. Showing the canvas prompt if there was a file upload (recently?), even if `privacy.resistFingerprinting.autoDeclineNoUserInputCanvasPrompts = false`. 2. UI changes so that it is suggested that the user reloads the page for the changes to be effective. For the first, I'm not sure if the use case is so prevalent that it justifies having this edge case (show prompt if user has uploaded a file). But I think it might be ok in practice, and I don't see any drawbacks (like cases where users might be annoyed by this change). For the second, I guess this belongs more to UX people. In my personal opinion, I'm not sure the UI change is really justified, at least as described in https://bugzilla.mozilla.org/show_bug.cgi?id=1633813. I don't think it's true that a page reload is needed for the canvas permission changes to take effect (in general). For example, in the WhatsApp case described in that bug it's enough to reupload the file, without reloading the page. And reuploading the file is needed in any case, even if you reload the page. I just don't know if that's the suggestion that should be given to users in general after allowing canvas permissions. Besides, would we show the reload suggestion after all canvas accepts, or only those that were allowed thanks to the "file upload" exemption? --- With respect to 2), I think it's interesting, but I also don't know whether it's feasible in practice. Specifically, I was thinking of Gijs idea of trying to keep state about whether the canvas is safe to read or not, fingerprinting-wise. I assume that there is a (non-empty) subset of canvas write operations that are "fingerprinting-safe". Probably a bit naively, I'd like to think that `canvas.drawImage` is "fp-safe" (irrespective of the image source). But even if we have to check the image source, I think implementing this could potentially unbreak some of these common legit canvas use cases. For example, in the WhatsApp case mentioned above, I'm quite sure it's just used for image format conversion, since the bug does not occur when uploading "jpeg" images. So, that would be something like `canvas.drawImage(pngImage, 0, 0);` plus `canvas.toDataURL('image/jpeg');`, which should be covered if we implement the `canvas.drawImage` exemption when the image was uploaded by the user. This "fingerprinting-tainting" canvas logic might start with just the `drawImage` case, but perhaps it would be possible to extend little by little, if we know that some canvas write operation is safe and can help fixing breakage for legit use cases. Tom wrote:
So the question then is, it seems like given Tor's strict stance, the only way this could be implemented was if the data read from the canvas was an exact match on the uploaded data. Is that accurate? If so, the next step would be to test these websites, because if they don't behave that way it's probably not worth implementing this at all. Is it really needed to check that there is an exact match between the uploaded image and read canvas data? Even assuming `drawImage` differs between devices, *I think* it might be ok to allow canvas extraction if `drawImage` is performed with an image uploaded from a user. I don't see how the data could be used as an effective fingerprinting vector.
In any case, I think implementing this "fingerprinting-tainting" logic for canvas, starting with the `drawImage` case, may be interesting to pursue. This may also have privacy benefits for users in practice, since it can potentially avoid having to give full canvas access for cases where it's not needed. Of course, websites could always force canvas "fp-tainting", so that the only way to unbreak them is to allow canvas extraction, but that's a different story I think. Thanks, Alex
Alex: thanks for your thoughtful comments. --[ Response to comments on (1)
For the first, I'm not sure if the use case is so prevalent that it justifies having this edge case (show prompt if user has uploaded a file). But I think it might be ok in practice, and I don't see any drawbacks (like cases where users might be annoyed by this change).
I think a lot of people use WhatsApp Web (at least on Firefox) and, in the short term, this seems to be a nice way to unbreak it without turning off RFP or setting `privacy.resistFingerprinting.autoDeclineNoUserInputCanvasPrompts = false`.
Besides, would we show the reload suggestion after all canvas accepts, or only those that were allowed thanks to the "file upload" exemption?
I was thinking after all canvas accepts. But, first, you're right, one does not need to reload the page to be able to extract data from the canvas. Reloading is nice because it is a silver bullet and handles all edge cases where a request might already have been made. --[ Response to comments on (2) I really like the "fingerprinting-tainting" idea. Perhaps it would be worthwhile to implement this behind a default-off pref and conduct a study (and, of course, try to attack it.) Tom: What do you think? Best, Sanketh ?On 2020-04-29, 7:08 AM, "tbb-dev on behalf of Alex Catarineu" <tbb-dev-bounces@lists.torproject.org on behalf of acat@torproject.org> wrote: I'd also agree that uploading an image/file should not be taken as an implicit permission for reading the canvas. With respect to 1), I guess there are two parts: 1. Showing the canvas prompt if there was a file upload (recently?), even if `privacy.resistFingerprinting.autoDeclineNoUserInputCanvasPrompts = false`. 2. UI changes so that it is suggested that the user reloads the page for the changes to be effective. For the first, I'm not sure if the use case is so prevalent that it justifies having this edge case (show prompt if user has uploaded a file). But I think it might be ok in practice, and I don't see any drawbacks (like cases where users might be annoyed by this change). For the second, I guess this belongs more to UX people. In my personal opinion, I'm not sure the UI change is really justified, at least as described in https://bugzilla.mozilla.org/show_bug.cgi?id=1633813. I don't think it's true that a page reload is needed for the canvas permission changes to take effect (in general). For example, in the WhatsApp case described in that bug it's enough to reupload the file, without reloading the page. And reuploading the file is needed in any case, even if you reload the page. I just don't know if that's the suggestion that should be given to users in general after allowing canvas permissions. Besides, would we show the reload suggestion after all canvas accepts, or only those that were allowed thanks to the "file upload" exemption? --- With respect to 2), I think it's interesting, but I also don't know whether it's feasible in practice. Specifically, I was thinking of Gijs idea of trying to keep state about whether the canvas is safe to read or not, fingerprinting-wise. I assume that there is a (non-empty) subset of canvas write operations that are "fingerprinting-safe". Probably a bit naively, I'd like to think that `canvas.drawImage` is "fp-safe" (irrespective of the image source). But even if we have to check the image source, I think implementing this could potentially unbreak some of these common legit canvas use cases. For example, in the WhatsApp case mentioned above, I'm quite sure it's just used for image format conversion, since the bug does not occur when uploading "jpeg" images. So, that would be something like `canvas.drawImage(pngImage, 0, 0);` plus `canvas.toDataURL('image/jpeg');`, which should be covered if we implement the `canvas.drawImage` exemption when the image was uploaded by the user. This "fingerprinting-tainting" canvas logic might start with just the `drawImage` case, but perhaps it would be possible to extend little by little, if we know that some canvas write operation is safe and can help fixing breakage for legit use cases. Tom wrote: > So the question then is, it seems like given Tor's strict stance, the > only way this could be implemented was if the data read from the > canvas was an exact match on the uploaded data. Is that accurate? If > so, the next step would be to test these websites, because if they > don't behave that way it's probably not worth implementing this at > all. Is it really needed to check that there is an exact match between the uploaded image and read canvas data? Even assuming `drawImage` differs between devices, *I think* it might be ok to allow canvas extraction if `drawImage` is performed with an image uploaded from a user. I don't see how the data could be used as an effective fingerprinting vector. In any case, I think implementing this "fingerprinting-tainting" logic for canvas, starting with the `drawImage` case, may be interesting to pursue. This may also have privacy benefits for users in practice, since it can potentially avoid having to give full canvas access for cases where it's not needed. Of course, websites could always force canvas "fp-tainting", so that the only way to unbreak them is to allow canvas extraction, but that's a different story I think. Thanks, Alex _______________________________________________ tbb-dev mailing list tbb-dev@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tbb-dev
--[ Response to comments on (2)
I really like the "fingerprinting-tainting" idea. Perhaps it would be worthwhile to implement this behind a default-off pref and conduct a study (and, of course, try to attack it.)
Tom: What do you think?
I think the idea that the sites might be doing drawImage -> toDataURL is worth investigating. I think we should try to implement the most conservative choice that fixes these instances we know of; so I think the next step is to dig into what's happening on these sites: what's written to the canvas, and why/how they're extracting it. -tom
On Wed, Apr 29, 2020 at 01:07:40PM +0200, Alex Catarineu wrote:
With respect to 2), I think it's interesting, but I also don't know whether it's feasible in practice. Specifically, I was thinking of Gijs idea of trying to keep state about whether the canvas is safe to read or not, fingerprinting-wise. I assume that there is a (non-empty) subset of canvas write operations that are "fingerprinting-safe". Probably a bit naively, I'd like to think that `canvas.drawImage` is "fp-safe" (irrespective of the image source). But even if we have to check the image source, I think implementing this could potentially unbreak some of these common legit canvas use cases.
For example, in the WhatsApp case mentioned above, I'm quite sure it's just used for image format conversion, since the bug does not occur when uploading "jpeg" images. So, that would be something like `canvas.drawImage(pngImage, 0, 0);` plus `canvas.toDataURL('image/jpeg');`, which should be covered if we implement the `canvas.drawImage` exemption when the image was uploaded by the user. This "fingerprinting-tainting" canvas logic might start with just the `drawImage` case, but perhaps it would be possible to extend little by little, if we know that some canvas write operation is safe and can help fixing breakage for legit use cases.
I generally agree with your message, but I am curious about this idea. Are you saying that ctx.drawImage() is fingerprinting-safe, or are you saying that any "canvas extraction" from a canvas element initialized by ctx.drawImage is fingerprinting-safe? As far as I'm aware, drawImage() is not protected by the Canvas prompt (so that should never be a problem). If your comment was about "subsequent canvas extraction", then that is worth investigating. Are any of the conversions passed onto the GPU? Do we know if format conversation is deterministic?
participants (4)
-
Alex Catarineu -
Matthew Finkel -
Sanketh Menda -
Tom Ritter