Hello everyone! Today is the official "pencils down" date for Google Summer of Code, so I am going to outline the results of my work this Summer as a way of concluding my project. The first thing I had done at the start of my project was learn about Mozilla's XPCOM components and how they are used to interface with Gecko and other Firefox internals from extension code. I also learned about special files included with any extension such as install.rdf which are used to describe an extension's version information and signature-verifying public key, among other things. This lead me to understanding how extensions are updated and discover the kinds of limitations I would find in trying to update HTTPS Everywhere's rulesets library without changing the rest of the extension. The next thing I did was draft the specification for the format of the JSON manifest used to describe ruleset updates and explain how the process of updating the rulesets library would work. This work has since been built on and improved, and the existing specification document can be found in my fork of the extension, and is linked below[0]. This document describes everything one needs to know about the ruleset updater. It details: * How an object signing certificate can be generated using certutil from the NSS toolset. * How to create an update.json manifest file using the ruleset_update_manifest.py utility script I created.[1] * How to sign the update.json file using pk1sign and an object signing certificate. * How update information is fetched and how download failures are handled. * How the contents of update.json and the rulesets library database file are verified to be authentic. * What the significance of each field in update.json is. * Approximately how the ruleset updating mechanism functions in pseudocode. After the details of how updating would work were outlined in updateJSONSpec.md, I began implementing the ruleset updater module code for the extension. I also started working on a way to write tests for HTTPS Everywhere that I would initially use to test my code. I discovered that Firefox extensions could import XPCOM components exported by other extensions added to a user's browser, and so began work on a separate extension[2] that could be installed and used solely for testing HTTPS Everywhere. This work was soon improved upon by my mentor, Yan, who had the idea of creating a skeleton of an extension inside of HTTPS Everywhere's own codebase[3]. This extension used the more modern Addon SDK, which includes a unit testing suite, enabling HTTPS Everywhere developers to start writing proper unit tests that could be included in the HTTPS-Everywhere repository and run from the command line. While the unit testing environment I now had available was very effective, I had run into some surprising difficulty with Mozilla's nsIDataSignatureVerified XPCOM component, used to verify the signatures we were generating. By this point, Yan and I had agreed to try to compute the signature over the SHA256 hash of update.json's contents, so that this data could be more easily transcribed from a regular work computer to the airgapped machine EFF uses to house an offline signing key. It was at this point that we found that the key and signatures OpenSSL were generating were, somehow, incompatible with nsIDataSignatureVerifier, and Yan managed to get the verification unit test to pass by using certutil and pk1sign. The same problem of verifying signatures was encountered again when I got around to testing the actual ruleset updater code again, and had to generate new data, despite using certutil and pk1sign. Quite unfortunately, this problem drained almost four whole weeks of my time, as I had been looking into solutions involving porting a Firefox extension signing tool called Uhura (written in Perl), which meant also trying to understand what the tool was doing with ASN.1 specifications, among other things. Finally, I found that the nsIDataSignatureVerifier tool successfully verified the signature I had created for testing when it was not the hash of update.json's contents that were signed and supplied to the verifier, but the contents of the file themselves. The component is poorly documented and, after reaching out to developers in Mozilla's ecosystem, I found that it isn't very well understood in general. As such, even now I do not really understand why the verifier did not behave as expected in this scenario. Regardless, while it is somewhat less convenient here to sign update.json, the signature verification process works. Finally, I was able to move on to solve the problems of downloading and verifying the authenticity of the rulesets database file, and of reinitializing the extension's rulesets with those of the new database file. These problems were technically nuanced, but have been solved. The ruleset updater has been integrated into my fork of the extension's repository on github. My tests of its performance have passed, and I am pleased to say that it appears to be working. I have since submitted a pull request[4] to have my work reviewed and merged, and have even made changes requested by Yan, who has reviewed my work already. I'd like to conclude by giving my thanks to everyone in both of these mailing lists. I have received some invaluable support and advice that has had a profoundly positive impact on my experience. I learned a lot from everyone that lent a hand, and I feel the project has benefited greatly from each of the contributions I received to my efforts. My mentor, Yan, has also been a pleasure to work with. Our weekly meetings added very significant structure to my workflow, helped to keep me organized, and were also a great opportunity for me to pique her brain for ideas and possible solutions to some of the tougher problems I've encountered. I sincerely regard this as having been a positive experience, and I hope that everyone who has helped me until now is satisfied with my efforts and the actual results of my work. Thanks, everyone! [0] - https://github.com/redwire/https-everywhere/blob/master/doc/updateJSONSpec.m... [1] - https://github.com/redwire/https-everywhere/blob/master/utils/ruleset_update... [2] - https://github.com/redwire/https-everywhere-testing [3] - https://github.com/EFForg/https-everywhere/tree/master/https-everywhere-test... [4] - https://github.com/EFForg/https-everywhere/pull/437