July 2014 - tor-commits - lists.torproject.org

[obfsproxy/master] Improve packet morphing algorithm.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 0806a87f0183b3b74d20fe99184bdfb361d5f443 Author: Philipp Winter <phw(a)torproject.org> Date: Mon Mar 10 18:47:15 2014 +0100 Improve packet morphing algorithm. We only want to run the packet morphing algorithm when we really need it -- which is immediately before we send data. Previously, we would morph immediately upon receiving data which is not optimal. This should fix <https://bugs.torproject.org/10991>. Thanks to Yawning Angel who pointed out the problem. --- obfsproxy/transports/scramblesuit/packetmorpher.py | 37 +++++++++++++++++--- obfsproxy/transports/scramblesuit/scramblesuit.py | 31 +++++++--------- 2 files changed, 44 insertions(+), 24 deletions(-) diff --git a/obfsproxy/transports/scramblesuit/packetmorpher.py b/obfsproxy/transports/scramblesuit/packetmorpher.py index 4a72f0b..081d7dc 100644 --- a/obfsproxy/transports/scramblesuit/packetmorpher.py +++ b/obfsproxy/transports/scramblesuit/packetmorpher.py @@ -8,6 +8,7 @@ morphed network data should then match the probability distribution. import random +import message import probdist import const @@ -39,14 +40,37 @@ class PacketMorpher( object ): self.dist = probdist.new(lambda: random.randint(const.HDR_LENGTH, const.MTU)) + def getPadding( self, sendCrypter, sendHMAC, dataLen ): + """ + Based on the burst's size, return a ready-to-send padding blurb. + """ + + padLen = self.calcPadding(dataLen) + + assert const.HDR_LENGTH <= padLen < (const.MTU + const.HDR_LENGTH), \ + "Invalid padding length %d." % padLen + + # We have to use two padding messages if the padding is > MTU. + if padLen > const.MTU: + padMsgs = [message.new("", paddingLen=700 - const.HDR_LENGTH), + message.new("", paddingLen=padLen - 700 - \ + const.HDR_LENGTH)] + else: + padMsgs = [message.new("", paddingLen=padLen - const.HDR_LENGTH)] + + blurbs = [msg.encryptAndHMAC(sendCrypter, sendHMAC) for msg in padMsgs] + + return "".join(blurbs) + def calcPadding( self, dataLen ): """ - Based on `dataLen', determines the padding for a network packet. + Based on `dataLen', determine and return a burst's padding. - ScrambleSuit morphs packets which are smaller than the link's MTU. - This method draws a random sample from the probability distribution - which is used to determine and return the padding for such packets. - This effectively gets rid of Tor's 586-byte signature. + ScrambleSuit morphs the last packet in a burst, i.e., packets which + don't fill the link's MTU. This is done by drawing a random sample + from our probability distribution which is used to determine and return + the padding for such packets. This effectively gets rid of Tor's + 586-byte signature. """ # The `is' and `should-be' length of the burst's last packet. @@ -59,6 +83,9 @@ class PacketMorpher( object ): else: padLen = (const.MTU - dataLen) + sampleLen + if padLen < const.HDR_LENGTH: + padLen += const.MTU + log.debug("Morphing the last %d-byte packet to %d bytes by adding %d " "bytes of padding." % (dataLen % const.MTU, sampleLen, padLen)) diff --git a/obfsproxy/transports/scramblesuit/scramblesuit.py b/obfsproxy/transports/scramblesuit/scramblesuit.py index 8e04cc5..43b52b5 100644 --- a/obfsproxy/transports/scramblesuit/scramblesuit.py +++ b/obfsproxy/transports/scramblesuit/scramblesuit.py @@ -234,27 +234,12 @@ class ScrambleSuitTransport( base.BaseTransport ): # Wrap the application's data in ScrambleSuit protocol messages. messages = message.createProtocolMessages(data, flags=flags) - - # Let the packet morpher tell us how much we should pad. - paddingLen = self.pktMorpher.calcPadding(sum([len(msg) for - msg in messages])) - - # If padding > header length, a single message will do... - if paddingLen > const.HDR_LENGTH: - messages.append(message.new("", paddingLen=paddingLen - - const.HDR_LENGTH)) - - # ...otherwise, we use two padding-only messages. - else: - messages.append(message.new("", paddingLen=const.MPU - - const.HDR_LENGTH)) - messages.append(message.new("", paddingLen=paddingLen)) - blurb = "".join([msg.encryptAndHMAC(self.sendCrypter, self.sendHMAC) for msg in messages]) - # Flush data chunk for chunk to obfuscate inter arrival times. + # Flush data chunk for chunk to obfuscate inter-arrival times. if const.USE_IAT_OBFUSCATION: + if len(self.choppingBuf) == 0: self.choppingBuf.write(blurb) reactor.callLater(self.iatMorpher.randomSample(), @@ -262,8 +247,12 @@ class ScrambleSuitTransport( base.BaseTransport ): else: # flushPieces() is still busy processing the chopping buffer. self.choppingBuf.write(blurb) + else: - self.circuit.downstream.write(blurb) + padBlurb = self.pktMorpher.getPadding(self.sendCrypter, + self.sendHMAC, + len(blurb)) + self.circuit.downstream.write(blurb + padBlurb) def flushPieces( self ): """ @@ -283,7 +272,11 @@ class ScrambleSuitTransport( base.BaseTransport ): # Drain and send whatever is left in the output buffer. else: - self.circuit.downstream.write(self.choppingBuf.read()) + blurb = self.choppingBuf.read() + padBlurb = self.pktMorpher.getPadding(self.sendCrypter, + self.sendHMAC, + len(blurb)) + self.circuit.downstream.write(blurb + padBlurb) return reactor.callLater(self.iatMorpher.randomSample(), self.flushPieces)

1 0

[obfsproxy/master] Add ChangeLog entry about scramblesuit spec improvements.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 94089c08ddb9f3b050c2b3c3eeb80c451c5bb79d Author: George Kadianakis <desnacked(a)riseup.net> Date: Mon Apr 14 15:10:35 2014 +0300 Add ChangeLog entry about scramblesuit spec improvements. Conflicts: ChangeLog --- ChangeLog | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/ChangeLog b/ChangeLog index 920df42..76cdfc7 100644 --- a/ChangeLog +++ b/ChangeLog @@ -2,6 +2,11 @@ Changes in version 0.2.11 - UNRELEASED: - Fix bug to prevent denial-of-service attacks targeting the authentication mechanism. Fixes #11092. The problem was pointed out by Yawning Angel. + - Fix bug to prevent denial-of-service attacks targeting the + authentication mechanism. Fixes #11092. The problem was pointed + out by Yawning Angel. + - Improve scramblesuit spec and fix several issues. Fixes bug #10893. + Thanks to Yawning Angel. Changes in version 0.2.10 - 2014-06-05:

1 0

[obfsproxy/master] Add ChangeLog entry about scramblesuit's packetmorpher improvements.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 29781d77f63279fcff84aba9240d6ff0b1ca1eb9 Author: George Kadianakis <desnacked(a)riseup.net> Date: Mon Apr 14 15:15:22 2014 +0300 Add ChangeLog entry about scramblesuit's packetmorpher improvements. --- ChangeLog | 2 ++ 1 file changed, 2 insertions(+) diff --git a/ChangeLog b/ChangeLog index 76cdfc7..6615967 100644 --- a/ChangeLog +++ b/ChangeLog @@ -7,6 +7,8 @@ Changes in version 0.2.11 - UNRELEASED: out by Yawning Angel. - Improve scramblesuit spec and fix several issues. Fixes bug #10893. Thanks to Yawning Angel. + - Improve scramblesuit's packet morphing algorithm. Fixes bug #10991. + Thanks to Yawning Angel. Changes in version 0.2.10 - 2014-06-05:

1 0

[obfsproxy/master] Use more readable error messages.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit b498fe6feffda4a4d73112a099ecc9cdc134ffb6 Author: Philipp Winter <phw(a)torproject.org> Date: Sun Mar 9 00:45:27 2014 +0100 Use more readable error messages. To a user, it's not immediately clear that a given password was not well formed. The refactored error messages should fix that. See bug <https://bugs.torproject.org/11143> for context. --- obfsproxy/transports/scramblesuit/scramblesuit.py | 28 +++++++++++++++------ 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/obfsproxy/transports/scramblesuit/scramblesuit.py b/obfsproxy/transports/scramblesuit/scramblesuit.py index 87a7370..8e04cc5 100644 --- a/obfsproxy/transports/scramblesuit/scramblesuit.py +++ b/obfsproxy/transports/scramblesuit/scramblesuit.py @@ -117,8 +117,16 @@ class ScrambleSuitTransport( base.BaseTransport ): if cls.weAreServer and not cls.weAreExternal: cfg = transportConfig.getServerTransportOptions() if cfg and "password" in cfg: - cls.uniformDHSecret = base64.b32decode(util.sanitiseBase32( - cfg["password"])) + try: + cls.uniformDHSecret = base64.b32decode(util.sanitiseBase32( + cfg["password"])) + except TypeError as error: + log.error(error.message) + raise base.PluggableTransportError("Given password '%s' " \ + "is not valid Base32! Run " \ + "'generate_password.py' to generate a good " \ + "password." % cfg["password"]) + cls.uniformDHSecret = cls.uniformDHSecret.strip() @classmethod @@ -562,9 +570,9 @@ class ScrambleSuitTransport( base.BaseTransport ): args.uniformDHSecret)) except (TypeError, AttributeError) as error: log.error(error.message) - raise base.PluggableTransportError( - "UniformDH password '%s' isn't valid base32!" - % args.uniformDHSecret) + raise base.PluggableTransportError("Given password '%s' is not " \ + "valid Base32! Run 'generate_password.py' to generate " \ + "a good password." % args.uniformDHSecret) parentalApproval = super( ScrambleSuitTransport, cls).validate_external_mode_cli(args) @@ -611,8 +619,14 @@ class ScrambleSuitTransport( base.BaseTransport ): log.warning("A UniformDH password was already specified over " "the command line. Using the SOCKS secret instead.") - self.uniformDHSecret = base64.b32decode(util.sanitiseBase32( - args[0].split('=')[1].strip())) + try: + self.uniformDHSecret = base64.b32decode(util.sanitiseBase32( + args[0].split('=')[1].strip())) + except TypeError as error: + log.error(error.message) + raise base.PluggableTransportError("Given password '%s' is not " \ + "valid Base32! Run 'generate_password.py' to generate " \ + "a good password." % args[0].split('=')[1].strip()) rawLength = len(self.uniformDHSecret) if rawLength != const.SHARED_SECRET_LENGTH:

1 0

[obfsproxy/master] Add PacketMorpher unittests.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 55fa261c83174b3e241241cf7b66ed3f629c7f88 Author: George Kadianakis <desnacked(a)riseup.net> Date: Mon Apr 14 15:14:14 2014 +0300 Add PacketMorpher unittests. --- obfsproxy/test/transports/test_scramblesuit.py | 36 ++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/obfsproxy/test/transports/test_scramblesuit.py b/obfsproxy/test/transports/test_scramblesuit.py index 3392b5f..8481fab 100644 --- a/obfsproxy/test/transports/test_scramblesuit.py +++ b/obfsproxy/test/transports/test_scramblesuit.py @@ -21,6 +21,8 @@ import obfsproxy.transports.scramblesuit.scramblesuit as scramblesuit import obfsproxy.transports.scramblesuit.message as message import obfsproxy.transports.scramblesuit.state as state import obfsproxy.transports.scramblesuit.ticket as ticket +import obfsproxy.transports.scramblesuit.packetmorpher as packetmorpher +import obfsproxy.transports.scramblesuit.probdist as probdist # Disable all logging as it would yield plenty of warning and error @@ -418,5 +420,39 @@ class TicketTest( unittest.TestCase ): else: self.assertTrue(ss.receiveTicket(buf)) +class PacketMorpher( unittest.TestCase ): + + def test1_calcPadding( self ): + + def checkDistribution( dist ): + pm = packetmorpher.new(dist) + for i in xrange(0, const.MTU + 2): + padLen = pm.calcPadding(i) + self.assertTrue(const.HDR_LENGTH <= \ + padLen < \ + (const.MTU + const.HDR_LENGTH)) + + # Test randomly generated distributions. + for i in xrange(0, 100): + checkDistribution(None) + + # Test border-case distributions. + checkDistribution(probdist.new(lambda: 0)) + checkDistribution(probdist.new(lambda: 1)) + checkDistribution(probdist.new(lambda: const.MTU)) + checkDistribution(probdist.new(lambda: const.MTU + 1)) + + def test2_getPadding( self ): + pm = packetmorpher.new() + sendCrypter = mycrypto.PayloadCrypter() + sendCrypter.setSessionKey("A" * 32, "A" * 8) + sendHMAC = "A" * 32 + + for i in xrange(0, const.MTU + 2): + padLen = len(pm.getPadding(sendCrypter, sendHMAC, i)) + self.assertTrue(const.HDR_LENGTH <= padLen < const.MTU + \ + const.HDR_LENGTH) + + if __name__ == '__main__': unittest.main()

1 0

[obfsproxy/master] Fix problems in UniformDH spec.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 18db73dd4d3f5a3a35019474dc5e53b1ef2f14c3 Author: Philipp Winter <phw(a)torproject.org> Date: Sun Mar 2 00:01:03 2014 +0100 Fix problems in UniformDH spec. 1. Fix a typo by changing "X" to "Y". 2. Add "M_C" and "M_S" to the respective MAC function as they are also authenticated. The problem was pointed out by Yawning Angel in: <https://bugs.torproject.org/10893> --- doc/scramblesuit/scramblesuit-spec.txt | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/doc/scramblesuit/scramblesuit-spec.txt b/doc/scramblesuit/scramblesuit-spec.txt index 66c1ac8..45a35f5 100644 --- a/doc/scramblesuit/scramblesuit-spec.txt +++ b/doc/scramblesuit/scramblesuit-spec.txt @@ -80,15 +80,15 @@ 256-bit master key k_t. Session keys are then derived from k_t as discussed in Section 2.3. - Client Server Legend: - | X | P_C | M_C | MAC(X | P_C | E) | X: client public key - | ---------------------------------> | Y: server public key - | Y | P_S | M_S | MAC(X | P_S | E) | P_{C,S}: padding - | <--------------------------------- | M_{C,S}: mark to locate the MAC - | AEnc(k_t+1 | T_t+1) | E: approximate timestamp - | <--------------------------------- | k_t+1: future master key - | AEnc(Tor traffic) | T_t+1: future ticket - | <--------------------------------> | + Client Server Legend: + | X | P_C | M_C | MAC(X | P_C | M_C | E) | X: client public key + | ---------------------------------------> | Y: server public key + | Y | P_S | M_S | MAC(Y | P_S | M_S | E) | P_{C,S}: padding + | <--------------------------------------- | M_{C,S}: mark to locate MAC + | AEnc(k_t+1 | T_t+1) | E: approximate timestamp + | <--------------------------------------- | k_t+1: future master key + | AEnc(Tor traffic) | T_t+1: future ticket + | <--------------------------------------> | Immediately after the handshake succeeded, the server proceeds to issue and send a new session ticket T_t+1 together with the according master key

1 0

[obfsproxy/master] Close connection if authentication fails.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit d16d4ade7504e83eef6e7f34c7da827a8a76c80a Author: Philipp Winter <phw(a)torproject.org> Date: Sat Mar 1 23:29:04 2014 +0100 Close connection if authentication fails. Adapt the server's behaviour to the spec change of commit 892846eb. When a server bootstraps for the first time, it randomly generates a byte threshold after which unauthenticated connections are closed. This should fix <https://bugs.torproject.org/11092>. --- obfsproxy/transports/scramblesuit/scramblesuit.py | 7 +++++++ obfsproxy/transports/scramblesuit/state.py | 9 +++++++++ 2 files changed, 16 insertions(+) diff --git a/obfsproxy/transports/scramblesuit/scramblesuit.py b/obfsproxy/transports/scramblesuit/scramblesuit.py index 6279925..da2a32d 100644 --- a/obfsproxy/transports/scramblesuit/scramblesuit.py +++ b/obfsproxy/transports/scramblesuit/scramblesuit.py @@ -485,6 +485,13 @@ class ScrambleSuitTransport( base.BaseTransport ): self.sendTicketAndSeed() else: + if len(data) > self.srvState.closingThreshold: + log.info("Terminating connection after having received %d" + " bytes because client could not " + "authenticate." % len(data)) + self.circuit.close() + return + log.debug("Authentication unsuccessful so far. " "Waiting for more data.") return diff --git a/obfsproxy/transports/scramblesuit/state.py b/obfsproxy/transports/scramblesuit/state.py index 9d32d0c..384db97 100644 --- a/obfsproxy/transports/scramblesuit/state.py +++ b/obfsproxy/transports/scramblesuit/state.py @@ -76,6 +76,7 @@ class State( object ): self.pktDist = None self.iatDist = None self.fallbackPassword = None + self.closingThreshold = None def genState( self ): """ @@ -112,6 +113,14 @@ class State( object ): # did not set `ServerTransportOptions'. self.fallbackPassword = os.urandom(const.SHARED_SECRET_LENGTH) + # Unauthenticated connections are closed after having received the + # following amount of bytes. + max_handshake_len = const.MAX_PADDING_LENGTH + \ + const.MARK_LENGTH + \ + const.HMAC_SHA256_128_LENGTH + self.closingThreshold = prng.randint(max_handshake_len, + max_handshake_len * 2) + self.writeState() def isReplayed( self, hmac ):

1 0

[obfsproxy/master] Elaborate on server's behaviour.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 081e943987fad482786b1fa6fe3653672b22227d Author: Philipp Winter <phw(a)torproject.org> Date: Sat Mar 1 22:14:18 2014 +0100 Elaborate on server's behaviour. In particular, discuss strategies which servers should implement if clients cannot authenticate. Otherwise, malicious clients could run computation-based or memory-based denial-of-service attacks. This problem was reported by Yawning Angel. This should fix <https://bugs.torproject.org/11092>. --- doc/scramblesuit/scramblesuit-spec.txt | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/doc/scramblesuit/scramblesuit-spec.txt b/doc/scramblesuit/scramblesuit-spec.txt index de6b778..1de6263 100644 --- a/doc/scramblesuit/scramblesuit-spec.txt +++ b/doc/scramblesuit/scramblesuit-spec.txt @@ -35,10 +35,13 @@ secret should thwart active probing attacks. As stated in the research paper [1], a server only replies to a client if - the client can prove knowledge of the shared secret. If a client sends - data which lacks this knowledge, the server MUST NOT reply. It MAY - terminate the connection if the client could not prove knowledge of the - shared secret after a timeout has passed. + the client can prove knowledge of the shared secret. As long as clients + cannot prove knowledge of the shared secret, servers MUST NOT reply. If + authentication did not succeed after 1532 bytes have been received, the + server SHOULD stop processing incoming data to prevent denial-of-service + attacks. The server MAY close the TCP connection. Alternatively, the + server MAY proceed to accept data but it SHOULD stop buffering or + processing the data, thus effectively ignoring the client. 2.1 UniformDH Handshake

1 0

[obfsproxy/master] Add ChangeLog entry for #11092.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 1367b4979c75001fbb91f972ef673d819e2f588a Author: George Kadianakis <desnacked(a)riseup.net> Date: Mon Apr 14 14:47:20 2014 +0300 Add ChangeLog entry for #11092. Conflicts: ChangeLog --- ChangeLog | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/ChangeLog b/ChangeLog index 86c540e..920df42 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,9 @@ +Changes in version 0.2.11 - UNRELEASED: + - Fix bug to prevent denial-of-service attacks targeting the + authentication mechanism. Fixes #11092. The problem was pointed + out by Yawning Angel. + + Changes in version 0.2.10 - 2014-06-05: - Don't set the transport's circuit to None if we are closing a circuit; it was causing problems in obfs3. Fixes #11820. @@ -7,6 +13,7 @@ Changes in version 0.2.10 - 2014-06-05: printing it for every connection). Fixes #11768. - Log the pyptlib version in startup. Patch by John Giannelos. Fixes #9878. + Changes in version 0.2.9 - 2014-05-01: - Support connecting over a HTTPS CONNECT proxy. Patch by Yawning Angel. Fixes #11409.

1 0

[obfsproxy/master] Fix typo.
by asn＠torproject.org 15 Jul '14

15 Jul '14

commit 01eca9d516b2d4a8b47931d585f2b74417f058dd Author: Philipp Winter <phw(a)torproject.org> Date: Sat Mar 1 23:52:16 2014 +0100 Fix typo. The problem was pointed out by Yawning Angel in: <https://bugs.torproject.org/10893#comment:3> --- doc/scramblesuit/scramblesuit-spec.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/scramblesuit/scramblesuit-spec.txt b/doc/scramblesuit/scramblesuit-spec.txt index 1de6263..66c1ac8 100644 --- a/doc/scramblesuit/scramblesuit-spec.txt +++ b/doc/scramblesuit/scramblesuit-spec.txt @@ -191,7 +191,7 @@ padding which is used for packet length obfuscation. Note that both fields can be set to 0 which results in an empty protocol message. ScrambleSuit's maximum message length is 1448 bytes. Exluding the header, this results in - 1437 bytes for the transported data. + 1427 bytes for the transported data. The 1-byte flag field is used for protocol signalling. Below, all defined flags along with their semantics are explained.

1 0