[or-cvs] r11130: started tutorial, added hacked http/https proxy code (in topf/trunk: . config doc lib utils)

benedikt at seul.org benedikt at seul.org
Wed Aug 15 23:01:33 UTC 2007


Author: benedikt
Date: 2007-08-15 19:01:32 -0400 (Wed, 15 Aug 2007)
New Revision: 11130

Added:
   topf/trunk/config/
   topf/trunk/config/config.yml
   topf/trunk/doc/
   topf/trunk/doc/tutorial.tex
   topf/trunk/lib/cell.rb
   topf/trunk/utils/tor_http_proxy.rb
   topf/trunk/utils/tor_https_proxy.rb
Removed:
   topf/trunk/config.yml
Modified:
   topf/trunk/TODO
   topf/trunk/lib/dir.rb
   topf/trunk/lib/fuzz.rb
   topf/trunk/lib/topf.rb
   topf/trunk/tor-control-fuzz.rb
   topf/trunk/tor-dir-fuzz.rb
Log:
started tutorial, added hacked http/https proxy code

Modified: topf/trunk/TODO
===================================================================
--- topf/trunk/TODO	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/TODO	2007-08-15 23:01:32 UTC (rev 11130)
@@ -1,9 +1,12 @@
-- FIX BitStruct Rest-Field problem
+- FIX BitStruct Rest-Field problem => DONE
 
+IMPORTANT:
 - Fix pkcs1 signature
 - add more tests
-- implement other tor protocols 
+- implement other tor protocols => CONTROL STARTED; CELL TO GO!! 
 - calculate all possible permutations instead of random probing => DONE
+----------------------------------------------------------------------------
+LESS IMPORTANT: 
 - implement fancy web frontend
 - easy to install and setup testing network
 - distributed testing

Copied: topf/trunk/config/config.yml (from rev 11044, topf/trunk/config.yml)
===================================================================
--- topf/trunk/config/config.yml	                        (rev 0)
+++ topf/trunk/config/config.yml	2007-08-15 23:01:32 UTC (rev 11130)
@@ -0,0 +1,10 @@
+
+############################################################
+# This file holds information about our running tor-service
+############################################################
+
+DIRPORT: "2324"
+CONTROLPORT: "2323"
+HOST: "127.0.0.1" 
+KEYFILE: stuff/fuzz-private.pem
+DEBUG: false

Deleted: topf/trunk/config.yml
===================================================================
--- topf/trunk/config.yml	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/config.yml	2007-08-15 23:01:32 UTC (rev 11130)
@@ -1,10 +0,0 @@
-
-############################################################
-# This file holds information about our running tor-service
-############################################################
-
-DIRPORT: "2324"
-CONTROLPORT: "2323"
-HOST: "127.0.0.1" 
-KEYFILE: stuff/fuzz-private.pem
-DEBUG: true 

Added: topf/trunk/doc/tutorial.tex
===================================================================
--- topf/trunk/doc/tutorial.tex	                        (rev 0)
+++ topf/trunk/doc/tutorial.tex	2007-08-15 23:01:32 UTC (rev 11130)
@@ -0,0 +1,51 @@
+\documentclass[10pt,a4paper]{article}
+\usepackage[latin1]{inputenc}
+\usepackage{amsmath}
+\usepackage{amsfonts}
+\usepackage{amssymb}
+\author{Benedikt Boss}
+\title{Howto use (T)he (O)nion (P)rotocol (F)uzzer}
+\begin{document}
+\maketitle
+\newpage
+\tableofcontents
+\newpage
+\section{Introduction}
+T.O.P.F is a block-based fuzzing Framework developed to test the TOR protocol-suite.
+
+\section{Working with T.O.P.F}
+\subsection{Setting up a working Environment}
+T.O.P.F uses mainly Ruby Standart Librarys and supplies the rest through a typical subversion checkout so installing Ruby (Version 1.8) and checking out the latest T.O.P.F trunk should be enough to setup a working Test-Environment on the most Systems.
+
+\subsubsection{Checking out the current T.O.P.F trunk}
+Checking out T.O.P.F is as simple as starting a "svn co https://tor-svn.freehaven.net/svn/topf/trunk" on your command-shell of choice.
+
+\subsection{Implementing T.O.P.F Structures}
+T.O.P.F uses a modified version of the BitStruct Library to emulate a sort of c-like structures called fuzz-struct. A simple Example of such a structure implemented in Ruby looks like this:\\
+\begin{verbatim}
+class Example < BitStruct
+    text :example, 7
+    unsigned :version, 8
+    initial_value.example = "example"
+    initial_value.version   = 1
+end
+\end{verbatim}
+This creates a Class called "Example" with the contents of a 8*8Bit long String and a 8Bit unsigned integer. Through the initial\_value call default values for all entered fields can be set.
+For a more detailed description of all possible field-types please take a look at the FuzzStruct Reference.
+\subsection{Writing T.O.P.F Tests}
+T.O.P.F uses tests on a per-field type base. That means you are able to write Tests specific to a field-type the fuzz-struct library supplies (reference to the library reference).
+\subsection{Do the Fuzz!}
+
+\section{T.O.P.F Reference}
+\subsection{Fuzz}
+\subsubsection{Test-Cases}
+\subsubsection{Tests}
+\subsubsection{Connection}
+\subsubsection{Observer}
+\subsection{Dir}
+\subsection{Control}
+\subsection{Cell}
+\section{Fuzz-Struct Reference}
+\subsection{Types}
+
+\end{document}\documentclass[10pt]{article}
\ No newline at end of file

Added: topf/trunk/lib/cell.rb
===================================================================
--- topf/trunk/lib/cell.rb	                        (rev 0)
+++ topf/trunk/lib/cell.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -0,0 +1,83 @@
+module TOPF
+    module Cell
+        CELL_COMMANDS = [
+            [ 0, "" ],  # PADDING 
+            [ 1, "handshake challenge" ], # CREATE 
+            [ 2, "handshake response" ], # CREATED 
+            [ 3, "stop using a circuit" ], # RELAY 
+            [ 4, "creat a circuit not a pk" ], # DESTROY 
+            [ 5, "circuit created not a pk" ], # CREATE FAST 
+            [ 6, "" ] # CREATED FAST 
+        ]
+
+        class Packet < BitStruct
+            unsigned :circID, 16
+            unsigned :command, 16
+            rest     :payload
+        end
+
+        class Extended < BitStruct
+            hex_octets       :address, 32
+            unsigned    :port, 16
+            rest        :skin
+        end
+
+        ERRORS = {
+            0   => "NONE",
+            1   => "PROTOCOL",
+            2   => "INTERNAL",
+            3   => "REQUESTED",
+            4   => "HIBERNATING",
+            5   => "RESOURCELIMIT",
+            6   => "CONNECTFAILED",
+            7   => "OR_IDENTITY",
+            8   => "OR_CONN_CLOSED",
+            9   => "FINISHED",
+            10  => "TIMEOUT",
+            11  => "DESTROYED",
+            12  => "NOSUCHSERVICE"
+        }
+
+        class Relay < BitStruct
+            unsigned :command, 8
+            unsigned :recognized, 16
+            unsigned :streamID, 16
+            unsigned :digest, 32
+            unsigned :length, 16
+            rest     :data
+        end
+
+        RELAY_COMMANDS = [
+            [ 1, "RELAY_BEGIN" ],       # forward
+            [ 2, "RELAY_DATA" ],        # forward or backward
+            [ 3, "RELAY_END" ],         # forward or backward
+            [ 4, "RELAY_CONNECTED" ],   # backward
+            [ 5, "RELAY_SENDME" ],      # forward or backward
+            [ 6, "RELAY_EXTEND "],      # forward
+            [ 7, "RELAY_EXTENDED "],    # backward
+            [ 8, "RELAY_TRUNCATE" ],    # forward
+            [ 9, "RELAY_TRUNCATED" ],   # backward
+            [10, "RELAY_DROP" ],        # forward or backward
+            [11, "RELAY_RESOLVE" ],     # forward
+            [12, "RELAY_RESOLVED" ],    # backward
+            [13, "RELAY_BEGIN_DIR" ]    # forward
+        ]
+
+        CLOSING_REASONS = [
+            [ 1, "REASON_MISC" ],
+            [ 2, "REASON_RESOLVEFAILED" ],
+            [ 3, "REASON_CONNECTREFUSED" ],
+            [ 4, "REASON_EXITPOLICY" ],
+            [ 5, "REASON_DESTROY" ],
+            [ 6, "REASON_DONE" ],
+            [ 7, "REASON_TIMEOUT" ],
+            [ 8, "unallocated" ],
+            [ 9, "REASON_HIBERNATING" ],
+            [10, "REASON_INTERNAL" ],
+            [11, "REASON_RESOURCELIMIT" ],
+            [12, "REASON_CONNRESET" ],
+            [13, "REASON_TORPROTOCOL" ],
+            [14, "REASON_NOTDIRECTORY" ]
+        ]
+    end
+end 

Modified: topf/trunk/lib/dir.rb
===================================================================
--- topf/trunk/lib/dir.rb	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/lib/dir.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -22,9 +22,9 @@
 -----END RSA PUBLIC KEY-----}
 
         class RouterItem < BitStruct
-            char    :item, 6*8, :fuzzable => false 
-            char    :nickname, TOPF::Dir::DefaultNicknameLength*8 
-            octets  :address, 32
+            text    :item, 6, :fuzzable => false 
+            text    :nickname, TOPF::Dir::DefaultNicknameLength
+            hex_octets  :address, 32
             signed  :OrPort, 32
             signed  :SocksPort, 32
             signed  :DirPort, 32
@@ -38,15 +38,15 @@
         end
 
         class PublishedItem < BitStruct
-            char    :item, 9*8
-            char    :time, 19*8
+            text    :item, 9
+            text    :time, 19
 
             initial_value.item       = "published"
             initial_value.time          = "#{Date.today.to_s} #{ Time.now.strftime "%H:%M:%S"}" 
         end
 
         class OnionKeyItem < BitStruct
-            char    :item, 9*8
+            text    :item, 9
             rest    :publicKey      # key in PEM Format 
 
             initial_value.item       = "onion-key"
@@ -54,7 +54,7 @@
         end
 
         class SigningKeyItem < BitStruct
-            char    :item, 11*8
+            text    :item, 11
             rest    :publicKey      # key in PEM Format
 
             initial_value.item       = "signing-key"
@@ -62,7 +62,7 @@
         end
 
         class BandwidthItem < BitStruct
-            char    :item, 9*8
+            text    :item, 9
             signed  :avg,       128
             signed  :burst,     128
             signed  :observed,  128
@@ -74,7 +74,7 @@
         end
 
         class PlatformItem < BitStruct
-            char :item, 8*8
+            text :item, 8
             rest :string
 
             initial_value.item = "platform"
@@ -85,7 +85,7 @@
         end 
 
         class HibernateItem < BitStruct
-            char :item, 11*8
+            text :item, 11
             unsigned :state, 1
 
             initial_value.item   = "hibernating"
@@ -93,44 +93,44 @@
         end
 
         class UptimeItem < BitStruct
-            char :item, 6*8 
+            text :item, 6
             # FORMAT ?? 
             
             initial_value.item  = "uptime"
         end
 
         class ContactItem < BitStruct
-            char :item, 7*8 
+            text :item, 7
             rest :info
 
             initial_value.item  = "contact"
         end
 
         class FamilyItem < BitStruct
-            char :item, 6*8
+            text :item, 6
             rest :names
 
             initial_value.item = "family"
         end
 
         class ReadHistoryItem < BitStruct
-            char :item, 12*8 
-            char :time, 19*8
+            text :item, 12
+            text :time, 19
 
             initial_value.item = "read-history"
             initial_value.time          = "#{Date.today.to_s} #{ Time.now.strftime "%H:%M:%S"}" 
         end
 
         class WriteHistoryItem < BitStruct
-            char :item, 12*8
-            char :time, 19*8
+            text :item, 12
+            text :time, 19
 
             initial_value.item = "write-history"
             initial_value.time          = "#{Date.today.to_s} #{ Time.now.strftime "%H:%M:%S"}" 
         end
 
         class EventDnsItem < BitStruct
-            char :item, 8*8
+            text :item, 8
             unsigned :bool, 1
                 
             initial_value.item = "eventdns"

Modified: topf/trunk/lib/fuzz.rb
===================================================================
--- topf/trunk/lib/fuzz.rb	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/lib/fuzz.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -20,29 +20,26 @@
         end
     end
 
-    class Observer
-        def initialize
-        end
-    end
-
-    class BinaryObserver < Observer
+    class BinaryObserver 
         def initialize( programname, *arguments )
             @programname = programname
             @arguments = arguments.to_a
             @pid = nil
         end
         def observe!
-            Process.fork do 
-                @pid = Process.fork do
-                    exec("#{@programname} #{@arguments.join(" ")} ")
+            @pid = fork do 
+                trap("CLD") do
+                    pid = Process.wait
+                    puts "Child pid #{pid}: terminated"
+                    process_status = $?
                 end
-                Process.wait
-                process_status = $?
+                Fuzz::LOGGER.info "[x] starting #{@programname}"
+                system("#{@programname} #{@arguments.join(" ")} ")
+            end
                 #            Fuzz::LOGGER.debug "[!] %d coredumped? %s" % [@pid, process_status.coredump?]
                 #            Fuzz::LOGGER.debug "[!] %d get uncaught signal %d" % [@pid, process_status.termsig] if process_status.signaled?
                 #            Fuzz::LOGGER.debug "[!] %d stopped by signal %d" % [@pid, process_status.stopsig] if process_status.stopped?
                 #            Fuzz::LOGGER.debug "[!] %d exited %d" % [@pid, process_status.exitstatus] if process_status.exited?
-            end
         end
         def get_pid
             @pid
@@ -52,9 +49,6 @@
         end
     end
 
-    class ProcessObserver < Observer
-    end
-
     # This Class holds all tests to related to a given field-type
     class Tests
         attr_reader :tests, :permutations, :signed_tests, :char_tests, :unsigned_tests
@@ -247,18 +241,22 @@
             end
         end
 
-        def fuzz!(struct, join_character="", args={})
-            begin
-                while true
-                    data = struct.fuzz!.join( join_character )
-                    Fuzz::LOGGER.debug "sending data: %s" % data
-                    self.send data
-                    assert(args[:assert] )
+        def fuzz!(structs, join_character="", args={})
+            structs = [structs] if !structs.is_a? Array
+            structs.each do |struct|
+                begin
+                    while true
+                        data = struct.fuzz!.join( join_character )
+                        Fuzz::LOGGER.debug "sending data: %s" % data
+                        self.send data
+                        assert(args[:assert] )
+                    end
+                rescue Exception => exception
+                    Fuzz::LOGGER.debug "[!] ERROR: %s" % $!
+                    Fuzz::LOGGER.debug "[!] \n%s" % exception.backtrace.join("\n")
                 end
-            rescue Exception => exception
-                Fuzz::LOGGER.debug "[!] ERROR: %s" % $!
-                Fuzz::LOGGER.debug "[!] \n%s" % exception.backtrace.join("\n")
             end
+            raise "finished all tests"
         end
 
         # receives data from the socket and tests it against a supplied array of strings, values 
@@ -291,7 +289,7 @@
                         raise "not implemented yet"
                 end 
             rescue Exception
-                Fuzz::LOGGER.debug "[!] Exception: %s" % $!
+                Fuzz::LOGGER.info "[!] Exception: %s" % $!
                 exit 0
             end         
         end        
@@ -303,8 +301,8 @@
         DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  arg     } # Return string
         DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  ""      } # Return empty String 
         DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  "A"*arg.size    } # FAULTING ARGUMENT STRING
-        DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  "A"*Fuzz::MAX_RAND    } # LONG STRING
-        DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  "%n"*Fuzz::MAX_RAND   } # FORMAT STRING
+#        DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  "A"*Fuzz::MAX_RAND    } # LONG STRING
+#        DEFAULT_TESTS.register Fuzz::Test.new("char") {|arg, size|  "%n"*Fuzz::MAX_RAND   } # FORMAT STRING
 
         # Tests for signed numbers
         DEFAULT_TESTS.register Fuzz::Test.new("signed") {|arg, size|  0 } # return zero 

Modified: topf/trunk/lib/topf.rb
===================================================================
--- topf/trunk/lib/topf.rb	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/lib/topf.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -14,3 +14,4 @@
 
 require "dir"
 require "control"
+require "cell"

Modified: topf/trunk/tor-control-fuzz.rb
===================================================================
--- topf/trunk/tor-control-fuzz.rb	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/tor-control-fuzz.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -1,38 +1,40 @@
 require "lib/topf"
 
 begin
-    Fuzz::LOGGER.debug "[x] starting tor"
-    observer = Fuzz::BinaryObserver.new "tor", "-f torrc", "&> /dev/null"
-
+    observer = Fuzz::BinaryObserver.new "tor", "-f stuff/torrc", "&> /dev/null"
     observer.observe!
     sleep 2 
-    Fuzz::LOGGER.debug "[x] tor pid: %d" % observer.get_pid
-#    if !observer.get_pid
-#        started = false
-#        raise "Fork error"
-#    else
-#        started = true
-#    end
+    
+    Fuzz::LOGGER.info "[x] tor pid: %d" % observer.get_pid
+    if !observer.get_pid
+        started = false
+        raise "Fork error"
+    else
+        started = true
+    end
    
-    Fuzz::LOGGER.debug "[x] loading options"
+    Fuzz::LOGGER.info "[x] loading options"
 
-    config = YAML::load_file "config.yml"
+    config = YAML::load_file "config/config.yml"
 
     options = {
         :host => config["HOST"], 
         :port => config["CONTROLPORT"],
+        :debug => config["DEBUG"],
         :type => :tcp,
-        :debug => false
+        :timeout => 0.01,
+        :observer => observer
     }
 
-    Fuzz::LOGGER.debug "[x] connecting to control port"
+    Fuzz::LOGGER.info "[x] connecting to control port"
     fuzzer = Fuzz::Connection.new( options )
-    
-    Fuzz::LOGGER.debug "[x] sending authentication"
+
+    Fuzz::LOGGER.info "[x] sending authentication..."
     auth = TOPF::Control::AuthenticateItem.new
     
     fuzzer.send auth 
     fuzzer.assert ["250 OK"]
+    Fuzz::LOGGER.info "[x] authentication ok!"
     
     setconffuzz = TOPF::Control::ItemFuzz.new( TOPF::Control::SetConfItem.new, 
                                             Fuzz::DEFAULT_TESTS, 
@@ -51,18 +53,14 @@
                                             TOPF::Control::SETCONF_KEYS )
 
 
-    Fuzz::LOGGER.debug "[x] start fuzzing #{setconffuzz.class}"
+    Fuzz::LOGGER.info "[x] start fuzzing #{setconffuzz.class}"
 
-    fuzzer.fuzz!( setconffuzz, " ", { :append => "\r\n",
+    fuzzer.fuzz!( [ setconffuzz, resetconffuzz, getconffuzz ],  " ", { :append => "\r\n",
                                    :assert => TOPF::Control::SETCONF_REPLYS } )
-
-    fuzzer.close 
-
-    observer.exit
-rescue Exception => blah
-    Fuzz::LOGGER.debug "[!] ERROR: %s" % $!
-    Fuzz::LOGGER.debug "[!] \n%s" % blah.backtrace.join("\n")
-    Fuzz::LOGGER.debug "[!] closing everything down"
+    
+rescue Exception => error
+    #Fuzz::LOGGER.info "[x] %s\n%s\nclosing everything down" % [ $!, error.backtrace.join("\n") ]
+    Fuzz::LOGGER.info "[x] %s\nclosing everything down" % [ $! ]
     fuzzer.close if fuzzer
     observer.exit if observer and started
 end

Modified: topf/trunk/tor-dir-fuzz.rb
===================================================================
--- topf/trunk/tor-dir-fuzz.rb	2007-08-15 21:53:34 UTC (rev 11129)
+++ topf/trunk/tor-dir-fuzz.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -1,8 +1,13 @@
 require "lib/topf"
 
 begin
+    observer = Fuzz::BinaryObserver.new "tor", "-f torrc", "&> /dev/null"
+    observer.observe!
+    sleep 2
+
+
     config = YAML::load_file "config.yml"
-
+    
     rd = TOPF::Dir::RouterDescriptor.new( config["KEYFILE"])
     
     options = {
@@ -15,7 +20,8 @@
         :debug => config["DEBUG"] 
     }
 
-    fuzzer = Fuzz::Host.new( options ) 
+    fuzzer = Fuzz::Connection.new( options ) 
     fuzzer.fuzz!
-    fuzzer.stop
+    fuzzer.close
+    observer.exit
 end

Added: topf/trunk/utils/tor_http_proxy.rb
===================================================================
--- topf/trunk/utils/tor_http_proxy.rb	                        (rev 0)
+++ topf/trunk/utils/tor_http_proxy.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -0,0 +1,41 @@
+require 'socket'
+require 'net/http'
+
+
+begin
+    ARGV[0] ? port = ARGV[0] : port = 2343
+
+    default_connection_port = 80
+
+    server = TCPServer.new('localhost', port)
+
+    while (session = server.accept)
+        request = session.gets.split(" ")
+        request[1] =~ /(\d+\.\d+\.\d+\.\d+)+(:\d+)?([\/\w\.\-\_]+)+/
+        address = $1
+        url     = $3
+        $2 ? parsed_port    = $2.gsub(":", "") : parsed_port = default_connection_port
+       
+        pp [address, parsed_port, url]
+
+        begin
+            case request[0].to_s
+
+            when "GET"
+                response = Net::HTTP.start(address,parsed_port) do |http|
+                    http.get(url)
+                end
+            when "POST"
+                response = Net::HTTP.start(address, parsed_port) do |http|
+                    http.post(url)
+                end
+            end
+        rescue Exception => error
+            puts error
+            puts error.backtrace
+        end
+
+        session.write response
+        session.close
+    end
+end

Added: topf/trunk/utils/tor_https_proxy.rb
===================================================================
--- topf/trunk/utils/tor_https_proxy.rb	                        (rev 0)
+++ topf/trunk/utils/tor_https_proxy.rb	2007-08-15 23:01:32 UTC (rev 11130)
@@ -0,0 +1,32 @@
+require 'socket'
+require 'net/http'
+require '../lib/topf'
+
+begin
+    ARGV[0] ? port = ARGV[0] : port = 2343
+
+    default_connection_port = 80
+
+    server = TCPServer.new('localhost', port)
+
+    while (session = server.accept)
+        request = session.gets.split(" ")
+        pp request
+
+        request[1] =~ /(\d+\.\d+\.\d+\.\d+)+(:\d+)+/
+        address = $1
+        $2 ? parsed_port    = $2.gsub(":", "") : parsed_port = default_connection_port
+        pp [address, parsed_port]
+        pp session.gets
+        
+        session.write "HTTP/1.0 200 Connection established\r\nProxy-agent: topf-proxy" + "\r\n"*2
+
+        data = session.gets
+        puts "DATA:"
+        pp data
+        puts "PARSED:"
+        pp BitStruct.parse(data, TOPF::Cell::Packet)
+
+        session.close
+    end
+end



More information about the tor-commits mailing list