[or-cvs] r10466: Added Initial SoC Plan document (in libevent-urz/trunk: . doc)

[Urz at seul.org]

Author: Urz
Date: 2007-06-03 00:12:30 -0400 (Sun, 03 Jun 2007)
New Revision: 10466

Added:
   libevent-urz/trunk/doc/
   libevent-urz/trunk/doc/plan.txt
Log:
Added Initial SoC Plan document

Added: libevent-urz/trunk/doc/plan.txt
===================================================================
--- libevent-urz/trunk/doc/plan.txt	                        (rev 0)
+++ libevent-urz/trunk/doc/plan.txt	2007-06-03 04:12:30 UTC (rev 10466)
@@ -0,0 +1,133 @@
+Basic Plan:
+
+1) IOCP / Sockets dump data into evbuffers.
+2) evbuffer onread/onwrite call pushes data through a socketpair to let
+    the main thread know that data has arrived.
+3) main thread calls the real callback for the evbuffer (conn_[read|write]_callback)
+4) data which needs to be written back to the network is written to an evbuffer
+5) the evbuffer callback (a normal one, not a main-loop one) 
+    writes the data to the network in a network-api dependant way
+
+Things that need to be done / Schedule
+//testable, extensible, integratable and documented
+
+/////////////// Item 1: //////////////////////////
+Do 2) First.
+Justification:
+
+1 global socketpair, who has 1 read event which polls all evbuffers.
+evbuffers need to be made threadsafe...
+
+
+It it more generally useful than 1) or 3), and can stand alone better.
+
+Coding:
+Alter evbuffer (event.h:212) struct so it has a socketpair.
+Alter evbuffer new/free routines (buffer.c:61/71)to initialize/close socketpair.
+(On free/close we also need to remove the events for our socketpair)
+Add a new event_add / event_del etc which sets the read event for the event_loop half of the socketpair
+(so the user doesn't have to know about the socketpair trick).
+Write a new, simple function to be the evbuffer callback. This function needs to
+push something down the pipe to let the main-loop know an event has occured.
+('R'/'W')
+Write a short read event handler for the event_loop half of the socketpair which
+simply reads all there is to be read (no blocking!) and calls the real read/write callbacks
+depending on what it reads.
+
+Nick...
+I know this is more complex than what we talked about, but we can't really check the write
+part of a socketpair with select. Originally my plan was to only check the read part.
+This seems to be OK from our perspective, until we get to parts 4)
+and 5), because tor's main event loop wants to know when data is ready to be written.
+
+Testing:
+Add test cases to libevent testing code... test/regress.c I believe
+
+Extensibility:
+This is pretty generalized already, but we should add a sample to demonstrate it's usage
+(I'm thinking 2 threads, one writes to an evbuffer and the other has a main loop which
+checks for events on the evbuffer and does some other work)
+
+Documentation:
+Should be well commented, but the man pages probably need altering to explain the new api.
+
+Schedule:
+Should be done by 10th June.
+I have 3 big assignments due 30th May, 31st May, and 1st June, but I would still like to
+have most of it done by the 3rd of June.
+
+/////////////// Item 2: ///////////////////////////
+Do 1) Second
+Justification:
+With 2) done it is probably more useful to other projects than anything else, although
+not by much.
+
+Coding:
+Two equivalent sets of code need to be created:
+A *nix one, which uses select (in another thread) and evbuffer_read()
+(allowing evbuffer_read to figgure out how much data to pull with ioctl) and
+A Win32 one which uses IOCP and a thread pool to put data in the evbuffers with
+evbuffer_add().
+
+Testing:
+A simple, cross platform program which uses evbuffers to (eg) read data in from sockets
+and dumps it to screen.
+
+Extensibility:
+This code is not extensible in itself, but are backends to an interface. So long as the
+requirements are well documented, someone could extend this interface to add other
+network apis to it, or to add other stream-like things.
+
+Documentation:
+thourough internal documentation, plus a short requirements/spec document for creating
+new backends.
+
+Schedule:
+This is not a big section, but I have 4 exams shortly after the 10th, (12th, 18th, 18th, 20th)
+so I will say the 1st July.
+(Note: First evaluation begins 9th July)
+
+//////////////////////// Item 3: //////////////////////
+3, 4 and 5 really need to be done together.
+Justification:
+There isn't much difference for the rest, they all seem about the same...
+
+Coding:
+connection_t (or.h:740) needs to be altered to use evbuffers.
+Code where data is read from the sockets need to be altered.
+This includes (at least) the following:
+    common/compat.h:231 tor_socket_send and tor_socket_recv
+    common/tortls.c:48 tor_tls_t
+everywhere these are used should be replaced with evbuffer_add or evbuffer_drain on one
+of the input/output buffers.
+tor_tls_t and friends will need altering as well.
+TODO: I need to look into exactly how OpenSSL interacts with the sockets and what I can do
+to make this network api independant. This may need to be done with more evbuffers.
+
+Testing:
+To do this properly we really need to test the whole of tor, running it under a debugger until
+it crashes would seem the most sensible method.
+
+Extensibility:
+This is mostly editing to tor, extensibility is more related to the rest of the code,
+than these changes.
+
+Documentation:
+Comments should be sufficient.
+
+Schedule:
+This is a relatively large and ill defined section.
+I'm going to say 1st August as a ballpark figure.
+I would like to have as much done as possible before ~23rd July, when I return to Uni,
+but I don't expect it all to be finished.
+This ballpark date also gives some wiggle room in case of schedule creep.
+(Note: Second and Final Evaluation starts August 20th)
+
+//////////////////////////////////////////////////////////////
+
+Random notes:
+event.h has a whole lot of bufferevent stuff, but I can't find the functions themselves. 
+Where are they? do they even exist? are they related to this?
+
+What about closing the connection? How will this be done in a connection independent way?
+When the buffer is freed?
\ No newline at end of file