Survey on Tor Trac usage and how you manage your tasks
Hi everyone! At the last annual dev meeting in Waterloo we had a very productive discussion about managing the various channels of Tor communication: https://trac.torproject.org/projects/tor/wiki/org/meetings/2011TorAnnualDevM... Related to that discussion, we should talk about how we use Tor's Trac system, or more generally how we manage our Tor tasks. We use Trac as issue tracker, wiki, and for project management. But it seems that everyone uses Trac slightly different and has their own expectations how other people use it. This can slow us down. In addition to that, some people may not know how Trac can solve their problems. Knowing how others use Trac might help them manage their tasks more quickly. Let's conduct a survey! The survey below consists of two parts: The first part focuses on Trac's features and whether you're using them or not. The second part is about managing your tasks in general, either with or without Trac. If you are a Tor developer or volunteer or think you have some input on the topic, please fill out this survey until Sunday, August 28, 2011 and send it either to this list or to me. I'll try to summarize what was said and send the survey results to this list, hopefully before the end of the month. Thanks! Karsten 1 Using Trac features 1.1 Which of the reports (stored ticket queries) do you use most often? 1.2 What are typical custom queries that you run? 1.3 How do you use milestones and roadmaps? 1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists? 1.5 Which wiki pages do you read/edit most often? 1.6 How do you search for wiki pages? 1.7 What are typical search terms that you use when using the search features? 1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use? 1.9 How relevant are the following ticket fields for you? 1.9.1 Reported by 1.9.2 Owned by 1.9.3 Priority 1.9.4 Milestone 1.9.5 Component 1.9.6 Version 1.9.7 Keywords 1.9.8 Cc 1.9.9 Parent ID 1.9.10 Points 1.9.11 Actual Points 1.10 How relevant are the following ticket statuses for you? 1.10.1 accepted 1.10.2 assigned 1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened 1.11 What other features do you use in Trac? 1.12 What features are you missing in Trac? 1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)? 2 Solving typical software development tasks Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task. 2.1 How do you decide what to work on, both when fixing bugs or when implementing new features? 2.2 How do you keep track of what things you're supposed to do for sponsors and for when? 2.3 How do you memorize new ideas to work on when they come up? 2.4 How do you coordinate working together with someone on something? 2.5 How do you learn who you could work together with on something? 2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
1 Using Trac features 1.1 Which of the reports (stored ticket queries) do you use most often?
"{28} Open Arm Tickets", of course :P
1.2 What are typical custom queries that you run?
I don't since I found the search interface to be a pita. I track tickets via email instead, flagging those I'm interested in and using gmail's searching functionality.
1.3 How do you use milestones and roadmaps?
I keep a wiki with my todo plans for arm (https://trac.torproject.org/projects/tor/wiki/doc/arm).
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
I'm subscribed to tor-bugs but not tor-wiki-changes (I use my own rss2email instance since I want prompt notices but tor-wiki-changes seems to be on an hourly batch).
1.5 Which wiki pages do you read/edit most often?
I maintain... https://trac.torproject.org/projects/tor/wiki/doc/arm https://trac.torproject.org/projects/tor/wiki/doc/badRelays
1.6 How do you search for wiki pages?
Most of the time I use my browser's address autocomplete. Otherwise I've rarely had a need to search or navigate the wiki.
1.7 What are typical search terms that you use when using the search features?
N/A
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
Nope
1.9 How relevant are the following ticket fields for you? 1.9.1 Reported by 1.9.2 Owned by
Useful
1.9.3 Priority
Meh, it's a rough indicator of somebody's opinion on the importance but that's it.
1.9.4 Milestone
Not used for my project
1.9.5 Component
Most important field there is :)
1.9.6 Version
Not used for my project
1.9.7 Keywords 1.9.8 Cc
Not useful (I ignore this)
1.9.9 Parent ID
I don't use this
1.9.10 Points 1.9.11 Actual Points
I don't use this, iirc this is just some metadata field for Mike
1.10 How relevant are the following ticket statuses for you? 1.10.1 accepted 1.10.2 assigned 1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened
Several of the statuses (accepted/assigned/new/reopened) have identical meanings since trac assigns them automatically, so they're not an indicator of a human's intent. The only ones that matter are 'needs_information', 'needs_review', and 'closed'. I'd like to see this narrowed to just a small set of useful statuses with a combo box for the reason... - assigned - pending - requester information - code review - code push - schedule - ... etc - resolved - no issue - requester disappeared - duplicate - fixed / implemented - ... etc
1.11 What other features do you use in Trac?
I use trac as a basic bug tracker and wiki - that's it.
1.12 What features are you missing in Trac?
None, I'm happy with it for my use cases. Once upon a time it would have been nice to have MediaWiki syntax, but the trac derivative is fine.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
For my use cases I'm happy with it. Assuming that you already know what you're looking for trac is fine. That said, our landing page, searching functionality, etc makes it a nightmare for newcomers. From my point of view it's suitable for our current pool of developers and if adding something makes a person's life easier (for example, the "Points" field) then great, go for it. For everyone else trac serves just two purposes: 1. The wiki, which houses content (ie, we link to it from non-trac pages like the site and blog) but I don't think we expect people to navigate trac itself... at least I hope not! 2. Ticket reporting, which we should certainly improve. I've bolded the 'New Ticket' and cypherpunks info on the landing page, though creating a ticket via cypherpunks really should be a fist sized cherry red button on that page...
2 Solving typical software development tasks 2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
My priorities are as follows: 1. time critical issues facing arm users (hotfixes for recent releases and such) 2. features, fixes, or answering questions to make someone's life nicer (where the more involved the requester is, the higher priority it is for me) 3. whatever I feel like at the moment, usually something that works toward the next arm release
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
N/A
2.3 How do you memorize new ideas to work on when they come up?
Not sure if I understand the question. If you mean 'how do I keep track of todo notes' I usually email myself the relevant snippets of irc discussions so I remember to go back and address the issues they mentioned later.
2.4 How do you coordinate working together with someone on something?
There's precious few projects I work with others on, unfortunately. I'd love to change that but there's little interest in controllers or UI development (and what little there is focuses on Vidalia). Oh well... Mostly this question would just concern my work with TorCtl where I write a patch, add a link for the branch on a ticket, then pester Mike every so often until he reviews and merges it.
2.5 How do you learn who you could work together with on something?
Not sure I follow. If I have questions about another project I talk with the project owner, as listed on: https://www.torproject.org/getinvolved/volunteer.html.en#Projects
2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
I dislike the idea of overloading trac. For me it's just a bug tracker and wiki. If others want to use it for things like project management then great, but I'd be weary of trying to jerry rig solutions it's not intended for.
Thus spake Karsten Loesing (karsten.loesing@gmx.net):
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
https://trac.torproject.org/projects/tor/report/14 https://trac.torproject.org/projects/tor/report/38 https://trac.torproject.org/projects/tor/report/39
1.2 What are typical custom queries that you run?
All kinds, but mostly to add extra filters, columns, grouping, and sorting to existing reports and milestones.
1.3 How do you use milestones and roadmaps?
I use milestones to track TBB development and my deliverables for sponsors.
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
I am subscribed to tor-bugs. It goes into a folder and is unread. Trac mail sent directly to me as the Owner, Cc, and/or Reporter goes directly to my inbox, though.
1.5 Which wiki pages do you read/edit most often?
The abuse templates, chrome bugs, sponsor pages, HTTPS-Everywhere design.
1.6 How do you search for wiki pages?
Urlbar history search.
1.7 What are typical search terms that you use when using the search features?
I don't use the trac query to find tickets. Also don't use search engines, as I've not found one capable of deep-indexing trac. If I'm not at least Cc'd on a ticket, it does not exist to me.
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
Agile iteration tracking keywords.
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by
Only insofar as it makes trac email the account that is in this field.
1.9.2 Owned by
Email+knowing which bugs are mine, and who to pester about other bugs.
1.9.3 Priority
Useful for iteration and release planning.
1.9.4 Milestone
Useful for release planning and sponsor deliverables.
1.9.5 Component
Useful for reports, release planning, and filing.
1.9.6 Version
Don't use it.
1.9.7 Keywords
Useful for agile iteration tracking.
1.9.8 Cc
Useful for alerting other people to bugs (unless they send all trac mail to /dev/null).
1.9.9 Parent ID
Useful for tickets with many subtasks, either because the task is too large for one ticket, or because the task spans multiple components.
1.9.10 Points
Useful for recording an estimate of how long a ticket will take. Points are also useful for estimating how much work remains to be done on a given release. The Point completion rate minus the Point open rate for a release can be used to project when that release will be ready.
1.9.11 Actual Points
Useful for recording the actual amount of time+effort spent on a ticket.
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted
Useless.
1.10.2 assigned
Useless.
1.10.3 closed
Very useful.
1.10.4 needs_information
Useful.
1.10.5 needs_review
Somewhat useful.
1.10.6 new
Useless.
1.10.7 reopened
Useless.
1.11 What other features do you use in Trac?
Embedded queries. See for example: https://trac.torproject.org/projects/tor/ticket/3410
1.12 What features are you missing in Trac?
1. Git commit hooks. We should be able to mention "Bug #XXX" at the start of a commit message and have that commit get posted as a comment to trac w/ a link to the gitweb url for the commit. I believe we have a trac plugin for this, but it is not yet properly configured. At other employers, commits could only go into stable releases if they referenced a bug # in the first line. I thought this was a good policy. Back in the SVN days, this property was actually enforcable on the server. Git may make this bit more difficult.. 2. Trac 0.12-stable. I would like some of the embedded query syntax that is available in 0.12.x-stable of trac. In particular, 0.12 would make it much easier to compute the rate of Opened tickets against a given project in a certain period of time. 3. Better handling of duplicate bugs. The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to eachother. 4. Field updates should perhaps not send email Some field updates don't require emailing everybody on the ticket. I *think* 0.12 might provide the ability to solve this one, too, but I am not sure.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever.
2 Solving typical software development tasks
Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task.
2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
Every two weeks I sort the tickets opened against me by priority and by component. I then take approximately 20 Points worth of tickets and assign them to that iteration to work on. As emergency issues arise during this two week period, I give them a "Fire" keyword for that iteration. I tend to get anywhere from 5 to 20 extra Points of fires in a given two week iteration, with 10 being the most common.
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
I mostly use the Sponsor deliverable pages, but I plan to also use the milestone field.
2.3 How do you memorize new ideas to work on when they come up?
If it is development, it goes into trac immediately. Otherwise, it goes into my personal TODO list, which I try to run similarly to the trac iterations.
2.4 How do you coordinate working together with someone on something?
I first try to use the Cc field in trac. It doesn't always work. I fall back to IRC, and then email a couple days later. Sometimes I forget to email and the task falls on the floor and gets forgotten.
2.5 How do you learn who you could work together with on something?
The trac component owner.
2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
I could envision Trac having better support for agile, but really that would just amount to it creating graphs of iteration progress rates and auto-computing ticket completion rates and ticket open rates. Since all of the agile plugins for trac seem rather invasive, I can do this myself for now. -- Mike Perry Mad Computer Scientist fscked.org evil labs
Thus spake Mike Perry (mikeperry@fscked.org):
1.12 What features are you missing in Trac?
1. Git commit hooks.
We should be able to mention "Bug #XXX" at the start of a commit message and have that commit get posted as a comment to trac w/ a link to the gitweb url for the commit. I believe we have a trac plugin for this, but it is not yet properly configured.
At other employers, commits could only go into stable releases if they referenced a bug # in the first line. I thought this was a good policy. Back in the SVN days, this property was actually enforcable on the server. Git may make this bit more difficult..
2. Trac 0.12-stable.
I would like some of the embedded query syntax that is available in 0.12.x-stable of trac. In particular, 0.12 would make it much easier to compute the rate of Opened tickets against a given project in a certain period of time.
3. Better handling of duplicate bugs.
The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to eachother.
4. Field updates should perhaps not send email
Some field updates don't require emailing everybody on the ticket. I *think* 0.12 might provide the ability to solve this one, too, but I am not sure.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever.
I'd like to add the following feature requests to my poll results: 5. Prevent trac from emailing you about your own changes always_notify_updater = false in trac.ini is close to this, but it is not exactly what I want. I think it is useful to automatically notify previous updaters via email, but it seems silly to send me mail for my own changes. Yet there is only one option for both behaviors in trac.. I could see setting it to false and just being more careful about adding people to the Cc line as a stopgap. 6. Embargoed/Security tickets For serious security issues, it would be nice to be able to make tickets not viewable by accounts not on the Cc, owner, or reporter lines of the ticket. I don't think trac can do this at all. -- Mike Perry Mad Computer Scientist fscked.org evil labs
Hi, today, I read that blog post: http://www.guerilla-ciso.com/archives/2049 It is talking about the rise of slow DOS attacks and that Tor could play an important role in it (in fact the post links to an already existing tool for this attack properly configured to get used with Tor). As one of the defenses (granted the last one on his list) the author mentions: "Block TOR exit nodes before the traffic reaches your webservers (IE, at layer 3/4)." Well, as this is not good for the Tor network and makes it unnecessary easy for censors to argue for blocking Tor ("we just want to defend us against slow DOS attacks") I am wondering whether there is already some effort under way to detect and ban such kind of traffic. Or should there be such effort at all? Georg
On Wed, Aug 24, 2011 at 12:21 PM, Georg Koppen <g.koppen@jondos.de> wrote:
... [blah blah] about the rise of slow DOS attacks and that Tor could play an important role in it ... I am wondering whether there is already some effort under way to detect and ban such kind of traffic. Or should there be such effort at all?
there should not be an effort. or better said, slow DoS protections begin in design stage, continue through development and are supported by operations. in nearly every instance of a successful slow DoS there is a negligent or incompetent sysadmin or developer (or both!) at fault. the fact that millions of poorly configured webservers are unable to service requests to their poorly designed and implemented webapps should have no bearing on Tor efforts or exit traffic.
Thus spake Georg Koppen (g.koppen@jondos.de):
today, I read that blog post:
http://www.guerilla-ciso.com/archives/2049
It is talking about the rise of slow DOS attacks and that Tor could play an important role in it (in fact the post links to an already existing tool for this attack properly configured to get used with Tor). As one of the defenses (granted the last one on his list) the author mentions:
"Block TOR exit nodes before the traffic reaches your webservers (IE, at layer 3/4)."
Heh, guy describes an attack that can bring down a webserver from a coffeeshop and his solution is to block Tor. I love people like this. If he was just listing it for completeness, he should also have recommended a national open wireless and open proxy registry, so everyone can block that too. You know, to stop attacks. At least it was last on the list.. This attack definitely sounds like it should be mitigated by Apache config options, and possibly also some form of load-based connection pruning support in Apache itself for use when the server comes close to the MaxClients limit.
Well, as this is not good for the Tor network and makes it unnecessary easy for censors to argue for blocking Tor ("we just want to defend us against slow DOS attacks") I am wondering whether there is already some effort under way to detect and ban such kind of traffic. Or should there be such effort at all?
There have been proposals to run IDSs at exit nodes before. In theory, they can be supported by the Tor protocol without damaging traffic: https://lists.torproject.org/pipermail/tor-relays/2011-March/000675.html So far no one seems interested in doing exit IDS the right way though. We probably have a few exit operators running IDSs already, but they are doing so at risk of being BadExited if they are discovered to be interfering with *any* amount of normal traffic. In general though, the belief is that this is not really our job. If an attack is possible through Tor, blocking Tor or making Tor illegal is akin to burying your head in the ground. Sure, you might stop the script kiddie who ran their attack script with 'torsocks' today, but some other attacker will knock your site over from a coffee shop or open proxy tomorrow. This core philosophy is the basis behind the abuse template set for exit operators: https://trac.torproject.org/projects/tor/wiki/doc/TorAbuseTemplates This philosophy obviously puts us at odds with all the DNSRBL/honeypot folks out there that believe that vigilante justice should be metered out by threatening and spamming ISP abuse departments into pulling the plug on "noisy" IP addresses, but we believe they are just wasting their lives playing wack-a-mole. I guess if it makes them feel better, that's great for them. Everybody deserves their Prozac. But they're not really solving any real problems. Fix the software. Don't fight brain damge with network damage. -- Mike Perry Mad Computer Scientist fscked.org evil labs
On Mon, Aug 22, 2011 at 5:29 AM, Karsten Loesing <karsten.loesing@gmx.net> wrote:
Hi everyone!
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
None
1.2 What are typical custom queries that you run?
Only one: https://trac.torproject.org/projects/tor/query?status=accepted&status=assign...
1.3 How do you use milestones and roadmaps?
I do not.
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
tor-bugs only - I read the mail there looking for controller and TorCtl topics.
1.5 Which wiki pages do you read/edit most often?
None
1.6 How do you search for wiki pages?
I do not.
1.7 What are typical search terms that you use when using the search features?
N/A
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
No
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by 1.9.2 Owned by 1.9.3 Priority 1.9.5 Component
Very important
1.9.4 Milestone 1.9.6 Version 1.9.7 Keywords 1.9.8 Cc 1.9.9 Parent ID 1.9.10 Points 1.9.11 Actual Points
Not at all
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted 1.10.2 assigned
Not at all
1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened
Important
1.11 What other features do you use in Trac?
I am subscribed to a Trac-based RSS feed watching the above Custom Query.
1.12 What features are you missing in Trac?
I would have preferred the ability to receive emails for any changes related to a specific component (i.e. TorCtl).
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
2 Solving typical software development tasks
Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task.
2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
Whatever catches my interest.
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
N/A
2.3 How do you memorize new ideas to work on when they come up?
Local TODO list.
2.4 How do you coordinate working together with someone on something?
Mailing list and Trac
2.5 How do you learn who you could work together with on something?
Ask on the mailing list and read Trac tickets to see who is working on the component.
2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
None -- Sean Robinson WiFi Radar - http://wifi-radar.berlios.de Python WiFi - http://pythonwifi.wikispot.org pymnl - http://pymnl.wikispot.org
Thus spake Sean Robinson (seankrobinson@gmail.com):
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted 1.10.2 assigned
Not at all
1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened
Important
Out of curiosity, what makes having both "new" and "reopened" useful? I find the numerous ticket statuses to be a pain when using the query interface, and in my mind "assigned", "accepted", "new", and "reopened" basically all mean the same thing. -- Mike Perry Mad Computer Scientist fscked.org evil labs
On Wed, Aug 24, 2011 at 6:09 PM, Mike Perry <mikeperry@fscked.org> wrote:
Thus spake Sean Robinson (seankrobinson@gmail.com):
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted 1.10.2 assigned
Not at all
1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened
Important
Out of curiosity, what makes having both "new" and "reopened" useful?
I find the numerous ticket statuses to be a pain when using the query interface, and in my mind "assigned", "accepted", "new", and "reopened" basically all mean the same thing.
I like them as a way to track where the ticket is in its lifetime. The difference between "new" and "reopened" for me is whether this is a brand new ticket or a continuing problem. "Reopened", to me, is also a sign of disagreement between the reporter and the maintainer about the value or status of the ticket. Yes, I agree that "assigned", "accepted", and "new" seem too fine-grained in their meaning to an external audience. I suppose these might be useful in a hierarchically staffed project where one assigns/tracks tickets that one does not work on oneself. As an outsider to the TOR project, I don't know whether these make sense here. -- Sean Robinson
On 8/22/11 2:29 PM, Karsten Loesing wrote:
Hi everyone!
At the last annual dev meeting in Waterloo we had a very productive discussion about managing the various channels of Tor communication:
https://trac.torproject.org/projects/tor/wiki/org/meetings/2011TorAnnualDevM...
Related to that discussion, we should talk about how we use Tor's Trac system, or more generally how we manage our Tor tasks. We use Trac as issue tracker, wiki, and for project management. But it seems that everyone uses Trac slightly different and has their own expectations how other people use it. This can slow us down. In addition to that, some people may not know how Trac can solve their problems. Knowing how others use Trac might help them manage their tasks more quickly. Let's conduct a survey!
The survey below consists of two parts: The first part focuses on Trac's features and whether you're using them or not. The second part is about managing your tasks in general, either with or without Trac.
If you are a Tor developer or volunteer or think you have some input on the topic, please fill out this survey until
Sunday, August 28, 2011
and send it either to this list or to me. I'll try to summarize what was said and send the survey results to this list, hopefully before the end of the month.
So far, 7 people replied to the survey and I got 1 request to extend the deadline. I think it's best to wait for another week if that means we'll get feedback from more Tor developers and volunteers. After all, we might use the survey results to discuss changes to using Trac in the future. The new deadline is Sunday, September 4, 2011 Dear Tor developers and volunteers who haven't responded yet: If you have an opinion on using Trac, please let us know! Thanks, Karsten 1 Using Trac features 1.1 Which of the reports (stored ticket queries) do you use most often? 1.2 What are typical custom queries that you run? 1.3 How do you use milestones and roadmaps? 1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists? 1.5 Which wiki pages do you read/edit most often? 1.6 How do you search for wiki pages? 1.7 What are typical search terms that you use when using the search features? 1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use? 1.9 How relevant are the following ticket fields for you? 1.9.1 Reported by 1.9.2 Owned by 1.9.3 Priority 1.9.4 Milestone 1.9.5 Component 1.9.6 Version 1.9.7 Keywords 1.9.8 Cc 1.9.9 Parent ID 1.9.10 Points 1.9.11 Actual Points 1.10 How relevant are the following ticket statuses for you? 1.10.1 accepted 1.10.2 assigned 1.10.3 closed 1.10.4 needs_information 1.10.5 needs_review 1.10.6 new 1.10.7 reopened 1.11 What other features do you use in Trac? 1.12 What features are you missing in Trac? 1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)? 2 Solving typical software development tasks Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task. 2.1 How do you decide what to work on, both when fixing bugs or when implementing new features? 2.2 How do you keep track of what things you're supposed to do for sponsors and for when? 2.3 How do you memorize new ideas to work on when they come up? 2.4 How do you coordinate working together with someone on something? 2.5 How do you learn who you could work together with on something? 2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
On 8/29/11 10:48 AM, Karsten Loesing wrote:
The new deadline is
Sunday, September 4, 2011
Dear Tor developers and volunteers who haven't responded yet: If you have an opinion on using Trac, please let us know!
First of all, thanks to the 13 people for taking part in the survey! In this mail I'm trying to summarize the results from part 1 of the survey and derive some suggestions for making Trac better. (I'm leaving out the results from part 2 for now, because it turned out the questions weren't as useful as I thought. I may use the replies to design a second survey in a month or two from now, but for the moment I think it's best to just focus on our Trac usage.) Below are the results, plus some suggestions in the paragraphs starting with "[Suggestion: " to separate my suggestions from result summaries. Feel free to comment on the suggestions or add your own! Depending on whether this thread is going to explode, I'll write another summary in a few days to list the suggestions that people seem to agree on and that we're going to implement. Best, Karsten
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
From all the replies, there's 1 person using 5 reports, 2 persons using 3 reports, 1 person using 2 reports, 4 persons using 1 report, and 5 persons using no reports at all. There's only 1 report being viewed by more than one person. 5 persons are not using reports at all.
The following reports are viewed: 7, 8, 12 (mentioned twice), 14, 22, 23 (mentioned twice), 27, 28, 34, 35, 36, 38, 39, 40 (mentioned twice). Hence, the following reports are not viewed: 1, 2, 3, 4, 5, 6, 10, 11, 15, 16, 17, 18, 19, 20, 21, 24, 29, 30, 31, 32, 33, 37. [Suggestion: Backup and delete all reports with a component name in them. The current list of Available Reports list is mostly useless for newcomers who don't care much about components, but who are interested in finding something to work on. Developers and volunteers can bookmark custom queries or put links to them on a wiki page belonging to a component. For example, report 12 "Tor: Active Tickets by Milestone" is https://trac.torproject.org/projects/tor/query?status=!closed&group=mileston... ]
1.2 What are typical custom queries that you run?
There were 11 replies to this question. 4 persons start from a small set of queries that are always the same, similar to stored ticket queries, and refine them as needed. 4 persons run many different queries. 4 persons don't use the custom query feature at all.
1.3 How do you use milestones and roadmaps?
7 out of 11 people use milestones. The components for which milestones are used are Tor (4 mentions), Vidalia (1 mention), TBB (1 mention), and sponsors deadlines (1 mention). 4 persons use wiki pages or parent tickets instead of or in addition to milestones to classify when tickets should be completed. Nobody uses the roadmap view of milestones. Unrelated to this survey, here are some statistics on the milestones in our Trac: We have 224 milestones in total, 37 of which having at least 1 ticket assigned. That leaves 187 milestones with zero tickets assigned, most of which are a left-over from the flyspray migration. [Suggestion: Delete all milestones that have no tickets assigned to them or that seem useful even without assigned tickets.]
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
11 persons are subscribed to tor-bugs, but most people filter the mails to only learn about components they care about. 5 persons read tor-wiki-changes, but at least 3 persons would be glad if they contained diffs. [Suggestion: Try harder to include diffs in tor-wiki-changes notifications. Maybe Trac 0.12 will make this easier.]
1.5 Which wiki pages do you read/edit most often?
8 out of 11 persons replying to this question use the wiki. Mentioned wiki pages are (multiple selections allowed): sponsor pages (5 mentions), specific pages in doc/ (3 mentions), various FAQ (2 mentions), dev meeting pages (2 mentions), roadmaps in org/roadmaps (1 mention).
1.6 How do you search for wiki pages?
People use at least four methods to search the wiki (multiple selections allowed): TitleIndex (5 mentions), external search engine (4 mentions), browser auto-completion or history (3 mentions), Trac search (3 mentions).
1.7 What are typical search terms that you use when using the search features?
3 persons search Trac using key words and 1 person types in ticket numbers in the search field. The rest doesn't use the search feature.
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
1 person uses Agile iteration tracking keywords (D'oh, there goes the anonymity!) and 2 persons use the "easy" keyword sometimes. At least 1 person was surprised that there's a tag page. The rest doesn't use keywords, nor the tags page.
1.9 How relevant are the following ticket fields for you?
Here are the ticket fields sorted by conceived relevance in decreasing order, plus some comments. The numbers in brackets are (useful, somewhat useful, not useful): The Component field (10, 1, 1) is used to find/filter tickets and guess who's paying attention to a ticket. 1 person said that the many Tor components make it hard to refer to the software "tor" and that it's easier to look at milestones itself. The Owned by field (10, 1, 1) is important for most people to decide whether they're responsible for fixing or finding someone else to fix something. The Milestone field (9, 0, 2) is used for release planning and sponsor deliverables, and it also determines the ticket priority to some extent. The Reported by field (8, 2, 2) is used by some to estimate how useful a reported defect or suggested enhancement is, and it is useful to know whether someone will be mailed by Trac when the ticket changes. The Cc field (7, 1, 3) is used to add people whose input is desired and to learn who will be mailed when the ticket changes. There's the problem that only users with certain permissions can add other persons to the Cc field of existing tickets. [Suggestion: Allow all developers and volunteers to edit the Cc field.] The Priority field (6, 5, 1) is important to some people, but is highly dependent on who sets the priority. The Parent ID field (6, 1, 4) is used for complex tickets with many subtasks and when a task spans multiple components. The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list. [Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.] The Keywords field (2, 1, 8) is only used for agile iteration tracking and to tag tickets as "easy." The Points (1, 0, 10) and Actual Points (1, 0, 10) fields are actively used by only 1 person and read by 1 other person. [Suggestion: Investigate whether it's possible to suppress email notifications for changes to the Points and Actual Points fields.]
1.10 How relevant are the following ticket statuses for you?
Here are the ticket statuses sorted by relevance from most important to least important. Numbers in brackets are (useful, somewhat useful, not useful): The closed status (8, 0, 2) is considered useful by pretty much everyone. The needs_review status (7, 2, 1) is used to determine which tickets to read and respond to first. The needs_information status (6, 2, 0) is new, but considered promising and says that one cannot do more without feedback. The new (2, 1, 4), assigned (1, 3, 5), reopened (1, 2, 5), and accepted status (1, 1, 6) have slightly different meanings for some people, but are mostly equivalent for the majority. There are 5 persons who'd like to see these four statuses merged into one, e.g., assigned. 1 person suggests to narrow down statuses to three statuses: assigned, pending (with a combo box for reasons: requester information, code review, code push, schedule), and resolved. [Suggestion: Investigate whether we can merge "new," "assigned," "reopened," and "accepted" into a single "assigned" status.]
1.11 What other features do you use in Trac?
4 out of 11 persons mention use of extended features. These features include admin functions, e.g., deleting users and spam tickets, embedded queries (see #3410), and RSS feeds for custom queries, tickets, wiki pages, and the timeline (2 mentions).
1.12 What features are you missing in Trac?
Better search: The current search is ridiculous. I need to specify things like "I want these three keywords definitely, only in torbutton tickets that are in status needs-review". Git commit hooks (2 mentions): We should be able to mention "Bug #XXX" at the start of a commit message and have that commit get posted as a comment to trac w/ a link to the gitweb url for the commit. I believe we have a trac plugin for this, but it is not yet properly configured. At other employers, commits could only go into stable releases if they referenced a bug # in the first line. I thought this was a good policy. Back in the SVN days, this property was actually enforcable on the server. Git may make this bit more difficult.. [Suggestion: Investigate Git integration in Trac 0.12.] Trac 0.12-stable: I would like some of the embedded query syntax that is available in 0.12.x-stable of trac. In particular, 0.12 would make it much easier to compute the rate of Opened tickets against a given project in a certain period of time. [Suggestion: Try out Trac 0.12 on a VM and find out.] Better handling of duplicate bugs: The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to each other. Field updates should perhaps not send email: Some field updates don't require emailing everybody on the ticket. I *think* 0.12 might provide the ability to solve this one, too, but I am not sure. Prevent trac from emailing you about your own changes (2 mentions): always_notify_updater = false in trac.ini is close to this, but it is not exactly what I want. I think it is useful to automatically notify previous updaters via email, but it seems silly to send me mail for my own changes. Yet there is only one option for both behaviors in trac.. I could see setting it to false and just being more careful about adding people to the Cc line as a stopgap. Embargoed/Security tickets: For serious security issues, it would be nice to be able to make tickets not viewable by accounts not on the Cc, owner, or reporter lines of the ticket. I don't think trac can do this at all. Component-specific email notifications: I would have preferred the ability to receive emails for any changes related to a specific component (i.e. TorCtl). [Suggestion: Find out if r2e can do the trick and explain this on the wiki somewhere.] Better anonymous reporting (3 mentions): (First reporter) I'd like better anonymous reporting and commenting. (Second reporter) I've bolded the 'New Ticket' and cypherpunks info on the landing page, though creating a ticket via cypherpunks really should be a fist sized cherry red button on that page... (Third reporter) I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever. Voting on issues. Tor-detection: It would be nice to have Tor detection built into tor - so we can see if a reporting user is actually using Tor! Multiple milestones: I wish I could assign bugs to multiple milestones. [Suggestion: Only assign defects and enhancements to software product milestones and only assign tasks and projects to sponsor milestones.] Large-scale ticket management: One of the main problems we have in general is the proliferation of tickets. Originally that was a good thing -- "have something we need to do? Make a ticket, then we'll remember." But now we have so many darn tickets, especially in some components, that they're tough to manage, which leads to less active maintenance, which leads to having it get even more out of control. I'm not sure what to do about it, but sometimes it makes me less excited to add more tickets since I feel like I'm adding to the problem. So some feature to do large-scale ticket management would be good.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
No email notifications for certain fields: points, actual points are annoying because changes in them cause email to be sent as an update. cc list is the same. Separate system for users and developers: Trac is used for user support too much. I wish we had a system that was JUST developer centric, no user wiki pages or other crap that turns up in searches. Landing page: Our landing page is a nightmare for newcomers. [Suggestion: Write a new landing page.] Single "tor" component: I don't know what confuses others, but IMO the proliferation of components that are all "tor" doesn't help me, and makes stuff slightly harder. Get rid of wiki: Most of the contents of the wiki consist of obsolete and/or bad advice. These pages should be archived in a tarball where they will not be readily accessible to web browsers and search engines, then purged from our Trac installation. The rest of the pages in the Trac wiki should be moved to a wiki which allows a user who edit a page concurrently with another user to merge his/her/its changes into the wiki page. Perhaps we should just use Git and give up on browser-based wikis.
Hi Karsten, On Tue, Sep 6, 2011 at 10:44 AM, Karsten Loesing <karsten.loesing@gmx.net> wrote:
First of all, thanks to the 13 people for taking part in the survey!
Thanks for your effort to make Trac life easier for everyone!
[Suggestion: Backup and delete all reports with a component name in them. The current list of Available Reports list is mostly useless for newcomers who don't care much about components, but who are interested in finding something to work on. Developers and volunteers can bookmark custom queries or put links to them on a wiki page belonging to a component. For example, report 12 "Tor: Active Tickets by Milestone" is https://trac.torproject.org/projects/tor/query?status=!closed&group=mileston... ]
This is something I'm not entirely happy with. Using the saved query to look up GetTor, BridgeDB and Weather bugs is like the one feature I use in Trac. Sure, I can use bookmarks for this in different browsers I use. On the other hand, I doubt the projected benefit: I might be wrong, but I wonder if deleting the saved reports will result in even a single new volunteer starting to fix bugs for us. /C
On 9/6/11 12:06 PM, Christian Fromme wrote:
[Suggestion: Backup and delete all reports with a component name in them. The current list of Available Reports list is mostly useless for newcomers who don't care much about components, but who are interested in finding something to work on. Developers and volunteers can bookmark custom queries or put links to them on a wiki page belonging to a component. For example, report 12 "Tor: Active Tickets by Milestone" is https://trac.torproject.org/projects/tor/query?status=!closed&group=mileston... ]
This is something I'm not entirely happy with. Using the saved query to look up GetTor, BridgeDB and Weather bugs is like the one feature I use in Trac. Sure, I can use bookmarks for this in different browsers I use. On the other hand, I doubt the projected benefit: I might be wrong, but I wonder if deleting the saved reports will result in even a single new volunteer starting to fix bugs for us.
Okay. Leaving the list of Available Reports unchanged. Best, Karsten
On 2011-09-06, Karsten Loesing <karsten.loesing@gmx.net> wrote:
1.1 Which of the reports (stored ticket queries) do you use most often?
From all the replies, there's 1 person using 5 reports, 2 persons using 3 reports, 1 person using 2 reports, 4 persons using 1 report, and 5 persons using no reports at all. There's only 1 report being viewed by more than one person. 5 persons are not using reports at all.
The following reports are viewed: 7, 8, 12 (mentioned twice), 14, 22, 23 (mentioned twice), 27, 28, 34, 35, 36, 38, 39, 40 (mentioned twice). Hence, the following reports are not viewed: 1, 2, 3, 4, 5, 6, 10, 11, 15, 16, 17, 18, 19, 20, 21, 24, 29, 30, 31, 32, 33, 37.
[Suggestion: Backup and delete all reports with a component name in them. The current list of Available Reports list is mostly useless for newcomers who don't care much about components, but who are interested in finding something to work on. Developers and volunteers can bookmark custom queries or put links to them on a wiki page belonging to a component. For example, report 12 "Tor: Active Tickets by Milestone" is https://trac.torproject.org/projects/tor/query?status=!closed&group=mileston... ]
This is unnecessary. New developers need to find the 'Custom Query' page anyway; leaving reports on the 'Available Reports' page that no one reported using in this survey will not make that any harder. Making the 'Search the Tor bug tracker' link on the wiki main page bold might help. Also, at least one of the reports on that page (https://trac.torproject.org/projects/tor/report/24 (Archived Mixminion-* Tasks)) is for a query that we can no longer perform using the 'Custom Query' page; many of the rest would be difficult to recreate as a custom query.
1.7 What are typical search terms that you use when using the search features?
3 persons search Trac using key words and 1 person types in ticket numbers in the search field. The rest doesn't use the search feature.
I type ticket numbers into my browser's search field, too. I don't consider typing a Trac link target specifier into the search field to be searching.
The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list.
[Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.]
This field might receive more useful input from users if it were an ordinary text field. There are already far too many possible values for this field to be useful in searches; allowing arbitrary strings here cannot make it less useful
Single "tor" component: I don't know what confuses others, but IMO the proliferation of components that are all "tor" doesn't help me, and makes stuff slightly harder.
If these components were merged, I would have much more trouble digging through a custom query to find a particular ticket. Robert Ransom
On 9/6/11 2:19 PM, Robert Ransom wrote:
On 2011-09-06, Karsten Loesing <karsten.loesing@gmx.net> wrote:
1.1 Which of the reports (stored ticket queries) do you use most often?
From all the replies, there's 1 person using 5 reports, 2 persons using 3 reports, 1 person using 2 reports, 4 persons using 1 report, and 5 persons using no reports at all. There's only 1 report being viewed by more than one person. 5 persons are not using reports at all.
The following reports are viewed: 7, 8, 12 (mentioned twice), 14, 22, 23 (mentioned twice), 27, 28, 34, 35, 36, 38, 39, 40 (mentioned twice). Hence, the following reports are not viewed: 1, 2, 3, 4, 5, 6, 10, 11, 15, 16, 17, 18, 19, 20, 21, 24, 29, 30, 31, 32, 33, 37.
[Suggestion: Backup and delete all reports with a component name in them. The current list of Available Reports list is mostly useless for newcomers who don't care much about components, but who are interested in finding something to work on. Developers and volunteers can bookmark custom queries or put links to them on a wiki page belonging to a component. For example, report 12 "Tor: Active Tickets by Milestone" is https://trac.torproject.org/projects/tor/query?status=!closed&group=mileston... ]
This is unnecessary. New developers need to find the 'Custom Query' page anyway; leaving reports on the 'Available Reports' page that no one reported using in this survey will not make that any harder. Making the 'Search the Tor bug tracker' link on the wiki main page bold might help.
Also, at least one of the reports on that page (https://trac.torproject.org/projects/tor/report/24 (Archived Mixminion-* Tasks)) is for a query that we can no longer perform using the 'Custom Query' page; many of the rest would be difficult to recreate as a custom query.
Yup. Not touching the list of Available Reports. :)
1.7 What are typical search terms that you use when using the search features?
3 persons search Trac using key words and 1 person types in ticket numbers in the search field. The rest doesn't use the search feature.
I type ticket numbers into my browser's search field, too. I don't consider typing a Trac link target specifier into the search field to be searching.
I mentioned this feature, because someone was asking for a way to type in ticket numbers somewhere.
The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list.
[Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.]
This field might receive more useful input from users if it were an ordinary text field. There are already far too many possible values for this field to be useful in searches; allowing arbitrary strings here cannot make it less useful
I don't think we can turn this field into a text field, unless we start hacking Trac. We should probably try harder to keep the versions up-to-date to make the field at least somewhat useful.
Single "tor" component: I don't know what confuses others, but IMO the proliferation of components that are all "tor" doesn't help me, and makes stuff slightly harder.
If these components were merged, I would have much more trouble digging through a custom query to find a particular ticket.
Yup. I can see both positions here. I don't think we'll find a solution in this thread. This was Nick's suggestion. Maybe discuss this with Nick directly? Best, Karsten
On Mon, 12 Sep 2011, Karsten Loesing wrote:
3 persons search Trac using key words and 1 person types in ticket numbers in the search field. The rest doesn't use the search feature.
I type ticket numbers into my browser's search field, too. I don't consider typing a Trac link target specifier into the search field to be searching.
I mentioned this feature, because someone was asking for a way to type in ticket numbers somewhere.
Does the brower's address bar count? Like in http://bugs.torproject.org/1234? -- | .''`. ** Debian ** Peter Palfrader | : :' : The universal http://www.palfrader.org/ | `. `' Operating System | `- http://www.debian.org/
On Tue, Sep 06, 2011 at 10:44:11AM +0200, Karsten Loesing wrote:
The Component field (10, 1, 1) is used to find/filter tickets and guess who's paying attention to a ticket. 1 person said that the many Tor components make it hard to refer to the software "tor" and that it's easier to look at milestones itself.
I guess that would be Nick. I wouldn't object to making some other Trac field besides 'component' to categorize things. But lumping 500+ Tor tickets into one component, with no way to break them down by category, is going to make it much harder to find tickets. (I search for tickets by pulling up all the tickets in a given component, and then using my browser's search feature to thumb through them.)
The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list.
[Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.]
When you delete a version from the list, does it vanish from tickets that reference it? That is, will we be destroying history by deleting an old version?
The Points (1, 0, 10) and Actual Points (1, 0, 10) fields are actively used by only 1 person and read by 1 other person.
[Suggestion: Investigate whether it's possible to suppress email notifications for changes to the Points and Actual Points fields.]
I actually find these mails about as useful as the other mails, insofar as they tell me that Mike is thinking about working on a given ticket. But I can see how people who don't care what Mike is going to work on soon would not want to see them.
The new (2, 1, 4), assigned (1, 3, 5), reopened (1, 2, 5), and accepted status (1, 1, 6) have slightly different meanings for some people, but are mostly equivalent for the majority. There are 5 persons who'd like to see these four statuses merged into one, e.g., assigned. 1 person suggests to narrow down statuses to three statuses: assigned, pending (with a combo box for reasons: requester information, code review, code push, schedule), and resolved.
[Suggestion: Investigate whether we can merge "new," "assigned," "reopened," and "accepted" into a single "assigned" status.]
If we merge them, why on earth would it be called 'assigned'? So new tickets start out assigned, but sometimes not to anybody? That's confusing. How about a word that captures them all, like 'open'?
Better handling of duplicate bugs: The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to each other.
Great idea. I sometimes do that manually for the other bug, but not consistently.
Better anonymous reporting (3 mentions): (First reporter) I'd like better anonymous reporting and commenting. (Second reporter) I've bolded the 'New Ticket' and cypherpunks info on the landing page, though creating a ticket via cypherpunks really should be a fist sized cherry red button on that page... (Third reporter) I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever.
Good idea. It is true that tickets created by cypherpunks very often do not get any good followup. I would be tempted to say that cypherpunks shouldn't be allowed to open tickets. I wonder if the real problem here is that it's such a hassle to make a trac account. (Is it?)
Voting on issues.
This isn't a democracy here. :)
Separate system for users and developers: Trac is used for user support too much. I wish we had a system that was JUST developer centric, no user wiki pages or other crap that turns up in searches.
Yes!
Landing page: Our landing page is a nightmare for newcomers.
[Suggestion: Write a new landing page.]
That sounds really valuable. In particular, the first flaw with our current landing page is that it is not clear that you are looking at a combination bugtracker and wiki. --Roger
On 9/12/11 3:51 PM, Roger Dingledine wrote:
On Tue, Sep 06, 2011 at 10:44:11AM +0200, Karsten Loesing wrote:
The Component field (10, 1, 1) is used to find/filter tickets and guess who's paying attention to a ticket. 1 person said that the many Tor components make it hard to refer to the software "tor" and that it's easier to look at milestones itself.
I guess that would be Nick. I wouldn't object to making some other Trac field besides 'component' to categorize things. But lumping 500+ Tor tickets into one component, with no way to break them down by category, is going to make it much harder to find tickets.
(I search for tickets by pulling up all the tickets in a given component, and then using my browser's search feature to thumb through them.)
Well, I don't know what to do here. I think changing the component of 500+ tickets and adding, say, keywords for the category is a lot of work and will generate a lot of mail for people on tor-bugs. Is it worth it?
The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list.
[Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.]
When you delete a version from the list, does it vanish from tickets that reference it? That is, will we be destroying history by deleting an old version?
I just tried this in #4005. The version in the ticket will remain the same, but one cannot create new tickets using this version or change the version field of existing tickets to it. So, breaking history shouldn't be a problem.
The Points (1, 0, 10) and Actual Points (1, 0, 10) fields are actively used by only 1 person and read by 1 other person.
[Suggestion: Investigate whether it's possible to suppress email notifications for changes to the Points and Actual Points fields.]
I actually find these mails about as useful as the other mails, insofar as they tell me that Mike is thinking about working on a given ticket.
But I can see how people who don't care what Mike is going to work on soon would not want to see them.
The question is: how much will you care if these notifications go away? I think the majority of people wouldn't mind if the Points and Actual Points fields would go away entirely, so suppressing notifications was already a compromise.
The new (2, 1, 4), assigned (1, 3, 5), reopened (1, 2, 5), and accepted status (1, 1, 6) have slightly different meanings for some people, but are mostly equivalent for the majority. There are 5 persons who'd like to see these four statuses merged into one, e.g., assigned. 1 person suggests to narrow down statuses to three statuses: assigned, pending (with a combo box for reasons: requester information, code review, code push, schedule), and resolved.
[Suggestion: Investigate whether we can merge "new," "assigned," "reopened," and "accepted" into a single "assigned" status.]
If we merge them, why on earth would it be called 'assigned'? So new tickets start out assigned, but sometimes not to anybody? That's confusing.
How about a word that captures them all, like 'open'?
'open' it is then. (Assuming we can even change this.)
Better handling of duplicate bugs: The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to each other.
Great idea. I sometimes do that manually for the other bug, but not consistently.
I'll see if there's a plugin for that. Or maybe 0.12 does that already.
Better anonymous reporting (3 mentions): (First reporter) I'd like better anonymous reporting and commenting. (Second reporter) I've bolded the 'New Ticket' and cypherpunks info on the landing page, though creating a ticket via cypherpunks really should be a fist sized cherry red button on that page... (Third reporter) I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever.
Good idea. It is true that tickets created by cypherpunks very often do not get any good followup. I would be tempted to say that cypherpunks shouldn't be allowed to open tickets.
I wonder if the real problem here is that it's such a hassle to make a trac account. (Is it?)
Voting on issues.
This isn't a democracy here. :)
Separate system for users and developers: Trac is used for user support too much. I wish we had a system that was JUST developer centric, no user wiki pages or other crap that turns up in searches.
Yes!
I'm not opposed. Should we set up a poll somewhere? :)
Landing page: Our landing page is a nightmare for newcomers.
[Suggestion: Write a new landing page.]
That sounds really valuable. In particular, the first flaw with our current landing page is that it is not clear that you are looking at a combination bugtracker and wiki.
Yup. We should decide first whether we want to solve that by moving away the wiki. Though I know at least one person who won't be thrilled that there will be new URLs for wiki pages---again. Best, Karsten
On Monday, September 12, 2011 11:27:51 Karsten Loesing wrote:
Well, I don't know what to do here. I think changing the component of 500+ tickets and adding, say, keywords for the category is a lot of work and will generate a lot of mail for people on tor-bugs. Is it worth it?
Is there any way to automate the keyword creation for tickets? -- Andrew pgp 0x74ED336B
I'm replying to Roger's email because he condensed Karsten's email, but I can probably answer lots of Trac questions in general. Will try to do so in this thread. * Roger Dingledine <arma@mit.edu> [2011:09:12 09:51 -0400]:
On Tue, Sep 06, 2011 at 10:44:11AM +0200, Karsten Loesing wrote:
The Version field (3, 3, 5) is not used by many components and is considered not very useful, because bug reporters get versions wrong in most cases anyway. Also, current versions of products are never in the list.
[Suggestion: Delete all obsolete versions from the list, and try harder to add new versions.]
When you delete a version from the list, does it vanish from tickets that reference it? That is, will we be destroying history by deleting an old version?
I just tested this: https://trac.torproject.org/projects/tor/ticket/2926#comment:1 I created a Version (called Test), assigned a bug to it, then deleted the Version. It appears to still be in the bug, so I think it is probably safe to delete old versions. We could try a few more times with versions we don't really care about anymore just to be sure. How useful are the versions when deciding a bug applied to a very old version of Tor and is still present in a current one? That mostly gets mentioned in the changelog and not on Trac, right?
The Points (1, 0, 10) and Actual Points (1, 0, 10) fields are actively used by only 1 person and read by 1 other person.
[Suggestion: Investigate whether it's possible to suppress email notifications for changes to the Points and Actual Points fields.]
I actually find these mails about as useful as the other mails, insofar as they tell me that Mike is thinking about working on a given ticket.
But I can see how people who don't care what Mike is going to work on soon would not want to see them.
I can't see an obvious way to suppress these, looking through our trac.ini. In general, I think we should let the firehose spew and try to give people methods for dealing with what comes out. Randomly 'hiding' parts of automatic communication seems sketchy, and doing it because people can't figure out how to configure their mail doesn't seem like a good reason. If we could figure out a way to differentiate between 'mostly useful to everyone' and 'only useful to information sponges' -- i.e., the latter gets Mike's emails, all ticket changes and notifications, and the former just gets whatever we decide is important -- then I could see an argument for making a separate tor-bugs-full that gets everything and letting tor-bugs keep the high-level stuff (opened tickets, replies, reassignment, closed tickets).
How about a word that captures them all, like 'open'?
I think we can do this in Trac. (And I like 'open').
Better handling of duplicate bugs: The "dup" feature should have a bug field that causes both bugs to be updated automatically with links to each other.
Great idea. I sometimes do that manually for the other bug, but not consistently.
There were a few plugins for this that we tried last year and they totally sucked. This is a very old problem with Trac, in fact: http://trac.edgewall.org/ticket/31 We could try again!
Better anonymous reporting (3 mentions): (First reporter) I'd like better anonymous reporting and commenting. (Second reporter) I've bolded the 'New Ticket' and cypherpunks info on the landing page, though creating a ticket via cypherpunks really should be a fist sized cherry red button on that page... (Third reporter) I think the 'cypherpunk' user should have to add an email address in the Cc field. If they put mailinator in there, so be it. But they at least need to know we need to talk to them, or their bugs will probably just sit around forever.
Good idea. It is true that tickets created by cypherpunks very often do not get any good followup. I would be tempted to say that cypherpunks shouldn't be allowed to open tickets.
I wonder if the real problem here is that it's such a hassle to make a trac account. (Is it?)
It isn't a hassle. It is literally one screen which asks for all your information, at which point you hit 'Submit' and you have a trac account. The whole process takes about 20 seconds. I don't like the cypherpunks account either, for all reasons mentioned above. But people creating new accounts in general are not required to provide an email address either, so even if we removed cypherpunks's ability to create tickets, we could potentially have the exact same problem anyway, unless we decide to enforce email as part of account creation.
Separate system for users and developers: Trac is used for user support too much. I wish we had a system that was JUST developer centric, no user wiki pages or other crap that turns up in searches.
Yes!
Can I take this opportunity to plug ikiwiki again, if we do make a separate wiki? It uses git, and we can use text editors to write documentation, rather than writing in web boxes. For people who travel a lot it would be very useful as an offline tool. (Inability to do bug-stuff offline is one thing that annoys Roger, but I can't remember if he already mentioned that.)
On 9/6/11 10:44 AM, Karsten Loesing wrote:
Below are the results, plus some suggestions in the paragraphs starting with "[Suggestion: " to separate my suggestions from result summaries.
Feel free to comment on the suggestions or add your own!
Depending on whether this thread is going to explode, I'll write another summary in a few days to list the suggestions that people seem to agree on and that we're going to implement.
Now that a few days have passed (43 to be precise), I'm writing the first summary of the suggested changes that we should implement. We can always implement more changes later, but let's start somewhere: 1. Delete all milestones that have no tickets assigned to them or that seem useful even without assigned tickets. We have 224 milestones in total, 37 of which having at least 1 ticket assigned. That leaves 187 milestones with zero tickets assigned, most of which are a left-over from the flyspray migration. 2. Try harder to include diffs in tor-wiki-changes notifications. I looked for an option that does this in Trac 0.12, but didn't find one. Erinn, do you know how we could implement this change? 3. Delete all obsolete versions from the list, and try harder to add new versions. Erinn and I tried to delete old versions and found that it's safe to do so. The deleted version string in a ticket will remain the same, but one cannot create new tickets using the deleted version or change the version field of existing tickets to it. 4. Investigate whether we can merge "new", "assigned", "reopened", and "accepted" into a single "open" status. Erinn said we can do this in Trac. I had some trouble getting rid of the "new" status when trying this in Trac 0.12, but didn't look too closely. Erinn, can you try to implement this in our Trac? I'm aware that the full list of changes is longer. These are just the suggestions that everyone seemed to agree on and that are (hopefully) easy to implement. Best, Karsten
On 2011-10-19, Karsten Loesing <karsten.loesing@gmx.net> wrote:
3. Delete all obsolete versions from the list, and try harder to add new versions. Erinn and I tried to delete old versions and found that it's safe to do so. The deleted version string in a ticket will remain the same, but one cannot create new tickets using the deleted version or change the version field of existing tickets to it.
If a vandal changes the version field of an existing ticket, we will be unable to undo that change. I would still prefer that we make the ‘Version’ field a plain text-entry field. I think we still won't add new versions to the list reliably, and I assume you don't plan to add the many different TBB version numbers to the list. Robert Ransom
On 10/19/11 5:50 PM, Robert Ransom wrote:
On 2011-10-19, Karsten Loesing <karsten.loesing@gmx.net> wrote:
3. Delete all obsolete versions from the list, and try harder to add new versions. Erinn and I tried to delete old versions and found that it's safe to do so. The deleted version string in a ticket will remain the same, but one cannot create new tickets using the deleted version or change the version field of existing tickets to it.
If a vandal changes the version field of an existing ticket, we will be unable to undo that change.
Ugh, you're right. Haven't thought of that case. I'll leave old versions untouched for the moment and will only add new versions to the list.
I would still prefer that we make the ‘Version’ field a plain text-entry field.
I think if we don't give users a list of choices, the field will become even less useful.
I think we still won't add new versions to the list reliably, and I assume you don't plan to add the many different TBB version numbers to the list.
Maybe we should add a line "Add version string to Trac" to the release process? Best, Karsten
* Karsten Loesing <karsten.loesing@gmx.net> [2011:10:19 16:12 +0200]:
2. Try harder to include diffs in tor-wiki-changes notifications. I looked for an option that does this in Trac 0.12, but didn't find one. Erinn, do you know how we could implement this change?
I found this: http://trac.edgewall.org/ticket/1660 If we are feeling adventurous, I suppose we could try the very old patches listed there. Otherwise we might just want to comment on the bug and say we wish to have this feature and ask what the status is.
4. Investigate whether we can merge "new", "assigned", "reopened", and "accepted" into a single "open" status. Erinn said we can do this in Trac. I had some trouble getting rid of the "new" status when trying this in Trac 0.12, but didn't look too closely. Erinn, can you try to implement this in our Trac?
How did you try it? I am nervous to try it on our current trac unless we schedule a 'maintenance' window where we tell everyone trac might be broken. I believe weasel has a test trac installed, I will look into testing it there instead.
On 10/20/11 5:34 PM, Erinn Clark wrote:
* Karsten Loesing <karsten.loesing@gmx.net> [2011:10:19 16:12 +0200]:
2. Try harder to include diffs in tor-wiki-changes notifications. I looked for an option that does this in Trac 0.12, but didn't find one. Erinn, do you know how we could implement this change?
I found this: http://trac.edgewall.org/ticket/1660
If we are feeling adventurous, I suppose we could try the very old patches listed there. Otherwise we might just want to comment on the bug and say we wish to have this feature and ask what the status is.
Hmm, not sure if I feel this adventurous. Patching Trac means we'll have to patch it every time there's an update, right? Probably not worth it for this change. I'd say it's better to let people click on a link in an email if they care about the wiki diff than to risk breaking Trac. Thanks for looking this up!
4. Investigate whether we can merge "new", "assigned", "reopened", and "accepted" into a single "open" status. Erinn said we can do this in Trac. I had some trouble getting rid of the "new" status when trying this in Trac 0.12, but didn't look too closely. Erinn, can you try to implement this in our Trac?
How did you try it? I am nervous to try it on our current trac unless we schedule a 'maintenance' window where we tell everyone trac might be broken. I believe weasel has a test trac installed, I will look into testing it there instead.
I tried this by editing trac.conf, probably similar to how you implemented our needs_review and needs_information statuses. I don't have a clean diff yet. If you give me our trac.conf (maybe via private mail), I'll try again next week. Thanks, Karsten
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
https://trac.torproject.org/projects/tor/report/8
1.2 What are typical custom queries that you run?
I sometimes search on a tag if one is being used for a particular multi-ticket task.
1.3 How do you use milestones and roadmaps?
not.
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
I am not.
1.5 Which wiki pages do you read/edit most often?
Pages that have to do with internal processes.
1.6 How do you search for wiki pages?
search window inside wiki.
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by
1.9.2 Owned by
Important
1.9.3 Priority
Important
1.9.4 Milestone
Important
1.9.5 Component
1.9.6 Version
1.9.7 Keywords
1.9.8 Cc
1.9.9 Parent ID
Important
1.9.10 Points
1.9.11 Actual Points
1.10 How relevant are the following ticket statuses for you?
All of those statuses are relevant to me.
2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
Discussion with the people working on the most closely related things.
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
In order: A personal list, trac, the master list of sponsor goals on the wiki.
2.4 How do you coordinate working together with someone on something?
I really do best coordinating over private email, which I am still working on changing because I perceive that it doesn't work well with how everyone else coordinates. I've been trying to find a balance between day to day coordination in private, and then making sure that major milestones get communicated to a list. I have started using this approach as with the GSoC coordination in the latter portion of the summer.
2.5 How do you learn who you could work together with on something?
Face to face interaction. Again, I know this is something I have to change because it doesn't work well in the context of Tor. -- Sincerely Yours, ---Thomas S. Benjamin
Hi Karsten, things I didn't answer either don't apply to me for some reason, or I don't use them (if they're Trac fields) or I have nothing to say for other reasons. On Mon, Aug 22, 2011 at 2:29 PM, Karsten Loesing <karsten.loesing@gmx.net> wrote:
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
BridgeDB Tickets GetTor Tickets Weather Tickets
1.2 What are typical custom queries that you run?
Depends. Nothing noteworthy I'd say
1.3 How do you use milestones and roadmaps?
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
I filter tor-bugs mails to look for BridgeDB, GetTor and Weather bugs/updates.
1.5 Which wiki pages do you read/edit most often?
1.6 How do you search for wiki pages?
1.7 What are typical search terms that you use when using the search features?
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by
Important
1.9.2 Owned by
Important
1.9.3 Priority
Somewhat important. This is sometimes used very subjectively by submitters. "NEEDS TO BE FIXED ASAP OR THE WORLD ENDS!!!!!111eleven" ;-)
1.9.4 Milestone
1.9.5 Component
Important
1.9.6 Version
1.9.7 Keywords
1.9.8 Cc
1.9.9 Parent ID
1.9.10 Points
1.9.11 Actual Points
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted
1.10.2 assigned
Important
1.10.3 closed
Important
1.10.4 needs_information
1.10.5 needs_review
Important
1.10.6 new
1.10.7 reopened
1.11 What other features do you use in Trac?
1.12 What features are you missing in Trac?
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
2 Solving typical software development tasks
Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task.
2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
By priority, usually putting out the most important fires on one of the services like BridgeDB/GetTor/Weather. Usually these important things come up via IRC conversations or Trac tickets. When I'm looking for work otherwise, I look through Trac.
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
Currently doesn't apply to me.
2.3 How do you memorize new ideas to work on when they come up?
Trac.
2.4 How do you coordinate working together with someone on something?
Usually via IRC or email.
2.5 How do you learn who you could work together with on something?
By chance, IRC or email.
2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
None. Hope this helps, /C
On Mon, Aug 22, 2011 at 8:29 AM, Karsten Loesing <karsten.loesing@gmx.net> wrote:
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
I don't use the reports; in the time that it takes me to confirm that a report will actually give me the ticket that I want, I
1.2 What are typical custom queries that you run?
Look for every open ticket in a Tor milestone. Look for every open ticket on a given non-Tor component Look for every open ticket on Tor client, Tor server, Tor bridge, Tor directory authority, Tor hidden services.
1.3 How do you use milestones and roadmaps?
I assign Tor tickets to milestones when I'm pretty sure it'd be a good idea to do them for that milestone. I've make a wiki page for the stuff I want to get done for 0.2.3.x.
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
Yes; I read all the mails that are sent to those lists that pertain to components I work on.
1.5 Which wiki pages do you read/edit most often?
Currently, https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/023 and the various sponsor deliverable pages.
1.6 How do you search for wiki pages?
I go to the Index page and scroll or search on it.
1.7 What are typical search terms that you use when using the search features?
n/a
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
no
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by
not really
1.9.2 Owned by
nope.
1.9.3 Priority
Within milestones and components, I frequently sort by priority. If it's "major", it's something I *definitely* want to get done for the next release if possible. If it's "minor" or lower, it's nice to have, but could slip without severe consequence. If it is "critical" or "blocker", then I assume either that it is a crash bug that makes the network unusable for people, that it is a security bug that is compromising users, or that somebody is being melodramatic.
1.9.4 Milestone
I use milestones a lot. Stuff that is assigned to a "Tor: x-final" milestone means that it's something we should consider for backport (if the milestone is older), or stuff that we should do for an upcoming release (if the milestone is newer)
1.9.5 Component
Somewhat important, but there are so many Tor components that just refer to the software "tor" that I often find myself just looking at milestones instead.
1.9.6 Version
Relevant, though I don't believe it unless the person reporting the bug actually says "I'm using version foo" in the comments, since people often get this wrong.
1.9.7 Keywords
I assign "easy" to bugs I think are easy to fix. I don't read or search on keywords much.
1.9.8 Cc
I get all the bug mail, so I don't mess with this.
1.9.9 Parent ID
Very important; this is how I cluster related bugs and implementation steps.
1.9.10 Points 1.9.11 Actual Points
I don't use these.
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted 1.10.2 assigned 1.10.6 new 1.10.7 reopened
These are all identical from my POV. The distinction between them is not something I pay attention to.
1.10.3 closed
This is the desired end state for all bugs. Very important.
1.10.4 needs_information
This state is kinda important. It means to me, "can't do more here without feedback."
1.10.5 needs_review
Very important. I try to respond to these first.
1.11 What other features do you use in Trac?
Just the tracker and the wiki
1.12 What features are you missing in Trac?
I wish I could assign bugs to multiple milestones.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
I don't know what confuses others, but IMO the proliferation of components that are all "tor" doesn't help me, and makes stuff slightly harder.
2 Solving typical software development tasks
Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task.
2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
I skim the stuff for the relevant milestones weekly, and look at the 023 roadmap when I'm done with a major task.
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
I'm trying to use https://trac.torproject.org/projects/tor/wiki/org/roadmaps/Tor/023, and periodically consult the per-sponsors deliverable pages.
2.3 How do you memorize new ideas to work on when they come up?
I don't think I memorize ideas to work on when they come up; I open a new ticket, or ask the person who's enthusiastic about the new idea to do so.
2.4 How do you coordinate working together with someone on something?
IRC and trac tickets, usually.
2.5 How do you learn who you could work together with on something?
IRC and needs_review tickets, usually
2.6 What other software development tasks do you have that may be supported by Trac or a Trac-like system?
It would be neat to have better code review tools.
On Monday, August 22, 2011 08:29:09 Karsten Loesing wrote:
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
All tickets, mine first
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
tor-bugs, as review of all tickets with changes.
1.5 Which wiki pages do you read/edit most often? 1.6 How do you search for wiki pages?
index page. trac search is useless for finding any information on the trac. ddg or google are better at finding info on our trac.
1.10 How relevant are the following ticket statuses for you? 1.10.1 accepted 1.10.4 needs_information 1.10.6 new
I use 'new' to mean 'haven't thought about it yet'. I use 'accepted' to mean I've thought about it and have a plan. I use 'needs_information' to mean the user needs to provide info to me before further progress.
1.11 What other features do you use in Trac?
I use the timeline heavily to see what's changed. I mostly subscribe to RSS feeds of tickets, wiki pages, and timeline to keep track of the various bits that are relevant to me. -- Andrew pgp 0x74ED336B
On Mon, Aug 22, 2011 at 02:29:09PM +0200, Karsten Loesing wrote:
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
Basically none of them. Every once in a while I use "{12} Tor: Active Tickets by Milestone"
1.2 What are typical custom queries that you run?
In almost every case it is "custom query", press the "-" to get rid of the "owner is arma" constraint, scroll down on add filter to 'component', click somewhere so the new box appears, then pick the component I wanted to get a list of trac tickets on. If it's a big component, I put a bigger number into "max items per page" so I can see all the tickets I asked for (and be able to search their titles using ^F).
1.3 How do you use milestones and roadmaps?
I use the Tor 0.2.x.y-final milestones to categorize Tor tickets by which version they should go in. I don't pick (or look at) milestones on non tor components, since I assume people have their own policies that vary by component. By 'roadmap', do you mean https://trac.torproject.org/projects/tor/roadmap ? If so, I don't use it. In fact, I went so far as to make my own roadmap-like thing for Tor 0.2.2: https://trac.torproject.org/projects/tor/ticket/3032
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
I'm subscribed to tor-bugs. I put them in a separate mailbox. If I'm not travelling, I read them all and use them to know which tickets other people are active on, so I can be most useful at helping them make progress. If I am travelling, I let them pile up and generally don't look back. I run a nightly r2e on the trac wiki rss. I generally delete the mails it sends me, unread -- mostly because the wiki notification mails don't have any useful content about what changed, and getting a useful diff is too klunky to be worth it. ("Besides, they're just wiki pages.")
1.5 Which wiki pages do you read/edit most often?
I use the wiki faq page to point people to faq entries that haven't been transitioned yet. (Basically nobody knows about the wiki faq now that we moved the faq over, but we left most of the answers on the wiki. Oops.) I use the sponsor pages, mostly sponsorf, to remind myself of the various things we've promised, and to help keep them updated on our progress. I'm happy to hand the 'keeping them updated' part to somebody else though.
1.6 How do you search for wiki pages?
In my browser history. All the important ones (there aren't many) are there.
1.7 What are typical search terms that you use when using the search features?
A few times I've tried searching for trac tickets using phrases I remembered from them. It always gave me way too many pages, and they were not easy to thumb through, so I stopped.
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
Woah, there's a tag page? I use the keyword 'easy' sometimes, but ignore keywords generally.
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by
Important, since it tells me how likely the trac ticket is to make sense.
1.9.2 Owned by
Important, since it tells me either who has promised to work on the ticket, or who somebody else hoped would work on the ticket.
1.9.3 Priority
Important. I can signal to other people whether I feel strongly that the task should be done (major) or think it's not a big deal or at least not a big deal soon (minor).
1.9.4 Milestone
Important for the Tor-component tickets. I don't pay attention to it for other components.
1.9.5 Component
Important since it tells me who is (hopefully) paying attention to the ticket.
1.9.6 Version
I try to fill this one in when I can, but the consistency here is so spotty that we would not lose much if it disappeared (especially since it's most useful for bug reports, which mostly come from users who don't successfully report the right version).
1.9.7 Keywords
I ignore it.
1.9.8 Cc
We've been using this one increasingly. People who've added themselves to the cc list indicate that they care about this ticket in particular. Sometimes I add people to the cc list to indicate that I think they should care -- but I try to do that only for people that I've witnessed signing themselves up to some cc list.
1.9.9 Parent ID
Rarely used, but relevant when it is used.
1.9.10 Points 1.9.11 Actual Points
Mike uses these to trac his agile hours. It's a little bit interesting to see him learn lessons from attempting estimates, but I wouldn't miss them if they disappeared.
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted
I treat this as equivalent to 'new' for most people, and equivalent to 'assigned' for a few people. Not all that useful.
1.10.2 assigned
If somebody assigned a ticket to themselves, it's meaningful. If somebody else assigned it, like happens automatically when you switch a ticket from one component to another, it's not all that useful.
1.10.3 closed
Very important. If we couldn't close tickets, where would we be? :)
1.10.4 needs_information
New but promising.
1.10.5 needs_review
Very useful. It means there's a patch I should read.
1.10.6 new 1.10.7 reopened
I wouldn't mind if these merged, but there is a little bit of information to be gained by knowing that somebody thought it was fixed and somebody thought it wasn't.
1.11 What other features do you use in Trac?
Nothing else comes to mind. I spend most of my trac time going to a specific trac ticket using a number I found in my tor-bugs mailbox, looking over the whole discussion, and adding a comment.
1.12 What features are you missing in Trac?
Listing tickets for a given component could sure be easier. One of the main problems we have in general is the proliferation of tickets. Originally that was a good thing -- "have something we need to do? Make a ticket, then we'll remember." But now we have so many darn tickets, especially in some components, that they're tough to manage, which leads to less active maintenance, which leads to having it get even more out of control. I'm not sure what to do about it, but sometimes it makes me less excited to add more tickets since I feel like I'm adding to the problem. So some feature to do large-scale ticket management would be good.
2 Solving typical software development tasks
Note that the questions below don't just focus on Trac, but on any tools or communication media you use for solving a task.
2.1 How do you decide what to work on, both when fixing bugs or when implementing new features?
I generally pull from a) whatever is on my todo list, b) whatever people are working on on irc, and c) whatever is arriving to my tor-bugs mailbox. Often the urgent items on my todo list are not development tasks, so I use development work as a break from the 'real' work.
2.2 How do you keep track of what things you're supposed to do for sponsors and for when?
They're on the sponsors wiki pages. As they get closer I move them to my internal todo file. Recently I've started keeping my todo list in three priority categories ("rsn, soon, and eventually") so hopefully others can have a better chance of following along or predicting me.
2.3 How do you memorize new ideas to work on when they come up?
Ideally I make a trac ticket about them right then. Otherwise (offline or no time), I put a line in a todo list somewhere, which sometimes works out and sometimes gets lost.
2.4 How do you coordinate working together with someone on something?
For a lot of the topics I work on (e.g. being at the center of the research web, or keeping people informed about other parts of the Tor world they need to know about), I coordinate via eventually sending emails. There isn't really that much near-term interaction. Aside from the (increasingly small amount of) development work I do, a lot of my interactions are helping everybody to move forward on their own tasks, helping people know the big picture of where we should go, keeping track of what everybody is up to, and helping to grow the community. A lot of this happens in person, and some happens over irc. I would be a more useful developer if I ignored the big picture more, and then picked out more deep Tor bugs/features to work on. But I can'd do that and also be keeping the big picture in my head to interact with other communities, funders, etc. In any case, that's a topic for a different thread. :)
2.5 How do you learn who you could work together with on something?
By watching trac, irc, the mailing lists, and interacting with people in person, I try to know generally what everybody is up to. Then *I* can be one of the people that people can come to when they want to know who else would be a good match for some project. --Roger
On 2011-08-22, Karsten Loesing <karsten.loesing@gmx.net> wrote:
1 Using Trac features
1.1 Which of the reports (stored ticket queries) do you use most often?
I just used https://trac.torproject.org/projects/tor/report/40 (Tor tickets by milestone) today as a starting point to find a submitted patch that I had forgotten to put in needs_review when I saw it in #tor-bots (#3923). I also used https://trac.torproject.org/projects/tor/report/23 (Tor tickets needing review), and I think I've used it more than once before today.
1.2 What are typical custom queries that you run?
I frequently search for tickets in a component with the default set of statuses, and sometimes search for all closed tickets in a component. I have also used the default custom query (Owner = me) fairly frequently. I occasionally search for tickets containing a keyword in the summary or description fields, mainly when I'm looking for a particular closed ticket whose number I have forgotten in a component containing many tickets.
1.3 How do you use milestones and roadmaps?
I use the 'Milestone' ticket field to indicate which version of Tor a bugfix or feature needs to be applied to. I would do that with Vidalia, too, but there is no Vidalia 0.2.x milestone. What are roadmaps?
1.4 Are you subscribed to tor-bugs and/or tor-wiki-changes, and how do you use the mails sent to these lists?
I am subscribed to tor-bugs. Back when I was using a tolerable MUA, I used unread tor-bugs messages to indicate tickets that I needed to work on or monitor. Now tor-bugs just fills my unusable inbox. I am not subscribed to tor-wiki-changes, and I would not subscribe to it even if I could read the messages.
1.5 Which wiki pages do you read/edit most often?
None.
1.6 How do you search for wiki pages?
I type "https://trac.torproject.org/projects/tor/wiki/TitleIndex" into Firefox's URL bar and vgrep the list. If I'm looking for a wiki page, I usually know the last component of its title, but I no longer know its exact URL.
1.7 What are typical search terms that you use when using the search features?
If a search term were typical, I would bookmark the wiki page or remember the ticket number.
1.8 Do you use keywords and the tags page, and if so, what are keywords that you typically use?
I use the 'easy' keyword for tickets that I suspect would be trivial for someone who knows how to use grep and a TAGS file to fix/implement. What is 'the tags page'?
1.9 How relevant are the following ticket fields for you?
1.9.1 Reported by
Very useful for finding tickets that I have filed.
1.9.2 Owned by
Useful.
1.9.3 Priority
I use this field to indicate the severity of a bug or lack of feature.
1.9.4 Milestone
See above. Very important for tickets on Tor itself; would be important for Vidalia tickets if I could use it; not useful for anything else.
1.9.5 Component
Very important.
1.9.6 Version
Useless. Current versions of products are never in the list.
1.9.7 Keywords
Useless. No one looks for 'easy' tickets. Other keywords are chosen too haphazardly to be useful (usually by users who have never used our bug tracker before).
1.9.8 Cc
Only useful for CC-ing people that notice when they are on a ticket's CC list (i.e. Erinn), and only useful since yesterday (when Erinn gave me the Trac permission needed to set the CC field arbitrarily). Previously, I could only set the CC field when creating a new ticket or add/remove myself in the CC list; the latter was useless.
1.9.9 Parent ID
Rarely useful.
1.9.10 Points
DIE DIE DIE
1.9.11 Actual Points
DIE DIE DIE
1.10 How relevant are the following ticket statuses for you?
1.10.1 accepted
Equivalent to 'assigned', 'new', and 'reopened'.
1.10.2 assigned
Indicates that a ticket's owner has been changed from the default owner for its component.
1.10.3 closed
Indicates that a ticket has been handled or discarded.
1.10.4 needs_information
Sometimes indicates that more information is needed before we can/should continue to handle a ticket. Sometimes does not (e.g. #1705).
1.10.5 needs_review
Very relevant. Indicates that a patch exists to fix a bug/add a feature.
1.10.6 new
Indicates that a ticket has not had its 'owner' field set explicitly since it was created.
1.10.7 reopened
Often indicates that a user is being an idiot or jerk. Otherwise, equivalent to 'accepted', 'assigned', 'new', and 'reopened'.
1.11 What other features do you use in Trac?
The 'Timeline' is occasionally useful for finding tickets opened or modified during some period of time.
1.12 What features are you missing in Trac?
A 'ban' feature to get rid of idiots/jerks.
1.13 What features would you want our Trac not to offer anymore, because they're making things only more confusing for you (and for people who are new to Tor)?
The Trac wiki should die. Most of the contents of the wiki consist of obsolete and/or bad advice. These pages should be archived in a tarball where they will not be readily accessible to web browsers and search engines, then purged from our Trac installation. The rest of the pages in the Trac wiki should be moved to a wiki which allows a user who edit a page concurrently with another user to merge his/her/its changes into the wiki page. Perhaps we should just use Git and give up on browser-based wikis. Robert Ransom
Sorry for the late reply — I haven't checked my email in a while — but I'm replying anyway because I've some strong feelings on this issue. In short, Trac sucks. My issues with it are as follows: 1. The design sucks. 2. There's no AJAX. Now, I know a lot of people on this list hate Web 2.0, but I really think its irrational. Especially over Tor, reloading pages as many times as Trac makes you do is time consuming and cumbersome. Of course, I'm not advocating making it *require* JavaScript, but it would certainly be a nice feature. 3. An unreasonable number of pages must be navigated through in order to perform simple functions. The difficult of doing this prevents people from finding bugs, and therefore discourages contribution. 4. There are too many options. There are oodles of options that must be fiddled with every single time a ticket is created, and if one makes a mistake, it's not possible to go back and edit the ticket. 5. Searching for tickets is a confusing and somewhat difficult task. In order to search for one's own tickets, for example, one must navigate a menu with FORTY choices with unclear labels. 6. Searching in general doesn't work very well. Whatever algorithm Trac uses for search is awful. 7. Trac sends email about your own modifications, and about a lot of things people probably don't want email about (i.e. every comment on a ticket). A significantly improved method would be to offer notifications on the website itself. 8. Trac does not integrate with Git. It should allow you to reference commits from tickets, and reference tickets from commit messages (viewed in the web interface). Being able to close tickets from commit messages would also be useful. 9. HTTP Basic authentication could be confusing for some users. A cookies-based system would probably be easier, and allow for persistent logins (which most people consider a good thing). 10. "Points" are annoying. I'm not personally a fan of 'agile development', and I really don't want to get notifications about it. Again, finer-grained control over notifications would probably solve this. Because of these reasons, I think Trac actually discourages contribution. It's *much* easier to report a bug, say, on GitHub than on Trac, and it's much easier to get updates about progress on it; both of which are things that are important to the average person reviewing a bug. ~ Kat
On 9/6/11 11:01 PM, katmagic wrote:
Sorry for the late reply — I haven't checked my email in a while — but I'm replying anyway because I've some strong feelings on this issue. In short, Trac sucks. My issues with it are as follows:
1. The design sucks.
You're right, it needs more green!
2. There's no AJAX. Now, I know a lot of people on this list hate Web 2.0, but I really think its irrational. Especially over Tor, reloading pages as many times as Trac makes you do is time consuming and cumbersome. Of course, I'm not advocating making it *require* JavaScript, but it would certainly be a nice feature.
This isn't something we can influence. Well, assuming that we want to keep using Trac for a while.
3. An unreasonable number of pages must be navigated through in order to perform simple functions. The difficult of doing this prevents people from finding bugs, and therefore discourages contribution.
Can you be more specific how we could fix this?
4. There are too many options. There are oodles of options that must be fiddled with every single time a ticket is created, and if one makes a mistake, it's not possible to go back and edit the ticket.
The survey had questions about all statuses and ticket fields to evaluate their usefulness. The fact that it's not possible to go back and edit a ticket may be a permission problem. What permissions are you missing? On the other hand, you can always add another comment saying what you really meant.
5. Searching for tickets is a confusing and somewhat difficult task. In order to search for one's own tickets, for example, one must navigate a menu with FORTY choices with unclear labels.
Do you mean the custom query form? Which parts of it do you find hard to understand for new users?
6. Searching in general doesn't work very well. Whatever algorithm Trac uses for search is awful.
I don't think we can change this.
7. Trac sends email about your own modifications, and about a lot of things people probably don't want email about (i.e. every comment on a ticket). A significantly improved method would be to offer notifications on the website itself.
I think two other people stated that they don't want to have emails for their own modifications. Actually, I find that useful, because I'm using my inbox as an archive that I can search even when I'm offline. I'm not sure what you mean with notifications on the website itself.
8. Trac does not integrate with Git. It should allow you to reference commits from tickets, and reference tickets from commit messages (viewed in the web interface). Being able to close tickets from commit messages would also be useful.
We need to install the Git plugin for that. This is one of the things we should change, yes.
9. HTTP Basic authentication could be confusing for some users. A cookies-based system would probably be easier, and allow for persistent logins (which most people consider a good thing).
I don't think we can or should change this.
10. "Points" are annoying. I'm not personally a fan of 'agile development', and I really don't want to get notifications about it. Again, finer-grained control over notifications would probably solve this.
Finer-grained control over notifications is probably the answer.
Because of these reasons, I think Trac actually discourages contribution. It's *much* easier to report a bug, say, on GitHub than on Trac, and it's much easier to get updates about progress on it; both of which are things that are important to the average person reviewing a bug.
I still believe we should first try to tweak Trac to fit our needs before moving on to the next tool. Thanks for your input! Best, Karsten
On Mon, 2011-09-12 at 15:57 +0200, Karsten Loesing wrote:
On 9/6/11 11:01 PM, katmagic wrote:
Sorry for the late reply — I haven't checked my email in a while — but I'm replying anyway because I've some strong feelings on this issue. In short, Trac sucks. My issues with it are as follows:
1. The design sucks.
You're right, it needs more green!
2. There's no AJAX. Now, I know a lot of people on this list hate Web 2.0, but I really think its irrational. Especially over Tor, reloading pages as many times as Trac makes you do is time consuming and cumbersome. Of course, I'm not advocating making it *require* JavaScript, but it would certainly be a nice feature.
This isn't something we can influence. Well, assuming that we want to keep using Trac for a while.
Why should Trac be assumed?
3. An unreasonable number of pages must be navigated through in order to perform simple functions. The difficult of doing this prevents people from finding bugs, and therefore discourages contribution.
Can you be more specific how we could fix this?
Maybe hierarchical CSS menus on the header? There are a myriad of ways to do it, though I don't think Trac can implement any of them.
4. There are too many options. There are oodles of options that must be fiddled with every single time a ticket is created, and if one makes a mistake, it's not possible to go back and edit the ticket.
The survey had questions about all statuses and ticket fields to evaluate their usefulness.
1.9 How relevant are the following ticket fields for you?
1.9.5 Component There are way too many of these for a flat list. A hierarchy of some sort might work, but the component could just as easily be a keyword.
1.9.6 Version Wow, that list is long.
1.9.7 Keywords It would be helpful to have a (perhaps hierarchical) list of previous keywords to choose from, in addition to creating custom ones.
1.9.9 Parent ID Tickets need to be in a hierarchy. Entering the parent ID manually like this is really cumbersome.
1.9.10 Points 1.9.11 Actual Points What are these things for, anyway? If only mikeperry uses them, maybe they should be local to a user? Maybe there should be a generic, user-local field for annotations.
The fact that it's not possible to go back and edit a ticket may be a permission problem. What permissions are you missing? On the other hand, you can always add another comment saying what you really meant.
How am I supposed to know what permissions I have? And having to correct things in separate comments will lead to a bunch of poorly worded, inaccurate, or superfluous comments or tickets.
5. Searching for tickets is a confusing and somewhat difficult task. In order to search for one's own tickets, for example, one must navigate a menu with FORTY choices with unclear labels.
Do you mean the custom query form? Which parts of it do you find hard to understand for new
Even the 'View Tickets' view is confusing. Firstly, sorting shouldn't be listed on the View Tickets page, but as an option chosen on that sub-page. Secondly, the option descriptions are ambiguous. What does 'My Tickets (all)' mean? Does it mean tickets I reported? Tickets assigned to me? Tickets I'm CCed on? Tickets I've commented on? or some combination of these?
6. Searching in general doesn't work very well. Whatever algorithm Trac uses for search is awful.
I don't think we can change this [without switching from Trac].
7. Trac sends email about your own modifications, and about a lot of things people probably don't want email about (i.e. every comment on a ticket). A significantly improved method would be to offer notifications on the website itself.
I think two other people stated that they don't want to have emails for their own modifications. Actually, I find that useful, because I'm using my inbox as an archive that I can search even when I'm offline.
I'm not sure what you mean with notifications on the website itself.
There'd be a notice on the website navigation header alerting you that you have messages. When you click on it, you'd be taken to a page with a list of new tickets, replies, etc. You could also have notifications for saved searches here, rather like Twitter.
8. Trac does not integrate with Git. It should allow you to reference commits from tickets, and reference tickets from commit messages (viewed in the web interface). Being able to close tickets from commit messages would also be useful.
We need to install the Git plugin for that. This is one of the things we should change, yes.
9. HTTP Basic authentication could be confusing for some users. A cookies-based system would probably be easier, and allow for persistent logins (which most people consider a good thing).
I don't think we can or should change this.
10. "Points" are annoying. I'm not personally a fan of 'agile development', and I really don't want to get notifications about it. Again, finer-grained control over notifications would probably solve this.
Finer-grained control over notifications is probably the answer.
Because of these reasons, I think Trac actually discourages contribution. It's *much* easier to report a bug, say, on GitHub than on Trac, and it's much easier to get updates about progress on it; both of which are things that are important to the average person reviewing a bug.
I still believe we should first try to tweak Trac to fit our needs before moving on to the next tool.
Thanks for your input!
Best, Karsten
On 9/13/11 5:20 AM, katmagic wrote:
On Mon, 2011-09-12 at 15:57 +0200, Karsten Loesing wrote:
On 9/6/11 11:01 PM, katmagic wrote:
Sorry for the late reply — I haven't checked my email in a while — but I'm replying anyway because I've some strong feelings on this issue. In short, Trac sucks. My issues with it are as follows:
1. The design sucks.
You're right, it needs more green!
2. There's no AJAX. Now, I know a lot of people on this list hate Web 2.0, but I really think its irrational. Especially over Tor, reloading pages as many times as Trac makes you do is time consuming and cumbersome. Of course, I'm not advocating making it *require* JavaScript, but it would certainly be a nice feature.
This isn't something we can influence. Well, assuming that we want to keep using Trac for a while.
Why should Trac be assumed?
This is primarily a question of effort. We should first try to improve the current situation by making simple changes to Trac. Then we can try more difficult changes to Trac. And then we can think about moving to another tool. It's also unclear though whether another tool would solve all our problems without introducing new ones.
3. An unreasonable number of pages must be navigated through in order to perform simple functions. The difficult of doing this prevents people from finding bugs, and therefore discourages contribution.
Can you be more specific how we could fix this?
Maybe hierarchical CSS menus on the header? There are a myriad of ways to do it, though I don't think Trac can implement any of them.
OK.
4. There are too many options. There are oodles of options that must be fiddled with every single time a ticket is created, and if one makes a mistake, it's not possible to go back and edit the ticket.
The survey had questions about all statuses and ticket fields to evaluate their usefulness.
1.9 How relevant are the following ticket fields for you?
1.9.5 Component There are way too many of these for a flat list. A hierarchy of some sort might work, but the component could just as easily be a keyword.
There's already a hierarchy in the list in the naming. Tor Client, Tor Relay, etc. are all related to (little-t) tor. What we can do is rename a few components to make it clearer what's a Tor sub-component and what's a different tool: "Tor Bridge" -> "Tor (Bridge)" "Tor Browser" -> "Browser" "Tor Check" -> "Check" "Tor Client" -> "Tor (Client)" "Tor Cloud" -> "ServerCloud" "Tor Directory Authority" -> "Tor (Directory Authority)" "Tor Hidden Services" -> "Tor (Hidden Services)" "Tor Relay" -> "Tor (Relay)" "Tor VM" -> "VM" "Tor Weather" -> "Weather" "Tor bundles/installation" -> "Bundles/Installation" I think I wanted to suggest something like this in my mail with the other suggestions, but forgot.
1.9.6 Version Wow, that list is long.
It is! Therefore the suggestion to throw out old versions.
1.9.7 Keywords It would be helpful to have a (perhaps hierarchical) list of previous keywords to choose from, in addition to creating custom ones.
I don't think we can restrict what people are writing there.
1.9.9 Parent ID Tickets need to be in a hierarchy. Entering the parent ID manually like this is really cumbersome.
I think we can assume that whoever wants to create child tickets is capable of typing in or pasting a ticket number. This is nothing that the average bug reporter needs to do.
1.9.10 Points 1.9.11 Actual Points What are these things for, anyway? If only mikeperry uses them, maybe they should be local to a user? Maybe there should be a generic, user-local field for annotations.
I don't think we can implement either of these two suggestions.
The fact that it's not possible to go back and edit a ticket may be a permission problem. What permissions are you missing? On the other hand, you can always add another comment saying what you really meant.
How am I supposed to know what permissions I have? And having to correct things in separate comments will lead to a bunch of poorly worded, inaccurate, or superfluous comments or tickets.
Well, your statement was that you cannot go back and edit a ticket. I didn't look into the permissions in detail and don't have a non-admin user to test this, which is why I asked which permissions you think you need. But I guess I can find out myself.
5. Searching for tickets is a confusing and somewhat difficult task. In order to search for one's own tickets, for example, one must navigate a menu with FORTY choices with unclear labels.
Do you mean the custom query form? Which parts of it do you find hard to understand for new
Even the 'View Tickets' view is confusing. Firstly, sorting shouldn't be listed on the View Tickets page, but as an option chosen on that sub-page. Secondly, the option descriptions are ambiguous. What does 'My Tickets (all)' mean? Does it mean tickets I reported? Tickets assigned to me? Tickets I'm CCed on? Tickets I've commented on? or some combination of these?
I agree that the current list of Active Reports is confusing. It seems that Tor devs have quite different opinions on how to fix this, so I'll save it for the second round of changes. Easy fixes first.
6. Searching in general doesn't work very well. Whatever algorithm Trac uses for search is awful.
I don't think we can change this [without switching from Trac].
7. Trac sends email about your own modifications, and about a lot of things people probably don't want email about (i.e. every comment on a ticket). A significantly improved method would be to offer notifications on the website itself.
I think two other people stated that they don't want to have emails for their own modifications. Actually, I find that useful, because I'm using my inbox as an archive that I can search even when I'm offline.
I'm not sure what you mean with notifications on the website itself.
There'd be a notice on the website navigation header alerting you that you have messages. When you click on it, you'd be taken to a page with a list of new tickets, replies, etc. You could also have notifications for saved searches here, rather like Twitter.
Sounds useful, but unless it's in Trac 0.12, we won't have it.
8. Trac does not integrate with Git. It should allow you to reference commits from tickets, and reference tickets from commit messages (viewed in the web interface). Being able to close tickets from commit messages would also be useful.
We need to install the Git plugin for that. This is one of the things we should change, yes.
9. HTTP Basic authentication could be confusing for some users. A cookies-based system would probably be easier, and allow for persistent logins (which most people consider a good thing).
I don't think we can or should change this.
10. "Points" are annoying. I'm not personally a fan of 'agile development', and I really don't want to get notifications about it. Again, finer-grained control over notifications would probably solve this.
Finer-grained control over notifications is probably the answer.
Because of these reasons, I think Trac actually discourages contribution. It's *much* easier to report a bug, say, on GitHub than on Trac, and it's much easier to get updates about progress on it; both of which are things that are important to the average person reviewing a bug.
I still believe we should first try to tweak Trac to fit our needs before moving on to the next tool.
Thanks for your input!
Again, thanks for taking the time! Best, Karsten
participants (15)
-
Andrew Lewman -
Christian Fromme -
coderman -
Damian Johnson -
Erinn Clark -
Georg Koppen -
Karsten Loesing -
katmagic -
Mike Perry -
Nick Mathewson -
Peter Palfrader -
Robert Ransom -
Roger Dingledine -
Sean Robinson -
Thomas S. Benjamin