[tor-bugs] #30857 [Internal Services/Services Admin Team]: migrate (some projects? everything?) from trac to gitlab

[Tor Bug Tracker & Wiki]

#30857: migrate (some projects? everything?) from trac to gitlab
-------------------------------------------------+-------------------------
 Reporter:  anarcat                              |          Owner:  (none)
     Type:  project                              |         Status:  new
 Priority:  Medium                               |      Milestone:
Component:  Internal Services/Services Admin     |        Version:
  Team                                           |
 Severity:  Normal                               |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:  #29400                               |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by anarcat):

 there are three possible ways to go forward here:

  1. status quo: no or piecemeal migration. dip/gitlab exists and teams
 migrate organically or not at all, when/if they want, and we keep trac
 forever. probably not acceptable.

  2. migrate team by team: a) pick a team. b) convince team to migrate. c)
 migrate all issues, code and wiki pages to gitlab d) move on to next team
 e) congrats, you're done. seems like the ideal plan to me, because it can
 be done incrementally and allows for progressive testing and ironing out
 of issues. might be difficult to automate.

  3. migrate in one shot. we just bite the bullet and migrate everything
 and everyone all at once, with a flag day when Trac becomes readonly.
 radical solution. might be faster and easier to perform than the other
 solution (less labor) but is much riskier because, if things break, we
 need to fix them VERY FAST NOW and people will/may be upset

 In the parent ticket, I mentioned [https://github.com/tracboat/tracboat
 tracboat] as a tool that might be used to migrate from Trac to GitLab. I
 am not sure it supports migrating one project/component at a time, at
 least it's not obvious how to do so in the documentation.

 Another problem is how to deal with Trac in the long term. A complete
 migration wouldn't be complete if Trac still requires maintenance. For
 this, I see those options:

  1. the golden redirect set: every migrated ticket and wiki page has a
 corresponding ticket/wiki page in GitLab and a gigantic set of redirection
 rules makes sure they are mapped correctly. probably impractical, but
 solves the maintenance problem possibly forever.

  2. read-only Trac: user creation is disabled and existing users are
 locked from making any change to the site. only a temporary or
 intermediate measure.

  3. fossilization: Trac is turned into a static HTML site that can be
 mirrored like any other site. can be a long term solution and a good
 compromise with a possibly impossible to design and therefore failing
 (because incomplete) set of redirection rules.

  4. destruction: we hate the web and pretend link rot is not a problem and
 just get rid of the old site, assuming everything is migrated and people
 will find their stuff eventually. probably not an option.

 As a safety precaution, I have already started step 3, in a way. I am
 working with [https://archiveteam.org/ Archive Team] to send a copy of the
 Trac website into the internet archive, thanks to
 [https://archivebot.readthedocs.io/ archivebot]. This will also allow us
 to build a good "ignore set", a list of patterns to ignore to avoid
 getting lost in the website when/if we decide to create a static HTML
 copy. It's also a good practice to have a backup of all of our stuff in
 the internet archive.

 This currently consists of two crawl jobs:

  * https://trac.torproject.org/ - just feed the site into `wpull` (this is
 what archivebot does, basically) and tweak the ignores to skip the nastier
 stuff. Ignore set currently includes:
    *
 `^https?://trac\.torproject\.org/projects/tor/wiki/.*[?&]action=diff(&|$)`:
 diffs covered by previous revisions
    * `^https?://trac\.torproject\.org/projects/tor/timeline\?`: requires
 login
    *
 `^https?://trac\.torproject\.org/projects/tor/query.*[?&]order=(?!priority)`:
 tries to avoid cardinality explosion in sorted results
  * a single pull of each ticket, from 1 to the latest ticket (#30856 at
 the time of writing, [https://transfer.notkiska.pw/yJMSY/trac.torproject
 .org-tickets full list])

 This is being coordinated on the `#archivebot` channel in efnet.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30857#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online