[tor-bugs] #33042 [Internal Services/Tor Sysadmin Team]: switch servers to a UTF-8 locale by default

[Tor Bug Tracker & Wiki]

#33042: switch servers to a UTF-8 locale by default
-------------------------------------------------+-------------------------
 Reporter:  anarcat                              |          Owner:  anarcat
     Type:  task                                 |         Status:
                                                 |  needs_review
 Priority:  High                                 |      Milestone:
Component:  Internal Services/Tor Sysadmin Team  |        Version:
 Severity:  Major                                |     Resolution:
 Keywords:                                       |  Actual Points:
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------
Changes (by anarcat):

 * status:  accepted => needs_review


Comment:

 i've pushed this to the `default-utf8` branch on puppet, and i'll merge it
 on monday unless there are any objections:

 {{{
 commit 35a01dc2c0b45a9b5e63a83533f1562cc3520e1f
 Author: Antoine Beaupré <anarcat at debian.org>
 Date:   Thu Jan 23 15:43:20 2020 -0500

     enable default LANG=C.UTF-8 locale

     Ever since this commit:

     1e1fc35e do accept LANG or LC_* variables in sshd

     i've had those weird warnings when loading the vimrc. I finally
     figured out why that was happening: it's because we don't get a UTF-8
     locale anymore.

     To fix this, we set the default locale to the base default locale (C)
     and unicode (so C.UTF-8). The only difference with the default locale
     is the addition of the UTF-8 codepages, which is backwards compatible
     with ASCII.

     I don't expect problems to come out of this: until 1e1fc35e various
     locales were in use on the servers and it didn't seem to cause
     significant problems. In particular, I've been using fr_CA.UTF-8 which
     did cause some messages to show up in french, but in general I expect
     software should tolerate locale changes like this without too much
     problems.

     Going *back* to a regular C locale could be harder: that could mean
     some characters, exactly like the ones in that .vimrc, could not
     display properly and cause the kind of errors I was seeing and that
     caused me to start going down the path of:

     0bcad064 remove listchars parameter

     So, in a way, this is complementary to the change that stops accepting
     locales in sshd: we don't accept those locales, but at least we have a
     sensible, UTF-8 ready default locale.

 diff --git a/modules/torproject_org/files/root-dotfiles/vimrc
 b/modules/torproject_org/files/root-dotfiles/vimrc
 index cf50c15f..d99e4d68 100644
 --- a/modules/torproject_org/files/root-dotfiles/vimrc
 +++ b/modules/torproject_org/files/root-dotfiles/vimrc
 @@ -10,8 +10,8 @@ set ai
  :syn on
  :set title
  :set pastetoggle=<F10>
 -":set listchars=tab:»·,trail:·
 -":set list
 +:set listchars=tab:»·,trail:·
 +:set list
  :nmap <F11> :set invlist<return>
  :imap <F11> <C-O>:set invlist<return>
  :set clipboard^=autoselectml guioptions+=A
 diff --git a/modules/torproject_org/manifests/init.pp
 b/modules/torproject_org/manifests/init.pp
 index 568dba81..f19b5605 100644
 --- a/modules/torproject_org/manifests/init.pp
 +++ b/modules/torproject_org/manifests/init.pp
 @@ -33,7 +33,7 @@ class torproject_org {
              source  => "puppet:///modules/torproject_org/etc/cron.weekly
 /make-dh-params",
              ;
          "/etc/default/locale":
 -            content => "",
 +            content => "LANG=C.UTF-8\n",
              ;
          "/etc/nsswitch.conf":
              mode   => '0755',
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33042#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online