13/03/14 20:59, Mike Perry wrote:
anonym:
[...]
To make problem 1 less of a problem we could use a tool which can sync all non-English DTD:s to the English one. It be invoked via something like `make update-translations`, making it possible refresh the DTD files so one can package a Tor Launcher that works for all locales without having to involve Transifex (that can wait until right before release time). I'm quite ignorant of XML/DTD stuff, but clearly Transifex must make use of a tool like that. Any ideas about this?
Yes, however, it takes a few hours, and has had some issues (which they are in the process of resolving. See https://trac.torproject.org/projects/tor/ticket/9144).
I am not sure if their DTD merge tool is open source/available. Probably?
Problem 2: Another issue is that `chrome.manifest` doesn't list English as the first locale. If it did, English would be the fallback for locales not listed in `chrome.manifest` (note that this would not solve problem 1!). For the record, the current fallback is `af`, which doesn't even have a directory in `chrome/locale`, so all unlisted locales are broken.
Ah, I was actually not aware that the first locale was the fallback.
To be honest I was a bit frustrated by the lack of in-depth documentation about this from Mozilla (or if it exists, that they've hidden it so well), so I just went with what I had seen working through experimentation. On second thought, we shouldn't rely on the order. I believe it's just a coincidence that this works at the moment.
I've yet to see it properly documented (except on a third-party site [1]) but it seems that "en-US" is first tried as a fallback. Tor Launcher currently registers "en", which isn't good enough, apparently. Renaming it to "en-US" makes it the fallback no matter in which order the locales are registered.
Because of this, you may want to rename "en" to "en-US" in your translation branches (possibly in Torbutton and any other Firefox/xulrunner repo too, didn't check).
[1] http://babelwiki.babelzilla.org/index.php?title=Create_a_Locale_Support_for_...
Is it per-entity, or is it only used as a fallback if a locale is entirely absent? It sounds like only if a locale is absent?
Yes, only if the locale is absent. There's no per-entity fallback, which is the root cause of problem 1.
Problem 3: However, `chrome.manifest` has a pretty exhaustive list of locales, so the broken fallback isn't used in many cases. Unfortunately most of them are pointing either to non-existing directories in `chrome/locale` (fatal error similar to the one from problem 1), and to existing directories which contain translations that are pretty much guaranteed to be outdated (and hence probably hit by problem 1) because they're not updated by `localization/import-translations.sh`. Hence all these locales will experience similar errors to what lunar reported.
For problems 2 and 3 I think some `make` automation can improve the situation quite a bit. See the attached patch for something like that.
Right. This sounds like a good start. I would prefer only one list of locales to update though.. Ideally in a form that makes it easy to cut+paste from the TBB list (ie an env var updated in two places like we have now).
In either case, we should only list locales supported in transifex though, it sounds like.
To avoid problem 3, yes.
An alternative approach to maintaining a static list of supported locales (which is error prone, see commit f9759b6 :)) would be to import and bundle/register all locales available in Transifex:
1. In `import-translations.sh` we import all locales available in Transifex into `chrome/locale`. 2. When packaging we register (in `chrome.manifest`) each locale (i.e. folder) present in `chrome/locale` that has all required translation files (i.e. the same files as the English locale). At the moment e.g. `ca_ES` and `ms` do not have all, so this check is necessary to not break those locales.
This would result in Tor Launcher bundling translations with lots of untranslated strings (possibly 100% untranslated in some) which I'm not sure how you feel about. OTOH, your current process doesn't guarantee anything about that even for the explicitly listed supported locales (except if you manually check). Still, one can easily make it so that only the locales specified by some env var are packaged, which seems to be what you want.
[...]
I am very interested in arriving at a solution that works for both of us out of the box, but I also want to maintain the ability to make it straightforward to verify that the list of locales in TBB, Torbutton, and Tor Launcher are the same. This is most easily done if we have a single line of locales in an env var.
Otherwise, your patch looks like a good direction to head in.
See the `locale_fix` branch on git://git.tails.boum.org/tor-launcher for an improvement using the "alternative approach" mentioned above.
In that branch, building a working package that includes all locales from Transifex' translations amounts to:
make import-translations git add src/chrome/locale git commit -m "Import translations." make package
Note that we have to assume that Transifex' translation repos are up-to-date; we only ensure that each bundled locale has the necessary .dtd and .properties files, not that they have all required entities/strings. That would be a nice fail-safe against problem 1 but we'll have to find the appropriate tools first.
To only bundle selected locales, as you requested, one exports `BUNDLE_LOCALES` appropriately, e.g. with your old list:
export BUNDLE_LOCALES="ar de es fa fr it ko nl pl pt ru vi zh-CN" make package
Unlike the rather optimistic "bundle everything" case, this will throw an error if any of them couldn't be bundled for whatever reason, as a fail-safe. Also note that the "en-US" locale is always automatically included since it's the fallback.
Hopefully this suits your needs.
Cheers!