Greetings, I noticed the rsync job had some issues and it was due to 100% disk usage on the dedicated volume for the mirror. Currently it's at 24GB. Would be nice to know about those changes in advance to prepare disk space if needed. Is it expected to grow more? Best regards
arma announced it in September https://lists.torproject.org/pipermail/tor-mirrors/2014-September/000675.htm... - I forgot and my mirror was full too, so thanks for the email :) Can someone update https://www.torproject.org/docs/running-a-mirror.html.en with expected size? currently "The Tor website and distribution directory currently require roughly 6 GB of disk space." On Tue, Dec 9, 2014 at 8:44 PM, goll <goll@kset.org> wrote:
Greetings,
I noticed the rsync job had some issues and it was due to 100% disk usage on the dedicated volume for the mirror.
Currently it's at 24GB. Would be nice to know about those changes in advance to prepare disk space if needed.
Is it expected to grow more?
Best regards _______________________________________________ tor-mirrors mailing list tor-mirrors@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors
On Tue, Dec 09, 2014 at 10:19:14PM +0100, StalkR wrote:
arma announced it in September https://lists.torproject.org/pipermail/tor-mirrors/2014-September/000675.htm... - I forgot and my mirror was full too, so thanks for the email :)
Can someone update https://www.torproject.org/docs/running-a-mirror.html.en with expected size? currently "The Tor website and distribution directory currently require roughly 6 GB of disk space."
Is it accurate now to say "roughly 9 GB"? Or are we using more than that? I wonder if, even if that's our current amount, we will later briefly have three versions of Tor Browser up there at once, and it will be up to 13-14 briefly. That is, what balance should we strike between being accurate for now, vs not having this thread start up again in a few months? :) --Roger
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Actually it is more like 24 GB right now for me. % du -sch * 50M about 8.0K cgi-bin 88K css 24G dist 1.9M docs 148K donate 148K download 72K eff 20K en 4.0K favicon.ico 344K getinvolved 15M images 132K include 16K index.html.en 124K js 4.0K Makefile 16K Makefile.common 4.0K Makefile.local 4.0K Makefile.local.sample 20K po2wml.sh 8.5M press 16K project 632K projects 8.0K README.md 4.0K robots.txt 8.0K tordnsel 8.0K update-mirrors.pl 12K wml2po.sh 24G total Martin On 10.12.2014 05:12, Roger Dingledine wrote: > On Tue, Dec 09, 2014 at 10:19:14PM +0100, StalkR wrote: >> arma announced it in September >> https://lists.torproject.org/pipermail/tor-mirrors/2014-September/000675.html >> >> >> - - I forgot and my mirror was full too, so thanks for the email :) >> >> Can someone update >> https://www.torproject.org/docs/running-a-mirror.html.en with >> expected size? currently "The Tor website and distribution >> directory currently require roughly 6 GB of disk space." > > Is it accurate now to say "roughly 9 GB"? Or are we using more than > that? I wonder if, even if that's our current amount, we will later > briefly have three versions of Tor Browser up there at once, and it > will be up to 13-14 briefly. That is, what balance should we strike > between being accurate for now, vs not having this thread start up > again in a few months? :) > > --Roger > > _______________________________________________ tor-mirrors > mailing list tor-mirrors@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors > -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJUh8nGAAoJEL1KoFKPY/F+KGAIAKi20C9cc0TLMxI1hf4s7LhB rCE6kLrhehuYerShbox6CcOml8JpFiPCE3GjuhFgGT9GNFWe6hXv5KqZkbu7+rhC dwojiQIh68GULpXoy7EISV4ajUa0LOHYWTW8ET55+0u+t8bUCYaLtcKxY+yPGaW3 TlhOL9UcjqWjqWmiYHJVhtMV7VEURghmb3LUwYEndvPt9+FgYFJ+HOx4rXmc6rUP icoMKlAQcUcuXl1Ehc+49hWm+brWuRPyNhawnqpmNnbIUD3LLu1F1iplVtG6wnkW jdZlSKN93J0e1eNHlfZO6ouaGbiikP5pdma/Z9MF7apzYJz2SAzeWdGIBGUL5F8= =MCeY -----END PGP SIGNATURE-----
Hi, On 10 Dec 2014, at 05:19, Martin Weinelt <tor@linuxlounge.net> wrote:
Actually it is more like 24 GB right now for me.
Yuck. The thing that's roughly 6 GB is *one* copy of TorBrowser, built for everything. Right now it seems we have ~6 copies sitting around. I've updated the website with the current requirements, but I'm also trying to get those responsible for uploading a new TorBrowser release to implement an automatic mechanism preventing such issues before they can happen. Cheers Sebastian
Awful lazy way to handle a file system, public or not. On 12/9/2014 11:47 PM, Sebastian Hahn wrote:
Hi,
On 10 Dec 2014, at 05:19, Martin Weinelt <tor@linuxlounge.net> wrote:
Actually it is more like 24 GB right now for me. Yuck. The thing that's roughly 6 GB is *one* copy of TorBrowser, built for everything. Right now it seems we have ~6 copies sitting around. I've updated the website with the current requirements, but I'm also trying to get those responsible for uploading a new TorBrowser release to implement an automatic mechanism preventing such issues before they can happen.
Cheers Sebastian
_______________________________________________ tor-mirrors mailing list tor-mirrors@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors
50 biggest files in pwd: find . -empty -o -printf '%.5m %10M %#9u %-9g %TY-%Tm-%Td+%Tr [%Y] %#20s %p\n' 2>/dev/null | sort -nrk8 | head -n50
On Wed, 10 Dec 2014 05:47:14 +0100 Sebastian Hahn <sebastian@torproject.org> wrote:
Hi,
On 10 Dec 2014, at 05:19, Martin Weinelt <tor@linuxlounge.net> wrote:
Actually it is more like 24 GB right now for me.
Yuck. The thing that's roughly 6 GB is *one* copy of TorBrowser, built for everything. Right now it seems we have ~6 copies sitting around. I've updated the website with the current requirements, but I'm also trying to get those responsible for uploading a new TorBrowser release to implement an automatic mechanism preventing such issues before they can happen.
Cheers Sebastian
Greetings Sebastian, Unfortunately it happened again. Current usage of dist/torbrowser directory: 4.7G ./5.0a1 5.6G ./4.0.8 6.2G ./4.5a5 6.4G ./4.5.1 6.5G ./4.5 30G . Should the mirrors increase the dist volume size to more than 30G? Best regards
Hi,
On 26 May 2015, at 21:41, goll <goll@kset.org> wrote: On Wed, 10 Dec 2014 05:47:14 +0100 Sebastian Hahn <sebastian@torproject.org> wrote:
Greetings Sebastian,
Unfortunately it happened again.
Current usage of dist/torbrowser directory:
4.7G ./5.0a1 5.6G ./4.0.8 6.2G ./4.5a5 6.4G ./4.5.1 6.5G ./4.5 30G .
Should the mirrors increase the dist volume size to more than 30G?
It seems so, according to bug 15530[0]. That’s a lot of disk space for everything that isn’t part of the mad rush whenever a new TB is released, I’m not sure what good options are atm :( I think the recommendation will be 50GB for now. Even tho I have no idea how many days/months/years that’ll be OK for… Cheers Sebastian [0]: https://trac.torproject.org/projects/tor/ticket/15530
I'm good with 50GB, appreciate you guys keeping an eye on this. On 5/27/2015 3:37 AM, Sebastian Hahn wrote:
Hi,
On 26 May 2015, at 21:41, goll <goll@kset.org> wrote: On Wed, 10 Dec 2014 05:47:14 +0100 Sebastian Hahn <sebastian@torproject.org> wrote:
Greetings Sebastian,
Unfortunately it happened again.
Current usage of dist/torbrowser directory:
4.7G ./5.0a1 5.6G ./4.0.8 6.2G ./4.5a5 6.4G ./4.5.1 6.5G ./4.5 30G .
Should the mirrors increase the dist volume size to more than 30G? It seems so, according to bug 15530[0]. That’s a lot of disk space for everything that isn’t part of the mad rush whenever a new TB is released, I’m not sure what good options are atm :(
I think the recommendation will be 50GB for now. Even tho I have no idea how many days/months/years that’ll be OK for…
Cheers Sebastian
[0]: https://trac.torproject.org/projects/tor/ticket/15530
_______________________________________________ tor-mirrors mailing list tor-mirrors@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors
I noticed disk full too last week, I don't recall seeing an announcement about it but at least https://www.torproject.org/docs/running-a-mirror.html.en was updated to mention "Expect space requirements of up to 50 GB" so I knew what to do. It'd be great if future changes in expectation could be announced to the list, as arma did last time. On Thu, May 28, 2015 at 8:13 PM, AskApache <webmaster@askapache.com> wrote:
I'm good with 50GB, appreciate you guys keeping an eye on this.
On 5/27/2015 3:37 AM, Sebastian Hahn wrote:
Hi,
On 26 May 2015, at 21:41, goll <goll@kset.org> <goll@kset.org> wrote: On Wed, 10 Dec 2014 05:47:14 +0100 Sebastian Hahn <sebastian@torproject.org> <sebastian@torproject.org> wrote:
Greetings Sebastian,
Unfortunately it happened again.
Current usage of dist/torbrowser directory:
4.7G ./5.0a1 5.6G ./4.0.8 6.2G ./4.5a5 6.4G ./4.5.1 6.5G ./4.5 30G .
Should the mirrors increase the dist volume size to more than 30G?
It seems so, according to bug 15530[0]. That’s a lot of disk space for everything that isn’t part of the mad rush whenever a new TB is released, I’m not sure what good options are atm :(
I think the recommendation will be 50GB for now. Even tho I have no idea how many days/months/years that’ll be OK for…
Cheers Sebastian
[0]: https://trac.torproject.org/projects/tor/ticket/15530
_______________________________________________ tor-mirrors mailing listtor-mirrors@lists.torproject.orghttps://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors
_______________________________________________ tor-mirrors mailing list tor-mirrors@lists.torproject.org https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Can confirm this, a full mirror is using 24GB. $ du -sh torproject.org/ 24G torproject.org/ Cheers, Christian Transition statement: http://blog.ph3x.at/2014/11/13/new-pgp-keys/ - -- Christian Krbusek m: christian@ph3x.at Am 2014-12-10 um 05:19 schrieb Martin Weinelt: > Actually it is more like 24 GB right now for me. > > % du -sch * 50M about 8.0K cgi-bin 88K css 24G dist 1.9M docs 148K > donate 148K download 72K eff 20K en 4.0K favicon.ico 344K > getinvolved 15M images 132K include 16K index.html.en 124K js 4.0K > Makefile 16K Makefile.common 4.0K Makefile.local 4.0K > Makefile.local.sample 20K po2wml.sh 8.5M press 16K project 632K > projects 8.0K README.md 4.0K robots.txt 8.0K tordnsel 8.0K > update-mirrors.pl 12K wml2po.sh 24G total > > > Martin > > On 10.12.2014 05:12, Roger Dingledine wrote: >> On Tue, Dec 09, 2014 at 10:19:14PM +0100, StalkR wrote: >>> arma announced it in September >>> https://lists.torproject.org/pipermail/tor-mirrors/2014-September/000675.html >>> >>> >>> > >>> - - I forgot and my mirror was full too, so thanks for the email :) >>> >>> Can someone update >>> https://www.torproject.org/docs/running-a-mirror.html.en with >>> expected size? currently "The Tor website and distribution >>> directory currently require roughly 6 GB of disk space." > >> Is it accurate now to say "roughly 9 GB"? Or are we using more >> than that? I wonder if, even if that's our current amount, we >> will later briefly have three versions of Tor Browser up there at >> once, and it will be up to 13-14 briefly. That is, what balance >> should we strike between being accurate for now, vs not having >> this thread start up again in a few months? :) > >> --Roger > >> _______________________________________________ tor-mirrors >> mailing list tor-mirrors@lists.torproject.org >> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors > >> > > _______________________________________________ tor-mirrors mailing > list tor-mirrors@lists.torproject.org > https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-mirrors > > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQGcBAEBAgAGBQJUh9BaAAoJEDuBlk6mYW0iEPQMAK4ylXfsGJeJFuPmx3OiS2Jx E6Gr87KbLXlQgq0C1PWjPex7BbM5TOZ4kpp2HJMsHArFe7i0gvLU2ENU/MlebKZi EXB1vokVBFZxcmlT9RlOEw+rMunIYsPdje1nxMpBhPEAyZfr28Qt7Kn74e7nA0HM kjBWViRvyWmMGV4PO3eRbqymbs8apGVTOdroMFwqxcahqry8/j8dvcdHoH4f2wTo VqunDNS8xEvIZzDfwmRcEWkbzD6IQ7Pu4JOkFxW8OECAoInvFSFW2BSWtQgeF+p3 ZO3IhI93tisdcm53BR8NDn3YYaZRPLQ088ZOa0ri3D21MRkznOwfuq5BmX7tx0OZ +ut9obGCo86x4JuPZWLrhQfNT18Xi+uzEHg/ovairBx3r+IbAp4CUL7po8kB/g/9 iQrTdRlD2UEXNSneHpLVEYpMU0R5faHNXjYCk11N+Y2HfqVRNPcXfwnEHuefNH7R ync0Co2KAU2qBmUlAfY2JSXfxFHLiGM3yHA0V6wJBw== =udqI -----END PGP SIGNATURE-----
participants (8)
-
AskApache -
Christian Krbusek -
goll -
Martin Weinelt -
Roger Dingledine -
Sebastian Hahn -
Sebastian Hahn -
StalkR