[tor-talk] Craigslist now giving Tor the slows, lol

[Mirimir]

On 06/01/2014 05:11 PM, Mirimir wrote:
> On 06/01/2014 01:46 PM, grarpamp wrote:
>> On Sun, Jun 1, 2014 at 2:04 PM, Michael Dupree Jr.
>> <michael at michaeldupree.net> wrote:
>>> Seems to be about normal(slow tor) speed for me.
>>
>> I should try to work on some tools to test against sites people
>> report as blocking tor, since as in your note re: this one,
>> often other people say it 'works for them'. To be fair,
>> it could be higher exit probabilities resulting in abuse
>> blocking for those nodes. Or other such things that
>> could be checked.
> 
> Here's a crude hack:
> 
> Craigslist_test.sh
> ...............................................................
> #!/bin/bash
> 
> run=1
> while [ $run -le N ]
> ## where N is approximately the run time (hours) ##
> do
>    echo "run $run starting at `date +%F_%H-%M`"
>    for CLURL in $(cat ~/Craigslist_URLs)
>    ## don't include "http://" in URLs ##
>    do
>       NOW=`date +%F_%H-%M`
>       echo "   dump IP Chicken at $NOW"
>       links2 -g -dump http://ipchicken.com/ > ~/dumps/IP-$NOW &
>       NOW=`date +%F_%H-%M`
>       echo "   dump Craigslist URL http://$CLURL at $NOW"
>       links2 -g -dump http://$CLURL > ~/dumps/$CLURL-$NOW &
>       sleep $[($RANDOM % 5)+5]m
>    done
>    sleep $[($RANDOM % 60)+30]m
>    let "run+=1"
> done
> exit 0
> ...............................................................
> 
> Craigslist_URLs
> .......................
> berlin.de.craigslist.de
> ...
> .......................

Upon reflection, it should be NOW=`date +%F_%H-%M-%S`.

Links2 doesn't write the file until it's been fully loaded. So the time
required to load equals the difference between the "dump Craigslist URL
..." and file timestamps.