[tor-bugs] #18910 [Metrics/CollecTor]: distributing descriptors accross CollecTor instances
Tor Bug Tracker & Wiki
blackhole at torproject.org
Thu Oct 20 11:40:07 UTC 2016
#18910: distributing descriptors accross CollecTor instances
-------------------------------+---------------------------------
Reporter: iwakeh | Owner: iwakeh
Type: enhancement | Status: needs_review
Priority: High | Milestone: CollecTor 1.1.0
Component: Metrics/CollecTor | Version:
Severity: Normal | Resolution:
Keywords: ctip | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-------------------------------+---------------------------------
Comment (by karsten):
Please take a look at the new commits in my updated task-18910-4 branch.
Remaining tasks:
- Find out why unit tests fail. (See below.)
- Test on a server.
- Squash commits and merge.
Here's the test output with 4 failing tests, both on OS X and on Linux:
{{{
[junit] Testsuite: org.torproject.collector.sync.SyncPersistenceTest
[junit] Tests run: 40, Failures: 4, Errors: 0, Skipped: 0, Time
elapsed: 30.708 sec
[junit]
[junit] Testcase: testDescWriteOutput[0] took 2.044 sec
[junit] Testcase: testDescWriteRecent[0] took 0.523 sec
[junit] Testcase: testOutFileContent[0] took 0.396 sec
[junit] Testcase: testRecentFileContent[0] took 0.389 sec
[junit] Testcase: testDescWriteOutput[1] took 0.454 sec
[junit] FAILED
[junit] data used: relay-descriptors/server-
descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
relay-2016-10-02-16-05-00-server-descriptors, resulting list:
[/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/4/1/4179c50d3c764bc85c9d719e14e55a6cc232a10d,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/2/0/2091f76a8256e479cbe4f57be85f87909af07236,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/b/b/bbca7ed70ba6ea88f995b067a004f5a4d0903d6e,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/d/a/dae8966ca600b46bc75ed5efb97286481e9a6876,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/e/1/e1142337dee5b890393a0891acbde51577c2b743,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/a/0/a0ed9227a9413f140445002ce412f8828591e7ec,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/5/b/5b202650802a916f1ec3a1ef36b98706e3747701,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/5/a/5a536243bf056cd7177ddfd8eb363fec978f3343]
expected:<...-descriptor/2016/10/[e/3/e381ce74a1a592f6d375706665aba6d4d22923f1]>
but
was:<...-descriptor/2016/10/[c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea]>
[junit] junit.framework.AssertionFailedError: data used: relay-
descriptors/server-
descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
relay-2016-10-02-16-05-00-server-descriptors, resulting list:
[/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/4/1/4179c50d3c764bc85c9d719e14e55a6cc232a10d,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/2/0/2091f76a8256e479cbe4f57be85f87909af07236,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/b/b/bbca7ed70ba6ea88f995b067a004f5a4d0903d6e,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/d/a/dae8966ca600b46bc75ed5efb97286481e9a6876,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/e/3/e381ce74a1a592f6d375706665aba6d4d22923f1,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/e/1/e1142337dee5b890393a0891acbde51577c2b743,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/a/0/a0ed9227a9413f140445002ce412f8828591e7ec,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/5/b/5b202650802a916f1ec3a1ef36b98706e3747701,
/tmp/junit1420314177894412746/out/relay-descriptors/server-
descriptor/2016/10/5/a/5a536243bf056cd7177ddfd8eb363fec978f3343]
expected:<...-descriptor/2016/10/[e/3/e381ce74a1a592f6d375706665aba6d4d22923f1]>
but
was:<...-descriptor/2016/10/[c/8/c8c3588019f7c896eb4185cfc1074cfe5eb405ea]>
[junit] at
org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
[junit]
[junit] Testcase: testDescWriteRecent[1] took 0.057 sec
[junit] Testcase: testOutFileContent[1] took 0.1 sec
[junit] Testcase: testRecentFileContent[1] took 0.058 sec
[junit] Testcase: testDescWriteOutput[2] took 1.398 sec
[junit] Testcase: testDescWriteRecent[2] took 0.505 sec
[junit] Testcase: testOutFileContent[2] took 0.858 sec
[junit] Testcase: testRecentFileContent[2] took 0.722 sec
[junit] Testcase: testDescWriteOutput[3] took 0.434 sec
[junit] Testcase: testDescWriteRecent[3] took 0.43 sec
[junit] Testcase: testOutFileContent[3] took 0.673 sec
[junit] Testcase: testRecentFileContent[3] took 0.67 sec
[junit] Testcase: testDescWriteOutput[4] took 1.336 sec
[junit] Testcase: testDescWriteRecent[4] took 0.861 sec
[junit] Testcase: testOutFileContent[4] took 1.484 sec
[junit] Testcase: testRecentFileContent[4] took 1.431 sec
[junit] Testcase: testDescWriteOutput[5] took 0.867 sec
[junit] Testcase: testDescWriteRecent[5] took 0.8 sec
[junit] Testcase: testOutFileContent[5] took 1.601 sec
[junit] Testcase: testRecentFileContent[5] took 1.7 sec
[junit] Testcase: testDescWriteOutput[6] took 0.19 sec
[junit] FAILED
[junit] data used: relay-descriptors/extra-
info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
relay-2016-10-02-08-05-00-extra-infos, resulting list:
[/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/3/1/317586098443ed19b200417556a08ebc42133521,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/3/6/36691feb7cec6a9630b9ecd11a9b5dc61c147c5d,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/4/e/4ef90738e54a403b265120dcbab7b494e0c68d3b,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/8/2/82471deac7b251089a0878d29a228d4e323b823f,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/0/4/04219ada0be922fa7518d36b0d8e66afc55e8603,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/e/b/eb73b59951bc1b0403be81220fb75be464954c31,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/6/a/6a36d4ac36447e645c91ed63633a09197b7ad97e] expected
:<.../extra-info/2016/10/[9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c]>
but was:<.../extra-
info/2016/10/[c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516]>
[junit] junit.framework.AssertionFailedError: data used: relay-
descriptors/extra-
info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
relay-2016-10-02-08-05-00-extra-infos, resulting list:
[/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/3/1/317586098443ed19b200417556a08ebc42133521,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/3/6/36691feb7cec6a9630b9ecd11a9b5dc61c147c5d,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/4/e/4ef90738e54a403b265120dcbab7b494e0c68d3b,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/8/2/82471deac7b251089a0878d29a228d4e323b823f,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/0/4/04219ada0be922fa7518d36b0d8e66afc55e8603,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/e/b/eb73b59951bc1b0403be81220fb75be464954c31,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c,
/tmp/junit4173269913956546333/out/relay-descriptors/extra-
info/2016/10/6/a/6a36d4ac36447e645c91ed63633a09197b7ad97e] expected
:<.../extra-info/2016/10/[9/a/9a4b819baeeeb6952ba737b752471b8637e75a5c]>
but was:<.../extra-
info/2016/10/[c/a/ca86eb96d22d188bb574b6b329ab21e0d9243516]>
[junit] at
org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
[junit]
[junit] Testcase: testDescWriteRecent[6] took 0.038 sec
[junit] Testcase: testOutFileContent[6] took 0.048 sec
[junit] Testcase: testRecentFileContent[6] took 0.092 sec
[junit] Testcase: testDescWriteOutput[7] took 0.086 sec
[junit] FAILED
[junit] data used: bridge-descriptors/2016/10/extra-
infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
bridge-2016-10-02-08-09-00-extra-infos, resulting list:
[/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/3/4/3412a1dccd183a1c0bd1b748f34d88594be6ea52,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/3/1/31bcea576e77ba66150f7903b588c919adad849c,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/1/6/16d2b79fbd0a8567c6afd7585c775ac7745561e3,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/f/a/fabd8f614633ec2d2d405f2554e14381bc33d9cb,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/b/c/bcfcbb38b15e9b500b1a6e9b0bcbbce858660f17,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/e/1/e11c5239494bad2f6f3759f1104a2f6182beab4d,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/e/e/eee0dc51b9a0a71ba73610123b13cea212b5cf83,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/7/a/7a93ca1edc543e747f1157bc3a557890335311a4] expected:<...2016/10
/extra-infos/[9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829]> but
was:<...2016/10/extra-
infos/[c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a]>
[junit] junit.framework.AssertionFailedError: data used: bridge-
descriptors/2016/10/extra-
infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
bridge-2016-10-02-08-09-00-extra-infos, resulting list:
[/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/3/4/3412a1dccd183a1c0bd1b748f34d88594be6ea52,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/3/1/31bcea576e77ba66150f7903b588c919adad849c,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/1/6/16d2b79fbd0a8567c6afd7585c775ac7745561e3,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/f/a/fabd8f614633ec2d2d405f2554e14381bc33d9cb,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/b/c/bcfcbb38b15e9b500b1a6e9b0bcbbce858660f17,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/e/1/e11c5239494bad2f6f3759f1104a2f6182beab4d,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/e/e/eee0dc51b9a0a71ba73610123b13cea212b5cf83,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829,
/tmp/junit4453404715943167674/out/bridge-descriptors/2016/10/extra-
infos/7/a/7a93ca1edc543e747f1157bc3a557890335311a4] expected:<...2016/10
/extra-infos/[9/f/9f88a7c2abe6665d204137ba8c2661d42e7c2829]> but
was:<...2016/10/extra-
infos/[c/a/cab78ea0ffe9a7bc00527fef19f546c47d59f01a]>
[junit] at
org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
[junit]
[junit] Testcase: testDescWriteRecent[7] took 0.027 sec
[junit] Testcase: testOutFileContent[7] took 0.071 sec
[junit] Testcase: testRecentFileContent[7] took 0.054 sec
[junit] Testcase: testDescWriteOutput[8] took 0.07 sec
[junit] FAILED
[junit] data used: bridge-descriptors/2016/10/server-
descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
bridge-2016-10-02-16-09-00-server-descriptors, resulting list:
[/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/c/3/c32acc52826b37e5319f1bac2f8812b33a772540,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/0/7/07d952e9020cb68a63d9156653a2e41af4af4d44,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/0/a/0a65c636a20631bd5deb2f10dc664d2032303c46,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/e/5/e5d46e88cb52d4fc0524398cfb7a6754394bc5e9,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/6/1/614414898ee133ee9bf6b10a9898cab518c5453e,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/7/8/784d6f9e82426072fbfc7a42f8f7133ba6ec1453,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/5/b/5b20316b03afc98a165a219044b6fa6ca34c58ab] expected
:<.../server-descriptors/[a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872]>
but was:<.../server-
descriptors/[c/3/c32acc52826b37e5319f1bac2f8812b33a772540]>
[junit] junit.framework.AssertionFailedError: data used: bridge-
descriptors/2016/10/server-
descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
bridge-2016-10-02-16-09-00-server-descriptors, resulting list:
[/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/c/3/c32acc52826b37e5319f1bac2f8812b33a772540,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/0/7/07d952e9020cb68a63d9156653a2e41af4af4d44,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/0/a/0a65c636a20631bd5deb2f10dc664d2032303c46,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/e/5/e5d46e88cb52d4fc0524398cfb7a6754394bc5e9,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/6/1/614414898ee133ee9bf6b10a9898cab518c5453e,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/7/8/784d6f9e82426072fbfc7a42f8f7133ba6ec1453,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872,
/tmp/junit6983894424845948870/out/bridge-descriptors/2016/10/server-
descriptors/5/b/5b20316b03afc98a165a219044b6fa6ca34c58ab] expected
:<.../server-descriptors/[a/8/a8a5509ad1393c8f36abd2d8f0de1bb751926872]>
but was:<.../server-
descriptors/[c/3/c32acc52826b37e5319f1bac2f8812b33a772540]>
[junit] at
org.torproject.collector.sync.SyncPersistenceTest.testDescWriteOutput(SyncPersistenceTest.java:179)
[junit]
[junit] Testcase: testDescWriteRecent[8] took 0.025 sec
[junit] Testcase: testOutFileContent[8] took 0.036 sec
[junit] Testcase: testRecentFileContent[8] took 0.036 sec
[junit] Testcase: testDescWriteOutput[9] took 0.192 sec
[junit] Testcase: testDescWriteRecent[9] took 0.086 sec
[junit] Testcase: testOutFileContent[9] took 0.1 sec
[junit] Testcase: testRecentFileContent[9] took 0.109 sec
}}}
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/18910#comment:67>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list