February 2018 - tor-commits - lists.torproject.org

[webwml/master] Another subtask idea for 'python tor client' project
by atagar＠torproject.org 28 Feb '18

28 Feb '18

commit 0193699060b314ae5247e9a6de2cdd844cc4626f Author: Damian Johnson <atagar(a)torproject.org> Date: Wed Feb 28 12:25:25 2018 -0800 Another subtask idea for 'python tor client' project Neat idea from David so people can write a 'is my hidden service up?' check. --- getinvolved/en/volunteer.wml | 1 + 1 file changed, 1 insertion(+) diff --git a/getinvolved/en/volunteer.wml b/getinvolved/en/volunteer.wml index 617e5ff5..ea7e8e87 100644 --- a/getinvolved/en/volunteer.wml +++ b/getinvolved/en/volunteer.wml @@ -757,6 +757,7 @@ This opens interesting opportunities we've never had before such as... <li>Download hidden service descriptors. Unlike relays, the descriptors for hidden services are only available over the ORPort. Once <a href="https://trac.torproject.org/projects/tor/ticket/17945">ticket 17945</a> is merged v3 HS descriptor downloads will require a multi-hop circuit. This requires an understanding of Tor's <a href="https://gitweb.torproject.org/torspec.git/tree/rend-spec-v3.txt">hidden service</a> specifications, particularly the HSDir hash ring.</li> <li>Authenticate our ORPort connection, checking that ORPorts we connect to have the right key fingerprint.</li> <li>Add protocol level integration test of Tor's ORPort, for instance exercising edge cases in circuit construction and extension.</li> + <li>Connect to a hidden service to check if it's up or not.</li> </ul>

1 0

[metrics-web/master] Make all graph data available as CSV.
by karsten＠torproject.org 28 Feb '18

28 Feb '18

commit b75358eace4e8e0a75c60fe0ea1814aa479ecb9f Author: Karsten Loesing <karsten.loesing(a)gmx.net> Date: Wed Feb 28 20:55:10 2018 +0100 Make all graph data available as CSV. Previously, we provided links to CSV files that graphs are based on. But in some cases it would require some data wrangling to obtain the data in the graph, which is less usable than it could be. Now we're generating CSV files based on the graph and selected parameters. This will enable users to quickly obtain the data in a graph and further process it using tools of their choice. Implements #25382. --- src/main/R/rserver/graphs.R | 562 +++++++++++++++++---- .../torproject/metrics/web/GraphImageServlet.java | 3 +- .../org/torproject/metrics/web/GraphServlet.java | 1 - .../org/torproject/metrics/web/LinkServlet.java | 1 - .../java/org/torproject/metrics/web/Metric.java | 6 - .../org/torproject/metrics/web/MetricServlet.java | 5 - .../torproject/metrics/web/RObjectGenerator.java | 9 +- .../org/torproject/metrics/web/TableServlet.java | 1 - src/main/resources/web.xml | 35 +- src/main/resources/web/json/metrics.json | 149 +----- src/main/resources/web/jsps/graph.jsp | 10 +- src/main/resources/web/jsps/table.jsp | 9 - 12 files changed, 533 insertions(+), 258 deletions(-) diff --git a/src/main/R/rserver/graphs.R b/src/main/R/rserver/graphs.R index ab1a60d..fd5201d 100644 --- a/src/main/R/rserver/graphs.R +++ b/src/main/R/rserver/graphs.R @@ -328,7 +328,7 @@ stats_dir = "/srv/metrics.torproject.org/metrics/shared/stats/" rdata_dir = "/srv/metrics.torproject.org/metrics/shared/RData/" -plot_networksize <- function(start, end, path) { +prepare_networksize <- function(start, end) { s <- read.csv(paste(stats_dir, "servers.csv", sep = ""), stringsAsFactors = FALSE) s <- s[s$date >= start & s$date <= end & s$flag == '' & @@ -336,6 +336,11 @@ plot_networksize <- function(start, end, path) { s$ec2bridge == '', ] s <- data.frame(date = as.Date(s$date, "%Y-%m-%d"), relays = s$relays, bridges = s$bridges) + s +} + +plot_networksize <- function(start, end, path) { + s <- prepare_networksize(start, end) dates <- seq(from = as.Date(start, "%Y-%m-%d"), to = as.Date(end, "%Y-%m-%d"), by="1 day") missing <- setdiff(dates, as.Date(s$date, origin = "1970-01-01")) @@ -356,7 +361,12 @@ plot_networksize <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_versions <- function(start, end, path) { +write_networksize <- function(start, end, path) { + prepare_networksize(start, end) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_versions <- function(start, end) { s <- read.csv(paste(stats_dir, "servers.csv", sep = ""), stringsAsFactors = FALSE) s <- s[s$date >= start & s$date <= end & s$flag == '' & @@ -364,6 +374,11 @@ plot_versions <- function(start, end, path) { s$ec2bridge == '', ] s <- data.frame(date = as.Date(s$date, "%Y-%m-%d"), version = s$version, relays = s$relays) + s +} + +plot_versions <- function(start, end, path) { + s <- prepare_versions(start, end) known_versions <- c("Other", "0.1.0", "0.1.1", "0.1.2", "0.2.0", "0.2.1", "0.2.2", "0.2.3", "0.2.4", "0.2.5", "0.2.6", "0.2.7", "0.2.8", "0.2.9", "0.3.0", "0.3.1", "0.3.2", "0.3.3") @@ -388,14 +403,26 @@ plot_versions <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_platforms <- function(start, end, path) { +write_versions <- function(start, end, path) { + prepare_versions(start, end) %>% + spread(key = "version", value = "relays", fill = 0) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_platforms <- function(start, end) { s <- read.csv(paste(stats_dir, "servers.csv", sep = ""), stringsAsFactors = FALSE) s <- s[s$date >= start & s$date <= end & s$flag == '' & s$country == '' & s$version == '' & s$platform != '' & s$ec2bridge == '', ] platforms <- data.frame(date = as.Date(s$date, "%Y-%m-%d"), - variable = s$platform, value = s$relays) + variable = ifelse(s$platform == "Darwin", "macOS", s$platform), + value = s$relays) + platforms +} + +plot_platforms <- function(start, end, path) { + platforms <- prepare_platforms(start, end) ggplot(platforms, aes(x = as.Date(date, "%Y-%m-%d"), y = value, colour = variable)) + geom_line() + @@ -403,25 +430,35 @@ plot_platforms <- function(start, end, path) { labels = custom_labels, minor_breaks = custom_minor_breaks) + scale_y_continuous(name = "", labels = formatter, limits = c(0, NA)) + scale_colour_manual(name = "Platform", - breaks = c("Linux", "Darwin", "BSD", "Windows", "Other"), - labels = c("Linux", "macOS", "BSD", "Windows", "Other"), + breaks = c("Linux", "macOS", "BSD", "Windows", "Other"), values = c("#E69F00", "#56B4E9", "#009E73", "#0072B2", "#333333")) + ggtitle("Relay platforms") + labs(caption = copyright_notice) ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_bandwidth <- function(start, end, path) { +write_platforms <- function(start, end, path) { + prepare_platforms(start, end) %>% + spread(variable, value) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_bandwidth <- function(start, end) { b <- read.csv(paste(stats_dir, "bandwidth.csv", sep = ""), stringsAsFactors = FALSE) b <- b[b$date >= start & b$date <= end & b$isexit == '' & b$isguard == '', ] b <- data.frame(date = as.Date(b$date, "%Y-%m-%d"), - bwadv = b$advbw, - bwhist = (b$bwread + b$bwwrite) / 2) + bwadv = b$advbw * 8 / 1e9, + bwhist = (b$bwread + b$bwwrite) * 8 / 2e9) + b +} + +plot_bandwidth <- function(start, end, path) { + b <- prepare_bandwidth(start, end) bandwidth <- melt(b, id = "date") ggplot(bandwidth, aes(x = as.Date(date, "%Y-%m-%d"), - y = value * 8 / 1e9, colour = variable)) + + y = value, colour = variable)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, labels = custom_labels, minor_breaks = custom_minor_breaks) + @@ -436,7 +473,12 @@ plot_bandwidth <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_bwhist_flags <- function(start, end, path) { +write_bandwidth <- function(start, end, path) { + prepare_bandwidth(start, end) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_bwhist_flags <- function(start, end) { b <- read.csv(paste(stats_dir, "bandwidth.csv", sep = ""), stringsAsFactors = FALSE) b <- b[b$date >= start & b$date <= end & b$isexit != '' & @@ -458,10 +500,15 @@ plot_bwhist_flags <- function(start, end, path) { data.frame(date = as.Date(missing, origin = "1970-01-01"), isexit = TRUE, isguard = TRUE, read = NA, written = NA)) bw <- data.frame(date = bw$date, variable = ifelse(bw$isexit, - ifelse(bw$isguard, "Guard & Exit", "Exit only"), - ifelse(bw$isguard, "Guard only", "Middle only")), - value = (bw$read + bw$written) / 2) - ggplot(bw, aes(x = as.Date(date, "%Y-%m-%d"), y = value * 8 / 1e9, + ifelse(bw$isguard, "guard_and_exit", "exit_only"), + ifelse(bw$isguard, "guard_only", "middle_only")), + value = (bw$read + bw$written) * 8 / 2e9) + bw +} + +plot_bwhist_flags <- function(start, end, path) { + bw <- prepare_bwhist_flags(start, end) + ggplot(bw, aes(x = as.Date(date, "%Y-%m-%d"), y = value, colour = variable)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, @@ -469,22 +516,36 @@ plot_bwhist_flags <- function(start, end, path) { scale_y_continuous(name = "", labels = unit_format(unit = "Gbit/s"), limits = c(0, NA)) + scale_colour_manual(name = "", - values = c("#E69F00", "#56B4E9", "#009E73", "#0072B2")) + + breaks = c("exit_only", "guard_and_exit", "guard_only", "middle_only"), + labels = c("Exit only", "Guard & Exit", "Guard only", "Middle only"), + values = c("#E69F00", "#56B4E9", "#009E73", "#0072B2")) + ggtitle("Bandwidth history by relay flags") + labs(caption = copyright_notice) + theme(legend.position = "top") ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_dirbytes <- function(start, end, path) { +write_bwhist_flags <- function(start, end, path) { + prepare_bwhist_flags(start, end) %>% + spread(variable, value) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_dirbytes <- function(start, end, path) { b <- read.csv(paste(stats_dir, "bandwidth.csv", sep = ""), stringsAsFactors = FALSE) b <- b[b$date >= start & b$date <= end & b$isexit == '' & b$isguard == '', ] b <- data.frame(date = as.Date(b$date, "%Y-%m-%d"), - dirread = b$dirread, dirwrite = b$dirwrite) + dirread = b$dirread * 8 / 1e9, + dirwrite = b$dirwrite * 8 / 1e9) + b +} + +plot_dirbytes <- function(start, end, path) { + b <- prepare_dirbytes(start, end) dir <- melt(b, id = "date") - ggplot(dir, aes(x = as.Date(date, "%Y-%m-%d"), y = value * 8 / 1e9, + ggplot(dir, aes(x = as.Date(date, "%Y-%m-%d"), y = value, colour = variable)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, @@ -500,7 +561,12 @@ plot_dirbytes <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_relayflags <- function(start, end, flags, path) { +write_dirbytes <- function(start, end, path) { + prepare_dirbytes(start, end) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_relayflags <- function(start, end, flags) { s <- read.csv(paste(stats_dir, "servers.csv", sep = ""), stringsAsFactors = FALSE) s <- s[s$date >= start & s$date <= end & s$country == '' & @@ -509,6 +575,11 @@ plot_relayflags <- function(start, end, flags, path) { variable = ifelse(s$flag == '', 'Running', s$flag), value = s$relays) networksize <- s[s$variable %in% flags, ] + networksize +} + +plot_relayflags <- function(start, end, flags, path) { + networksize <- prepare_relayflags(start, end, flags) networksize <- rbind(data.frame( date = as.Date(end) + 1, variable = c("Running", "Exit", "Guard", "Fast", "Stable", "HSDir"), @@ -534,6 +605,13 @@ plot_relayflags <- function(start, end, flags, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } +write_relayflags <- function(start, end, flags, path) { + prepare_relayflags(start, end, flags) %>% + mutate(variable = tolower(variable)) %>% + spread(variable, value) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + plot_torperf <- function(start, end, source, server, filesize, path) { filesizeVal <- ifelse(filesize == '50kb', 50 * 1024, ifelse(filesize == '1mb', 1024 * 1024, 5 * 1024 * 1024)) @@ -580,35 +658,51 @@ plot_torperf <- function(start, end, source, server, filesize, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_torperf_failures <- function(start, end, source, server, filesize, path) { +# Ideally, this function would share code with plot_torperf by using a +# common prepare_torperf function. This just turned out to be a bit +# harder than for other functions, because plot_torperf uses different +# colours based on which sources exist, unrelated to which source is +# plotted. Left as future work. +write_torperf <- function(start, end, source_, server_, filesize_, path) { + read.csv(paste(stats_dir, "torperf-1.1.csv", sep = ""), + colClasses = c("date" = "Date")) %>% + filter(date >= as.Date(start), date <= as.Date(end), + filesize == ifelse(filesize_ == '50kb', 50 * 1024, + ifelse(filesize_ == '1mb', 1024 * 1024, 5 * 1024 * 1024)), + source == ifelse(source_ == 'all', '', source_), + server == server_) %>% + select(date, q1, md, q3) %>% + mutate(q1 = q1 / 1e3, md = md / 1e3, q3 = q3 / 1e3) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_torperf_failures <- function(start, end, source, server, filesize) { filesizeVal <- ifelse(filesize == '50kb', 50 * 1024, ifelse(filesize == '1mb', 1024 * 1024, 5 * 1024 * 1024)) t <- read.csv(paste(stats_dir, "torperf-1.1.csv", sep = ""), stringsAsFactors = FALSE) t <- t[t$date >= start & t$date <= end & t$filesize == filesizeVal & t$source == ifelse(source == 'all', '', source) & - t$server == server, ] + t$server == server & t$requests > 0, ] torperf <- data.frame(date = as.Date(t$date, "%Y-%m-%d"), - timeouts = t$timeouts, failures = t$failures, - requests = t$requests) + timeouts = t$timeouts / t$requests, + failures = t$failures / t$requests) + torperf +} + +plot_torperf_failures <- function(start, end, source, server, filesize, path) { + torperf <- prepare_torperf_failures(start, end, source, server, filesize) dates <- seq(from = as.Date(start, "%Y-%m-%d"), to = as.Date(end, "%Y-%m-%d"), by="1 day") missing <- setdiff(dates, torperf$date) if (length(missing) > 0) torperf <- rbind(torperf, data.frame(date = as.Date(missing, origin = "1970-01-01"), - timeouts = NA, failures = NA, requests = NA)) + timeouts = NA, failures = NA)) + torperf <- melt(torperf, id = "date") filesizes <- data.frame(filesizes = c("5mb", "1mb", "50kb"), label = c("5 MiB", "1 MiB", "50 KiB"), stringsAsFactors = FALSE) filesizeStr <- filesizes[filesizes$filesize == filesize, "label"] - torperf <- rbind(data.frame(date = torperf$date, - value = ifelse(torperf$requests > 0, - torperf$timeouts / torperf$requests, 0), - variable = "timeouts"), - data.frame(date = torperf$date, - value = ifelse(torperf$requests > 0, - torperf$failures / torperf$requests, 0), - variable = "failures")) ggplot(torperf, aes(x = as.Date(date, "%Y-%m-%d"), y = value, colour = variable)) + geom_point(size = 2) + @@ -626,7 +720,12 @@ plot_torperf_failures <- function(start, end, source, server, filesize, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_connbidirect <- function(start, end, path) { +write_torperf_failures <- function(start, end, source, server, filesize, path) { + prepare_torperf_failures(start, end, source, server, filesize) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_connbidirect <- function(start, end) { c <- read.csv(paste(stats_dir, "connbidirect2.csv", sep = ""), stringsAsFactors = FALSE) c <- c[c$date >= start & c$date <= end, ] @@ -636,6 +735,11 @@ plot_connbidirect <- function(start, end, path) { quantile = paste("X", c$quantile, sep = ""), fraction = c$fraction / 100) c <- cast(c, date + direction ~ quantile, value = "fraction") + c +} + +plot_connbidirect <- function(start, end, path) { + c <- prepare_connbidirect(start, end) ggplot(c, aes(x = date, y = X0.5, colour = direction)) + geom_line(size = 0.75) + geom_ribbon(aes(x = date, ymin = X0.25, ymax = X0.75, @@ -657,28 +761,33 @@ plot_connbidirect <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_bandwidth_flags <- function(start, end, path) { +write_connbidirect <- function(start, end, path) { + prepare_connbidirect(start, end) %>% + rename(q1 = X0.25, md = X0.5, q3 = X0.75) %>% + gather(variable, value, -(date:direction)) %>% + unite(temp, direction, variable) %>% + spread(temp, value) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_bandwidth_flags <- function(start, end) { b <- read.csv(paste(stats_dir, "bandwidth.csv", sep = ""), stringsAsFactors = FALSE) b <- b[b$date >= start & b$date <= end & b$isexit != '' & b$isguard != '', ] b <- data.frame(date = as.Date(b$date, "%Y-%m-%d"), isexit = b$isexit == 't', isguard = b$isguard == 't', - advbw = b$advbw, - bwhist = floor((b$bwread + b$bwwrite) / 2)) + advbw = b$advbw * 8 / 1e9, + bwhist = (b$bwread + b$bwwrite) * 8 / 2e9) b <- rbind( - data.frame(b[b$isguard == TRUE, ], flag = "Guard"), - data.frame(b[b$isexit == TRUE, ], flag = "Exit")) + data.frame(b[b$isguard == TRUE, ], flag = "guard"), + data.frame(b[b$isexit == TRUE, ], flag = "exit")) b <- data.frame(date = b$date, advbw = b$advbw, bwhist = b$bwhist, flag = b$flag) b <- aggregate(list(advbw = b$advbw, bwhist = b$bwhist), by = list(date = b$date, flag = b$flag), FUN = sum, na.rm = TRUE, na.action = NULL) - b <- melt(b, id.vars = c("date", "flag")) - b <- data.frame(date = b$date, - type = ifelse(b$variable == 'advbw', 'advertised bandwidth', - 'bandwidth history'), - flag = b$flag, value = b$value) + b <- melt(b, id.vars = c("date", "flag"), variable_name = "type") bandwidth <- b[b$value > 0, ] dates <- seq(from = as.Date(start, "%Y-%m-%d"), to = as.Date(end, "%Y-%m-%d"), by = "1 day") @@ -687,27 +796,35 @@ plot_bandwidth_flags <- function(start, end, path) { if (length(missing) > 0) { bandwidth <- rbind(bandwidth, data.frame(date = as.Date(missing, origin = "1970-01-01"), - type = "advertised bandwidth", flag = "Exit", value = NA), + type = "advbw", flag = "exit", value = NA), data.frame(date = as.Date(missing, origin = "1970-01-01"), - type = "bandwidth history", flag = "Exit", value = NA), + type = "bwhist", flag = "exit", value = NA), data.frame(date = as.Date(missing, origin = "1970-01-01"), - type = "advertised bandwidth", flag = "Guard", value = NA), + type = "advbw", flag = "guard", value = NA), data.frame(date = as.Date(missing, origin = "1970-01-01"), - type = "bandwidth history", flag = "Guard", value = NA)) + type = "bwhist", flag = "guard", value = NA)) } bandwidth <- data.frame(date = bandwidth$date, - variable = as.factor(paste(bandwidth$flag, ", ", bandwidth$type, + variable = as.factor(paste(bandwidth$flag, "_", bandwidth$type, sep = "")), value = bandwidth$value) bandwidth$variable <- factor(bandwidth$variable, levels = levels(bandwidth$variable)[c(3, 4, 1, 2)]) + bandwidth +} + +plot_bandwidth_flags <- function(start, end, path) { + bandwidth <- prepare_bandwidth_flags(start, end) ggplot(bandwidth, aes(x = as.Date(date, "%Y-%m-%d"), - y = value * 8 / 1e9, colour = variable)) + + y = value, colour = variable)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, labels = custom_labels, minor_breaks = custom_minor_breaks) + scale_y_continuous(name = "", labels = unit_format(unit = "Gbit/s"), limits = c(0, NA)) + scale_colour_manual(name = "", + breaks = c("guard_advbw", "guard_bwhist", "exit_advbw", "exit_bwhist"), + labels = c("Guard, advertised bandwidth", "Guard, bandwidth history", + "Exit, advertised bandwidth", "Exit, bandwidth history"), values = c("#E69F00", "#D6C827", "#009E73", "#00C34F")) + ggtitle(paste("Advertised bandwidth and bandwidth history by", "relay flags")) + @@ -716,6 +833,12 @@ plot_bandwidth_flags <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } +write_bandwidth_flags <- function(start, end, path) { + prepare_bandwidth_flags(start, end) %>% + spread(variable, value) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + plot_userstats <- function(start, end, node, variable, value, events, path) { load(paste(rdata_dir, "clients-", node, ".RData", sep = "")) @@ -861,20 +984,96 @@ plot_userstats_bridge_version <- function(start, end, version, path) { plot_userstats(start, end, 'bridge', 'version', version, 'off', path) } +write_userstats_relay_country <- function(start, end, country_, events, + path) { + load(paste(rdata_dir, "clients-relay.RData", sep = "")) + u <- data %>% + filter(date >= as.Date(start), date <= as.Date(end), + country == ifelse(country_ == 'all', '', country_), transport == '', + version == '') + if (country_ != 'all' && events == 'on') { + u <- u %>% + mutate(downturns = clients < u$lower, upturns = clients > upper) %>% + select(date, clients, downturns, upturns, lower, upper) + } else if (country_ != 'all' && events != 'off') { + u <- u %>% + mutate(downturns = clients < u$lower, upturns = clients > upper) %>% + select(date, clients, downturns, upturns) + } else { + u <- u %>% + select(date, clients) + } + u %>% + rename(users = clients) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +write_userstats_bridge_country <- function(start, end, country_, path) { + load(paste(rdata_dir, "clients-bridge.RData", sep = "")) + data %>% + filter(date >= as.Date(start), date <= as.Date(end), + country == ifelse(country_ == 'all', '', country_), transport == '', + version == '') %>% + select(date, clients) %>% + rename(users = clients) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +write_userstats_bridge_transport <- function(start, end, transports, path) { + load(paste(rdata_dir, "clients-bridge.RData", sep = "")) + u <- data %>% + filter(date >= as.Date(start), date <= as.Date(end), + country == '', version == '', transport != '') %>% + select(date, transport, clients) + if ('!<OR>' %in% transports) { + n <- u %>% + filter(transport != '<OR>') %>% + group_by(date) %>% + summarize(clients = sum(clients)) + u <- rbind(u, data.frame(date = n$date, transport = '!<OR>', + clients = n$clients)) + } + u %>% + filter(transport %in% transports) %>% + mutate(transport = ifelse(transport == '<OR>', 'default_or_protocol', + ifelse(transport == '!<OR>', 'any_pt', + ifelse(transport == '<??>', 'unknown_pluggable_transports', + transport)))) %>% + group_by(date, transport) %>% + select(date, transport, clients) %>% + spread(transport, clients) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +write_userstats_bridge_version <- function(start, end, version_, path) { + load(paste(rdata_dir, "clients-bridge.RData", sep = "")) + data %>% + filter(date >= as.Date(start), date <= as.Date(end), + country == '', transport == '', version == version_) %>% + select(date, clients) %>% + rename(users = clients) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_userstats_bridge_combined <- function(start, end, country) { + top <- 3 + country <- ifelse(country == "all", NA, country) + load(paste(rdata_dir, "userstats-bridge-combined.RData", sep = "")) + u <- data + u <- u[u$date >= start & u$date <= end + & (is.na(country) | u$country == country), ] + a <- aggregate(list(mid = (u$high + u$low) / 2), + by = list(transport = u$transport), FUN = sum) + a <- a[order(a$mid, decreasing = TRUE)[1:top], ] + u <- u[u$transport %in% a$transport, ] + u +} + plot_userstats_bridge_combined <- function(start, end, country, path) { if (country == "all") { plot_userstats_bridge_country(start, end, country, path) } else { - top <- 3 - country <- ifelse(country == "all", NA, country) - load(paste(rdata_dir, "userstats-bridge-combined.RData", sep = "")) - u <- data - u <- u[u$date >= start & u$date <= end - & (is.na(country) | u$country == country), ] - a <- aggregate(list(mid = (u$high + u$low) / 2), - by = list(transport = u$transport), FUN = sum) - a <- a[order(a$mid, decreasing = TRUE)[1:top], ] - u <- u[u$transport %in% a$transport, ] + u <- prepare_userstats_bridge_combined(start, end, country) title <- paste("Bridge users by transport from ", countryname(country), sep = "") ggplot(u, aes(x = as.Date(date), ymin = low, ymax = high, @@ -883,8 +1082,8 @@ plot_userstats_bridge_combined <- function(start, end, country, path) { scale_x_date(name = "", breaks = custom_breaks, labels = custom_labels, minor_breaks = custom_minor_breaks) + scale_y_continuous(name = "", limits = c(0, NA), labels = formatter) + - scale_colour_hue(paste("Top-", top, " transports", sep = "")) + - scale_fill_hue(paste("Top-", top, " transports", sep = "")) + + scale_colour_hue("Top-3 transports") + + scale_fill_hue("Top-3 transports") + ggtitle(title) + labs(caption = copyright_notice) + theme(legend.position = "top") @@ -892,15 +1091,36 @@ plot_userstats_bridge_combined <- function(start, end, country, path) { } } -plot_advbwdist_perc <- function(start, end, p, path) { +write_userstats_bridge_combined <- function(start, end, country, path) { + if (country == "all") { + write_userstats_bridge_country(start, end, country, path) + } else { + prepare_userstats_bridge_combined(start, end, country) %>% + select(date, transport, low, high) %>% + mutate(transport = ifelse(transport == '<OR>', + 'default_or_protocol', transport)) %>% + gather(variable, value, -(date:transport)) %>% + unite(temp, transport, variable) %>% + spread(temp, value) %>% + write.csv(path, quote = FALSE, row.names = FALSE) + } +} + +prepare_advbwdist_perc <- function(start, end, p) { t <- read.csv(paste(stats_dir, "advbwdist.csv", sep = ""), stringsAsFactors = FALSE) t <- t[t$date >= start & t$date <= end & t$percentile %in% as.numeric(p), ] - t <- data.frame(date = t$date, advbw = t$advbw * 8 / 1e9, - variable = ifelse(t$isexit != "t", "All relays", - "Exits only"), - percentile = as.factor(t$percentile)) + t <- data.frame(date = t$date, percentile = as.factor(t$percentile), + variable = ifelse(t$isexit != "t", "all", "exits"), + advbw = t$advbw * 8 / 1e9) + t +} + +plot_advbwdist_perc <- function(start, end, p, path) { + t <- prepare_advbwdist_perc(start, end, p) + t$variable <- ifelse(t$variable == "all", "All relays", + "Exits only") ggplot(t, aes(x = as.Date(date), y = advbw, colour = percentile)) + facet_grid(variable ~ .) + geom_line() + @@ -915,14 +1135,27 @@ plot_advbwdist_perc <- function(start, end, p, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_advbwdist_relay <- function(start, end, n, path) { +write_advbwdist_perc <- function(start, end, p, path) { + prepare_advbwdist_perc(start, end, p) %>% + unite(temp, variable, percentile) %>% + spread(temp, advbw) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_advbwdist_relay <- function(start, end, n) { t <- read.csv(paste(stats_dir, "advbwdist.csv", sep = ""), stringsAsFactors = FALSE) t <- t[t$date >= start & t$date <= end & t$relay %in% as.numeric(n), ] - t <- data.frame(date = t$date, advbw = t$advbw * 8 / 1e9, - variable = ifelse(t$isexit != "t", "All relays", - "Exits only"), - relay = as.factor(t$relay)) + t <- data.frame(date = t$date, relay = as.factor(t$relay), + variable = ifelse(t$isexit != "t", "all", "exits"), + advbw = t$advbw * 8 / 1e9) + t +} + +plot_advbwdist_relay <- function(start, end, n, path) { + t <- prepare_advbwdist_relay(start, end, n) + t$variable <- ifelse(t$variable == "all", "All relays", + "Exits only") ggplot(t, aes(x = as.Date(date), y = advbw, colour = relay)) + facet_grid(variable ~ .) + geom_line() + @@ -936,14 +1169,25 @@ plot_advbwdist_relay <- function(start, end, n, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_hidserv_dir_onions_seen <- function(start, end, path) { +write_advbwdist_relay <- function(start, end, n, path) { + prepare_advbwdist_relay(start, end, n) %>% + unite(temp, variable, relay) %>% + spread(temp, advbw) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_hidserv_dir_onions_seen <- function(start, end) { h <- read.csv(paste(stats_dir, "hidserv.csv", sep = ""), stringsAsFactors = FALSE) h <- h[h$date >= start & h$date <= end & h$type == "dir-onions-seen", ] - h <- rbind(data.frame(date = NA, wiqm = 0), - data.frame(date = as.Date(h$date, "%Y-%m-%d"), - wiqm = ifelse(h$frac >= 0.01, h$wiqm, NA))) - ggplot(h, aes(x = as.Date(date, origin = "1970-01-01"), y = wiqm)) + + h <- data.frame(date = as.Date(h$date, "%Y-%m-%d"), + onions = ifelse(h$frac >= 0.01, h$wiqm, NA)) + h +} + +plot_hidserv_dir_onions_seen <- function(start, end, path) { + h <- prepare_hidserv_dir_onions_seen(start, end) + ggplot(h, aes(x = as.Date(date, origin = "1970-01-01"), y = onions)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, labels = custom_labels, minor_breaks = custom_minor_breaks) + @@ -953,16 +1197,25 @@ plot_hidserv_dir_onions_seen <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_hidserv_rend_relayed_cells <- function(start, end, path) { +write_hidserv_dir_onions_seen <- function(start, end, path) { + prepare_hidserv_dir_onions_seen(start, end) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_hidserv_rend_relayed_cells <- function(start, end) { h <- read.csv(paste(stats_dir, "hidserv.csv", sep = ""), stringsAsFactors = FALSE) h <- h[h$date >= start & h$date <= end & h$type == "rend-relayed-cells", ] - h <- rbind(data.frame(date = NA, wiqm = 0), - data.frame(date = as.Date(h$date, "%Y-%m-%d"), - wiqm = ifelse(h$frac >= 0.01, h$wiqm, NA))) - ggplot(h, aes(x = as.Date(date, origin = "1970-01-01"), - y = wiqm * 8 * 512 / (86400 * 1e9))) + + h <- data.frame(date = as.Date(h$date, "%Y-%m-%d"), + relayed = ifelse(h$frac >= 0.01, + h$wiqm * 8 * 512 / (86400 * 1e9), NA)) + h +} + +plot_hidserv_rend_relayed_cells <- function(start, end, path) { + h <- prepare_hidserv_rend_relayed_cells(start, end) + ggplot(h, aes(x = as.Date(date, origin = "1970-01-01"), y = relayed)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, labels = custom_labels, minor_breaks = custom_minor_breaks) + @@ -973,15 +1226,22 @@ plot_hidserv_rend_relayed_cells <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_hidserv_frac_reporting <- function(start, end, path) { +write_hidserv_rend_relayed_cells <- function(start, end, path) { + prepare_hidserv_rend_relayed_cells(start, end) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_hidserv_frac_reporting <- function(start, end) { h <- read.csv(paste(stats_dir, "hidserv.csv", sep = ""), stringsAsFactors = FALSE) h <- h[h$date >= start & h$date <= end, ] - h <- rbind(data.frame(date = NA, frac = 0, - type = c("rend-relayed-cells", - "dir-onions-seen")), - data.frame(date = as.Date(h$date, "%Y-%m-%d"), - frac = h$frac, type = h$type)) + h <- data.frame(date = as.Date(h$date, "%Y-%m-%d"), + frac = h$frac, type = h$type) + h +} + +plot_hidserv_frac_reporting <- function(start, end, path) { + h <- prepare_hidserv_frac_reporting(start, end) ggplot(h, aes(x = as.Date(date, origin = "1970-01-01"), y = frac, colour = type)) + geom_line() + @@ -1000,11 +1260,23 @@ plot_hidserv_frac_reporting <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_webstats_tb <- function(start, end, path) { +write_hidserv_frac_reporting <- function(start, end, path) { + prepare_hidserv_frac_reporting(start, end) %>% + mutate(type = ifelse(type == "dir-onions-seen", "onions", "relayed")) %>% + spread(type, frac) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_webstats_tb <- function(start, end) { load(paste(rdata_dir, "webstats-tb.RData", sep = "")) d <- data d <- d[d$log_date >= start & d$log_date <= end, ] d$request_type <- factor(d$request_type) + d +} + +plot_webstats_tb <- function(start, end, path) { + d <- prepare_webstats_tb(start, end) levels(d$request_type) <- list( 'Initial downloads' = 'tbid', 'Signature downloads' = 'tbsd', @@ -1024,12 +1296,26 @@ plot_webstats_tb <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_webstats_tb_platform <- function(start, end, path) { +write_webstats_tb <- function(start, end, path) { + prepare_webstats_tb(start, end) %>% + rename(date = log_date) %>% + spread(request_type, count) %>% + rename(initial_downloads = tbid, signature_downloads = tbsd, + update_pings = tbup, update_requests = tbur) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_webstats_tb_platform <- function(start, end) { d <- read.csv(paste(stats_dir, "webstats.csv", sep = ""), stringsAsFactors = FALSE) d <- d[d$log_date >= start & d$log_date <= end & d$request_type == 'tbid', ] d <- aggregate(list(count = d$count), by = list(log_date = as.Date(d$log_date), platform = d$platform), FUN = sum) + d +} + +plot_webstats_tb_platform <- function(start, end, path) { + d <- prepare_webstats_tb_platform(start, end) ggplot(d, aes(x = log_date, y = count, colour = platform)) + geom_point() + geom_line() + @@ -1046,6 +1332,14 @@ plot_webstats_tb_platform <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } +write_webstats_tb_platform <- function(start, end, path) { + prepare_webstats_tb_platform(start, end) %>% + rename(date = log_date) %>% + spread(platform, count) %>% + rename(linux = l, macos = m, windows = w) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + plot_webstats_tb_locale <- function(start, end, path) { d <- read.csv(paste(stats_dir, "webstats.csv", sep = ""), stringsAsFactors = FALSE) @@ -1072,11 +1366,38 @@ plot_webstats_tb_locale <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_webstats_tm <- function(start, end, path) { +# Ideally, this function would share code with plot_webstats_tb_locale +# by using a common prepare_webstats_tb_locale function. This just +# turned out to be a bit harder than for other functions, because +# plot_webstats_tb_locale needs the preliminary data frame e for its +# breaks and labels. Left as future work. +write_webstats_tb_locale <- function(start, end, path) { + d <- read.csv(paste(stats_dir, "webstats.csv", sep = ""), + stringsAsFactors = FALSE) + d <- d[d$log_date >= start & d$log_date <= end & d$request_type == 'tbid', ] + e <- d + e <- aggregate(list(count = e$count), by = list(locale = e$locale), FUN = sum) + e <- e[order(e$count, decreasing = TRUE), ] + e <- e[1:5, ] + d <- aggregate(list(count = d$count), by = list(log_date = as.Date(d$log_date), + locale = ifelse(d$locale %in% e$locale, d$locale, 'other')), FUN = sum) + d %>% + mutate(locale = tolower(locale)) %>% + rename(date = log_date) %>% + spread(locale, count) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_webstats_tm <- function(start, end) { load(paste(rdata_dir, "webstats-tm.RData", sep = "")) d <- data d <- d[d$log_date >= start & d$log_date <= end, ] d$request_type <- factor(d$request_type) + d +} + +plot_webstats_tm <- function(start, end, path) { + d <- prepare_webstats_tm(start, end) levels(d$request_type) <- list( 'Initial downloads' = 'tmid', 'Update pings' = 'tmup') @@ -1094,7 +1415,15 @@ plot_webstats_tm <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_relays_ipv6 <- function(start, end, path) { +write_webstats_tm <- function(start, end, path) { + prepare_webstats_tm(start, end) %>% + rename(date = log_date) %>% + spread(request_type, count) %>% + rename(initial_downloads = tmid, update_pings = tmup) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_relays_ipv6 <- function(start, end) { read.csv(paste(stats_dir, "ipv6servers.csv", sep = ""), colClasses = c("valid_after_date" = "Date")) %>% filter(valid_after_date >= as.Date(start), @@ -1106,7 +1435,11 @@ plot_relays_ipv6 <- function(start, end, path) { exiting = sum(server_count_sum_avg[exiting_ipv6_relay == 't'])) %>% complete(valid_after_date = full_seq(valid_after_date, period = 1)) %>% gather(total, announced, reachable, exiting, key = "category", - value = "count") %>% + value = "count") +} + +plot_relays_ipv6 <- function(start, end, path) { + prepare_relays_ipv6(start, end) %>% ggplot(aes(x = valid_after_date, y = count, colour = category)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, @@ -1122,7 +1455,14 @@ plot_relays_ipv6 <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_bridges_ipv6 <- function(start, end, path) { +write_relays_ipv6 <- function(start, end, path) { + prepare_relays_ipv6(start, end) %>% + rename(date = valid_after_date) %>% + spread(category, count) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_bridges_ipv6 <- function(start, end) { read.csv(paste(stats_dir, "ipv6servers.csv", sep = ""), colClasses = c("valid_after_date" = "Date")) %>% filter(valid_after_date >= as.Date(start), @@ -1131,7 +1471,11 @@ plot_bridges_ipv6 <- function(start, end, path) { summarize(total = sum(server_count_sum_avg), announced = sum(server_count_sum_avg[announced_ipv6 == 't'])) %>% complete(valid_after_date = full_seq(valid_after_date, period = 1)) %>% - gather(total, announced, key = "category", value = "count") %>% + gather(total, announced, key = "category", value = "count") +} + +plot_bridges_ipv6 <- function(start, end, path) { + prepare_bridges_ipv6(start, end) %>% ggplot(aes(x = valid_after_date, y = count, colour = category)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, @@ -1146,7 +1490,14 @@ plot_bridges_ipv6 <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } -plot_advbw_ipv6 <- function(start, end, path) { +write_bridges_ipv6 <- function(start, end, path) { + prepare_bridges_ipv6(start, end) %>% + rename(date = valid_after_date) %>% + spread(category, count) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + +prepare_advbw_ipv6 <- function(start, end) { read.csv(paste(stats_dir, "ipv6servers.csv", sep = ""), colClasses = c("valid_after_date" = "Date")) %>% filter(valid_after_date >= as.Date(start), @@ -1163,9 +1514,13 @@ plot_advbw_ipv6 <- function(start, end, path) { exiting_ipv6_relay != 'f'])) %>% complete(valid_after_date = full_seq(valid_after_date, period = 1)) %>% gather(total, total_guard, total_exit, reachable_guard, reachable_exit, - exiting, key = "category", value = "count") %>% - ggplot(aes(x = valid_after_date, y = (count * 8) / 1e9, - colour = category)) + + exiting, key = "category", value = "advbw") %>% + mutate(advbw = advbw * 8 / 1e9) +} + +plot_advbw_ipv6 <- function(start, end, path) { + prepare_advbw_ipv6(start, end) %>% + ggplot(aes(x = valid_after_date, y = advbw, colour = category)) + geom_line() + scale_x_date(name = "", breaks = custom_breaks, labels = custom_labels, minor_breaks = custom_minor_breaks) + @@ -1183,3 +1538,10 @@ plot_advbw_ipv6 <- function(start, end, path) { ggsave(filename = path, width = 8, height = 5, dpi = 150) } +write_advbw_ipv6 <- function(start, end, path) { + prepare_advbw_ipv6(start, end) %>% + rename(date = valid_after_date) %>% + spread(category, advbw) %>% + write.csv(path, quote = FALSE, row.names = FALSE) +} + diff --git a/src/main/java/org/torproject/metrics/web/GraphImageServlet.java b/src/main/java/org/torproject/metrics/web/GraphImageServlet.java index 0644fa2..f24d698 100644 --- a/src/main/java/org/torproject/metrics/web/GraphImageServlet.java +++ b/src/main/java/org/torproject/metrics/web/GraphImageServlet.java @@ -41,7 +41,8 @@ public class GraphImageServlet extends HttpServlet { String requestedGraph = request.getRequestURI(); String fileType = null; if (requestedGraph.endsWith(".png") - || requestedGraph.endsWith(".pdf")) { + || requestedGraph.endsWith(".pdf") + || requestedGraph.endsWith(".csv")) { fileType = requestedGraph.substring(requestedGraph.length() - 3); requestedGraph = requestedGraph.substring(0, requestedGraph.length() - 4); diff --git a/src/main/java/org/torproject/metrics/web/GraphServlet.java b/src/main/java/org/torproject/metrics/web/GraphServlet.java index 31116b4..2781be0 100644 --- a/src/main/java/org/torproject/metrics/web/GraphServlet.java +++ b/src/main/java/org/torproject/metrics/web/GraphServlet.java @@ -151,7 +151,6 @@ public class GraphServlet extends MetricServlet { } request.setAttribute("description", this.descriptions.get(requestedId)); - request.setAttribute("data", this.data.get(requestedId)); SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd"); dateFormat.setTimeZone(TimeZone.getTimeZone("UTC")); Date defaultEndDate = new Date(); diff --git a/src/main/java/org/torproject/metrics/web/LinkServlet.java b/src/main/java/org/torproject/metrics/web/LinkServlet.java index b2687f1..3dcaeb1 100644 --- a/src/main/java/org/torproject/metrics/web/LinkServlet.java +++ b/src/main/java/org/torproject/metrics/web/LinkServlet.java @@ -50,7 +50,6 @@ public class LinkServlet extends MetricServlet { } request.setAttribute("description", this.descriptions.get(requestedId)); - request.setAttribute("data", this.data.get(requestedId)); request.getRequestDispatcher("WEB-INF/link.jsp").forward(request, response); } diff --git a/src/main/java/org/torproject/metrics/web/Metric.java b/src/main/java/org/torproject/metrics/web/Metric.java index a5604eb..701351f 100644 --- a/src/main/java/org/torproject/metrics/web/Metric.java +++ b/src/main/java/org/torproject/metrics/web/Metric.java @@ -21,8 +21,6 @@ public class Metric { private String[] parameters; - private String[] data; - private String[] table_headers; private String[] table_cell_formats; @@ -75,10 +73,6 @@ public class Metric { return this.data_column_spec; } - public String[] getData() { - return this.data; - } - public boolean getIncludeRelatedEvents() { return this.includeRelatedEvents; } diff --git a/src/main/java/org/torproject/metrics/web/MetricServlet.java b/src/main/java/org/torproject/metrics/web/MetricServlet.java index f5b29dd..b3de046 100644 --- a/src/main/java/org/torproject/metrics/web/MetricServlet.java +++ b/src/main/java/org/torproject/metrics/web/MetricServlet.java @@ -31,8 +31,6 @@ public abstract class MetricServlet extends AnyServlet { protected Map<String, String[]> tableCellFormats = new HashMap<>(); - protected Map<String, String[]> data = new HashMap<>(); - protected Map<String, Category> categoriesById = new HashMap<>(); protected Set<String> includeRelatedEvents = new HashSet<>(); @@ -62,9 +60,6 @@ public abstract class MetricServlet extends AnyServlet { if (metric.getTableCellFormats() != null) { this.tableCellFormats.put(id, metric.getTableCellFormats()); } - if (metric.getData() != null) { - this.data.put(id, metric.getData()); - } if (metric.getIncludeRelatedEvents()) { this.includeRelatedEvents.add(id); } diff --git a/src/main/java/org/torproject/metrics/web/RObjectGenerator.java b/src/main/java/org/torproject/metrics/web/RObjectGenerator.java index c7d0041..5d9b29c 100644 --- a/src/main/java/org/torproject/metrics/web/RObjectGenerator.java +++ b/src/main/java/org/torproject/metrics/web/RObjectGenerator.java @@ -124,8 +124,13 @@ public class RObjectGenerator implements ServletContextListener { if (checkedParameters == null) { return null; } - StringBuilder queryBuilder = - new StringBuilder().append(function).append("("); + StringBuilder queryBuilder = new StringBuilder(); + if ("csv".equalsIgnoreCase(fileType)) { + queryBuilder.append("write_"); + } else { + queryBuilder.append("plot_"); + } + queryBuilder.append(function).append("("); StringBuilder imageFilenameBuilder = new StringBuilder(requestedGraph); for (Map.Entry<String, String[]> parameter diff --git a/src/main/java/org/torproject/metrics/web/TableServlet.java b/src/main/java/org/torproject/metrics/web/TableServlet.java index d37852f..84f46ee 100644 --- a/src/main/java/org/torproject/metrics/web/TableServlet.java +++ b/src/main/java/org/torproject/metrics/web/TableServlet.java @@ -62,7 +62,6 @@ public class TableServlet extends MetricServlet { this.descriptions.get(requestedId)); request.setAttribute("tableheader", this.tableHeaders.get(requestedId)); - request.setAttribute("data", this.data.get(requestedId)); SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd"); dateFormat.setTimeZone(TimeZone.getTimeZone("UTC")); Date defaultEndDate = new Date(); diff --git a/src/main/resources/web.xml b/src/main/resources/web.xml index 99df1db..a5c3212 100644 --- a/src/main/resources/web.xml +++ b/src/main/resources/web.xml @@ -121,62 +121,91 @@ <servlet-name>GraphImage</servlet-name> <url-pattern>/networksize.png</url-pattern> <url-pattern>/networksize.pdf</url-pattern> + <url-pattern>/networksize.csv</url-pattern> <url-pattern>/relaycountries.png</url-pattern> <url-pattern>/relaycountries.pdf</url-pattern> + <url-pattern>/relaycountries.csv</url-pattern> <url-pattern>/relayflags.png</url-pattern> <url-pattern>/relayflags.pdf</url-pattern> + <url-pattern>/relayflags.csv</url-pattern> <url-pattern>/versions.png</url-pattern> <url-pattern>/versions.pdf</url-pattern> + <url-pattern>/versions.csv</url-pattern> <url-pattern>/platforms.png</url-pattern> <url-pattern>/platforms.pdf</url-pattern> + <url-pattern>/platforms.csv</url-pattern> <url-pattern>/bandwidth.png</url-pattern> <url-pattern>/bandwidth.pdf</url-pattern> + <url-pattern>/bandwidth.csv</url-pattern> <url-pattern>/bwhist-flags.png</url-pattern> <url-pattern>/bwhist-flags.pdf</url-pattern> + <url-pattern>/bwhist-flags.csv</url-pattern> <url-pattern>/bandwidth-flags.png</url-pattern> <url-pattern>/bandwidth-flags.pdf</url-pattern> + <url-pattern>/bandwidth-flags.csv</url-pattern> <url-pattern>/dirbytes.png</url-pattern> <url-pattern>/dirbytes.pdf</url-pattern> + <url-pattern>/dirbytes.csv</url-pattern> <url-pattern>/torperf.png</url-pattern> <url-pattern>/torperf.pdf</url-pattern> + <url-pattern>/torperf.csv</url-pattern> <url-pattern>/torperf-failures.png</url-pattern> <url-pattern>/torperf-failures.pdf</url-pattern> + <url-pattern>/torperf-failures.csv</url-pattern> <url-pattern>/connbidirect.png</url-pattern> <url-pattern>/connbidirect.pdf</url-pattern> + <url-pattern>/connbidirect.csv</url-pattern> <url-pattern>/userstats-relay-country.png</url-pattern> <url-pattern>/userstats-relay-country.pdf</url-pattern> + <url-pattern>/userstats-relay-country.csv</url-pattern> <url-pattern>/userstats-bridge-country.png</url-pattern> <url-pattern>/userstats-bridge-country.pdf</url-pattern> + <url-pattern>/userstats-bridge-country.csv</url-pattern> <url-pattern>/userstats-bridge-transport.png</url-pattern> <url-pattern>/userstats-bridge-transport.pdf</url-pattern> + <url-pattern>/userstats-bridge-transport.csv</url-pattern> <url-pattern>/userstats-bridge-combined.png</url-pattern> <url-pattern>/userstats-bridge-combined.pdf</url-pattern> + <url-pattern>/userstats-bridge-combined.csv</url-pattern> <url-pattern>/userstats-bridge-version.png</url-pattern> <url-pattern>/userstats-bridge-version.pdf</url-pattern> + <url-pattern>/userstats-bridge-version.csv</url-pattern> <url-pattern>/advbwdist-perc.png</url-pattern> <url-pattern>/advbwdist-perc.pdf</url-pattern> + <url-pattern>/advbwdist-perc.csv</url-pattern> <url-pattern>/advbwdist-relay.png</url-pattern> <url-pattern>/advbwdist-relay.pdf</url-pattern> + <url-pattern>/advbwdist-relay.csv</url-pattern> <url-pattern>/hidserv-dir-onions-seen.png</url-pattern> <url-pattern>/hidserv-dir-onions-seen.pdf</url-pattern> + <url-pattern>/hidserv-dir-onions-seen.csv</url-pattern> <url-pattern>/hidserv-rend-relayed-cells.png</url-pattern> <url-pattern>/hidserv-rend-relayed-cells.pdf</url-pattern> + <url-pattern>/hidserv-rend-relayed-cells.csv</url-pattern> <url-pattern>/hidserv-frac-reporting.png</url-pattern> <url-pattern>/hidserv-frac-reporting.pdf</url-pattern> + <url-pattern>/hidserv-frac-reporting.csv</url-pattern> <url-pattern>/webstats-tb.png</url-pattern> <url-pattern>/webstats-tb.pdf</url-pattern> + <url-pattern>/webstats-tb.csv</url-pattern> <url-pattern>/webstats-tb-platform.png</url-pattern> <url-pattern>/webstats-tb-platform.pdf</url-pattern> + <url-pattern>/webstats-tb-platform.csv</url-pattern> <url-pattern>/webstats-tb-locale.png</url-pattern> <url-pattern>/webstats-tb-locale.pdf</url-pattern> + <url-pattern>/webstats-tb-locale.csv</url-pattern> <url-pattern>/webstats-tm.png</url-pattern> <url-pattern>/webstats-tm.pdf</url-pattern> - <url-pattern>/relays-ipv6.pdf</url-pattern> + <url-pattern>/webstats-tm.csv</url-pattern> <url-pattern>/relays-ipv6.png</url-pattern> - <url-pattern>/bridges-ipv6.pdf</url-pattern> + <url-pattern>/relays-ipv6.pdf</url-pattern> + <url-pattern>/relays-ipv6.csv</url-pattern> <url-pattern>/bridges-ipv6.png</url-pattern> - <url-pattern>/advbw-ipv6.pdf</url-pattern> + <url-pattern>/bridges-ipv6.pdf</url-pattern> + <url-pattern>/bridges-ipv6.csv</url-pattern> <url-pattern>/advbw-ipv6.png</url-pattern> + <url-pattern>/advbw-ipv6.pdf</url-pattern> + <url-pattern>/advbw-ipv6.csv</url-pattern> </servlet-mapping> <servlet> diff --git a/src/main/resources/web/json/metrics.json b/src/main/resources/web/json/metrics.json index 0b8aca7..0739b88 100644 --- a/src/main/resources/web/json/metrics.json +++ b/src/main/resources/web/json/metrics.json @@ -4,13 +4,10 @@ "title": "Relays and bridges", "type": "Graph", "description": "This graph shows the number of running <a href=\"glossary.html#relay\">relays</a> and <a href=\"glossary.html#bridge\">bridges</a> in the network.", - "function": "plot_networksize", + "function": "networksize", "parameters": [ "start", "end" - ], - "data": [ - "servers" ] }, { @@ -18,14 +15,11 @@ "title": "Relays by relay flag", "type": "Graph", "description": "This graph shows the number of running <a href=\"glossary.html#relay\">relays</a> that have had certain <a href=\"glossary.html#relay-flag\">flags</a> assigned by the <a href=\"glossary.html#directory-authority\">directory authorities</a>. These flags indicate that a relay should be preferred for either guard (\"Guard\") or exit positions (\"Exit\"), that a relay is suitable for high-bandwidth (\"Fast\") or long-lived circuits (\"Stable\"), or that a relay is considered a onion service directory (\"HSDir\").", - "function": "plot_relayflags", + "function": "relayflags", "parameters": [ "start", "end", "flag" - ], - "data": [ - "servers" ] }, { @@ -33,13 +27,10 @@ "title": "Relays by tor version", "type": "Graph", "description": "This graph shows the number of running <a href=\"glossary.html#relay\">relays</a> by tor software version. Relays report their tor software version when they announce themselves in the network. More details on when these versions were declared stable or unstable can be found on the <a href=\"https://www.torproject.org/download/download.html\">download page</a> and in the <a href=\"https://gitweb.torproject.org/tor.git/tree/ChangeLog\">changes file</a>.", - "function": "plot_versions", + "function": "versions", "parameters": [ "start", "end" - ], - "data": [ - "servers" ] }, { @@ -47,13 +38,10 @@ "title": "Relays by platform", "type": "Graph", "description": "This graph shows the number of running <a href=\"glossary.html#relay\">relays</a> by operating system. Relays report their operating system when they announce themselves in the network.", - "function": "plot_platforms", + "function": "platforms", "parameters": [ "start", "end" - ], - "data": [ - "servers" ] }, { @@ -61,13 +49,10 @@ "title": "Relays by IP version", "type": "Graph", "description": "This graph shows the number of <a href=\"glossary.html#relay\">relays</a> supporting IPv6 as compared to all relays. A relay can support IPv6 by announcing an IPv6 address and port for the OR protocol, which may then be confirmed as reachable by the <a href=\"glossary.html#directory-authority\">directory authorities</a>, and by permitting exiting to IPv6 targets. These sets are not distinct, because relays can have various combinations of announced/confirmed OR ports and exit policies.", - "function": "plot_relays_ipv6", + "function": "relays_ipv6", "parameters": [ "start", "end" - ], - "data": [ - "ipv6servers" ] }, { @@ -75,13 +60,10 @@ "title": "Bridges by IP version", "type": "Graph", "description": "This graph shows the number of <a href=\"glossary.html#bridge\">bridges</a> supporting IPv6 as compared to all bridges. A bridge can support IPv6 by announcing an IPv6 address and port for the OR protocol.", - "function": "plot_bridges_ipv6", + "function": "bridges_ipv6", "parameters": [ "start", "end" - ], - "data": [ - "ipv6servers" ] }, { @@ -89,13 +71,10 @@ "title": "Total relay bandwidth", "type": "Graph", "description": "This graph shows the total <a href=\"glossary.html#advertised-bandwidth\">advertised</a> and <a href=\"glossary.html#bandwidth-history\">consumed bandwidth</a> of all <a href=\"glossary.html#relay\">relays</a> in the network.", - "function": "plot_bandwidth", + "function": "bandwidth", "parameters": [ "start", "end" - ], - "data": [ - "bandwidth" ] }, { @@ -103,13 +82,10 @@ "title": "Consumed bandwidth by Exit/Guard flag combination", "type": "Graph", "description": "This graph shows the <a href=\"glossary.html#bandwidth-history\">consumed bandwidth</a> reported by relays, subdivided into four distinct subsets by assigned \"Exit\" and/or \"Guard\" <a href=\"glossary.html#relay-flag\">flags</a>.", - "function": "plot_bwhist_flags", + "function": "bwhist_flags", "parameters": [ "start", "end" - ], - "data": [ - "bandwidth" ] }, { @@ -117,13 +93,10 @@ "title": "Advertised and consumed bandwidth by relay flag", "type": "Graph", "description": "This graph shows <a href=\"glossary.html#advertised-bandwidth\">advertised</a> and <a href=\"glossary.html#bandwidth-history\">consumed bandwidth</a> of relays with either \"Exit\" or \"Guard\" <a href=\"glossary.html#relay-flag\">flags</a> assigned by the directory authorities. These sets are not distinct, because a relay that has both the \"Exit\" and \"Guard\" flags assigned will be included in both sets.", - "function": "plot_bandwidth_flags", + "function": "bandwidth_flags", "parameters": [ "start", "end" - ], - "data": [ - "bandwidth" ] }, { @@ -131,13 +104,10 @@ "title": "Bandwidth spent on answering directory requests", "type": "Graph", "description": "This graph shows the portion of <a href=\"glossary.html#bandwidth-history\">consumed bandwidth</a> that <a href=\"glossary.html#directory-authority\">directory authorities</a> and <a href=\"glossary.html#directory-mirror\">mirrors</a> have spent on answering directory requests. Not all directories report these statistics, so the graph shows an estimation of total consumed bandwidth as it would be observed if all directories reported these statistics.", - "function": "plot_dirbytes", + "function": "dirbytes", "parameters": [ "start", "end" - ], - "data": [ - "bandwidth" ] }, { @@ -145,13 +115,10 @@ "title": "Advertised bandwidth by IP version", "type": "Graph", "description": "This graph shows total <a href=\"glossary.html#advertised-bandwidth\">advertised bandwidth</a> by relays supporting IPv6 as compared to all relays. A relay can support IPv6 by announcing an IPv6 address and port for the OR protocol, which may then be confirmed as reachable by the <a href=\"glossary.html#directory-authority\">directory authorities</a>, and by permitting exiting to IPv6 targets. In some cases, relay sets are broken down by whether relays got the \"Guard\" and/or \"Exit\" <a href=\"glossary.html#relay-flag\">relay flags</a> indicating their special qualification for the first or last position in a <a href=\"glossary.html#circuit\">circuit</a>. These sets are not distinct, because relays can have various combinations of announced/confirmed OR ports, exit policies, and relay flags.", - "function": "plot_advbw_ipv6", + "function": "advbw_ipv6", "parameters": [ "start", "end" - ], - "data": [ - "ipv6servers" ] }, { @@ -159,14 +126,11 @@ "title": "Advertised bandwidth distribution", "type": "Graph", "description": "This graph shows the distribution of the <a href=\"glossary.html#advertised-bandwidth\">advertised bandwidth</a> of relays in the network. Each percentile represents the advertised bandwidth that a given percentage of relays does not exceed (and that in turn the remaining relays either match or exceed). For example, 99% of relays advertise at most the bandwidth value shown in the 99th percentile line (and the remaining 1% advertise at least that amount).", - "function": "plot_advbwdist_perc", + "function": "advbwdist_perc", "parameters": [ "start", "end", "p" - ], - "data": [ - "advbwdist" ] }, { @@ -174,14 +138,11 @@ "title": "Advertised bandwidth of n-th fastest relays", "type": "Graph", "description": "This graph shows the <a href=\"glossary.html#advertised-bandwidth\">advertised bandwidth</a> of the n-th fastest relays in the network for different values of n.", - "function": "plot_advbwdist_relay", + "function": "advbwdist_relay", "parameters": [ "start", "end", "n" - ], - "data": [ - "advbwdist" ] }, { @@ -194,16 +155,13 @@ "title": "Relay users", "type": "Graph", "description": "This graph shows the estimated number of directly-connecting <a href=\"glossary.html#client\">clients</a>; that is, it excludes clients connecting via <a href=\"glossary.html#bridge\">bridges</a>. These estimates are derived from the number of directory requests counted on <a href=\"glossary.html#directory-authority\">directory authorities</a> and <a href=\"glossary.html#directory-mirror\">mirrors</a>. Relays resolve client IP addresses to country codes, so that graphs are available for most countries. Furthermore, it is possible to display indications of censorship events as obtained from an anomaly-based censorship-detection system (for more details, see this <a href=\"https://research.torproject.org/techreports/detector-2011-09-09.pdf\">technical report</a>). For further details see these <a href=\"https://gitweb.torproject.org/metrics-web.git/tree/src/main/resources/doc/u…">questions and answers about user statistics</a>.", - "function": "plot_userstats_relay_country", + "function": "userstats_relay_country", "parameters": [ "start", "end", "country", "events" ], - "data": [ - "clients" - ], "include_related_events": true }, { @@ -223,9 +181,6 @@ "table_cell_formats": [ "<a href=\"userstats-relay-country.html?graph=userstats-relay-country&country=${cc}\">${country}</a>&emsp;", "${abs} (${rel} %)" - ], - "data": [ - "clients" ] }, { @@ -247,9 +202,6 @@ "<a href=\"userstats-relay-country.html?graph=userstats-relay-country&country=${cc}&events=on\">${country}</a>&emsp;", "${downturns}", "${upturns}" - ], - "data": [ - "clients" ] }, { @@ -257,15 +209,12 @@ "title": "Bridge users by country", "type": "Graph", "description": "This graph shows the estimated number of <a href=\"glossary.html#client\">clients</a> connecting via <a href=\"glossary.html#bridge\">bridges</a>. These numbers are derived from directory requests counted on bridges. Bridges resolve client IP addresses of incoming directory requests to country codes, so that graphs are available for most countries. For further details see these <a href=\"https://gitweb.torproject.org/metrics-web.git/tree/src/main/resources/doc/u…">questions and answers about user statistics</a>.", - "function": "plot_userstats_bridge_country", + "function": "userstats_bridge_country", "parameters": [ "start", "end", "country" ], - "data": [ - "clients" - ], "include_related_events": true }, { @@ -285,9 +234,6 @@ "table_cell_formats": [ "<a href=\"userstats-bridge-country.html?graph=userstats-bridge-country&country=${cc}\">${country}</a>&emsp;", "${abs} (${rel} %)" - ], - "data": [ - "clients" ] }, { @@ -295,15 +241,12 @@ "title": "Bridge users by transport", "type": "Graph", "description": "This graph shows the estimated number of <a href=\"glossary.html#client\">clients</a> connecting via <a href=\"glossary.html#bridge\">bridges</a>. These numbers are derived from directory requests counted on bridges. Bridges distinguish connecting clients by transport protocol, which may include <a href=\"glossary.html#pluggable-transport\">pluggable transports</a>, so that graphs are available for different transports. For further details see these <a href=\"https://gitweb.torproject.org/metrics-web.git/tree/src/main/resources/doc/u…">questions and answers about user statistics</a>.", - "function": "plot_userstats_bridge_transport", + "function": "userstats_bridge_transport", "parameters": [ "start", "end", "transport" ], - "data": [ - "clients" - ], "include_related_events": true }, { @@ -311,15 +254,12 @@ "title": "Bridge users by country and transport", "type": "Graph", "description": "This graph shows the estimated number of <a href=\"glossary.html#client\">clients</a> connecting via <a href=\"glossary.html#bridge\">bridges</a>. These numbers are derived from directory requests counted on bridges. Bridges resolve client IP addresses of incoming directory requests to country codes, and they distinguish connecting clients by transport protocol, which may include <a href=\"glossary.html#pluggable-transport\">pluggable transports</a>. Even though bridges don't report a combination of clients by country and transport, it's possible to derive and graph lower and upper bounds from existing usage statistics. For further details see these <a href=\"https://gitweb.torproject.org/metrics-web.git/tree/src/main/resources/doc/u…">questions and answers about user statistics</a>.", - "function": "plot_userstats_bridge_combined", + "function": "userstats_bridge_combined", "parameters": [ "start", "end", "country" ], - "data": [ - "userstats-combined" - ], "include_related_events": true }, { @@ -327,15 +267,12 @@ "title": "Bridge users by IP version", "type": "Graph", "description": "This graph shows the estimated number of <a href=\"glossary.html#client\">clients</a> connecting via <a href=\"glossary.html#bridge\">bridges</a>. These numbers are derived from directory requests counted on bridges. Bridges distinguish connecting clients by IP version, so that graphs are available for both IP versions 4 and 6. For further details see these <a href=\"https://gitweb.torproject.org/metrics-web.git/tree/src/main/resources/doc/u…">questions and answers about user statistics</a>.", - "function": "plot_userstats_bridge_version", + "function": "userstats_bridge_version", "parameters": [ "start", "end", "version" ], - "data": [ - "clients" - ], "include_related_events": true }, { @@ -349,16 +286,13 @@ "title": "Time to download files over Tor", "type": "Graph", "description": "This graph shows overall performance when downloading static files of different sizes over Tor, either from a server on the public internet or from a version 2 onion server. The graph shows the range of measurements from first to third quartile, and highlights the median. The slowest and fastest quarter of measurements are omitted from the graph.", - "function": "plot_torperf", + "function": "torperf", "parameters": [ "start", "end", "source", "server", "filesize" - ], - "data": [ - "torperf-1.1" ] }, { @@ -366,16 +300,13 @@ "title": "Timeouts and failures of downloading files over Tor", "type": "Graph", "description": "This graph shows the fraction of timeouts and failures when downloading static files of different sizes over Tor, either from a server on the public internet or from a version 2 onion server. A timeout occurs when a download does not complete within the scheduled time, in which case it is aborted in order not to overlap with the next scheduled download. A failure occurs when the download completes, but the response is smaller than expected.", - "function": "plot_torperf_failures", + "function": "torperf_failures", "parameters": [ "start", "end", "source", "server", "filesize" - ], - "data": [ - "torperf-1.1" ] }, { @@ -383,13 +314,10 @@ "title": "Fraction of connections used uni-/bidirectionally", "type": "Graph", "description": "This graph shows the fraction of direct connections between a <a href=\"glossary.html#relay\">relay</a> and other nodes in the network that are used uni- or bi-directionally. Every 10 seconds, relays determine for every direct connection whether they read and wrote less than a threshold of 20 KiB. Connections below this threshold are excluded from the graph. For the remaining connections, relays determine whether they read/wrote at least 10 times as many bytes as they wrote/read. If so, they classify a connection as \"Mostly reading\" or \"Mostly writing\", respectively. All other connections are classified as \"Both reading and writing\". After classifying connections, read and write counters are reset for the next 10-second interval. The graph shows daily medians and inter-quartile ranges of reported fractions.", - "function": "plot_connbidirect", + "function": "connbidirect", "parameters": [ "start", "end" - ], - "data": [ - "connbidirect2" ] }, { @@ -397,13 +325,10 @@ "title": "Unique .onion addresses (version 2 only)", "type": "Graph", "description": "This graph shows the number of unique .onion addresses for version 2 onion services in the network per day. These numbers are extrapolated from aggregated statistics on unique version 2 .onion addresses reported by single <a href=\"glossary.html#relay\">relays</a> acting as <a href=\"glossary.html#onion-service\">onion-service</a> directories, if at least 1% of relays reported these statistics. For more details on the extrapolation algorithm, see <a href=\"https://blog.torproject.org/blog/some-statistics-about-onions\">this blog post</a> and <a href=\"https://research.torproject.org/techreports/extrapolating-hidserv-stats-201…">this technical report</a>.", - "function": "plot_hidserv_dir_onions_seen", + "function": "hidserv_dir_onions_seen", "parameters": [ "start", "end" - ], - "data": [ - "hidserv" ] }, { @@ -411,13 +336,10 @@ "title": "Onion-service traffic (versions 2 and 3)", "type": "Graph", "description": "This graph shows the amount of onion-service traffic from version 2 and version 3 onion services in the network per day. This number is extrapolated from aggregated statistics on onion-service traffic reported by single <a href=\"glossary.html#relay\">relays</a> acting as rendezvous points for version 2 and 3 <a href=\"glossary.html#onion-service\">onion services</a>, if at least 1% of relays reported these statistics. For more details on the extrapolation algorithm, see <a href=\"https://blog.torproject.org/blog/some-statistics-about-onions\">this blog post</a> and <a href=\"https://research.torproject.org/techreports/extrapolating-hidserv-stats-201…">this technical report</a>.", - "function": "plot_hidserv_rend_relayed_cells", + "function": "hidserv_rend_relayed_cells", "parameters": [ "start", "end" - ], - "data": [ - "hidserv" ] }, { @@ -425,13 +347,10 @@ "title": "Fraction of relays reporting onion-service statistics", "type": "Graph", "description": "This graph shows the fraction of <a href=\"glossary.html#relay\">relays</a> that report statistics on <a href=\"glossary.html#onion-service\">onion service</a> usage. If at least 1% of relays report a statistic, it gets extrapolated towards a network total, where higher fractions are produce more accurate results. For more details on the extrapolation algorithm, see <a href=\"https://blog.torproject.org/blog/some-statistics-about-onions\">this blog post</a> and <a href=\"https://research.torproject.org/techreports/extrapolating-hidserv-stats-201…">this technical report</a>.", - "function": "plot_hidserv_frac_reporting", + "function": "hidserv_frac_reporting", "parameters": [ "start", "end" - ], - "data": [ - "hidserv" ] }, { @@ -457,13 +376,10 @@ "title": "Tor Browser downloads and updates", "type": "Graph", "description": "This graph shows absolute numbers of requests to Tor's web servers related to Tor Browser. Initial downloads and signature downloads are requests made by the user to download a Tor Browser executable or a corresponding signature file from the Tor website. Update pings and update requests are requests made by Tor Browser to check whether a newer version is available or to download a newer version.", - "function": "plot_webstats_tb", + "function": "webstats_tb", "parameters": [ "start", "end" - ], - "data": [ - "webstats" ] }, { @@ -471,13 +387,10 @@ "title": "Tor Browser downloads by platform", "type": "Graph", "description": "This graph shows absolute numbers of requests to Tor's web servers to download a Tor Browser executable, broken down by platform (Windows, macOS, Linux) of the requested executable. Note that this graph does not show the platform used to download Tor Browser but the platform that it was downloaded for.", - "function": "plot_webstats_tb_platform", + "function": "webstats_tb_platform", "parameters": [ "start", "end" - ], - "data": [ - "webstats" ] }, { @@ -485,13 +398,10 @@ "title": "Tor Browser downloads by locale", "type": "Graph", "description": "This graph shows absolute numbers of requests to Tor's web servers to download a Tor Browser executable, broken down by requested locale.", - "function": "plot_webstats_tb_locale", + "function": "webstats_tb_locale", "parameters": [ "start", "end" - ], - "data": [ - "webstats" ] }, { @@ -499,13 +409,10 @@ "title": "Tor Messenger downloads and updates", "type": "Graph", "description": "This graph shows absolute numbers of requests to Tor's web servers related to Tor Messenger. Initial downloads are requests made by the user to download a Tor Messenger executable from the Tor website. Update pings are requests made by Tor Messenger to check whether a newer version is available.", - "function": "plot_webstats_tm", + "function": "webstats_tm", "parameters": [ "start", "end" - ], - "data": [ - "webstats" ] } ] diff --git a/src/main/resources/web/jsps/graph.jsp b/src/main/resources/web/jsps/graph.jsp index 238f6d5..41e751d 100644 --- a/src/main/resources/web/jsps/graph.jsp +++ b/src/main/resources/web/jsps/graph.jsp @@ -158,14 +158,8 @@ <a href="${id}.png${parameters}">PNG</a> or <a href="${id}.pdf${parameters}">PDF</a>. -<c:if test="${fn:length(data) > 0}"> -Download underlying data: -<ul> -<c:forEach var="row" items="${data}"> -<li><a href="stats/${row}.csv">CSV</a> (<a href="stats.html#${row}">format</a>)</li> -</c:forEach> -</ul> -</c:if> +Download data as +<a href="${id}.csv${parameters}">CSV</a>. </div> </div> diff --git a/src/main/resources/web/jsps/table.jsp b/src/main/resources/web/jsps/table.jsp index cf26ab2..fc8117d 100644 --- a/src/main/resources/web/jsps/table.jsp +++ b/src/main/resources/web/jsps/table.jsp @@ -64,15 +64,6 @@ </form> -<c:if test="${fn:length(data) > 0}"> -Download underlying data: -<ul> -<c:forEach var="row" items="${data}"> -<li><a href="stats/${row}.csv">CSV</a> (<a href="stats.html#${row}">format</a>)</li> -</c:forEach> -</ul> -</c:if> - </div> </div> </div>

1 0

[tor/release-0.3.3] defer the mystery date to thu or fri.
by nickm＠torproject.org 27 Feb '18

27 Feb '18

commit c0c487073ac28946af86ff984f29e6d7064dfb00 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Feb 27 18:39:08 2018 -0500 defer the mystery date to thu or fri. --- ChangeLog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 76c1ccc42..76a001a81 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,4 +1,4 @@ -Changes in version 0.3.3.3-alpha - 2018-02-2? +Changes in version 0.3.3.3-alpha - 2018-03-?? Tor 0.3.3.3-alpha is the third alpha release for the 0.3.3.x series. (TROVE BLURB HERE.)

1 0

[tor/release-0.2.9] backport a changelog change from arma to 0.2.9
by nickm＠torproject.org 27 Feb '18

27 Feb '18

commit 789a360e0a05764d398b248ac6fe4f049a5841ed Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Feb 27 18:38:36 2018 -0500 backport a changelog change from arma to 0.2.9 --- ChangeLog | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ChangeLog b/ChangeLog index e7bbe2705..6a3332f39 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,4 +1,4 @@ -Changes in version 0.2.9.15 - 2018-03-xx + Changes in version 0.2.9.15 - 2018-03-xx Tor 0.2.9.15 backports important security and stability bugfixes from later Tor releases. All Tor users should upgrade to this release, or to another of the releases coming out today. @@ -44,7 +44,7 @@ Changes in version 0.2.9.15 - 2018-03-xx - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. Previous versions of Tor would not have worked with OpenSSL 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. Closes ticket 24978. o Minor features (denial-of-service avoidance):

1 0

[tor/release-0.3.1] backport a changelog change from arma to 0.3.1
by nickm＠torproject.org 27 Feb '18

27 Feb '18

commit ce9df228a5cbbf6f53a118b571c7cb521343ace4 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Feb 27 18:38:21 2018 -0500 backport a changelog change from arma to 0.3.1 --- ChangeLog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 59fd32a8c..f13c8a193 100644 --- a/ChangeLog +++ b/ChangeLog @@ -56,7 +56,7 @@ Changes in version 0.3.1.10 - 2018-03-?? - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. Previous versions of Tor would not have worked with OpenSSL 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. Closes ticket 24978. o Minor features (fallback directory mirrors, backport from 0.3.2.9):

1 0

[tor/release-0.3.2] backport a changelog change from arma to 0.3.2
by nickm＠torproject.org 27 Feb '18

27 Feb '18

commit 43d44c04ff3af57d92e9c69e3fcaeba9ed79b1f3 Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Feb 27 18:38:04 2018 -0500 backport a changelog change from arma to 0.3.2 --- ChangeLog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index bad9f1b93..dcae057a2 100644 --- a/ChangeLog +++ b/ChangeLog @@ -57,7 +57,7 @@ Changes in version 0.3.2.10 - 2018-03-?? - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. Previous versions of Tor would not have worked with OpenSSL 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. Closes ticket 24978. o Minor features (geoip):

1 0

[tor/release-0.3.3] tiny tweaks to upcoming 0.3.3.3 changelog
by arma＠torproject.org 27 Feb '18

27 Feb '18

commit dc603be9080925f97f74b42b9c712d6ef385a0ad Author: Roger Dingledine <arma(a)torproject.org> Date: Tue Feb 27 17:21:10 2018 -0500 tiny tweaks to upcoming 0.3.3.3 changelog --- ChangeLog | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/ChangeLog b/ChangeLog index b04d3e603..76c1ccc42 100644 --- a/ChangeLog +++ b/ChangeLog @@ -10,7 +10,7 @@ Changes in version 0.3.3.3-alpha - 2018-02-2? - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. Previous versions of Tor would not have worked with OpenSSL 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + ciphersuites it requires. Now we enable the TLS 1.3 ciphersuites. Closes ticket 24978. o Minor features (logging): @@ -30,13 +30,12 @@ Changes in version 0.3.3.3-alpha - 2018-02-2? on 0.3.3.2-alpha. o Minor bugfixes (man page, SocksPort): - - Remove dead code about the old "SocksSockets" option, and rename - SocksSocketsGroupWritable to UnixSockssGroupWritable which does - the same exact thing. The old option still works, but is - deprecated. Fixes bug 24343; bugfix on 0.2.6.3. + - Remove dead code from the old "SocksSocket" option, and rename + SocksSocketsGroupWritable to UnixSocksGroupWritable. The old option + still works, but is deprecated. Fixes bug 24343; bugfix on 0.2.6.3. o Minor bugfixes (performance): - - Reduce the number of circuits that can be opened at once during + - Reduce the number of circuits that will be opened at once during the circuit build timeout phase. This is done by increasing the idle timeout to 3 minutes, and lowering the maximum number of concurrent learning circuits to 10. Fixes bug 24769; bugfix

1 0

[tor/release-0.3.1] Create initial 0.3.1.10 changelog.
by nickm＠torproject.org 27 Feb '18

27 Feb '18

commit ff679ee066e52adb09863f1a49396e609c1e3cde Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Feb 27 17:11:16 2018 -0500 Create initial 0.3.1.10 changelog. --- ChangeLog | 196 +++++++++++++++++++++++++++++++++++++++++++++ changes/bug21074_downgrade | 4 - changes/bug23318 | 11 --- changes/bug23817 | 3 - changes/bug23862 | 5 -- changes/bug23985 | 9 --- changes/bug24086 | 7 -- changes/bug24099 | 4 - changes/bug24167 | 7 -- changes/bug24170 | 3 - changes/bug24198 | 4 - changes/bug24480 | 3 - changes/bug24633 | 5 -- changes/bug24666 | 7 -- changes/bug24736 | 6 -- changes/bug24826_031 | 4 - changes/bug24859 | 4 - changes/bug24895 | 8 -- changes/bug24898 | 8 -- changes/bug24898-029 | 6 -- changes/bug24952 | 5 -- changes/bug24978 | 7 -- changes/bug25070 | 3 - changes/bug25223 | 4 - changes/geoip-2017-12-06 | 4 - changes/geoip-2018-01-05 | 4 - changes/geoip-2018-02-07 | 4 - changes/ticket23856 | 4 - changes/ticket24315 | 3 - changes/ticket24681 | 6 -- changes/ticket24902 | 13 --- changes/ticket25122 | 4 - changes/ticket25170 | 5 -- changes/ticket25202 | 4 - changes/ticket25323 | 4 - changes/ticket_24801 | 5 -- 36 files changed, 196 insertions(+), 187 deletions(-) diff --git a/ChangeLog b/ChangeLog index 0e3a738e3..59fd32a8c 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,199 @@ +Changes in version 0.3.1.10 - 2018-03-?? + Tor 0.3.1.10 backports a number of bugfixes, including important fixes for + security issues. + + BLURB HERE. + + o Major features (denial-of-service mitigation, backport from 0.3.3.2-alpha): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Minor features (linux seccomp2 sandbox, backport from 0.3.2.5-alpha): + - Update the sandbox rules so that they should now work correctly + with Glibc 2.26. Closes ticket 24315. + + o Major bugfixes (onion services, retry behavior, backport from 0.3.3.1-alpha): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + + o Major bugfixes (protocol versions, backport from 0.3.3.2-alpha): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (relay, backport from 0.3.3.1-alpha): + - Fix a set of false positives where relays would consider + connections to other relays as being client-only connections (and + thus e.g. deserving different link padding schemes) if those + relays fell out of the consensus briefly. Now we look only at the + initial handshake and whether the connection authenticated as a + relay. Fixes bug 24898; bugfix on 0.3.1.1-alpha. + + o Minor features (denial-of-service avoidance, backport from 0.3.3.2-alpha): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor feature (relay statistics, backport from 0.3.2.6-alpha): + - Change relay bandwidth reporting stats interval from 4 hours to 24 + hours in order to reduce the efficiency of guard discovery + attacks. Fixes ticket 23856. + + o Minor features (compatibility, OpenSSL, backport from 0.3.3.3-alpha): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (fallback directory mirrors, backport from 0.3.2.9): + - The fallback directory list has been re-generated based on the + current status of the network. Tor uses fallback directories to + bootstrap when it doesn't yet have up-to-date directory + information. Closes ticket 24801. + - Make the default DirAuthorityFallbackRate 0.1, so that clients + prefer to bootstrap from fallback directory mirrors. This is a + follow-up to 24679, which removed weights from the default + fallbacks. Implements ticket 24681. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor bugfix (channel connection, backport from 0.3.3.2-alpha): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority, backport from 0.3.3.2-alpha): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (address selection, backport from 0.3.2.9): + - When the fascist_firewall_choose_address_ functions don't find a + reachable address, set the returned address to the null address + and port. This is a precautionary measure, because some callers do + not check the return value. Fixes bug 24736; bugfix + on 0.2.8.2-alpha. + + o Major bugfixes (bootstrapping, backport from 0.3.2.5-alpha): + - Fetch descriptors aggressively whenever we lack enough to build + circuits, regardless of how many descriptors we are missing. + Previously, we would delay launching the fetch when we had fewer + than 15 missing descriptors, even if some of those descriptors + were blocking circuits from building. Fixes bug 23985; bugfix on + 0.1.1.11-alpha. The effects of this bug became worse in + 0.3.0.3-alpha, when we began treating missing descriptors from our + primary guards as a reason to delay circuits. + - Don't try fetching microdescriptors from relays that have failed + to deliver them in the past. Fixes bug 23817; bugfix + on 0.3.0.1-alpha. + + o Minor bugfixes (compilation, backport from 0.3.2.7-rc): + - Fix a signed/unsigned comparison warning introduced by our fix to + TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. + + o Minor bugfixes (control port, linux seccomp2 sandbox, backport from 0.3.2.5-alpha): + - Avoid a crash when attempting to use the seccomp2 sandbox together + with the OwningControllerProcess feature. Fixes bug 24198; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (directory cache, backport from 0.3.2.5-alpha): + - Recover better from empty or corrupt files in the consensus cache + directory. Fixes bug 24099; bugfix on 0.3.1.1-alpha. + - When a consensus diff calculation is only partially successful, + only record the successful parts as having succeeded. Partial + success can happen if (for example) one compression method fails + but the others succeed. Previously we misrecorded all the + calculations as having succeeded, which would later cause a + nonfatal assertion failure. Fixes bug 24086; bugfix + on 0.3.1.1-alpha. + + o Minor bugfixes (entry guards, backport from 0.3.2.3-alpha): + - Tor now updates its guard state when it reads a consensus + regardless of whether it's missing descriptors. That makes tor use + its primary guards to fetch descriptors in some edge cases where + it would previously have used fallback directories. Fixes bug + 23862; bugfix on 0.3.0.1-alpha. + + o Minor bugfixes (logging, backport from 0.3.3.2-alpha): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (memory usage, backport from 0.3.2.8-rc): + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (network layer, backport from 0.3.2.5-alpha): + - When closing a connection via close_connection_immediately(), we + mark it as "not blocked on bandwidth", to prevent later calls from + trying to unblock it, and give it permission to read. This fixes a + backtrace warning that can happen on relays under various + circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. + + o Minor bugfixes (path selection, backport from 0.3.2.4-alpha): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. + Previously, we would always round upwards; now, we round towards + the nearest integer. This had the biggest effect when a relay's + weight adjustments should have given it weight 0, but it got + weight 1 instead. Fixes bug 23318; bugfix on 0.2.4.3-alpha. + - When calculating the fraction of nodes that have descriptors, and + all nodes in the network have zero bandwidths, count the number of + nodes instead. Fixes bug 23318; bugfix on 0.2.4.10-alpha. + - Actually log the total bandwidth in compute_weighted_bandwidths(). + Fixes bug 24170; bugfix on 0.2.4.3-alpha. + + o Minor bugfixes (performance, fragile-hardening, backport from 0.3.3.1-alpha): + - Improve the performance of our consensus-diff application code + when Tor is built with the --enable-fragile-hardening option set. + Fixes bug 24826; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (OSX, backport from 0.3.3.1-alpha): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (portability, msvc, backport from 0.3.2.9): + - Fix a bug in the bit-counting parts of our timing-wheel code on + MSVC. (Note that MSVC is still not a supported build platform, due + to cyptographic timing channel risks.) Fixes bug 24633; bugfix + on 0.2.9.1-alpha. + + o Minor bugfixes (relay, partial backport): + - Make the internal channel_is_client() function look at what sort + of connection handshake the other side used, rather than whether + the other side ever sent a create_fast cell to us. Backports part + of the fixes from bugs 22805 and 24898. + + o Code simplification and refactoring (backport from 0.3.3.3-alpha): + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + + + + + Changes in version 0.3.1.9 - 2017-12-01: Tor 0.3.1.9 backports important security and stability fixes from the 0.3.2 development series. All Tor users should upgrade to this diff --git a/changes/bug21074_downgrade b/changes/bug21074_downgrade deleted file mode 100644 index 1bc1f8523..000000000 --- a/changes/bug21074_downgrade +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (portability): - - Don't exit the Tor process if setrlimit() fails to change the file - limit (which can happen sometimes on some versions of OSX). Fixes - bug 21074; bugfix on 0.0.9pre5. diff --git a/changes/bug23318 b/changes/bug23318 deleted file mode 100644 index 7fcb8d448..000000000 --- a/changes/bug23318 +++ /dev/null @@ -1,11 +0,0 @@ - o Minor bugfixes (path selection): - - When selecting relays by bandwidth, avoid a rounding error that - could sometimes cause load to be imbalanced incorrectly. Previously, - we would always round upwards; now, we round towards the nearest - integer. This had the biggest effect when a relay's weight adjustments - should have given it weight 0, but it got weight 1 instead. - Fixes bug 23318; bugfix on 0.2.4.3-alpha. - - When calculating the fraction of nodes that have descriptors, and all - all nodes in the network have zero bandwidths, count the number of nodes - instead. - Fixes bug 23318; bugfix on 0.2.4.10-alpha. diff --git a/changes/bug23817 b/changes/bug23817 deleted file mode 100644 index 474094279..000000000 --- a/changes/bug23817 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (descriptors): - - Don't try fetching microdescriptors from relays that have failed to - deliver them in the past. Fixes bug 23817; bugfix on 0.3.0.1-alpha. diff --git a/changes/bug23862 b/changes/bug23862 deleted file mode 100644 index 301ce7367..000000000 --- a/changes/bug23862 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (entry guards): - - Tor now updates its guard state when it reads a consensus regardless of - whether it's missing descriptors. That makes tor use its primary guards - to fetch descriptors in some edge cases where it would have used fallback - directories in the past. Fixes bug 23862; bugfix on 0.3.0.1-alpha. \ No newline at end of file diff --git a/changes/bug23985 b/changes/bug23985 deleted file mode 100644 index 9cb593796..000000000 --- a/changes/bug23985 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes (bootstrapping): - - Fetch descriptors aggressively whenever we lack enough - to build circuits, regardless of how many descriptors we are missing. - Previously, we would delay launching the fetch when we had fewer than - 15 missing descriptors, even if some of those descriptors were - blocking circuits from building. Fixes bug 23985; bugfix on - 0.1.1.11-alpha. The effects of this bug became worse in 0.3.0.3-alpha, - when we began treating missing descriptors from our primary guards - as a reason to delay circuits. diff --git a/changes/bug24086 b/changes/bug24086 deleted file mode 100644 index 2ae0b37e6..000000000 --- a/changes/bug24086 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (directory cache): - - When a consensus diff calculation is only partially successful, only - record the successful parts as having succeeded. Partial success - can happen if (for example) one compression method fails but - the others succeed. Previously we misrecorded all the calculations as - having succeeded, which would later cause a nonfatal assertion failure. - Fixes bug 24086; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24099 b/changes/bug24099 deleted file mode 100644 index dca399266..000000000 --- a/changes/bug24099 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (directory cache): - - Recover better from empty or corrupt files in the consensus cache - directory. Fixes bug 24099; bugfix on 0.3.1.1-alpha. - diff --git a/changes/bug24167 b/changes/bug24167 deleted file mode 100644 index fd0d87eff..000000000 --- a/changes/bug24167 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (network layer): - - When closing a connection via close_connection_immediately(), we - mark it as "not blocked on bandwidth", to prevent later calls - from trying to unblock it, and give it permission to read. This - fixes a backtrace warning that can happen on relays under various - circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. - diff --git a/changes/bug24170 b/changes/bug24170 deleted file mode 100644 index d3d734769..000000000 --- a/changes/bug24170 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (path selection): - - Actually log the total bandwidth in compute_weighted_bandwidths(). - Fixes bug 24170; bugfix on 0.2.4.3-alpha. diff --git a/changes/bug24198 b/changes/bug24198 deleted file mode 100644 index 679070687..000000000 --- a/changes/bug24198 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (controller, linux seccomp2 sandbox): - - Avoid a crash when attempting to use the seccomp2 sandbox - together with the OwningControllerProcess feature. - Fixes bug 24198; bugfix on 0.2.5.1-alpha. diff --git a/changes/bug24480 b/changes/bug24480 deleted file mode 100644 index 94e5b91a0..000000000 --- a/changes/bug24480 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (compilation): - - Fix a signed/unsigned comparison warning introduced by our - fix to TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. diff --git a/changes/bug24633 b/changes/bug24633 deleted file mode 100644 index 028c7cc14..000000000 --- a/changes/bug24633 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (portability, msvc): - - Fix a bug in the bit-counting parts of our timing-wheel code on - MSVC. (Note that MSVC is still not a supported build platform, - due to cyptographic timing channel risks.) Fixes bug 24633; - bugfix on 0.2.9.1-alpha. diff --git a/changes/bug24666 b/changes/bug24666 deleted file mode 100644 index 830775f5f..000000000 --- a/changes/bug24666 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (memory usage): - - - When queuing DESTROY cells on a channel, only queue the - circuit-id and reason fields: not the entire 514-byte - cell. This fix should help mitigate any bugs or attacks that - fill up these queues, and free more RAM for other uses. Fixes - bug 24666; bugfix on 0.2.5.1-alpha. diff --git a/changes/bug24736 b/changes/bug24736 deleted file mode 100644 index 632560932..000000000 --- a/changes/bug24736 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (address selection): - - When the fascist_firewall_choose_address_ functions don't find a - reachable address, set the returned address to the null address and port. - This is a precautionary measure, because some callers do not check the - return value. - Fixes bug 24736; bugfix on 0.2.8.2-alpha. diff --git a/changes/bug24826_031 b/changes/bug24826_031 deleted file mode 100644 index 3d4a66184..000000000 --- a/changes/bug24826_031 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (performance, fragile-hardening): - - Improve the performance of our consensus-diff application code when Tor - is built with the --enable-fragile-hardening option set. Fixes bug - 24826; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24859 b/changes/bug24859 deleted file mode 100644 index 122109d65..000000000 --- a/changes/bug24859 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (logging): - - Don't treat inability to store a cached consensus object as a - bug: it can happen normally when we are out of disk space. - Fixes bug 24859; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24895 b/changes/bug24895 deleted file mode 100644 index 7edde94a0..000000000 --- a/changes/bug24895 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes (onion services): - - Fix an "off by 2" error in counting rendezvous failures on the onion - service side. While we thought we would stop the rendezvous attempt - after one failed circuit, we were actually making three circuit attempts - before giving up. Now switch to a default of 2, and allow the consensus - parameter "hs_service_max_rdv_failures" to override. Fixes bug 24895; - bugfix on 0.0.6. - diff --git a/changes/bug24898 b/changes/bug24898 deleted file mode 100644 index f64340d71..000000000 --- a/changes/bug24898 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes (relays): - - Fix a set of false positives where relays would consider connections - to other relays as being client-only connections (and thus e.g. - deserving different link padding schemes) if those relays fell out - of the consensus briefly. Now we look only at the initial handshake - and whether the connection authenticated as a relay. Fixes bug - 24898; bugfix on 0.3.1.1-alpha. - diff --git a/changes/bug24898-029 b/changes/bug24898-029 deleted file mode 100644 index b33f09384..000000000 --- a/changes/bug24898-029 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (relay): - - Make the internal channel_is_client() function look at what sort - of connection handshake the other side used, rather than whether - the other side ever sent a create_fast cell to us. Backports part - of the fixes from bugs 22805 and 24898. - diff --git a/changes/bug24952 b/changes/bug24952 deleted file mode 100644 index 93174c04f..000000000 --- a/changes/bug24952 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (channel connection): - - The accurate address of a connection is real_addr, not the addr member. - TLS Channel remote address is now real_addr content instead of addr - member. Fixes bug 24952; bugfix on 707c1e2e26 in 0.2.4.11-alpha. - Patch by "ffmancera". diff --git a/changes/bug24978 b/changes/bug24978 deleted file mode 100644 index 5dc45c744..000000000 --- a/changes/bug24978 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor features (compatibility, OpenSSL): - - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. - Previous versions of Tor would not have worked with OpenSSL - 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. - Closes ticket 24978. - diff --git a/changes/bug25070 b/changes/bug25070 deleted file mode 100644 index c2f4e58c4..000000000 --- a/changes/bug25070 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes (protocol versions): - - Add Link protocol version 5 to the supported protocols list. - Fixes bug 25070; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug25223 b/changes/bug25223 deleted file mode 100644 index fdd556350..000000000 --- a/changes/bug25223 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Make sure we don't modify consensus parameters if we aren't a public - relay when a new consensus arrives. Fixes bug 25223; bugfix on - 0.3.3.2-alpha. diff --git a/changes/geoip-2017-12-06 b/changes/geoip-2017-12-06 deleted file mode 100644 index ae4fb1149..000000000 --- a/changes/geoip-2017-12-06 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the December 6 2017 Maxmind GeoLite2 - Country database. - diff --git a/changes/geoip-2018-01-05 b/changes/geoip-2018-01-05 deleted file mode 100644 index 59aba02d0..000000000 --- a/changes/geoip-2018-01-05 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the January 5 2018 Maxmind GeoLite2 - Country database. - diff --git a/changes/geoip-2018-02-07 b/changes/geoip-2018-02-07 deleted file mode 100644 index f45228fd7..000000000 --- a/changes/geoip-2018-02-07 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 - Country database. - diff --git a/changes/ticket23856 b/changes/ticket23856 deleted file mode 100644 index 049da18d0..000000000 --- a/changes/ticket23856 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor feature (relay statistics): - - Change relay bandwidth reporting stats interval from 4 hours to 24 hours - in order to reduce the efficiency of guard discovery attacks. Fixes - ticket 23856. diff --git a/changes/ticket24315 b/changes/ticket24315 deleted file mode 100644 index df34dbf41..000000000 --- a/changes/ticket24315 +++ /dev/null @@ -1,3 +0,0 @@ - o Major features (linux seccomp2 sandbox): - - Update the sandbox rules so that they should now work correctly with - Glibc 2.26. Closes ticket 24315. diff --git a/changes/ticket24681 b/changes/ticket24681 deleted file mode 100644 index cc0a42b2e..000000000 --- a/changes/ticket24681 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor features (fallback directory mirrors): - - Make the default DirAuthorityFallbackRate 0.1, so that clients on the - public tor network prefer to bootstrap off fallback directory mirrors. - This is a follow-up to 24679, which removed weights from the default - fallbacks. - Implements ticket 24681. diff --git a/changes/ticket24902 b/changes/ticket24902 deleted file mode 100644 index 1a2ef95cc..000000000 --- a/changes/ticket24902 +++ /dev/null @@ -1,13 +0,0 @@ - o Major features (denial of service mitigation): - - Give relays some defenses against the recent network overload. We start - with three defenses (default parameters in parentheses). First: if a - single client address makes too many concurrent connections (>100), hang - up on further connections. Second: if a single client address makes - circuits too quickly (more than 3 per second, with an allowed burst of - 90) while also having too many connections open (3), refuse new create - cells for the next while (1-2 hours). Third: if a client asks to - establish a rendezvous point to you directly, ignore the request. These - defenses can be manually controlled by new torrc options, but relays - will also take guidance from consensus parameters, so there's no need to - configure anything manually. Implements ticket 24902. - diff --git a/changes/ticket25122 b/changes/ticket25122 deleted file mode 100644 index 2921811b2..000000000 --- a/changes/ticket25122 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor feature (geoip cache): - - Make our OOM handler aware of the geoip client history cache so it - doesn't fill up the memory which is especially important for IPv6 and - our DoS mitigation subsystem. Closes ticket 25122. diff --git a/changes/ticket25170 b/changes/ticket25170 deleted file mode 100644 index 065213940..000000000 --- a/changes/ticket25170 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (directory authority, documentation): - - When a fingerprint or network address is marked as rejected, the - returned message by the authority now explicitly mention to set a valid - ContactInfo address and contact the bad-relays@ mailing list. Fixes bug - 25170; bugfix on 0.2.9.1. diff --git a/changes/ticket25202 b/changes/ticket25202 deleted file mode 100644 index ba64abad7..000000000 --- a/changes/ticket25202 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Add extra safety checks when refilling the circuit creation bucket to - ensure we never set a value that is above the allowed burst. Fixes - bug 25202; bugfix on 0.3.3.2-alpha. diff --git a/changes/ticket25323 b/changes/ticket25323 deleted file mode 100644 index 836825de5..000000000 --- a/changes/ticket25323 +++ /dev/null @@ -1,4 +0,0 @@ - o Code simplification and refactoring: - - Update the "rust dependencies" submodule to be an project-level - repository, rather than a user repository. Closes ticket 25323. - diff --git a/changes/ticket_24801 b/changes/ticket_24801 deleted file mode 100644 index f5f6c831a..000000000 --- a/changes/ticket_24801 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor features (new fallback directories): - - The fallback directory list has been re-generated based on the - current status of the network. Tor uses fallback directories to - bootstrap it doesn't yet have up-to-date directory - information. Closes ticket 24801.

1 0

[tor/release-0.3.2] Make a changelog for 0.3.2.10
by nickm＠torproject.org 27 Feb '18

27 Feb '18

commit 3acf6cafbb655a73f0a52b38a0dbd94eb5befa3f Author: Nick Mathewson <nickm(a)torproject.org> Date: Tue Feb 27 16:55:08 2018 -0500 Make a changelog for 0.3.2.10 (Note that two entries are marked OMIT: they are bugfixes on #24902 that we're backporting along with the #24902 code. I think that means that we don't backport their changelog entries, since they are bugfixes on a later version of Tor?) --- ChangeLog | 145 +++++++++++++++++++++++++++++++++++++++++++++ changes/bug21074_downgrade | 4 -- changes/bug24526 | 4 -- changes/bug24652 | 6 -- changes/bug24700 | 4 -- changes/bug24826_031 | 4 -- changes/bug24859 | 4 -- changes/bug24894 | 5 -- changes/bug24895 | 8 --- changes/bug24898 | 8 --- changes/bug24952 | 5 -- changes/bug24972 | 4 -- changes/bug24975 | 6 -- changes/bug24976 | 5 -- changes/bug24978 | 7 --- changes/bug25005 | 4 -- changes/bug25070 | 3 - changes/bug25105 | 5 -- changes/bug25223 | 4 -- changes/geoip-2018-02-07 | 4 -- changes/ticket24902 | 13 ---- changes/ticket25122 | 4 -- changes/ticket25170 | 5 -- changes/ticket25202 | 4 -- changes/ticket25323 | 4 -- 25 files changed, 145 insertions(+), 124 deletions(-) diff --git a/ChangeLog b/ChangeLog index da312f2d8..bad9f1b93 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,148 @@ +Changes in version 0.3.2.10 - 2018-03-?? + Tor 0.3.2.10 is the second stable release in the 0.3.2 series. It + backports a number of bugfixes, including important fixes for security + issues. + + BLURB HERE. + + o Major features (denial-of-service mitigation, backport from 0.3.3.2-alpha): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (onion services, retry behavior, backport from 0.3.3.1-alpha): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + - New-style (v3) onion services now obey the "max rendezvous circuit + attempts" logic. Previously they would make as many rendezvous + circuit attempts as they could fit in the MAX_REND_TIMEOUT second + window before giving up. Fixes bug 24894; bugfix on 0.3.2.1-alpha. + + o Major bugfixes (protocol versions, backport from 0.3.3.2-alpha): + - Add Link protocol version 5 to the supported protocols list. Fixes + bug 25070; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (relay, backport from 0.3.3.1-alpha): + - Fix a set of false positives where relays would consider + connections to other relays as being client-only connections (and + thus e.g. deserving different link padding schemes) if those + relays fell out of the consensus briefly. Now we look only at the + initial handshake and whether the connection authenticated as a + relay. Fixes bug 24898; bugfix on 0.3.1.1-alpha. + + o Major bugfixes (scheduler, consensus, backport from 0.3.3.2-alpha): + - The scheduler subsystem was failing to promptly notice changes in + consensus parameters, making it harder to switch schedulers + network-wide. Fixes bug 24975; bugfix on 0.3.2.1-alpha. + + o Minor features (denial-of-service avoidance, backport from 0.3.3.2-alpha): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (compatibility, OpenSSL, backport from 0.3.3.3-alpha): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (logging, diagnostic, backport from 0.3.3.2-alpha): + - When logging a failure to check a hidden service's certificate, + also log what the problem with the certificate was. Diagnostic + for ticket 24972. + + o Minor bugfix (channel connection, backport from 0.3.3.2-alpha): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority, backport from 0.3.3.2-alpha): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (build, rust, backport from 0.3.3.1-alpha): + - When building with Rust on OSX, link against libresolv, to work + around the issue at https://github.com/rust-lang/rust/issues/46797. + Fixes bug 24652; bugfix on 0.3.1.1-alpha. + + + [[[[ OMIT + o Minor bugfixes (DoS mitigation): + - Add extra safety checks when refilling the circuit creation bucket to + ensure we never set a value that is above the allowed burst. Fixes + bug 25202; bugfix on 0.3.3.2-alpha. + - Make sure we don't modify consensus parameters if we aren't a public + relay when a new consensus arrives. Fixes bug 25223; bugfix on + 0.3.3.2-alpha. + OMIT]]]] + + o Minor bugfixes (onion services, backport from 0.3.3.2-alpha): + - Remove a BUG() statement when a client fetches an onion descriptor + that has a lower revision counter than the one in its cache. This + can happen in normal circumstances due to HSDir desync. Fixes bug + 24976; bugfix on 0.3.2.1-alpha. + + o Minor bugfixes (logging, backport from 0.3.3.2-alpha): + - Don't treat inability to store a cached consensus object as a bug: + it can happen normally when we are out of disk space. Fixes bug + 24859; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (performance, fragile-hardening, backport from 0.3.3.1-alpha): + - Improve the performance of our consensus-diff application code + when Tor is built with the --enable-fragile-hardening option set. + Fixes bug 24826; bugfix on 0.3.1.1-alpha. + + o Minor bugfixes (OSX, backport from 0.3.3.1-alpha): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (scheduler, KIST, backport from 0.3.3.2-alpha): + - Avoid adding the same channel twice in the KIST scheduler pending + list, which would waste CPU cycles. Fixes bug 24700; bugfix + on 0.3.2.1-alpha. + + o Minor bugfixes (testing, backport from 0.3.3.1-alpha): + - Fix a memory leak in the scheduler/loop_kist unit test. Fixes bug + 25005; bugfix on 0.3.2.7-rc. + + o Minor bugfixes (v3 onion services, backport from 0.3.3.2-alpha): + - Look at the "HSRend" protocol version, not the "HSDir" protocol + version, when deciding whether a consensus entry can support the + v3 onion service protocol as a rendezvous point. Fixes bug 25105; + bugfix on 0.3.2.1-alpha. + + o Code simplification and refactoring (backport from 0.3.3.3-alpha): + - Update the "rust dependencies" submodule to be a project-level + repository, rather than a user repository. Closes ticket 25323. + + o Documentation (backport from 0.3.3.1-alpha) + - Document that operators who run more than one relay or bridge are + expected to set MyFamily and ContactInfo correctly. Closes + ticket 24526. + + Changes in version 0.3.2.9 - 2018-01-09 Tor 0.3.2.9 is the first stable release in the 0.3.2 series. diff --git a/changes/bug21074_downgrade b/changes/bug21074_downgrade deleted file mode 100644 index 1bc1f8523..000000000 --- a/changes/bug21074_downgrade +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (portability): - - Don't exit the Tor process if setrlimit() fails to change the file - limit (which can happen sometimes on some versions of OSX). Fixes - bug 21074; bugfix on 0.0.9pre5. diff --git a/changes/bug24526 b/changes/bug24526 deleted file mode 100644 index 4d69defa9..000000000 --- a/changes/bug24526 +++ /dev/null @@ -1,4 +0,0 @@ - o Documentation: - - Document that operators who run more than one relay or bridge are - expected to set MyFamily and ContactInfo correctly. Closes ticket - 24526. diff --git a/changes/bug24652 b/changes/bug24652 deleted file mode 100644 index 6e35e259e..000000000 --- a/changes/bug24652 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (build, compatibility, rust, OSX): - - - When building with Rust on OSX, link against libresolv, to - work around the issue at - https://github.com/rust-lang/rust/issues/46797. Fixes bug - 24652; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24700 b/changes/bug24700 deleted file mode 100644 index 74dc581a0..000000000 --- a/changes/bug24700 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (scheduler, KIST): - - Avoid adding the same channel twice in the KIST scheduler pending list - wasting CPU cycles at handling the same channel twice. Fixes bug 24700; - bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24826_031 b/changes/bug24826_031 deleted file mode 100644 index 3d4a66184..000000000 --- a/changes/bug24826_031 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (performance, fragile-hardening): - - Improve the performance of our consensus-diff application code when Tor - is built with the --enable-fragile-hardening option set. Fixes bug - 24826; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24859 b/changes/bug24859 deleted file mode 100644 index 122109d65..000000000 --- a/changes/bug24859 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (logging): - - Don't treat inability to store a cached consensus object as a - bug: it can happen normally when we are out of disk space. - Fixes bug 24859; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug24894 b/changes/bug24894 deleted file mode 100644 index b08cdce1f..000000000 --- a/changes/bug24894 +++ /dev/null @@ -1,5 +0,0 @@ - o Major bugfixes (v3 onion services): - - New-style (v3) onion services now obey the "max rendezvous circuit - attempts" logic. Previously they would make as many rendezvous - circuit attempts as they could fit in the MAX_REND_TIMEOUT second - window before giving up. Fixes bug 24894; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24895 b/changes/bug24895 deleted file mode 100644 index 7edde94a0..000000000 --- a/changes/bug24895 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes (onion services): - - Fix an "off by 2" error in counting rendezvous failures on the onion - service side. While we thought we would stop the rendezvous attempt - after one failed circuit, we were actually making three circuit attempts - before giving up. Now switch to a default of 2, and allow the consensus - parameter "hs_service_max_rdv_failures" to override. Fixes bug 24895; - bugfix on 0.0.6. - diff --git a/changes/bug24898 b/changes/bug24898 deleted file mode 100644 index f64340d71..000000000 --- a/changes/bug24898 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes (relays): - - Fix a set of false positives where relays would consider connections - to other relays as being client-only connections (and thus e.g. - deserving different link padding schemes) if those relays fell out - of the consensus briefly. Now we look only at the initial handshake - and whether the connection authenticated as a relay. Fixes bug - 24898; bugfix on 0.3.1.1-alpha. - diff --git a/changes/bug24952 b/changes/bug24952 deleted file mode 100644 index 93174c04f..000000000 --- a/changes/bug24952 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (channel connection): - - The accurate address of a connection is real_addr, not the addr member. - TLS Channel remote address is now real_addr content instead of addr - member. Fixes bug 24952; bugfix on 707c1e2e26 in 0.2.4.11-alpha. - Patch by "ffmancera". diff --git a/changes/bug24972 b/changes/bug24972 deleted file mode 100644 index 5adf970ab..000000000 --- a/changes/bug24972 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (logging, diagnostic): - - When logging a failure to check a hidden service's certificate, - also log what the problem with the certificate was. Diagnostic - for ticket 24972. diff --git a/changes/bug24975 b/changes/bug24975 deleted file mode 100644 index 32a5dfc92..000000000 --- a/changes/bug24975 +++ /dev/null @@ -1,6 +0,0 @@ - o Major bugfixes (scheduler, consensus): - - A logic in the code was preventing the scheduler subystem to properly - make a decision based on the latest consensus when it arrives. This lead - to the scheduler failing to notice any consensus parameters that might - have changed between consensuses. Fixes bug 24975; bugfix on - 0.3.2.1-alpha. diff --git a/changes/bug24976 b/changes/bug24976 deleted file mode 100644 index 9c3be86ea..000000000 --- a/changes/bug24976 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (hidden service v3 client): - - Remove a BUG() statement which can be triggered in normal circumstances - where a client fetches a descriptor that has a lower revision counter - than the one in its cache. This can happen due to HSDir desync. Fixes - bug 24976; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug24978 b/changes/bug24978 deleted file mode 100644 index 5dc45c744..000000000 --- a/changes/bug24978 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor features (compatibility, OpenSSL): - - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. - Previous versions of Tor would not have worked with OpenSSL - 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. - Closes ticket 24978. - diff --git a/changes/bug25005 b/changes/bug25005 deleted file mode 100644 index dedf283aa..000000000 --- a/changes/bug25005 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (unit tests): - - Fix a memory leak in the scheduler/loop_kist unit test. Fixes bug - 25005; bugfix on 0.3.2.7-rc. - diff --git a/changes/bug25070 b/changes/bug25070 deleted file mode 100644 index c2f4e58c4..000000000 --- a/changes/bug25070 +++ /dev/null @@ -1,3 +0,0 @@ - o Major bugfixes (protocol versions): - - Add Link protocol version 5 to the supported protocols list. - Fixes bug 25070; bugfix on 0.3.1.1-alpha. diff --git a/changes/bug25105 b/changes/bug25105 deleted file mode 100644 index 36d1a5f16..000000000 --- a/changes/bug25105 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (v3 onion services): - - Look at the "HSRend" protocol version, not the "HSDir" protocol - version, when deciding whether a consensus entry can support - the v3 onion service protocol as a rendezvous point. - Fixes bug 25105; bugfix on 0.3.2.1-alpha. diff --git a/changes/bug25223 b/changes/bug25223 deleted file mode 100644 index fdd556350..000000000 --- a/changes/bug25223 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Make sure we don't modify consensus parameters if we aren't a public - relay when a new consensus arrives. Fixes bug 25223; bugfix on - 0.3.3.2-alpha. diff --git a/changes/geoip-2018-02-07 b/changes/geoip-2018-02-07 deleted file mode 100644 index f45228fd7..000000000 --- a/changes/geoip-2018-02-07 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 - Country database. - diff --git a/changes/ticket24902 b/changes/ticket24902 deleted file mode 100644 index 1a2ef95cc..000000000 --- a/changes/ticket24902 +++ /dev/null @@ -1,13 +0,0 @@ - o Major features (denial of service mitigation): - - Give relays some defenses against the recent network overload. We start - with three defenses (default parameters in parentheses). First: if a - single client address makes too many concurrent connections (>100), hang - up on further connections. Second: if a single client address makes - circuits too quickly (more than 3 per second, with an allowed burst of - 90) while also having too many connections open (3), refuse new create - cells for the next while (1-2 hours). Third: if a client asks to - establish a rendezvous point to you directly, ignore the request. These - defenses can be manually controlled by new torrc options, but relays - will also take guidance from consensus parameters, so there's no need to - configure anything manually. Implements ticket 24902. - diff --git a/changes/ticket25122 b/changes/ticket25122 deleted file mode 100644 index 2921811b2..000000000 --- a/changes/ticket25122 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor feature (geoip cache): - - Make our OOM handler aware of the geoip client history cache so it - doesn't fill up the memory which is especially important for IPv6 and - our DoS mitigation subsystem. Closes ticket 25122. diff --git a/changes/ticket25170 b/changes/ticket25170 deleted file mode 100644 index 065213940..000000000 --- a/changes/ticket25170 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (directory authority, documentation): - - When a fingerprint or network address is marked as rejected, the - returned message by the authority now explicitly mention to set a valid - ContactInfo address and contact the bad-relays@ mailing list. Fixes bug - 25170; bugfix on 0.2.9.1. diff --git a/changes/ticket25202 b/changes/ticket25202 deleted file mode 100644 index ba64abad7..000000000 --- a/changes/ticket25202 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Add extra safety checks when refilling the circuit creation bucket to - ensure we never set a value that is above the allowed burst. Fixes - bug 25202; bugfix on 0.3.3.2-alpha. diff --git a/changes/ticket25323 b/changes/ticket25323 deleted file mode 100644 index 836825de5..000000000 --- a/changes/ticket25323 +++ /dev/null @@ -1,4 +0,0 @@ - o Code simplification and refactoring: - - Update the "rust dependencies" submodule to be an project-level - repository, rather than a user repository. Closes ticket 25323. -

1 0

[tor/release-0.2.9] start a new 029 changelog
by arma＠torproject.org 27 Feb '18

27 Feb '18

commit 2813180ab5a77195289e858e5b8ff2c4642e599e Author: Roger Dingledine <arma(a)torproject.org> Date: Tue Feb 27 14:41:13 2018 -0500 start a new 029 changelog created by catting all the changes/ files together, then overwriting each entry with how it was written in a later release, then running the format_changelog.py script hopefully this will save nickm a bit of time -- if not feel free to overwrite. :) --- ChangeLog | 147 ++++++++++++++++++++++++++++++++++++++++++++- ReleaseNotes | 2 +- changes/bug21074_downgrade | 4 -- changes/bug23318 | 11 ---- changes/bug23985 | 9 --- changes/bug24167 | 7 --- changes/bug24170 | 3 - changes/bug24198 | 4 -- changes/bug24480 | 3 - changes/bug24633 | 5 -- changes/bug24666 | 7 --- changes/bug24736 | 6 -- changes/bug24895 | 8 --- changes/bug24898-029 | 6 -- changes/bug24952 | 5 -- changes/bug24978 | 7 --- changes/bug25223 | 4 -- changes/geoip-2017-12-06 | 4 -- changes/geoip-2018-01-05 | 4 -- changes/geoip-2018-02-07 | 4 -- changes/ticket23856 | 4 -- changes/ticket24315 | 3 - changes/ticket24681 | 6 -- changes/ticket24902 | 13 ---- changes/ticket25122 | 4 -- changes/ticket25170 | 5 -- changes/ticket25202 | 4 -- changes/ticket_24801 | 5 -- 28 files changed, 147 insertions(+), 147 deletions(-) diff --git a/ChangeLog b/ChangeLog index 5c565fb36..e7bbe2705 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,5 +1,150 @@ +Changes in version 0.2.9.15 - 2018-03-xx + Tor 0.2.9.15 backports important security and stability bugfixes from + later Tor releases. All Tor users should upgrade to this release, or + to another of the releases coming out today. + + o Major features (denial-of-service mitigation): + - Give relays some defenses against the recent network overload. We + start with three defenses (default parameters in parentheses). + First: if a single client address makes too many concurrent + connections (>100), hang up on further connections. Second: if a + single client address makes circuits too quickly (more than 3 per + second, with an allowed burst of 90) while also having too many + connections open (3), refuse new create cells for the next while + (1-2 hours). Third: if a client asks to establish a rendezvous + point to you directly, ignore the request. These defenses can be + manually controlled by new torrc options, but relays will also + take guidance from consensus parameters, so there's no need to + configure anything manually. Implements ticket 24902. + + o Major bugfixes (bootstrapping): + - Fetch descriptors aggressively whenever we lack enough to build + circuits, regardless of how many descriptors we are missing. + Previously, we would delay launching the fetch when we had fewer + than 15 missing descriptors, even if some of those descriptors + were blocking circuits from building. Fixes bug 23985; bugfix on + 0.1.1.11-alpha. The effects of this bug became worse in + 0.3.0.3-alpha, when we began treating missing descriptors from our + primary guards as a reason to delay circuits. + + o Major bugfixes (onion services, retry behavior): + - Fix an "off by 2" error in counting rendezvous failures on the + onion service side. While we thought we would stop the rendezvous + attempt after one failed circuit, we were actually making three + circuit attempts before giving up. Now switch to a default of 2, + and allow the consensus parameter "hs_service_max_rdv_failures" to + override. Fixes bug 24895; bugfix on 0.0.6. + + o Minor feature (relay statistics): + - Change relay bandwidth reporting stats interval from 4 hours to 24 + hours in order to reduce the efficiency of guard discovery + attacks. Fixes ticket 23856. + + o Minor features (compatibility, OpenSSL): + - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. + Previous versions of Tor would not have worked with OpenSSL 1.1.1, + since they neither disabled TLS 1.3 nor enabled any of the + ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. + Closes ticket 24978. + + o Minor features (denial-of-service avoidance): + - Make our OOM handler aware of the geoip client history cache so it + doesn't fill up the memory. This check is important for IPv6 and + our DoS mitigation subsystem. Closes ticket 25122. + + o Minor features (fallback directory mirrors): + - The fallback directory list has been re-generated based on the + current status of the network. Tor uses fallback directories to + bootstrap when it doesn't yet have up-to-date directory + information. Closes ticket 24801. + - Make the default DirAuthorityFallbackRate 0.1, so that clients + prefer to bootstrap from fallback directory mirrors. This is a + follow-up to 24679, which removed weights from the default + fallbacks. Implements ticket 24681. + + o Minor features (geoip): + - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 + Country database. + + o Minor features (linux seccomp2 sandbox): + - Update the sandbox rules so that they should now work correctly + with Glibc 2.26. Closes ticket 24315. + + o Minor bugfix (channel connection): + - Use the actual observed address of an incoming relay connection, + not the canonical address of the relay from its descriptor, when + making decisions about how to handle the incoming connection. + Fixes bug 24952; bugfix on 0.2.4.11-alpha. Patch by "ffmancera". + + o Minor bugfix (directory authority): + - Directory authorities, when refusing a descriptor from a rejected + relay, now explicitly tell the relay (in its logs) to set a valid + ContactInfo address and contact the bad-relays@ mailing list. + Fixes bug 25170; bugfix on 0.2.9.1. + + o Minor bugfixes (address selection): + - When the fascist_firewall_choose_address_ functions don't find a + reachable address, set the returned address to the null address + and port. This is a precautionary measure, because some callers do + not check the return value. Fixes bug 24736; bugfix + on 0.2.8.2-alpha. + + o Minor bugfixes (compilation): + - Fix a signed/unsigned comparison warning introduced by our fix to + TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. + + o Minor bugfixes (control port, linux seccomp2 sandbox): + - Avoid a crash when attempting to use the seccomp2 sandbox together + with the OwningControllerProcess feature. Fixes bug 24198; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (memory usage): + - When queuing DESTROY cells on a channel, only queue the circuit-id + and reason fields: not the entire 514-byte cell. This fix should + help mitigate any bugs or attacks that fill up these queues, and + free more RAM for other uses. Fixes bug 24666; bugfix + on 0.2.5.1-alpha. + + o Minor bugfixes (network layer): + - When closing a connection via close_connection_immediately(), we + mark it as "not blocked on bandwidth", to prevent later calls from + trying to unblock it, and give it permission to read. This fixes a + backtrace warning that can happen on relays under various + circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. + + o Minor bugfixes (OSX): + - Don't exit the Tor process if setrlimit() fails to change the file + limit (which can happen sometimes on some versions of OSX). Fixes + bug 21074; bugfix on 0.0.9pre5. + + o Minor bugfixes (path selection): + - When selecting relays by bandwidth, avoid a rounding error that + could sometimes cause load to be imbalanced incorrectly. + Previously, we would always round upwards; now, we round towards + the nearest integer. This had the biggest effect when a relay's + weight adjustments should have given it weight 0, but it got + weight 1 instead. Fixes bug 23318; bugfix on 0.2.4.3-alpha. + - When calculating the fraction of nodes that have descriptors, and + all nodes in the network have zero bandwidths, count the number of + nodes instead. Fixes bug 23318; bugfix on 0.2.4.10-alpha. + - Actually log the total bandwidth in compute_weighted_bandwidths(). + Fixes bug 24170; bugfix on 0.2.4.3-alpha. + + o Minor bugfixes (portability, msvc): + - Fix a bug in the bit-counting parts of our timing-wheel code on + MSVC. (Note that MSVC is still not a supported build platform, due + to cryptographic timing channel risks.) Fixes bug 24633; bugfix + on 0.2.9.1-alpha. + + o Minor bugfixes (relay): + - Make the internal channel_is_client() function look at what sort + of connection handshake the other side used, rather than whether + the other side ever sent a create_fast cell to us. Backports part + of the fixes from bugs 22805 and 24898. + + Changes in version 0.2.9.14 - 2017-12-01 - Tor 0.3.0.13 backports important security and stability bugfixes from + Tor 0.2.9.14 backports important security and stability bugfixes from later Tor releases. All Tor users should upgrade to this release, or to another of the releases coming out today. diff --git a/ReleaseNotes b/ReleaseNotes index 3253bd2ba..e691e092b 100644 --- a/ReleaseNotes +++ b/ReleaseNotes @@ -3,7 +3,7 @@ of Tor. If you want to see more detailed descriptions of the changes in each development snapshot, see the ChangeLog file. Changes in version 0.2.9.14 - 2017-12-01 - Tor 0.3.0.13 backports important security and stability bugfixes from + Tor 0.2.9.14 backports important security and stability bugfixes from later Tor releases. All Tor users should upgrade to this release, or to another of the releases coming out today. diff --git a/changes/bug21074_downgrade b/changes/bug21074_downgrade deleted file mode 100644 index 1bc1f8523..000000000 --- a/changes/bug21074_downgrade +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (portability): - - Don't exit the Tor process if setrlimit() fails to change the file - limit (which can happen sometimes on some versions of OSX). Fixes - bug 21074; bugfix on 0.0.9pre5. diff --git a/changes/bug23318 b/changes/bug23318 deleted file mode 100644 index 7fcb8d448..000000000 --- a/changes/bug23318 +++ /dev/null @@ -1,11 +0,0 @@ - o Minor bugfixes (path selection): - - When selecting relays by bandwidth, avoid a rounding error that - could sometimes cause load to be imbalanced incorrectly. Previously, - we would always round upwards; now, we round towards the nearest - integer. This had the biggest effect when a relay's weight adjustments - should have given it weight 0, but it got weight 1 instead. - Fixes bug 23318; bugfix on 0.2.4.3-alpha. - - When calculating the fraction of nodes that have descriptors, and all - all nodes in the network have zero bandwidths, count the number of nodes - instead. - Fixes bug 23318; bugfix on 0.2.4.10-alpha. diff --git a/changes/bug23985 b/changes/bug23985 deleted file mode 100644 index 9cb593796..000000000 --- a/changes/bug23985 +++ /dev/null @@ -1,9 +0,0 @@ - o Minor bugfixes (bootstrapping): - - Fetch descriptors aggressively whenever we lack enough - to build circuits, regardless of how many descriptors we are missing. - Previously, we would delay launching the fetch when we had fewer than - 15 missing descriptors, even if some of those descriptors were - blocking circuits from building. Fixes bug 23985; bugfix on - 0.1.1.11-alpha. The effects of this bug became worse in 0.3.0.3-alpha, - when we began treating missing descriptors from our primary guards - as a reason to delay circuits. diff --git a/changes/bug24167 b/changes/bug24167 deleted file mode 100644 index fd0d87eff..000000000 --- a/changes/bug24167 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (network layer): - - When closing a connection via close_connection_immediately(), we - mark it as "not blocked on bandwidth", to prevent later calls - from trying to unblock it, and give it permission to read. This - fixes a backtrace warning that can happen on relays under various - circumstances. Fixes bug 24167; bugfix on 0.1.0.1-rc. - diff --git a/changes/bug24170 b/changes/bug24170 deleted file mode 100644 index d3d734769..000000000 --- a/changes/bug24170 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (path selection): - - Actually log the total bandwidth in compute_weighted_bandwidths(). - Fixes bug 24170; bugfix on 0.2.4.3-alpha. diff --git a/changes/bug24198 b/changes/bug24198 deleted file mode 100644 index 679070687..000000000 --- a/changes/bug24198 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (controller, linux seccomp2 sandbox): - - Avoid a crash when attempting to use the seccomp2 sandbox - together with the OwningControllerProcess feature. - Fixes bug 24198; bugfix on 0.2.5.1-alpha. diff --git a/changes/bug24480 b/changes/bug24480 deleted file mode 100644 index 94e5b91a0..000000000 --- a/changes/bug24480 +++ /dev/null @@ -1,3 +0,0 @@ - o Minor bugfixes (compilation): - - Fix a signed/unsigned comparison warning introduced by our - fix to TROVE-2017-009. Fixes bug 24480; bugfix on 0.2.5.16. diff --git a/changes/bug24633 b/changes/bug24633 deleted file mode 100644 index 028c7cc14..000000000 --- a/changes/bug24633 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfixes (portability, msvc): - - Fix a bug in the bit-counting parts of our timing-wheel code on - MSVC. (Note that MSVC is still not a supported build platform, - due to cyptographic timing channel risks.) Fixes bug 24633; - bugfix on 0.2.9.1-alpha. diff --git a/changes/bug24666 b/changes/bug24666 deleted file mode 100644 index 830775f5f..000000000 --- a/changes/bug24666 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor bugfixes (memory usage): - - - When queuing DESTROY cells on a channel, only queue the - circuit-id and reason fields: not the entire 514-byte - cell. This fix should help mitigate any bugs or attacks that - fill up these queues, and free more RAM for other uses. Fixes - bug 24666; bugfix on 0.2.5.1-alpha. diff --git a/changes/bug24736 b/changes/bug24736 deleted file mode 100644 index 632560932..000000000 --- a/changes/bug24736 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (address selection): - - When the fascist_firewall_choose_address_ functions don't find a - reachable address, set the returned address to the null address and port. - This is a precautionary measure, because some callers do not check the - return value. - Fixes bug 24736; bugfix on 0.2.8.2-alpha. diff --git a/changes/bug24895 b/changes/bug24895 deleted file mode 100644 index 7edde94a0..000000000 --- a/changes/bug24895 +++ /dev/null @@ -1,8 +0,0 @@ - o Major bugfixes (onion services): - - Fix an "off by 2" error in counting rendezvous failures on the onion - service side. While we thought we would stop the rendezvous attempt - after one failed circuit, we were actually making three circuit attempts - before giving up. Now switch to a default of 2, and allow the consensus - parameter "hs_service_max_rdv_failures" to override. Fixes bug 24895; - bugfix on 0.0.6. - diff --git a/changes/bug24898-029 b/changes/bug24898-029 deleted file mode 100644 index b33f09384..000000000 --- a/changes/bug24898-029 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor bugfixes (relay): - - Make the internal channel_is_client() function look at what sort - of connection handshake the other side used, rather than whether - the other side ever sent a create_fast cell to us. Backports part - of the fixes from bugs 22805 and 24898. - diff --git a/changes/bug24952 b/changes/bug24952 deleted file mode 100644 index 93174c04f..000000000 --- a/changes/bug24952 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (channel connection): - - The accurate address of a connection is real_addr, not the addr member. - TLS Channel remote address is now real_addr content instead of addr - member. Fixes bug 24952; bugfix on 707c1e2e26 in 0.2.4.11-alpha. - Patch by "ffmancera". diff --git a/changes/bug24978 b/changes/bug24978 deleted file mode 100644 index 5dc45c744..000000000 --- a/changes/bug24978 +++ /dev/null @@ -1,7 +0,0 @@ - o Minor features (compatibility, OpenSSL): - - Tor will now support TLS1.3 once OpenSSL 1.1.1 is released. - Previous versions of Tor would not have worked with OpenSSL - 1.1.1, since they neither disabled TLS 1.3 nor enabled any of the - ciphersuites it requires. Here we enable the TLS 1.3 ciphersuites. - Closes ticket 24978. - diff --git a/changes/bug25223 b/changes/bug25223 deleted file mode 100644 index fdd556350..000000000 --- a/changes/bug25223 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Make sure we don't modify consensus parameters if we aren't a public - relay when a new consensus arrives. Fixes bug 25223; bugfix on - 0.3.3.2-alpha. diff --git a/changes/geoip-2017-12-06 b/changes/geoip-2017-12-06 deleted file mode 100644 index ae4fb1149..000000000 --- a/changes/geoip-2017-12-06 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the December 6 2017 Maxmind GeoLite2 - Country database. - diff --git a/changes/geoip-2018-01-05 b/changes/geoip-2018-01-05 deleted file mode 100644 index 59aba02d0..000000000 --- a/changes/geoip-2018-01-05 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the January 5 2018 Maxmind GeoLite2 - Country database. - diff --git a/changes/geoip-2018-02-07 b/changes/geoip-2018-02-07 deleted file mode 100644 index f45228fd7..000000000 --- a/changes/geoip-2018-02-07 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor features (geoip): - - Update geoip and geoip6 to the February 7 2018 Maxmind GeoLite2 - Country database. - diff --git a/changes/ticket23856 b/changes/ticket23856 deleted file mode 100644 index 049da18d0..000000000 --- a/changes/ticket23856 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor feature (relay statistics): - - Change relay bandwidth reporting stats interval from 4 hours to 24 hours - in order to reduce the efficiency of guard discovery attacks. Fixes - ticket 23856. diff --git a/changes/ticket24315 b/changes/ticket24315 deleted file mode 100644 index df34dbf41..000000000 --- a/changes/ticket24315 +++ /dev/null @@ -1,3 +0,0 @@ - o Major features (linux seccomp2 sandbox): - - Update the sandbox rules so that they should now work correctly with - Glibc 2.26. Closes ticket 24315. diff --git a/changes/ticket24681 b/changes/ticket24681 deleted file mode 100644 index cc0a42b2e..000000000 --- a/changes/ticket24681 +++ /dev/null @@ -1,6 +0,0 @@ - o Minor features (fallback directory mirrors): - - Make the default DirAuthorityFallbackRate 0.1, so that clients on the - public tor network prefer to bootstrap off fallback directory mirrors. - This is a follow-up to 24679, which removed weights from the default - fallbacks. - Implements ticket 24681. diff --git a/changes/ticket24902 b/changes/ticket24902 deleted file mode 100644 index 1a2ef95cc..000000000 --- a/changes/ticket24902 +++ /dev/null @@ -1,13 +0,0 @@ - o Major features (denial of service mitigation): - - Give relays some defenses against the recent network overload. We start - with three defenses (default parameters in parentheses). First: if a - single client address makes too many concurrent connections (>100), hang - up on further connections. Second: if a single client address makes - circuits too quickly (more than 3 per second, with an allowed burst of - 90) while also having too many connections open (3), refuse new create - cells for the next while (1-2 hours). Third: if a client asks to - establish a rendezvous point to you directly, ignore the request. These - defenses can be manually controlled by new torrc options, but relays - will also take guidance from consensus parameters, so there's no need to - configure anything manually. Implements ticket 24902. - diff --git a/changes/ticket25122 b/changes/ticket25122 deleted file mode 100644 index 2921811b2..000000000 --- a/changes/ticket25122 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor feature (geoip cache): - - Make our OOM handler aware of the geoip client history cache so it - doesn't fill up the memory which is especially important for IPv6 and - our DoS mitigation subsystem. Closes ticket 25122. diff --git a/changes/ticket25170 b/changes/ticket25170 deleted file mode 100644 index 065213940..000000000 --- a/changes/ticket25170 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor bugfix (directory authority, documentation): - - When a fingerprint or network address is marked as rejected, the - returned message by the authority now explicitly mention to set a valid - ContactInfo address and contact the bad-relays@ mailing list. Fixes bug - 25170; bugfix on 0.2.9.1. diff --git a/changes/ticket25202 b/changes/ticket25202 deleted file mode 100644 index ba64abad7..000000000 --- a/changes/ticket25202 +++ /dev/null @@ -1,4 +0,0 @@ - o Minor bugfixes (DoS mitigation): - - Add extra safety checks when refilling the circuit creation bucket to - ensure we never set a value that is above the allowed burst. Fixes - bug 25202; bugfix on 0.3.3.2-alpha. diff --git a/changes/ticket_24801 b/changes/ticket_24801 deleted file mode 100644 index f5f6c831a..000000000 --- a/changes/ticket_24801 +++ /dev/null @@ -1,5 +0,0 @@ - o Minor features (new fallback directories): - - The fallback directory list has been re-generated based on the - current status of the network. Tor uses fallback directories to - bootstrap it doesn't yet have up-to-date directory - information. Closes ticket 24801.

1 0