[tor-bugs] #22594 [Metrics/Onionoo]: Escape characters in contact lines break hourly updater

Tor Bug Tracker & Wiki blackhole at torproject.org
Tue Jun 13 15:39:53 UTC 2017


#22594: Escape characters in contact lines break hourly updater
---------------------------------+--------------------------
     Reporter:  karsten          |      Owner:  metrics-team
         Type:  defect           |     Status:  new
     Priority:  High             |  Milestone:
    Component:  Metrics/Onionoo  |    Version:
     Severity:  Normal           |   Keywords:
Actual Points:                   |  Parent ID:
       Points:                   |   Reviewer:
      Sponsor:                   |
---------------------------------+--------------------------
 Onionoo's hourly updater broke on June 12 at around 13:30 UTC.  Turns out
 it couldn't process the following server descriptor (look out for the
 contact line):

 {{{
 @type server-descriptor 1.0
 router HarukaMiddle 46.101.62.152 9001 0 0
 identity-ed25519
 -----BEGIN ED25519 CERT-----
 AQQABlsyAa97mM9YlSM9a8RHbgNwUduV8zMYrUs/uXhfk3fg7ZPEAQAgBAA/B+AS
 R+3S5M09GNQjE9EzvGR/FS6s+WjMs9bNdxTideK2fjKqU3mR+QqCvOP7DYEh8/2w
 VMChyxEjyKWBRo4iFyVTICqeuStIRLqPAVY/ODcvHbQNbzOY1F8OZSEWWwc=
 -----END ED25519 CERT-----
 master-key-ed25519 PwfgEkft0uTNPRjUIxPRM7xkfxUurPlozLPWzXcU4nU
 platform Tor 0.2.9.11 on Linux
 proto Cons=1-2 Desc=1-2 DirCache=1 HSDir=1 HSIntro=3 HSRend=1-2 Link=1-4
 LinkAuth=1 Microdesc=1-2 Relay=1-2
 published 2017-06-12 12:39:39
 fingerprint EABD 6C28 2F28 C6F3 EBB1 AA59 3B50 2071 3B33 D131
 uptime 0
 bandwidth 2048000 2048000 2220032
 extra-info-digest 290AF585354448C448A748CBC632D933CD2880D6
 s5rHOgcE+7SAeDUUiQbb9OaYgpfPFOBgW/uJ4SVuXuk
 onion-key
 -----BEGIN RSA PUBLIC KEY-----
 MIGJAoGBAMaI+nbAiMpcMZRXhV6ai9ccHwhp9mr5nNwYunUzdNSyCIB0N56ODZ3c
 xme3mG1QQ08um1ewXb6vMxxsiZzzYVxdyawOa+oHMYQEISWyZlvPw+7PDtAg5bGR
 8gWqUAIXSTe3K1YXnaXpNcEiwVOO35jDih5HkzkVCvnzz8BCYDUHAgMBAAE=
 -----END RSA PUBLIC KEY-----
 signing-key
 -----BEGIN RSA PUBLIC KEY-----
 MIGJAoGBAMygGYMk7d+pYQu5JyCROtRS2LtURSSgfaAGsPUblTGD+ZoSlDDGIu2Q
 AUC7QzQxO1HxbUqkoQwo0Im3jsQPcHznIuKevJEzMCPkErn1DIwOidYdW9tTLgcG
 4+q5pvwpyd0uDTV2Z8SqXnIyVfAFpAGt8LOOFahJiTMHRRI8ABsFAgMBAAE=
 -----END RSA PUBLIC KEY-----
 onion-key-crosscert
 -----BEGIN CROSSCERT-----
 HAn8AxeqoyWnI0h/JhnKoEEs6Lgz/JwmhruEjOC488bOF5G1HJrkM675Sj4B8A3Z
 5pGPn0hi1Owbza72nlUtxPuK8pa6ItP/A8Q1v9AvwYhIkmEvHVaCH/FzthW+8AzG
 lGiH6FcP/VQ3htQ2kpNUyMrkqLVgDHmVXgT/bXBGsto=
 -----END CROSSCERT-----
 ntor-onion-key-crosscert 1
 -----BEGIN ED25519 CERT-----
 AQoABllNAT8H4BJH7dLkzT0Y1CMT0TO8ZH8VLqz5aMyz1s13FOJ1ADVfDOXFoxvI
 Zre9gmhKEuPq10ioGbGaKtvj/YrdVsFwNc76mGRSpaM2Ar2DBkxpZBh/p3Cwqe0V
 Bbcre5sQOgo=
 -----END ED25519 CERT-----
 hidden-service-dir
 contact Haruka iampsychopath/\@/\live/\./\co/\./\uk
 ntor-onion-key 98H8kDWwomC0z+goDIgI2MH0fMkW1I1vbme2dBoo8TY=
 reject *:*
 router-sig-ed25519
 UFYUr1vTeiRwA4grYD/LGLHC8xuzKMSNDijHIcZkckqXdxiXLwRasQagyXuKxwKS+q6A7uIxRUcwdjxq7t1sBA
 router-signature
 -----BEGIN SIGNATURE-----
 wngC1BBTQUrSkZejdr9pYEGJmAreph8x0g0UvA5jjTX7do/ltRrmPN6VSgxbd36y
 nHEe+cL8jYXUyLBENxnD4cA4pVxYgqFYWlhgtrDIonmeMWGXfirJBHIbZG/zKeVv
 EXRdkh13OHEtUWU1PEGRSNNt7oSQf6rl//Qwz3Andx8=
 -----END SIGNATURE-----
 }}}

 Here's the exception:

 {{{
 java.lang.NumberFormatException: \uk","
         at
 com.google.gson.stream.JsonReader.readEscapeCharacter(JsonReader.java:1466)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 com.google.gson.stream.JsonReader.nextQuotedValue(JsonReader.java:993)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 com.google.gson.stream.JsonReader.nextString(JsonReader.java:811)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:358)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 com.google.gson.internal.bind.TypeAdapters$13.read(TypeAdapters.java:346)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.read(ReflectiveTypeAdapterFactory.java:93)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:172)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at com.google.gson.Gson.fromJson(Gson.java:803)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at com.google.gson.Gson.fromJson(Gson.java:768)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at com.google.gson.Gson.fromJson(Gson.java:717)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at com.google.gson.Gson.fromJson(Gson.java:689)
 ~[onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.docs.DocumentStore.retrieveParsedDocumentFile(DocumentStore.java:539)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.docs.DocumentStore.retrieveDocumentFile(DocumentStore.java:505)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.docs.DocumentStore.retrieve(DocumentStore.java:378)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.updater.NodeDetailsStatusUpdater.processRelayServerDescriptor(NodeDetailsStatusUpdater.java:151)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.updater.NodeDetailsStatusUpdater.processDescriptor(NodeDetailsStatusUpdater.java:130)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.updater.DescriptorSource.readDescriptors(DescriptorSource.java:132)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.updater.DescriptorSource.readDescriptors(DescriptorSource.java:97)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at org.torproject.onionoo.cron.Main.updateStatuses(Main.java:180)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at org.torproject.onionoo.cron.Main.run(Main.java:129)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at
 org.torproject.onionoo.cron.Main.runOrScheduleExecutions(Main.java:103)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
         at org.torproject.onionoo.cron.Main.main(Main.java:35)
 [onionoo-4.0-1.2.0-dev.jar:4.0-1.2.0-dev-595f87a]
 }}}

 This issue was quite well hidden, because `Gson.fromJson` apparently
 doesn't catch `NumberFormatException`, so the main thread just dies.

 Here's the (pretty-printed) string that it attempted to parse:

 {{{
 {
   "desc_published": "2017-06-12 12:39:39",
   "last_restarted": "2017-06-12 12:39:39",
   "bandwidth_rate": 2048000,
   "bandwidth_burst": 2048000,
   "observed_bandwidth": 2220032,
   "advertised_bandwidth": 2048000,
   "exit_policy": [
     "reject *:*"
   ],
   "contact":"Haruka
 iampsychopath/\\\\@/\\\\live/\\\\./\\\\co/\\\\./\\\uk",
   "platform": "Tor 0.2.9.11 on Linux",
   "alleged_family": [],
   "effective_family": [],
   "indirect_family": [],
   "is_relay": true,
   "running": true,
   "nickname": "HarukaMiddle",
   "address": "46.101.62.152",
   "or_addresses_and_ports": [],
   "first_seen_millis": 1497031200000,
   "last_seen_millis": 1497268800000,
   "or_port": 9001,
   "dir_port": 0,
   "relay_flags": [
     "Fast",
     "Running",
     "Valid"
   ],
   "consensus_weight": 483,
   "default_policy": "reject",
   "port_list": "1-65535",
   "last_changed_or_address_or_port": 1497031200000,
   "recommended_version": true,
   "measured": true,
   "exit_addresses": {},
   "consensus_weight_fraction": 1.1896908e-05,
   "guard_probability": 0,
   "middle_probability": 3.122049e-05,
   "exit_probability": 0,
   "latitude": 51.5092,
   "longitude": -0.0955,
   "country_code": "gb",
   "country_name": "United Kingdom",
   "region_name": "England",
   "city_name": "London",
   "as_name": "Digital Ocean, Inc.",
   "as_number": "AS14061",
   "host_name": "46.101.62.152"
 }
 }}}

 I deployed the following hotfix:

 {{{
 diff --git a/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
 b/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
 index 34bc8ef..246c02b 100644
 --- a/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
 +++ b/src/main/java/org/torproject/onionoo/docs/DocumentStore.java
 @@ -537,8 +537,9 @@ public class DocumentStore {
      Gson gson = new Gson();
      try {
        result = gson.fromJson(documentString, documentType);
 -    } catch (JsonParseException e) {
 +    } catch (JsonParseException | NumberFormatException e) {
        /* Handle below. */
 +      log.error(documentString);
        log.error(e.getMessage(), e);
      }
      if (result == null) {
 }}}

 The hourly update is still running, but I believe the result will be that
 the relay publishing this descriptor will simply not show up in Onionoo
 results, or at least not with recent data.  Should be fine for the moment,
 but deserves a better fix.  The real fix is to check our logic for
 escaping/unescaping JSON strings, yet once more.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22594>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list