[tor-bugs] #31197 [Metrics/Website]: Upgrade metrics-web to Debian buster libraries

Tor Bug Tracker & Wiki blackhole at torproject.org
Thu Jul 25 10:44:48 UTC 2019


#31197: Upgrade metrics-web to Debian buster libraries
-----------------------------+------------------------------
 Reporter:  karsten          |          Owner:  metrics-team
     Type:  enhancement      |         Status:  new
 Priority:  High             |      Milestone:
Component:  Metrics/Website  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:  #31193           |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------
Changes (by karsten):

 * priority:  Medium => High


Comment:

 I'm hitting a brick wall here.

 Upgrading embedded Jetty and its internal dependencies on Tomcat9 turned
 out to be as complicated as expected. Here's what I have right now, and
 it's not working:

 {{{
 diff --git a/build.xml b/build.xml
 index 0c9fa0e4..0d4979cb 100644
 --- a/build.xml
 +++ b/build.xml
 @@ -9,9 +9,9 @@
    <property name="javadoc-title" value="MetricsWeb API Documentation"/>
    <property name="implementation-title" value="metrics-web" />
    <property name="release.version" value="1.2.0-dev" />
 -  <property name="metricslibversion" value="2.4.0" />
 +  <property name="metricslibversion" value="2.6.2" />
    <property name="exoneratorversion" value="4.1.0" />
 -  <property name="jetty.version" value="-9.2.21.v20170120" />
 +  <property name="jetty.version" value="-9.4.15.v20190215" />
    <property name="warfile"
              value="metrics-web-${release.version}.war"/>

 @@ -38,27 +38,27 @@
    </target>

    <patternset id="common" >
 -    <include name="commons-codec-1.10.jar"/>
 -    <include name="commons-lang3-3.5.jar"/>
 +    <include name="commons-codec-1.11.jar"/>
 +    <include name="commons-lang3-3.8.jar"/>
      <include name="commons-math3-3.6.1.jar"/>
 -    <include name="jackson-annotations-2.8.6.jar"/>
 -    <include name="jackson-core-2.8.6.jar"/>
 -    <include name="jackson-databind-2.8.6.jar"/>
 -    <include name="logback-core-1.1.9.jar" />
 -    <include name="logback-classic-1.1.9.jar" />
 +    <include name="jackson-annotations-2.9.8.jar"/>
 +    <include name="jackson-core-2.9.8.jar"/>
 +    <include name="jackson-databind-2.9.8.jar"/>
 +    <include name="logback-core-1.2.3.jar" />
 +    <include name="logback-classic-1.2.3.jar" />
      <include name="REngine.jar"/>
      <include name="Rserve.jar"/>
 -    <include name="slf4j-api-1.7.22.jar"/>
 +    <include name="slf4j-api-1.7.25.jar"/>
    </patternset>

    <patternset id="library" >
      <patternset refid="common" />
 -    <include name="metrics-lib-${metricslibversion}.jar"/>
 -    <include name="commons-compress-1.13.jar"/>
 +    <include name="metrics-lib-${metricslibversion}-thin.jar"/>
 +    <include name="commons-compress-1.18.jar"/>
      <include name="commons-math3-3.6.1.jar"/>
 -    <include name="postgresql-9.4.1212.jar"/>
 +    <include name="postgresql-42.2.5.jar"/>
      <include name="servlet-api-3.1.jar"/>
 -    <include name="xz-1.6.jar"/>
 +    <include name="xz-1.8.jar"/>
    </patternset>

    <patternset id="web" >
 @@ -76,18 +76,18 @@
 @@ -76,18 +76,18 @@
    </patternset>

    <patternset id="webruntime" >
 -    <include name="asm-5.2.jar" />
 -    <include name="asm-commons-5.2.jar" />
 -    <include name="commons-compress-1.13.jar"/>
 +    <include name="asm-all-7.0.jar" />
 +    <include name="commons-compress-1.18.jar"/>
      <include name="jetty9-plus${jetty.version}.jar"/>
      <include name="jetty9-jndi${jetty.version}.jar"/>
      <include name="jetty9-apache-jsp${jetty.version}-tweaked.jar" />
      <include name="metrics-lib-${metricslibversion}.jar"/>
      <include name="taglibs-standard-spec-1.2.5.jar"/>
 -    <include name="tomcat8-embed-jasper-8.5.14.jar" />
 -    <include name="tomcat8-embed-el-8.5.14.jar" />
 -    <include name="tomcat8-embed-core-8.5.14.jar" />
 -    <include name="eclipse-ecj-3.11.1.jar" />
 +    <include name="tomcat9-annotations-api-9.0.16.jar" />
 +    <include name="tomcat9-embed-jasper-9.0.16-tweaked.jar" />
 +    <include name="tomcat9-embed-el-9.0.16.jar" />
 +    <include name="tomcat9-embed-core-9.0.16.jar" />
 +    <include name="eclipse-ecj-3.16.0.jar" />
    </patternset>

    <patternset id="runtime" >
 @@ -139,16 +139,22 @@
    <target name="war"
            depends="submoduleupdate,jar">
      <echo message="Removing problematic service definitions from"/>
 -    <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and
 using "/>
 -    <echo message="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar
 below." />
 +    <echo message="${libs}/jetty9-apache-jsp${jetty.version}.jar and "/>
 +    <echo message="${libs}/tomcat9-embed-jasper-9.0.16.jar." />
      <delete file="${libs}/jetty9-apache-jsp${jetty.version}-tweaked.jar"
              quiet="true" />
 -
 +    <delete file="${libs}/tomcat9-embed-jasper-9.0.16-tweaked.jar"
 +            quiet="true" />
      <jar destfile="${libs}/jetty9-apache-
 jsp${jetty.version}-tweaked.jar">
        <zipfileset src="${libs}/jetty9-apache-jsp${jetty.version}.jar" >
          <exclude name="**/javax.servlet.ServletContainerInitializer"/>
        </zipfileset>
      </jar>
 +    <jar destfile="${libs}/tomcat9-embed-jasper-9.0.16-tweaked.jar">
 +      <zipfileset src="${libs}/tomcat9-embed-jasper-9.0.16.jar" >
 +        <exclude name="**/javax.servlet.ServletContainerInitializer"/>
 +      </zipfileset>
 +    </jar>

      <antcall target="generate-ml-javadoc" />
 }}}

 The error messages aren't very helpful, but I think it's related to some
 internal changes in Jetty or embedded Tomcat using a different library for
 JSP compilation.

 I feel like we're doing it wrong.

 The original idea of using libraries shipped with Debian stable was a good
 one, because it would allow anyone on a Debian machine to build our
 software with minimal effort.

 But this doesn't come for free. We're basically doing manual dependency
 management not only for our dependencies but also for their dependencies.

 Another major drawback, in addition to having to fix the issue above, is
 that it's almost prohibitively expensive to add new dependencies. For
 example, I'd very much want to add something like FindBugs. But I'm very
 much afraid of adding all its dependencies, which is why I'm not doing it.
 This hurts us, because we cannot improve our code quality.

 Suggestion: We look into tools for managing dependencies. This could
 include Ant Ivy or Maven or others, we discuss what we learned, and then
 we switch. The goal would still be to run our .jar and .war files on a
 Debian stable machine only with standard packages. But building would
 require more than just Debian packages.

 Let's discuss this at today's meeting. This is urgent, because we need to
 resolve this before merging other patches. Ugh.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/31197#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online


More information about the tor-bugs mailing list