[tor-bugs] #24659 [Core Tor/Tor]: Wrap our sha2 interface in Rust which implements the appropriate traits

[Tor Bug Tracker & Wiki]

#24659: Wrap our sha2 interface in Rust which implements the appropriate traits
-------------------------------+------------------------------------
 Reporter:  isis               |          Owner:  (none)
     Type:  enhancement        |         Status:  needs_review
 Priority:  Medium             |      Milestone:  Tor: 0.3.4.x-final
Component:  Core Tor/Tor       |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  rust, tor-crypto,  |  Actual Points:
Parent ID:                     |         Points:  1
 Reviewer:                     |        Sponsor:  Sponsor3-can
-------------------------------+------------------------------------

Comment (by isis):

 I've done the cleanup work, and also gone ahead and implemented SHA2-512
 wrappers. XOFs still need to be done, and maybe a bit of utility work to
 wrap C functions that don't fit nicely into any of the provided traits.

 There's a problem with linking. We already have some Rust code which calls
 C code in `src/rust/external`: it is used in `protover.rs` and it calls
 `tor_version_as_new_as`, the latter is in `src/or/protover.c` and is built
 into `src/or/libtor.a`. My new code needs to link to the functions in
 `src/common/crypto_digest.h` (from #24658), which are eventually built
 into `src/common/libor-crypto.a`. The Rust code is compiled eventually
 into `src/rust/target/*/libtor-rust.a`, and at the very end of the build
 (I think?) it is linked to both `libtor.a` and `libor-crypto.a` to make
 the final executable. The issue was that it does not have symbols
 available for the functions in `crypto_digest.h`:

 {{{
   = note: /home/travis/build/isislovecruft/tor/src/rust/target/debug/deps
 /libexternal-792b24e27067f1d1.rlib(external-
 792b24e27067f1d1.3fgl34tvxe66etyo.rcgu.o): In function
 `external::crypto_digest::CryptoDigest::new::h0e98658512d8b319':
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:225:
 undefined reference to `crypto_digest_new'
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:233:
 undefined reference to `crypto_digest_new'
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:234:
 undefined reference to `crypto_digest256_new'
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:235:
 undefined reference to `crypto_digest256_new'
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:236:
 undefined reference to `crypto_digest512_new'
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:237:
 undefined reference to `crypto_digest512_new'
           /home/travis/build/isislovecruft/tor/src/rust/target/debug/deps
 /libexternal-792b24e27067f1d1.rlib(external-
 792b24e27067f1d1.3fgl34tvxe66etyo.rcgu.o): In function
 `external::crypto_digest::CryptoDigest::add_bytes::h3401b4d82d21aa04':
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:280:
 undefined reference to `crypto_digest_add_bytes'
           /home/travis/build/isislovecruft/tor/src/rust/target/debug/deps
 /libexternal-792b24e27067f1d1.rlib(external-
 792b24e27067f1d1.3fgl34tvxe66etyo.rcgu.o): In function
 `external::crypto_digest::get_256_bit_digest::h126330ad84e55865':
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:318:
 undefined reference to `crypto_digest_get_digest'
           /home/travis/build/isislovecruft/tor/src/rust/target/debug/deps
 /libexternal-792b24e27067f1d1.rlib(external-
 792b24e27067f1d1.3fgl34tvxe66etyo.rcgu.o): In function
 `external::crypto_digest::get_512_bit_digest::h0ac55a1ee3c0a1d3':
 /home/travis/build/isislovecruft/tor/src/rust/external/crypto_digest.rs:360:
 undefined reference to `crypto_digest_get_digest'
           collect2: error: ld returned 1 exit status
 }}}


 If I
 [https://github.com/isislovecruft/tor/commit/c9e098f8639bbf5db6976247128bab7cb9a3491b
 add] a `#[link!]` directive to my code, like so, in
 `src/rust/external/crypto_digest.rs`:

 {{{
 +#[link(name = "or-crypto")]
  extern "C" {
      fn crypto_digest(digest: *mut c_char, m: *const c_char, len: size_t)
 -> c_int;
      fn crypto_digest256(digest: *mut c_char, m: *const c_char, len:
 size_t,
 }}}

 Then komlo's protover.rs code was missing symbols. So, I added another
 `#[link]` directive to komlo's code, in `src/rust/external/lib.rs`:

 {{{
 +#[link(name = "tor")]
  extern "C" {
      fn tor_version_as_new_as(
          platform: *const c_char,
 }}}

 This also did not work ([https://travis-
 ci.org/isislovecruft/tor/jobs/345790478#L2605 TravisCI results]):

 {{{
 = note: /usr/bin/ld: cannot find -ltor
           /usr/bin/ld: cannot find -lor-crypto
           collect2: error: ld returned 1 exit status
 }}}

 So, as far as I understand it (I'm not the greatest at build systems
 engineering), what probably needs to happen is something like this:

  1. `crypto_digest.[ch]` are built into a new library, say `libor-crypto-
 digest.a`.
  2. `libor-crypto-digest.a` is linked to when `libtor-rust.a` is being
 built.
  3. `libtor-rust.a` still needs to somehow be able to find the symbol for
 `tor_version_as_new_as` from `libtor.a`??? (''I have no idea how it was
 ever doing this!!'')
  4. `libor-crypto.a`, `libor-crypto-digest.a`, `libtor-rust.a`, and
 `libtor.a` all get linked together in the final stage.

 (Also at some point we may need to have talks about avoiding circular
 dependencies if we have Rust calling C and C calling Rust. However, this
 doesn't currently involve any circular dependencies which can be avoided
 through further library modularisation.)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24659#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online