Ken Keys:
The point is that one can't[*] extract a private key from a smartcard and because of that even if machine is compromised your private key stays safe.
If the machine is going to use the HS key, the actual HS key has to be visible to it.
Nope. If the machine is going to use the HS key it can ask a smartcard to do so. Of course private key is visible to something/someone anyway. But in case of smartcards it is visible to a smartcard only.
An encrypted container holding a VM could use RSA-style public/private key encryption so that it never has to see the private key used to unlock it. You would still need to trust the VM, but the encrypted container would allow you to establish a chain of custody.
It's OK to unlock some encrypted block device/VM with some 'unpluggable' key. But it does nothing to protect your HS' identity.