Note that most clients use the ORPort for fetching directory stuff, and that's heading towards "all clients" as people upgrade and stop using weird configurations.
If you're worried about denial of service issues on the DirPort, maybe the simple answer is to turn off the DirPort? I think the only real impact might have something to do with whether old clients believe that you're a usable guard.
What about fallback directory mirrors? Does fallback traffic go over the ORPort too? Is it safe to disable the DirPort on a fallback relay?