After a lot of debugging, it seems that the reason for streams failing is as follows :
As soon as a new stream is made, it goes into the NEW state according to Torctl logs.
Then with stem or carml running, they try to attach it to the specified circuit.
As soon as the stream is attached, it moves to the SENTCONNECT, and if it doesn't succeed, the state of that stream again changes to NEW after which the connection fails (the stream waits for the 120s then gets closed automatically with error code REASON_TIMEOUT).
A point i noted was that, a stream is in the SENTCONNECT state for a very short duration (fraction of seconds).
Other times, when the above process does not happen, the streams successfully move to the SUCCEEDED state after SENTCONNECT and are served through the specified circuit.
But this behavior seems to be completely random. Is there a way to resolve this issue?
@meejah : carml had the same problem as with stem as already explained above.
@Nick Mathewson : Yes there are some instances where the circuit creation itself takes a lot of time and fails sometimes. But those cases are already handled. The main problem as we could drill down is explained above.