We have discussed how Snowflake uses WebRTC data channels (DTLS), while some other WebRTC apps use media channels (DTLS-STRP), and how that could be used in a classifier. See for example Section 2 of https://censorbib.nymity.ch/#Fifield2016b. We have also entertained the possibility of encoding Snowflake data into an audio/video stream, à la FreeWave (https://censorbib.nymity.ch/#Houmansadr2013a) or CovertCast (https://censorbib.nymity.ch/#McPherson2016a), in order to use media channels in Snowflake.
I came across an API that may permit us to use media channel, without the trouble of an audio/video codec. WebRTC Encoded Transform (formerly Insertable Streams, I think) provides a way to get at and manipulate the raw bytes of media frames.
https://w3c.github.io/webrtc-encoded-transform/
This API defines an API surface for manipulating the bits on MediaStreamTracks being sent via an RTCPeerConnection.
The API defines RTCEncodedVideoFrame and RTCEncodedAudioFrame types that give you an ArrayBuffer of the encoded media data:
https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedVideoFrame-interfa... // New interfaces to define encoded video and audio frames. Will eventually // re-use or extend the equivalent defined in WebCodecs. [Exposed=(Window,DedicatedWorker)] interface RTCEncodedVideoFrame { readonly attribute RTCEncodedVideoFrameType type; readonly attribute unsigned long timestamp; attribute ArrayBuffer data; RTCEncodedVideoFrameMetadata getMetadata(); }; https://w3c.github.io/webrtc-encoded-transform/#RTCEncodedAudioFrame-interfa... [Exposed=(Window,DedicatedWorker)] interface RTCEncodedAudioFrame { readonly attribute unsigned long timestamp; attribute ArrayBuffer data; RTCEncodedAudioFrameMetadata getMetadata(); };
One of the intended use cases of Encoded Transform is a end-to-end encryption when then media streams pass through a middlebox that removes and re-adds hop-by-hop DTLS-STRP layers:
https://webrtchacks.com/true-end-to-end-encryption-with-webrtc-insertable-st... https://jitsi.org/security/
You can turn on end-to-end encryption (e2ee) as long as you are using Jitsi Meet on a browser with support for insertable streams.
The explainer document has a sample of inverting all the bits in an encoded media frame:
https://github.com/w3c/webrtc-encoded-transform/blob/5c7ab84f4ce338f299172fa... // Receiver transform function createReceiverTransform() { return new TransformStream({ start() {}, flush() {}, async transform(encodedFrame, controller) { // Reconstruct the original frame. const view = new DataView(encodedFrame.data);
// Ignore the last 4 bytes const newData = new ArrayBuffer(encodedFrame.data.byteLength - 4); const newView = new DataView(newData);
// Negate all bits in the incoming frame, ignoring the // last 4 bytes for (let i = 0; i < encodedFrame.data.byteLength - 4; ++i) newView.setInt8(i, ~view.getInt8(i));
encodedFrame.data = newData; controller.enqueue(encodedFrame); } }); }
I am not clear on the difference between Encoded Transform, Insertable Streams, and WebCodecs, but here are some possible browser support tables:
https://chromestatus.com/feature/6321945865879552 https://caniuse.com/?search=webcodec https://developer.mozilla.org/en-US/docs/Web/API/WebCodecs_API