Your ssh -L 8080:app:80 bastion command looks like it creates a dedicated pipe between two ports. It does not. What actually happens is that every new TCP connection to localhost:8080 opens a numbered channel inside the same encrypted SSH stream, and all those channels share one TCP connection to the bastion host. If you also have an interactive shell running over that same ssh session, your keystrokes and the forwarded HTTP traffic are interleaved as tagged binary packets, demultiplexed by channel number on the other side. The illusion of separate pipes is a careful piece of protocol engineering that most people never need to think about until something breaks.
The four layers you need to separate
SSH is not one protocol. It is a stack of three protocols riding on top of a single TCP connection. The TCP socket between your client and port 22 is the physical pipe. That pipe carries one ordered byte stream and knows nothing about encryption or multiplexing. The SSH transport layer, defined in RFC 4253, takes that raw byte stream and turns it into an encrypted binary packet stream with negotiated algorithms and keys. Each packet has a length field, padding to hide traffic patterns, and a message authentication code covering everything. The transport layer sees only SSH messages. It has no concept of a shell, a port forward, or a SOCKS proxy.
Above transport sits the user authentication protocol. The client proves its identity. Once that handshake finishes, both sides switch to the SSH connection protocol, identified by the service name "ssh-connection". This layer, specified in RFC 4254, is where channels exist. Channels are logical streams that are multiplexed inside the one encrypted connection. Every interactive shell, every port forward, every X11 session gets its own channel. Authentication happens once per TCP connection. After that, the client can open dozens of channels without reauthenticating.
Port forwarding and dynamic SOCKS tunnels live at the edge, inside the ssh client and server processes. They map local TCP sockets onto channels of specific types. The SSH protocol itself never sees SOCKS or HTTP. It sees channel open requests, data messages, and window adjustments. The bytes inside those messages happen to be HTTP requests or database queries, but the transport and connection layers treat them as opaque payloads.
The channel abstraction does the real multiplexing
Think of the SSH connection like a shipping container carried on a freight train. The train is the TCP connection. The container is the encrypted SSH transport. Inside the container, you pack multiple smaller boxes, each with a numbered label. Those boxes are channels. A box labeled session might hold a shell. A box labeled direct-tcpip might hold forwarded HTTP traffic. The train does not know what is in any box. It only moves the container from A to B. At the destination, you unpack the container and route each box to the right recipient based on its label.
On the wire, this works through a small set of message types. When the client wants a new logical stream, it allocates a local channel number and sends SSH_MSG_CHANNEL_OPEN with a type string like "session" or "direct-tcpip". The server receives this, decides whether to accept, and if so replies with SSH_MSG_CHANNEL_OPEN_CONFIRMATION. The confirmation includes the server's own channel number for this same logical stream. From that point forward, each side uses the peer's channel number in all messages related to that stream.
Channel numbers are independent on each side. The client's channel 3 might map to the server's channel 17. The mapping is established at open time and stored in per-channel state on both ends. Every data message, every window adjustment, every close notification carries the recipient's channel number. The receiving side looks up that number in a table of active channels and routes the payload to whatever subsystem is bound to it. A channel carrying shell bytes and a channel carrying forwarded MySQL traffic are indistinguishable at the dispatch level. The only difference is what file descriptor or socket the bytes get written to after demultiplexing.
Channel lifecycle in a single SSH connection
All channels follow the same lifecycle. Open. Transfer data. Optionally signal end of file. Close. The messages that drive this are defined in RFC 4254 section 5.
A channel starts with SSH_MSG_CHANNEL_OPEN. The payload contains the channel type, the sender's channel number, an initial window size, and a maximum packet size. For port forwarding channels, the open message also carries target host and port information. The peer responds with either SSH_MSG_CHANNEL_OPEN_CONFIRMATION or SSH_MSG_CHANNEL_OPEN_FAILURE. Confirmation binds the peer's channel number to the same logical stream.
Once confirmed, data flows through SSH_MSG_CHANNEL_DATA messages. These messages carry arbitrary byte payloads tagged with the recipient's channel number. There is no framing within a channel. If you send 5 bytes followed by 1000 bytes on the same channel, the receiver sees 1005 bytes of contiguous stream data. The SSH layer does not preserve message boundaries inside a channel. This makes channels behave like TCP streams sitting inside the encrypted tunnel.
Shutdown happens in two phases. When one side has no more data to send on a channel, it sends SSH_MSG_CHANNEL_EOF. This tells the peer "I am done writing, but I can still read." The peer can continue sending data. When both sides are finished, each sends SSH_MSG_CHANNEL_CLOSE. The channel is then freed on both endpoints. This two-phase shutdown mirrors TCP's half-close semantics and allows clean teardown of forwarded connections.
OpenSSH's channel implementation maintains a global list of Channel structures. Each structure tracks local and remote channel IDs, the current window size, the state of the underlying file descriptor or socket, and buffers for pending data. The main event loop iterates over these structures to decide which channels have data ready to send or receive.
Flow control prevents one channel from starving others
Without per-channel flow control, a high-throughput port forward could consume all available buffer space and block the interactive shell sharing the same SSH connection. SSH prevents this by giving each channel its own receive window. The mechanism is described in RFC 4254 section 5.2.
When a channel is opened, the opener advertises an initial window size. This is the number of bytes the peer is allowed to send on that channel before the opener must grant more window space. As the receiver delivers channel bytes to the local endpoint, it frees buffer space and sends SSH_MSG_CHANNEL_WINDOW_ADJUST to increase the sender's allowed window. If the receiver stops draining, because the downstream TCP socket for a forwarded connection is congested or the PTY for a shell is not being read, the receive buffer fills. Window adjustments stop. The sender's window drops to zero. The sender must stop transmitting data on that channel.
Crucially, this blockage applies only to the throttled channel. The sender maintains a per-channel window counter. When channel 5's window is zero, the sender stops putting SSH_MSG_CHANNEL_DATA for channel 5 into outgoing packets. But channel 7, which still has available window, can continue sending. Its data messages are interleaved into the same encrypted TCP stream. The SSH sender does not care that channel 5 is stalled. It simply skips that channel in its output polling loop until a window adjustment arrives.
This nested flow control operates inside the encrypted tunnel, independently of TCP's own flow control on the underlying socket. TCP manages end-to-end congestion between the SSH client and server. SSH's channel windows manage fairness between logical streams that share that TCP pipe. A file transfer saturating one forwarded port cannot starve your interactive shell because the shell channel's window is drained and replenished on its own schedule.
How local port forwarding actually works
Local forwarding is the -L flag. You run ssh -L 8080:internal-db:5432 bastion-host. The ssh client connects to bastion-host, authenticates, and starts the connection protocol. It also creates a TCP listening socket on the client machine bound to port 8080. Any local application that connects to localhost:8080 is connecting to the ssh client process, not directly to bastion-host.
When a local application connects, the ssh client accepts the TCP connection and immediately opens a new SSH channel with type "direct-tcpip". The channel open request includes four parameters defined in RFC 4254 section 7.2: the target host (internal-db), the target port (5432), and the originator's IP address and port. The originator information is the local application's connection details. It lets the remote side log who initiated the tunneled connection.
The ssh server receives this channel open request. If its configuration permits forwarding to internal-db:5432, it resolves that hostname and opens a new outbound TCP connection from itself to the target. If that TCP connection succeeds, the server sends back a channel open confirmation. The direct-tcpip channel is now bound to that outbound TCP socket.
This sequence creates a three-hop data path: local application TCP socket to ssh client, ssh client to channel data messages inside the encrypted SSH stream, and ssh server channel data to the outbound TCP socket on the remote side. Bytes flow in both directions. The local application writes to its socket, the ssh client reads, wraps bytes into SSH_MSG_CHANNEL_DATA, the server unwraps and writes to the target. The target responds, the server wraps, the client unwraps and writes to the local application. Neither the local application nor the remote database knows the SSH tunnel exists. They see ordinary TCP connections.
Every new connection to localhost:8080 opens a new direct-tcpip channel. Ten concurrent PostgreSQL connections from your application become ten separate channels inside one SSH TCP connection to bastion-host. Each gets its own channel ID, its own window, and its own lifecycle. They share nothing except the encrypted transport pipe.
Remote port forwarding inverts the direction
Remote forwarding uses the -R flag. The command ssh -R 9000:localhost:3306 bastion-host tells the ssh client to ask the server to listen on port 9000. When someone connects to port 9000 on the server, the server opens a channel back to the client, which then connects to localhost:3306 on the client machine.
This uses two protocol mechanisms. First, the client sends a global request of type "tcpip-forward" as defined in RFC 4254 section 7.1. Global requests are not tied to any channel. They affect the SSH connection as a whole. The tcpip-forward request tells the server to start listening on a specified address and port. The server acknowledges with a success or failure response.
When a third party connects to that listening port, the server accepts the TCP connection and opens a new channel of type "forwarded-tcpip". The channel open message includes the address and port that were connected to, plus the originator's address and port. The client receives this channel open request. If it accepts, it opens a new outbound TCP connection from itself to the target specified in the original -R command. In the example, that target is localhost:3306.
The resulting data path is the mirror of local forwarding. A remote user connects to port 9000 on the server. The server's bytes flow through an SSH channel to the client. The client writes them to localhost:3306. MySQL's responses travel back through the same channel to the server and out to the remote user. Multiple concurrent connections to port 9000 become multiple forwarded-tcpip channels multiplexed over the same SSH TCP connection.
Dynamic forwarding adds SOCKS5 without changing the SSH layer
Dynamic forwarding is the -D flag. ssh -D 1080 bastion-host starts a SOCKS5 proxy on the client machine listening on port 1080. The SSH server knows nothing about SOCKS. To the server, dynamic forwarding looks exactly like local forwarding with the target host and port determined dynamically per connection.
Applications that support SOCKS5 proxies connect to localhost:1080 and send SOCKS protocol messages. The SSH client implements a SOCKS5 server internally. When an application sends a SOCKS CONNECT request specifying a target like internal-api:443, the SSH client parses that request, extracts the host and port, and opens a direct-tcpip channel to the SSH server with those parameters. After the initial SOCKS handshake, the data path is identical to local forwarding. The SSH server connects to the specified target. The client relays bytes between the SOCKS application socket and the channel.
The key insight is that SOCKS is handled entirely by the SSH client. The SSH protocol does not have a SOCKS channel type. There is no SOCKS negotiation over the SSH connection. The channel type is still direct-tcpip. The only difference from local forwarding is that the target host and port come from runtime SOCKS requests rather than from a fixed -L argument. From the SSH server's perspective, dynamic forwarding and local forwarding produce identical channel open messages.
This means dynamic forwarding inherits all the same multiplexing behavior. Each SOCKS CONNECT creates a new direct-tcpip channel. Multiple concurrent proxied connections share the same SSH TCP connection. Flow control works the same way. Channel lifecycles work the same way. The SOCKS layer is an application concern that sits entirely outside the SSH protocol boundary.
OpenSSH ControlMaster adds a second multiplexing layer
Everything described so far is protocol-level multiplexing inside one SSH connection. OpenSSH adds another layer on top: process-level multiplexing through ControlMaster. This feature, documented in ssh_config(5), allows multiple independent ssh client processes to share a single SSH TCP connection.
The mechanism uses a Unix domain socket. You configure it with ControlPath in your ssh config. A typical setup:
Host *.example.com
ControlMaster auto
ControlPath ~/.ssh/controlmasters/%r@%h:%p
ControlPersist 10m
The first ssh invocation to a matching host becomes the master. It establishes the TCP connection, performs key exchange, and authenticates. It listens on a Unix domain socket at the configured ControlPath. Subsequent ssh invocations to the same host detect that the socket exists and connect to it as clients instead of opening a new TCP connection.
When a client connects to the control socket, it sends a multiplexing protocol message asking the master to perform an operation on its behalf. The operation might be "open a new session channel for an interactive shell" or "open a new direct-tcpip channel for a port forward." The master receives this request, opens a new channel over its existing SSH connection, and forwards data between the channel and the client's file descriptors. The client process never sees the SSH transport layer directly. It sees a local socket that the master proxies.
This is where the two layers of multiplexing become visible. The control socket multiplexes multiple ssh client processes onto one master process. The master process multiplexes multiple SSH channels onto one TCP connection. You could have three terminal windows, two scp transfers, and a SOCKS proxy all running through different ssh client processes, sharing one TCP connection through the control socket, with each logical stream mapped to its own SSH channel inside that connection.
ControlMaster also eliminates repeated authentication. The TCP connection is established once. Key exchange happens once. The master holds the authenticated session. New clients skip directly to requesting channel operations. This makes subsequent ssh commands nearly instantaneous. A remote command that would normally take 2 seconds for connection setup completes in milliseconds because the transport pipe is already open and authenticated.
The interaction between these layers matters for failure modes. If the master process dies, all channels close. Every client using that control socket loses its connection simultaneously. If the underlying TCP connection breaks, the same catastrophic failure propagates upward. The multiplexing layers give you concurrency, not fault isolation.
Quick Reference
| Property | Value |
|---|---|
| SSH transport protocol | RFC 4253 |
| SSH connection protocol | RFC 4254 |
| Channel type for local forwarding | direct-tcpip |
| Channel type for remote forwarding | forwarded-tcpip |
| Global request for remote listen | tcpip-forward |
| SOCKS channel type | none (uses direct-tcpip) |
| Flow control mechanism | Per-channel window (bytes) |
| Channel shutdown order | EOF then CLOSE (half-close) |
| ControlMaster socket type | Unix domain |
| Default ControlPersist | off (master exits with last session) |
Frequently Asked Questions
Q: Why does my port forward feel slow when I also have a large scp transfer running over the same SSH connection?
The channel window mechanism prevents starvation, but all channels share one TCP congestion control state. If scp fills the TCP send buffer, your interactive forward's packets experience increased latency because they queue behind bulk data in the kernel's TCP stack. SSH channel multiplexing does not give you independent TCP connections. It gives you independent logical streams inside one TCP stream. At the network layer, it is still one connection competing for bandwidth.
Q: Can I forward the same local port to different remote targets for different connections?
No. A local -L forwarding listener is bound to a fixed target when the listener is created. Every connection to that listener opens a direct-tcpip channel to the same host and port. If you need different targets per connection, use dynamic forwarding with -D and a SOCKS client. Each SOCKS CONNECT can specify a different destination, and each gets its own channel to that destination.
Q: What happens when the SSH TCP connection drops while I have active port forwards?
All channels die immediately. There is no per-channel reconnection mechanism in the SSH protocol. The transport layer failure tears down every channel. Your local application sees a TCP connection reset. ControlMaster does not help here because it shares the same underlying TCP connection. If you need resilience, you need an external connection manager that can reestablish the SSH tunnel and reconnect the forwarded sockets.
Q: Does SSH multiplexing add significant overhead compared to a raw TCP connection to the target?
The per-packet overhead is the SSH binary packet framing (5 bytes of length plus 1 byte of padding length minimum) plus encryption MAC (16 to 32 bytes depending on algorithm), plus the channel data message header (5 bytes for channel number and data length). This is typically under 60 bytes per packet. For bulk transfers, the overhead is negligible. For many tiny messages, the per-message cost can add up because SSH does not aggregate small channel data writes into larger packets automatically in all implementations.
Q: How many channels can one SSH connection support?
The protocol uses 32-bit channel numbers, so the theoretical limit is over 4 billion. Practical limits come from memory and file descriptors. Each channel requires kernel buffer space, an SSH channel structure, and potentially a TCP socket on one or both sides. OpenSSH's default is controlled by MaxSessions which defaults to 10 for the server-side session multiplexing. For port forwards, the limit is usually the server's MaxStartups and per-connection memory constraints. Hundreds of concurrent forwarded connections are routine. Thousands require tuning.
What comes next
SSH tunneling is a protocol stack, not a single feature. The transport encrypts. The connection protocol multiplexes. The forwarding types map TCP sockets onto channels. If you want this kind of breakdown every week, how real systems actually work under the hood instead of surface-level tutorials, subscribe to Internals Decoded at internalsdecoded.com. We write for the engineers who read RFCs and source code to understand what their tools are actually doing.