Difference in flushing behavior in OpenSSL and Apple Secure Transport

The whole API provided within Secure Transport is very limited and their workflow is listed here

Now let's look at the core method in c++ example which reproduces this issue by Neil in Arti's issue you provided.

// Secure Transport Write callback.
static OSStatus WriteFunc(SSLConnectionRef connection, const void* data, size_t* dataLength) {
    Conn* c = SSLConn<Conn>(connection);
    const uint8_t* p = static_cast<const uint8_t*>(data);
    size_t want = *dataLength;
    *dataLength = 0;

    if (c->buffered) {
        // Buffer implementation simulating what happens in Rust:
        // we buffer the data and return noErr pretending it was send (What Arti's DataStream does with the internal buffering).
        for (size_t i = 0; i < want; ++i) {
            c->outBuf.push_back(p[i]);
            if (c->outBuf.size() >= c->cap) {
                // Autoflush when capacity reached (equally as done in Rust).
                OSStatus st = c->flush();
                if (st != noErr) return st;
            }
        }
        *dataLength = want;
        
        return noErr; // Tells Secure Transport all bytes are delivered, but they're only delivered to the internal buffering.
    }
     
    // Direct send without buffering (E.g TcpConnect behavior).
    size_t off = 0;
    while (off < want) {
        ssize_t n = ::send(c->fd, p + off, want - off, 0);
        if (n < 0) {
            if (errno == EWOULDBLOCK || errno == EAGAIN) return errSSLWouldBlock;
            return errSSLClosedAbort;
        }
        off += static_cast<size_t>(n);
    }
    *dataLength = want;

    return noErr;
}

This function is registered by SSLSetIOFuncs(_:_:_:), and this callback is invoked during both the handshake phase (for TLS protocol messages) and application data phase (for encrypted payload)

If the c++ version does mimic the traffic flow; Then the implementation is missing protocol-aware flushing. In TLS, certain moments REQUIRE immediate transmission:

• Complete handshake messages (ClientHello, ServerHello, etc.)
• Before waiting for peer response
• Application data that expects a response

Ideally, we would need logic like this for proper TLS buffering with Secure Transport:

// Pseudo-code for proper TLS buffering
if (c->buffered) {
    buffer_data(data, length);
    
    // Smart flush triggers:
    if (is_handshake_message_complete() ||
        is_waiting_for_peer_response() ||
        if_buffer_full()) {
        
        flush_immediately();
    }
    
    return noErr;
}

without these 2 pseudo API (is_handshake_message_complete and is_waiting_for_peer_response) our test code in the c++ example will fail as:

• ClientHello gets buffered → never sent because buffer isn't full
• Secure Transport waits for ServerHello → will never come
• Handshake times out → connection fails

This explains why Arti's HTTPS requests failed after approximately 30 seconds with error -9806 (errSSLClosedAbort - "connection closed via error"), which typically indicates the client failed to interact with the server within a reasonable timeframe.

Lastly, the Secure Transport framework, in apple's doc:

Important This API is considered legacy. Use the Network framework instead.

And there are no such API like is_handshake_message_complete and is_waiting_for_peer_response in Secure Transport, which means Secure Transport's callback model doesn't provide the hooks needed for smart protocol-aware flushing, so the safest approach is to flush after every write on macOS.

Now let's look at the typical workflow with OpenSSL (with handshake/write application data/read application data stages), and compare the implementation differences to Secure Transport:

// after SSL_set_bio(ssl, rbio, wbio) or SSL_set_fd(ssl, fd) and nonblocking fd

// --- Handshake ---
for (;;) {
    int rc = SSL_connect(ssl);
    if (rc == 1) break;                         // handshake done

    int err = SSL_get_error(ssl, rc);
    if (err == SSL_ERROR_WANT_READ) {
        wait_until_readable(fd);                // select/poll/kevent
        continue;
    }
    if (err == SSL_ERROR_WANT_WRITE) {
        wait_until_writable(fd);                // (don't advance any app buffers)
        // Optionally: BIO_flush(wbio);         // if you *really* need to poke the BIO
        continue;
    }
    fail("handshake", err);                     // SSL_ERROR_SYSCALL, SSL_ERROR_SSL, etc.
}

// --- Write application data (handles partials) ---
size_t off = 0;
while (off < len) {
    int n = SSL_write(ssl, app_data + off, (int)(len - off));
    if (n > 0) { off += (size_t)n; continue; }

    int err = SSL_get_error(ssl, n);
    if (err == SSL_ERROR_WANT_WRITE) { wait_until_writable(fd); continue; }
    if (err == SSL_ERROR_WANT_READ)  { wait_until_readable(fd); continue; }
    fail("write", err);
}

// --- Read application data ---
for (;;) {
    int n = SSL_read(ssl, buf, sizeof buf);
    if (n > 0) { consume(buf, n); continue; }

    int err = SSL_get_error(ssl, n);
    if (err == SSL_ERROR_WANT_READ)  { wait_until_readable(fd); continue; }
    if (err == SSL_ERROR_WANT_WRITE) { wait_until_writable(fd); continue; }
    if (err == SSL_ERROR_ZERO_RETURN) break;    // peer performed a clean shutdown
    fail("read", err);
}

So, the key differences:

1. Callback vs Function based APIs:

• Secure Transport: The same WriteFunc callback handles both handshake messages and application data, with no way to distinguish between them or their urgency.
• OpenSSL: Uses separate functions (SSL_connect for handshake, SSL_write for application data), allowing different handling strategies for each phase.

2. State Communication:

• OpenSSL: Returns SSL_ERROR_WANT_WRITE to explicitly signal "flush needed" state. Applications can then either wait_until_writable(fd) or call BIO_flush() to handle buffered data.
• Secure Transport: Provides no equivalent state signaling mechanism. The callback must return immediate success/failure with no way to communicate buffering requirements back to the caller.