Skip to content

Do not pin the DTLS key policy to SHA-256 on Mbed TLS 4 - #286

Open
dsugisawa-mixi wants to merge 1 commit into
sepfy:mainfrom
dsugisawa-mixi:mbedtls4-any-hash
Open

dsugisawa-mixi wants to merge 1 commit into
sepfy:mainfrom
dsugisawa-mixi:mbedtls4-any-hash

Conversation

@dsugisawa-mixi

Copy link
Copy Markdown

Problem

With third_party/mbedtls at v4.1.0, the DTLS handshake fails whenever the peer
selects a CertificateVerify hash other than SHA-256. mbedtls_ssl_handshake
returns -0x0087PSA_ERROR_INVALID_ARGUMENT (-135).

dtls_srtp_generate_keypair() creates the ephemeral DTLS key with a PSA policy
that permits exactly one hash:

psa_set_key_algorithm(&attr, MBEDTLS_PK_ALG_ECDSA(PSA_ALG_SHA_256));

But the hash used for CertificateVerify is not ours to choose — it comes out of
the peer's CertificateRequest. Against Cloudflare Calls the negotiated ciphersuite
is TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384 and the CertificateRequest offers

Supported Signature Algorithm found: 04 03 (sha256, ecdsa)
Supported Signature Algorithm found: 05 03 (sha384, ecdsa)
Supported Signature Algorithm found: 06 03 (sha512, ecdsa)
Supported Signature Algorithm found: 08 07 (ed25519)
Mbed TLS picks SHA-384, hands PSA a 48-byte hash, and the key policy rejects it:

ssl_tls12_client.c:2655: => write certificate verify
ssl_tls.c:6551: => PSA calc verify
ssl_tls.c:6562: dumping 'PSA calculated verify result' (48 bytes)
ssl_tls12_client.c:2745: mbedtls_pk_sign_restartable() returned -135 (-0x0087)
It fails late enough to be confusing: the master secret is exported and both SRTP
sessions are created before the handshake gives up, so the log reads

INFO dtls_srtp.c 539 Created inbound SRTP session
INFO dtls_srtp.c 559 Created outbound SRTP session
ERROR dtls_srtp.c 665 failed! mbedtls_ssl_handshake returned -0x0087
The Mbed TLS 3 path has no such restriction — mbedtls_ecp_gen_key() produces a
plain key that can sign under any hash — so only the 4.x branch added in #270 is
affected. That PR's validation was a syntax compile against the 3.x headers plus a
link of an ESP-IDF 6.0 application, so a handshake negotiating a hash other than
SHA-256 would not have been exercised.

Fix

PSA_ALG_ANY_HASH in a key policy is what PSA provides for exactly this case — the
hash is chosen per-operation by the protocol. The RSA branch has the same problem
and the same fix.

Validation

Built against mbedtls v4.1.0 and run against Cloudflare Calls as a DTLS client
(a=setup:active), receiving. The CertificateVerify hash is still SHA-384 — the
debug log still dumps a 48-byte verify result — and the handshake now completes:

dtls_srtp.c 668 DTLS client handshake done
peer_connection.c 318 DTLS-SRTP handshake done
[PC] state=connected
-0x0087 no longer appears anywhere in the log. Linux x86-64, gcc,
CONFIG_DTLS_USE_ECDSA enabled.

The RSA branch is unchanged in behaviour by inspection only — I did not build with
CONFIG_DTLS_USE_ECDSA off.

The PSA key generated for the ephemeral DTLS certificate permits exactly
one hash, but the hash used for CertificateVerify is not ours to choose
-- it comes out of the peer's CertificateRequest.

Against Cloudflare Calls the negotiated ciphersuite is
TLS-ECDHE-ECDSA-WITH-AES-256-GCM-SHA384 and the CertificateRequest offers
sha256, sha384, sha512 and ed25519. Mbed TLS picks SHA-384, hands PSA a
48-byte hash, and the key policy rejects it:

  ssl_tls12_client.c:2745: mbedtls_pk_sign_restartable() returned -135 (-0x0087)
  dtls_srtp.c 665 failed! mbedtls_ssl_handshake returned -0x0087

-135 is PSA_ERROR_INVALID_ARGUMENT. It fails late enough to be confusing:
the master secret is exported and both SRTP sessions are created before
the handshake gives up.

PSA_ALG_ANY_HASH in a key policy is what PSA provides for this case, the
hash being chosen per-operation by the protocol. The Mbed TLS 3 path has
no such restriction -- mbedtls_ecp_gen_key produces a plain key that can
sign under any hash -- so only the 4.x branch is affected.

Verified against Cloudflare Calls with mbedtls v4.1.0: the
CertificateVerify hash is still SHA-384, and the handshake now completes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant