{"id":66891,"date":"2016-12-12T17:54:37","date_gmt":"2016-12-13T01:54:37","guid":{"rendered":"https:\/\/securingtomorrow.mcafee.com\/?p=66891"},"modified":"2025-06-02T01:01:04","modified_gmt":"2025-06-02T08:01:04","slug":"how-to-protect-against-openssl-1-1-0a-vulnerability-cve-2016-6309","status":"publish","type":"post","link":"https:\/\/www.mcafee.com\/blogs\/other-blogs\/mcafee-labs\/how-to-protect-against-openssl-1-1-0a-vulnerability-cve-2016-6309\/","title":{"rendered":"How to Protect Against OpenSSL 1.1.0a Vulnerability CVE-2016-6309"},"content":{"rendered":"

This blog post was written by Rock Liu.<\/em><\/p>\n

Recently the OpenSSL security library gained a fix for a critical security issue (CVE-2016-6309) that affects OpenSSL Version 1.1.0a. The remote attackers can cause the OpenSSL server to crash, or execute arbitrary code on it, by simply sending a handshake packet with a message larger than 16KB. To defend against these attacks we analyzed the code, wrote a proof of concept, and prepared a signature for our customers.<\/p>\n

OpenSSL offers the following description:<\/p>\n

\u201cThe buffer to receive messages is initialized to 16KB. If a message is received that is larger than that, then the buffer is \u2018realloc’d.\u2019 This can cause the location of the underlying buffer to change. Anything that is referring to the old location will be referring to free’d data.\u201d<\/p>\n

From the OpenSSL source code, we know it uses the structure “ssl_st<\/a>” to store the SSL session information. Both init_msg and init_buf are members of the structure of ssl_st. Init_msg points to the handshake message body, which is contained in the buffer pointed to by init_buf. The variable \u201cs\u201d (s->init_msg) is a pointer to the structure of ssl_st.<\/p>\n

From this information, we suspect a single handshake message might trigger this vulnerability.<\/p>\n

This is the format of a handshake packet in the TLS protocol:<\/p>\n

———+———–+———–+———–+———–+——————————<\/p>\n

| 0x16 | 0x0301 | 0x4000 | 0x0100 | 0x5560 | DATA(length=0x5560) |<\/p>\n

———+———–+————+———–+———–+——————————<\/p>\n

|\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |————–Length of challenge<\/p>\n

|\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |<\/p>\n

|\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |—————————————-Length handshake message<\/p>\n

|\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 |—————————————————–Version<\/p>\n

|——————————————————————Content type\uff1aHandshake<\/p>\n

 <\/p>\n

We manually constructed a handshake packet of more than 16KB (0x5560).<\/p>\n

Figure 1: The TLS handshake packet.<\/em><\/p>\n

We sent the oversized handshake to a vulnerable OpenSSL server, and it crashed.<\/p>\n

Figure 2: <\/em>Crash information from the vulnerable OpenSSL server.<\/em><\/p>\n

Now for more details. First, let’s take a look at how the issue was fixed. The patch is available here.<\/a><\/p>\n

if (!SSL_IS_DTLS(s)<\/p>\n

&& s->s3->tmp.message_size > 0<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 && !BUF_MEM_grow_clean(s->init_buf,<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0(int)s->s3->tmp.message_size<\/p>\n

–\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 + SSL3_HM_HEADER_LENGTH)) {<\/p>\n

+\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 && !grow_init_buf(s, s->s3->tmp.message_size<\/p>\n

+\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 + SSL3_HM_HEADER_LENGTH)) {<\/p>\n

ssl3_send_alert(s, SSL3_AL_FATAL, SSL_AD_INTERNAL_ERROR);<\/p>\n

SSLerr(SSL_F_READ_STATE_MACHINE, ERR_R_BUF_LIB);<\/p>\n

return SUB_STATE_ERROR;<\/p>\n

The lines preceded by “-” are the old code, while the “+” lines are the patch. The patch introduced a new function, grow_init_buf, to replace the function BUF_MEM_grow_clean. This new function handles the chores of buffer reallocation to make sure the buffer is big enough to hold the handshake message.<\/p>\n

Now let\u2019s check out the new function: grow_init_buf:<\/p>\n

static int grow_init_buf(SSL *s, size_t size) {<\/p>\n

size_t msg_offset = (char *)s->init_msg – s->init_buf->data;<\/p>\n

if (!BUF_MEM_grow_clean(s->init_buf, (int)size))<\/p>\n

return 0;<\/p>\n

if (size < msg_offset)<\/p>\n

return 0;<\/p>\n

s->init_msg = s->init_buf->data + msg_offset; \u00a0\u00a0<——————reset the init_msg<\/p>\n

return 1;<\/p>\n

}<\/p>\n

This newly introduced function internally will still call the old function BUF_MEM_grow_clean; after that it resets the init_msg (marked above).<\/p>\n

Next, let\u2019s look into this piece of vulnerable code with a debugger.<\/p>\n

Figure 3: The unpatched code before the control flow reaches the function BUF_MEM_grow_clean.<\/em><\/p>\n

Figure 3 gives us the following status information:<\/p>\n