Team82 has researched commercial and open source implementations of the popular MMS protocol, which is widely used in power substations for machine-to-machine communication
We disclosed to affected vendors five vulnerabilities in their respective MMS implementations.
The vulnerabilities could allow an attacker to crash an industrial device or in some cases, enable remote code execution.
We are also making freely available a tool we developed during the course of our research called MMS Stack Detector.
The tool was invaluable in identifying specific implementations based on observed MMS payloads.
By the end of the 20th century as power consumption requirements increased, a need emerged for controlled, secure, and adaptive power systems. Analog infrastructure was replaced by digital architectures, and there was a need for standardized communication between substations to enable robust interoperability between the systems.
A standard was developed by the IEC (International Electrotechnical Commission) to support this requirement: IEC 61850, and it became the de facto way for substations to communicate.
Given the increased number of attacks against substations and other power generation facilities, Team82 decided to review the security of Manufacturing Messaging Specification (MMS); one (and most popular) of the three protocols that are defined by the IEC 61850 standard.
IEC 61850 ultimately defines communication between intelligent electronic devices (IEDs). IEDs are microprocessor-based controllers that communicate with lower level devices, such as sensors, as well as high level SCADA systems. The communication services within IEC 61850 are mapped to following protocols:
Manufacturing Message Specification (MMS): Used between the IEDs and SCADA devices to exchange control information, and to read and write values to and from IEDs.
GOOSE (Generic Object Oriented System Event): Used between IEDs to exchange information about specific substations controlled by the IED.
SV (Sampled Values): Used to exchange measurements in sensitive-to-delay environments between IEDs and measurement units.
We researched open source and commercial MMS implementations in the power sector and how they’re used for communication between control centers and modern, digital substation devices.
We are publishing details on five vulnerabilities discovered in two MMS protocol implementations; MZ Automation’s libIEC61850 library and Triangle Microworks’ TMW IEC 61850 library. Successful exploits of these flaws could allow an attacker to crash an industrial device or in some cases, enable remote code execution.
We have also made publicly available a tool called MMS Stack Detector that was invaluable in our vulnerability research that helped us identify specific implementations based on observed MMS payloads. The tool is available on the Team82 GitHub page.
Originally, the MMS protocol was developed on top of the OSI network model, developed by ISO. The OSI model, analogous to the TCP/IP model (the one the Internet built upon), describes the infrastructure for interconnecting multiple computer networks. In contrast to the four-layer TCP/IP model, OSI model has seven independent layers.
The following is a chart that depicts the two models side by side:
As the TCP/IP model surpassed the OSI model in popularity, it became necessary to adapt protocols originally developed for the OSI model, such as MMS, to operate on the TCP/IP network stack.
The procedure to achieve this transition (from OSI to TCP/IP model) is described in RFC 983 in April 1986 (which was later updated by RFC 1006 and RFC 2126). The general approach is based on the observation that OSI’s transport layer provides similar services as the transport layer of the TCP/IP model (i.e COTP vs TCP). Therefore, to minimize changes to the existing systems and due to the layer-independence in both models, all upper OSI layers (layer 5 to 7) can be “copied as is” on top of the TCP/IP “transport layer,” allowing them to operate without any awareness of the underlying transport protocol.
In other words, the transport layer serves as a bridge from the OSI model to the TCP/IP model, and all upper layers remain the same as they were defined for the OSI model.
The following is the brief description of those layers as they appear in TCP/IP model:
The TPDU (OSI Transport Layer Protocol Data Unit), was wrapped with discrete units called TPKT. COTP, protocol analogous to TCP, is the protocol that delivers TPDUs.
Lets see what it looks like in Wireshark for the S7Comm protocol, a proprietary Siemens protocol that operates on top of COTP):
While the OSI model had no concept of “ports” when the protocol stack of COTP was implemented over the TCP/IP equivalent model, port TCP 102 was selected to be used as written in RFC 983. That’s why there are multiple protocols that use this same TCP 102 port: MMS, Siemens S7-Comm, Siemens S7-CommPlus and others.
The higher layers (Session, Presentation, and Application) of the OSI Model, remained as is and delivered on top of TPKT data unit. Although those layers are not part of the MMS protocol, most MMS servers that we have observed implement the OSI layers along with MMS (probably to minimize the changes during the adaptation). This is why when MMS traffic is captured we will see the layer 5-7 below the MMS application data:
Implemented by ISO 8326/8327, the OSI Session layer is responsible for managing the sessions within the communication. In the OSI model, the transport layer delivers discrete data units (called TPDU) in a “connectionless” manner. The connection-oriented communication is implemented in the Session layer. This layer will be responsible for opening, managing and closing the sessions ( in TCP/IP model this task is performed by the TCP protocol).
The data delivered in this layer is called SPDU (Session Protocol Data Unit). SPDU can have multiple types. The types we have seen in the MMS communication are:
CONNECT (CN)
ACCEPT (AC)
Give Tokens / DATA TRANSFER (DT)
OSI Presentation Protocol
Implemented by ISO 8823. This layer negotiates through ASN.1 with BER encoding. This layer is responsible for defining the encoding and the format for the application layer.
Implemented by ISO 8650. ISO uses the “Association” concept to “associate” the presentation layer with a specific application by sending Application Association Request (AARQ) and receiving Application Association Response (AARE). Authentication can be implemented in this layer as well.
This association happens only on MMS initiation procedure.
The MMS protocol, referenced also as ISO 9506, was developed by ISO in collaboration with IEC back in the 90's. The protocol is widely used by electrical, automotive and other industries.
The main advantages of MMS over other messaging protocols is that on one hand the protocol has big layer of abstraction and can be used by various manufacturers to interconnect various types of devices, but on the other hand the protocol is well defined in a sense of how the communication should be constructed and represented which requires minimum agreement between different parties before they can be interconnected.
The standard is not open to the public, which explains the low amount of documentation found online. The full protocol specification is out of scope of this blog, but some basic concepts should be covered:
The core concept of MMS and with which the discussed abstraction is achieved is the VMD (Virtual Manufacturing Device) model. The model defines virtual objects, services that are provided by the object and the actions that the MMS server takes when a specific service on a specific object is requested. It is up to the implementer of the service to associate the object to the actual internal element.
The MMS PDU can be one of the following types:
—Enter/ Leave MMS session—
Initiate Request/Response/Error PDU
Cancel Request/Response/Error PDU
Conclude Request/Response/Error PDU
—Within the MMS session—
Confirmed Request/Response/Error PDU
Cancel Request/Response/Error PDU
Unconfirmed PDU
Reject PDU
A list of available confirmed and unconfirmed requests can be found within the MMSd ASN.1 definition under ConfirmedServiceRequest and UnconfirmedService respectively.
Our approach was to look for the vulnerabilities in popular implementations of the protocol. Discovering popular implementations is fairly straightforward. We can simply search for MMS server products online, because vendors that implement the MMS protocol typically specify it. Additionally, open-source versions are available. By analyzing internet-exposed devices found through search engines like Shodan and Censys, we compiled the following list of popular MMS servers:
Now that we have a basic understanding of the protocol and a list of open-source and closed-source implementations, the next step is to identify specific implementations based on observed MMS payloads; identifying these implementations is invaluable to our vulnerability research. To achieve this task, we developed a new tool called "MMS Stack Detector." The tool will examine the unique-for-the-implementation information that is sent from the device/server and prompt the name of the implementation along with other useful information.
MMS Protocol has a message type (Identify) that provides information about the MMS server:
However, not all vendors provide this information, forcing us to explore alternative methods.
Protocol stack detection is typically accomplished by identifying a message or request type that uniquely indicates a specific implementation. This message or request must be broad enough to be supported by all servers, yet distinct enough to elicit different responses from each server, allowing for differentiation.
MMS initiate-ResponsePDU is exactly such message type:
The message is part of MMS initiation process; all vendors support it
The message includes list of supported services; this list is diverse enough for our goal
According to the standard, the server should return the intersection of the services specified by the client and the services supported by the server. However, in practice, we noticed that most servers simply return the list of services they support, regardless of the services requested by the client.
By analyzing the bitmap of services that was returned by the server, we can build a signature (subset of the bitmap) matching to which will indicate the implementing server.
Consider the following simplified psuedocode produced by MMS Stack Detector:
def find_mms_implementation(mms_services_bitmap):
for known_signature in known_signatures:
if mms_services_bitmap & known_signature['bitmap']:
return known_signature['implemetation_stack_name']
return "Not Found"
Now we can write a simple python code that will send the MMS Initiate Request PDU request to the MMS device, parse the response, and will detect the MMS implementation stack:
Now that we understand the basics of the protocol and have identified some implementations, we can explore the attack surface. The goal is to find vulnerabilities that cause either information leakage, denial of service, or remote code execution on the server (device) side.
When looking for vulnerabilities within several protocol implementations, fuzzing is favorable approach because:
Corpuses generated from fuzzing one application, can be used to test other applications.
When a research is conducted on multiple implementation stacks, manual vulnerability research is less feasible.
We decided to fuzz two popular MMS implementations:
libiec61850 by MZ Automation (Open Source)
IEC 61850 Source Code Library by Triangle Microworks
We started our fuzzing attempts by targeting an open-source implementation of MMS protocol software implementation. The incentive to start with an open-source implementation was the fact that they are easier to prepare for fuzzing, this includes finding relevant fuzzing candidate functions and building a suitable harness.
After gaining some experience with MMS open-source software and obtaining a large set of corpus from existing fuzzing processes, we approached closed-source binary fuzzing which was more difficult to target.
This stack is open source and written in C so the AFL fuzzer was our best option for this use case. We first determined where within the implementation that we wanted to fuzz and then write a fuzzing harness that will help us to narrow the executed code only to the desired one. The function that we have found to be a good candidate for fuzzing is MmsServerConnection_parseMessage.
We used AFL-fast mode because it will dramatically speed up the process.
The harness looks something like this:
int main() { __AFL_INIT(); |
The code flow is very simple. The mutated payload is fed to the buffer from the file generated by AFL and then forwarded to the MmsServerConnection_parseMessage. This function is a front door for the payload parsing.
Next we run the AFL with multiple instances to speed up the fuzzing process. We left the fuzzer running for a few days and moved to the next target.
This one is more challenging because we do not have the source code and thus can not compile the project with AFL’s compiler. We decided to fuzz the library we used AFL in QEMU_MODE. This mode adds instrumentation to the code in runtime rather than in compile time. This technique is not as efficient as the previous one but since the code base is relatively narrow we decided to give it a try.
As before, we have to find a function that is a front door to the parsing procedure and write a harness to the fuzzer.
Within the code we found the following function—its name, MMSd_Decode, provides a hint we’re on the right track:
Fortunately, MMSd_Decode is an exported function so we can easily use it in our harness:
int main(int argc, char const *argv[]) { |
The fuzzer is ready to go:
We ran the fuzzers on the libraries for several days until no new paths were found. After stopping the fuzzing process, the next step was to triage the results:
Remember, our goal is to crash the device and not a specific function (i.e the one that we have fuzzed). That means that even if there is a payload that crashes the parsing-procedure function, it still does not mean that the payload will crash the entire device.
Fortunately, both implementations had a demo server provided which we could easily utilize for our needs.
To validate the crashes found by AFL we used a simple python script that does the following:
Start the target server with GDB and an exploitable plugin
Send the payload within the MMS session.
If the server crashes (Stable crash):
Retrieve the crash details from GDB along with exploitable’s output
Compare with others, if unique, save for later
Else, the crash is not stable, continue to next payload
Return all unique payloads
Within MZ Automation libiec61850, four vulnerabilities were discovered with AFL, let's talk about the most interesting one (CVE-2022-2972).
The vulnerability is a pretty simple stack buffer overflow. The problem occurs while parsing the GetNameList-Request, component of the ConfirmedServiceRequest (look here for whole ASN.1 MMS format), in mmsServer_handleGetNameListRequest function:
GetNameList-Request ::= SEQUENCE |
The continueAfter defined in ASN.1 as a VisibleString:
The problem however, is that the server expects the continueAfter element to not exceed the size of 130 bytes but does not verifies it:
char continueAfterIdMemory[130]; |
The continueAfterLength calculated by calling: BerDecoder_decodeLength(buffer, & continueAfterLength, bufPos, maxBufPos);
Stack buffer overflow caused simply by crafting a payload with continueAfter element greater than 130 bytes:
Note that the size of the overflow, along with the content is fully controlled by an attacker.
Therefore, the stack based buffer-overflow can be used to crash the server or even potentially execute code remotely.
An example of problematic payload (MMS layer only):
0000 a0 82 01 35 02 01 03 a1 82 01 2e a0 03 80 01 02 |
Since we have full control on the size of the overflowed buffer and its content, in the environments that were not compiled with modern mitigations like ASLR and stack canary (which happens a lot in embedded environments), the construction of ROP chain to gain RCE is trivial.
We disclosed the vulnerabilities to the maintaining community via GitHub.
The following CVEs were issued by ICS-CERT:
The AFL in QEMU_MODE had successfully found a crash within the library that led to the Denial Of Service:
The vulnerability exists in MMSd_prparse function due to incorrect call to Pres_lookup_oid.
The Pres_lookup_oid function is responsible for lookup of specific pres_oid within a parsed data.
The arguments to the function are the object to look for (ctx), a pointer to the array of pointers to oids (pres_oids) and the size of the array (num_oids).
At some point the function will iterate over the pres_oids array up until it’s size num_oids and compare the value at pres_oids with pointer of ctx:
__int64 Pres_lookup_oid(_DWORD *ctx, __int64 pres_oids, int num_oids) . . . |
The function Pres_lookup_oid is called from MMSd_prparse. Let’s focus on case 0x7e:
int64 fastcall MMSd_prparse(MmsContext a1) |
The problem is that v96 is an array of only one valid pointer within it, but calls Pres_lookup_oid with 3 as an array’s size.
So when the following code from Pres_lookup_oid will be executed with index greater than zero (e.g. i = 1), the function will try to dereference a pointer that does not exist, causing a crash.
v6 = (_DWORD *)(8LL * i + pres_oids); |
The issue was disclosed to Triangle MicroWorks, a CVE-2022-38138 was issued to track this vulnerability. CISA issued an advisory, recommending their customers to update the library.
Now that we have found vulnerabilities in the fuzzed implementations, can we do more?
Remember that we talked about the advantages of running a fuzzer when looking for vulnerabilities in multiple protocol implementations? While fuzzing the open-source libiec61850 library, the AFL generated multiple corpuses (input payloads) that now can be tested against other MMS implementations:
Fortunately we do not need to go very far. In our lab we have a pretty extensive number of controllers, some of them implement MMS protocol. We sent the generated payloads to all devices in our lab that support MMS and guess what? We have crashed some.
A quite popular and high profile controller by Siemens. SIPROTEC5 uses SISCO’s MMS stack for the MMS support.
Sending the following payload will crash the device:
0000 a0 02 02 00 |
The MMS packet:
We have disclosed this vulnerability to the vendor, the vendor was able to identify the vulnerability and mentioned that the vulnerability does exist in a third party SISCO MMS stack and tracked under CVE-2015-6574. Therefore what happened was that Siemens actually used an outdated version of the vulnerable MMS stack in its firmware. Due to our disclosure, Siemens updated its firmware with an updated version of the protocol stack and CISA published an advisory to alert their customers regarding the update.
During the procedure of sending the corpuses that were generated by the fuzzers towards devices in our lab, we have encountered a peculiar case. A general purpose controller by ABB - ABB AC 800M - was “crashed” with following payload:
0000 a0 1a 02 01 01 a5 15 a0 0e 30 0c a0 0a 80 08 24 |
The MMS packet:
This is an interesting scenario. We see here that the packet is correctly parsed by wireshark and nothing seems to be suspicious, however the TAG name looks interesting:
$$SysCmd |
It turns out that purely by chance we had encountered an undocumented functionality that restarted the device. ABB uses the MMS protocol to control the devices, rather than to exchange data as originally intended to be used over MMS. This functionality does not require authentication.
Meaning, by using the special $$SysCmd TAG we can interact with the device with various commands. For example, command 0x01 is Halt System and this is what caused the device to actually Halt (as intended by ABB). The problem of course is that no authentication/authorization was needed to cause this disruption.
We reached ABB and were told this is an intended behavior and they are planning to add authentication in the future.
This triggered our curiosity and we dug a bit in the firmware and found more options:
Opcode | Description |
0x01 | Halt System (*we used this in our example above) |
0x02 | Heap Statistics |
0x03 | Set Heap Contents Filter Array |
0x06 | Remove LicenseKey |
0x07 | Show License Key Map |
0x08 | TCP Buffer Info |
0x0b | MMS Info |
0x0c | Write Assoc Descs |
0x0d | Write MMS |
0x0e | Prot Heap Statistics |
0x9e | Write INCAMailCons |
0x9f | Write Transport |
Rather than focusing on a specific device or application to identify vulnerabilities, we took a different approach. We explored how protocols, in this case MMS, developed in the 1990s have been adapted for modern technology. We built a tool—and made it publicly available—called MMS Stack Detector that we used to identify specific MMS implementations that also proved invaluable in our research. We uncovered five vulnerabilities, and a couple of other issues of note that are explained in this research. The vulnerabilities were disclosed to the vendors and had been remediated.
We fuzzed two MMS implementations, uncovering issues ranging from denial of service to remote code execution. Using corpuses from the fuzzers, we identified vulnerabilities in other implementations, successfully crashing both MMS-supporting devices in our lab.
This again highlighted the gap between modern technology's security demands and the outdated, hard-to-replace protocols. We urge security researchers to continue efforts in securing ICS devices by examining popular protocol implementations, finding vulnerabilities through fuzzing or manual methods, and responsibly disclosing them to vendors.
We also encourage vendors to follow the security guidelines and advisories issued by CISA and take action accordingly.
CWE-121 Stack-Based Buffer Overflow
The affected product does not sanitize input before memcpy is used, which could allow an attacker to crash the device or remotely execute arbitrary code.
CVSS v3: 10
CWE-843 Access of Resource Using Incompatible Type
The affected product accesses a resource using an incompatible type, which could allow an attacker to crash the server with a malicious payload.
CVSS v3: 8.6
CWE-121 Stack-Based Buffer Overflow
The affected product is vulnerable to a stack-based buffer overflow, which could allow an attacker to crash the device or remotely execute arbitrary code.
CVSS v3: 10
CWE-476 Null Pointer Dereference
The affected product uses a NULL pointer in certain situations. which could allow an attacker to crash the server.
CVSS v3: 8.6
