Analysis: Inspecting Mach Messages in macOS Kernel-Mode Part I: Sniffing the sent Mach messages

In this blog we will review how to inspect the sent Mach messages by setting up a kernel inline hook for the function mach_msg_overwrite_trap() and ipc_kmsg_send().

Mach IPC and Mach message are the foundation for many communications that occur in macOS.  The question that many threat researchers ask is, “how can we inspect these Mach messages in user-mode or kernel-mode perspective?”  In this blog, FortiGuard Labs looks at how to inspect Mach message in kernel-mode perspective by setting up an inline hook on specific kernel APIs for handling Mach messages.

This idea was inspired by a tool from Blackhat USA 2018 Arsenal: “Kemon” that is an open-source pre and post callback-based framework for macOS kernel monitoring. Kemon provides some good features, the most important of which is how to monitor macOS kernel by setting up kernel inline hooks on the kernel APIs we are concerned about. We can do a number of interesting things through kernel inline hooks. For example, we can develop an in-memory kernel fuzzer to find bugs in macOS kernel. We can also develop a tool to monitor IPC data in macOS. So I extended this framework to implement an inline hook to inspect the Mach messages in macOS.


A quick look at Mach Messages


Mach ports are a kernel-provided inter-process communication (IPC) mechanism used heavily throughout the operating system. A Mach port is a unidirectional channel that can have multiple send endpoints and only one receive endpoint. Because they are unidirectional, a common way to use a Mach port is to send a message to the receiver, and to include another Mach port on which the receiver can reply to the sender.

Mach messages are used for inter-process communication (IPC). Programs on macOS can either send and receive Mach messages directly, or they can use remote procedure calls (RPCs) generated by MIG (Mach Interface Generator).


Messaging Implementation

Let’s first look at how the user-space program and kernel handles sending and receiving Mach messages.

Both functions can send or receive a message, and they are defined in xnu-4570.71.2/libsyscall/mach/mach_msg. These two functions are the APIs in user-space.  They can invoke the function mach_msg_overwrite_trap() inside them. The function mach_msg_overwrite_trap() is implemented in xnu-4570.71.2/osfmk/ipc/mach_msg.c, which resides in the kernel mode.

The following is the execution flow of the function mach_msg() and mach_msg_overwrite():

We can see that when they enter the kernel they invoke the function ipc_kmsg_send() to send a Mach message. The declaration of the function ipc_kmsg_send() is shown below:

Its first parameter, kmsg, is the type of ipc_kmsg_t that is a pointer to the ipc_kmsg structure.

This structure is only the header for a kmsg buffer; the actual buffer is normally larger. The rest of the buffer holds the body of the message, which is defined as follows:

As shown in Figure 5, we can see the ipc_kmsg structure’s member variable ikm_header is a pointer to mach_msg_header_t structure. which represents the header of a Mach message.

Next, let’s take a look at the structure of a Mach message.

The top bit of the member variable msgh_bits in mach_msg_header_t structure is the complex flag. If it’s equal to 1, it representthats  this Mach message is a complex message and not a simple message. In a complex mach message, the mach_msg_header_t is followed by a descriptor count(mach_msg_body_t), then an array of that number of descriptors(mach_msg_*_descriptor_t). The type field of mach_msg_type_descriptor_t indicates the flavor of the descriptor. The mach_msg_body_t structure is defined as follows:

Figure 8 shows the presently defined types of descriptors.

We can see the descriptor type is positioned on the 11th byte(index from 0).

Next, let’s look at the exact definition of each descriptor. 

To this point we have detailed the structure of a Mach message. Now we will look at kernel inline hooking.

Kernel Inline Hooking

Inline hooking is a method of intercepting calls to target functions. The general idea is to redirect a function to a function of our own so that we can perform processing before and/or after the function performs it. This could include checking parameters, logging, spoofing the returned data, and filtering calls.

The hooks are placed by directly modifying code within the target function, usually by overwriting the first few bytes with a jump instruction to allow execution to be redirected before the function does something. 

Let’s look at how general inline hooking works.

As shown in Figure 12, we can see that hooking is made up of three parts.

1.     The inline hook – we overwrite 12 bytes into the prologue of the target function. These 12 bytes includes two instructions. If they move the trampoline’s address to register RAX, the JMP instruction will jump from the hooked function to our code.

2.     The trampoline consists of exactly three parts: the original instructions, a call instruction to call the handler, and a JMP instruction to jump back to the target function to continue executing the remaining instructions.

3.     The handler takes the exact same parameters with the target function. It allows you to do things like log data, filtering parameters, etc.

Not that we have briefly outlined how inline hooking works, let’s move on to how to hook the function of sending Mach messages.

As shown in Figure 1, in kernel mode the function mach_msg_overwrite_trap() is used to send Mach messages. It can invoke the function ipc_kmsg_send() to perform sending Mach messages. This means we need to implement a kernel inline hook on these two functions.

Before detailing the hooking implementation, there are two things to be noted. This first is how to resolve the kernel API’s symbol, and the other is how to disassemble the instructions from the address of the symbol. As for how to resolve the kernel symbol, you can refer to this blog. For the second, I used Capstone as the disassembler, which is a lightweight multi-platform and multi-architecture disassembly framework. The most significant feature is support for embedding into firmware or an OS kernel.

Next we’ll look at the implementation of hooking the functions mach_msg_overwrite_trap() and ipc_kmsg_send().

The following is a snippet of the function mach_msg_overwrite_trap() from

This function can check to see if the parameter option has the MACH_SEND_MSG property. If yes, it can then invoke the function ipc_kmsg_send() to send the Mach messages. The function ipc_kmsg_send() takes three parameters: the first is the type of a pointer to the ipc_kmsg structure. What we need to do is log and parse the data of the ipc_kmsg structure when the function ipc_kmsg_send() is called.  

Next, let’s go into the assembly world of the function mach_msg_overwrite_trap().

We can see that the function starts with a prologue. Here I used the following inline hook with 12 bytes length. 

Now let’s look at the assembly instructions of the hooked mach_msg_overwrite_trap(). It can jump to the memory address 0xffffff7f87a76160, which is actually the address of its trampoline.

At offset 0x376 relative to the start address of mach_msg_overwrite_trap(), it can invoke the ipc_kmsg_send().  We next pick seven bytes to do the inline hooking for ipc_kmsg_send(), such as the following:

So far, we have selected the opcode instructions for inline hooking. Next, we need to construct the trampolines. The following is the trampoline of hooking mach_msg_overwrite_trap().

The first 23 bytes of instructions are copied from the prologue of the function mach_msg_overwrite_trap(). This trampoline first executes the prologue of the target function, then invokes its handler that sets up the hook of the function ipc_kmsg_send(). Finally, it jumps back to the target function to continue to execute the remaining instructions. We can calculate the indirect jumping address as follows:

We can see that the trampoline finally jumps to 0xffffff8003d6f797(offset:+23) to execute instructions.  

Next, let’s look at the handler(mach_msg_overwrite_trap_prologue_handler).  In this handler, we first copy the instructions starting at 0xffffff8003d6faf6 (seven bytes length) to the corresponding position in the trampoline of hooking  ipc_kmsg_send(). We then overwrite the inline hooking instructions on the address starting at 0xffffff8003d6faf6.

It contains two instructions, JMP and nop. The JMP instruction is an indirect jump. We can see it is able to jump to ipc_kmsg_send_trampoline.


The following is the trampoline of hooking ipc_kmsg_send():

In this trampoline, it first invokes the function ipc_kmsg_send_pre_handler(), which is intended to parse and log the Mach messages to be sent. It then invokes the target function ipc_kmsg_send(), followed by invoking the ipc_kmsg_send_post_handler(), which is intended to handle or subvert the return value of the target function ipc_kmsg_send(). Finally, it can then jump back to the function mach_msg_overwrite_trap().

In this section we have examined how to sniff the sending Mach messages by implementing an inline hook of the function mach_msg_overwrite_trap() and ipc_kmsg_send().

Next, I will draw a picture to depict the workflow of the kernel inline hooking.

We can briefly sum up the steps of inline hooking mach_msg_overwrite_trap() and ipc_kmsg_send() as follows:

a.    Copy the first 23 bytes of the function mach_msg_overwrite_trap’s prologue to mach_msg_overwrite_trap_trampoline.

b.    Overwrite the first 12 bytes with our inline hooking instructions, which contain a MOV instruction and a JMP instruction followed by the address (8 bytes)of ipc_kmsg_send_trampoline.

c.    When the function mach_msg_overwrite_trap() is called, the execution can be redirected to mach_msg_overwrite_trap_trampoline. In mach_msg_overwrite_trap_trampoline, it can invoke the function mach_msg_overwrite_trap_prologue_handler.

d.    In the function mach_msg_overwrite_trap_prologue_handler it sets up the inline hook for the function ipc_kmsg_send(). It then copies instructions starting at 0xffffff8003d6faf6 (seven bytes length) to the corresponding position in ipc_kmsg_send_trampoline. Note that you need to tweak the relative address of the call instruction, otherwise it can cause a kernel panic.

e.    Overwrite the instructions starting at 0xffffff8003d6faf6 with our inline hooking instructions that contain an indirect JMP instruction and a NOP instruction. The indirect address is stored as eight bytes behind the inline hooking instructions of the function mach_msg_overwrite_trap().

f.     At the end of mach_msg_overwrite_trap_trampoline, it can jump back to 0xffffff8003d6f797 to continue to execute the remaining instructions.

g.    When it executes at 0xffffff8003d6faf6, it can then jump to ipc_kmsg_send_trampoline. In ipc_kmsg_send_trampoline it first invokes the function ipc_kmsg_send_pre_handler(), which is used to parse and log the Mach messages to be sent. It then invokes ipc_kmsg_send(), followed by invoking ipc_kmsg_send_post_handler() ,which is used to handle or subvert the return value of the function ipc_kmsg_send(). Finally, it jumps back to 0xffffff8003d6fafd in the function mach_msg_overwrite_trap().

Sniffing the Mach Messages

In the previous sections, I detailed the implementation of inline hooking for the function mach_msg_overwrite_trap() and ipc_kmsg_send(). In this section I present the sniffed sending Mach messages. All core work of inline hooks are implemented in a KEXT in kernel mode. We use a client program to receive the messages from logging Mach messages from the KEXT. This tool can record a detailed structure for a Mach message sent by a specific process. Some screenshots are shown below.

We can see the tool recorded all fields of the ipc_kmsg structure, as well as the parsed Mach messages in detail.


In this blog, I demonstrated how to sniff the sending of a Mach message by setting up a kernel inline hook for the function mach_msg_overwrite_trap() and ipc_kmsg_send(). This is able to record all fields of an ipc_kmsg structure and each field of a Mach message. In Part II of this blog, I will present how to sniff the received Mach messages via a kernel inline hooking.

You’re invited to stay tuned! Read part II of this blog analysis

Special thanks to the researcher Wang Yu’s open source project “Kemon: An Open-Source Pre and Post Callback-Based Framework for macOS Kernel Monitoring”.



Download our latest Fortinet Global Threat Landscape Report to find out more detail about recent threat landscape trends.

Sign up for our weekly FortiGuard Threat Brief.

Know your vulnerabilities – get the facts about your network security. A Fortinet Cyber Threat Assessment can help you better understand: Security and Threat Prevention, User Productivity, and Network Utilization and Performance.

*** This is a Security Bloggers Network syndicated blog from Fortinet All Blogs authored by Fortinet All Blogs. Read the original post at: