Mac system extensions for threat detection: Part 3 | Elastic Blog

Mac system extensions for threat detection: Part 3

This is the third and final post of a three-part series on understanding kernel extension frameworks for Mac systems. In part 1, we reviewed the existing kernel extension frameworks and the information that these frameworks can provide. In part 2 we covered techniques that could be used in kernel to gather even more details on system events. In this post, we will go into the new EndpointSecurity and SystemExtensions frameworks.

Out of the kernel

At WWDC 2019, Apple announced that in the upcoming macOS 10.16 release, third party vendors will no longer be allowed to run in the kernel via kernel extensions. What this means for security vendors is that those who are currently using a kernel extension to provide protection or detection will essentially need to toss out their kernel component and use the new frameworks that Apple has provided. These new frameworks are:

In combination, these new frameworks are intended to provide functionality that can take the place of kernel extensions in userland. Let’s take a deeper look at each framework and see how they can be used to mirror the functionality previously provided by kernel extensions.

EndpointSecurity framework

The EndpointSecurity framework revealed in WWDC 2019 is the spiritual successor to the Kauth framework that provides introspection into various system events such as:

In comparison to Kauth and Mandatory Access Framework, EndpointSecurity framework provides very rich information regarding system events — often providing information that would otherwise require non-supported kernel techniques to retrieve. This information is packaged into a relatively straightforward API accessible in userland. This provides developers the ability to use modern tools to debug their userland EndpointSecurity client, making development easier and any crashes less catastrophic for the system.

EndpointSecurity Events are further broken down into Authorization Event types and Notification Event types. Authorization Events provide the ability for processes that have plugged into the EndpointSecurity framework to interdict on the event — either allowing it to go through or stopping it. These events are particularly useful in preventing malware from executing, stopping unauthorized modifications to the file system, or loading of possibly malicious dynamic libraries. These Authorization Event types can also be used for behavior-based detections of malicious behavior and notifying users of possible malicious activity.

For more alert-based detection and response, the Notification Event types provide information regarding system events that have happened without the chance to intervene. This subscribing to this event type is better for use cases where certain sequences of system events may correlate to malicious behavior and an alert should be sent to the user.

APIs

There are several notable APIs when using the EndpointSecurity framework.

Putting it all together

Given these APIs, how can these pieces be put together to create functionality for preventing or detecting threats?

Subscribing, receiving, and responding to events
// Globals, can be encapsulated if preferred
// typedef of event handler function
typedef void(*event_handler_t)(const es_message_t*);
// Our client instance, will be filled out by es_new_client
es_client_t* g_client = nullptr;
// Declaration for ES_EVENT_TYPE_AUTH_EXEC handler
static void HandleExecAuth(const es_message_t* p_msg);
// Define the events we want to subscribe to
es_event_type_t g_subscribed_events[] =
{
    // Place all es_event_type_t values you wish to subscribe to in
    // here
    ...
    ES_EVENT_TYPE_AUTH_EXEC,
    ...
};
// Define the handler for a given message type
std::map<es_event_type_t, event_handler_t> event_handler_map =
{
    ...
    {ES_EVENT_TYPE_AUTH_EXEC, HandleExecAuth},
    ...
};
...
// In the function that is responsible for hooking into the
// EndpointSecurity subsystem
// Define our block handler
es_handler_block_t handler = ^void(es_client_t *_Nonnull,
const es_message_t *_Nonnull p_msg)
{
    // Look up the event handler for a given message type
    if (event_handler_map.count(p_msg->event_type) == 1)
    {
        // Handle the message
        event_handler_map.at(p_msg->event_type)(p_msg);
    }
    else
    {
        // Log an error
    }
};
// Create a new client instance, this will connect us to the
// EndpointSecurity subsystem
es_new_client_result_t result = es_new_client(&g_client, handler);
// Cache needs to be explicitly cleared between program invocations
es_clear_cache_result_t resCache = es_clear_cache(g_client);
if (result == ES_NEW_CLIENT_RESULT_SUCCESS &&
    resCache == ES_CLEAR_CACHE_RESULT_SUCCESS)
{
    // Now subscribe to the events in g_subscribed_events
    es_return_t subResult = es_subscribe(g_client,
                            g_subscribed_events,
                 sizeof(g_subscribed_events)/sizeof(es_event_type_t));
    if (subResult == ES_RETURN_SUCCESS)
    {
        std::cout << "Subscription success" << std::endl;
    }
}
...
// ES_EVENT_TYPE_AUTH_EXEC handler
static void HandleExecAuth(const es_message_t* p_msg)
{
    const es_event_exec_t* p_exec_notification = &p_msg->event.exec;
    if (p_exec_notification->target)
    {
        if (p_exec_notification->target->executable)
        {
            // Analyze the binary, determine if this binary
            // is allowed to execute or not
        }
    }
    // After analysis is complete, respond to the message to
    // allow or forbid the execution from going forward. Can
    // either be ES_AUTH_RESULT_ALLOW or ES_AUTH_RESULT_DENY
    es_respond_auth_result(g_client, p_msg,
                           ES_AUTH_RESULT_ALLOW, false);
}

A more complete example of how to use the EndpointSecurity framework can be seen in the sample project attached with this post. It’s also important to know that any vendors who wish to distribute applications that use the EndpointSecurity framework must have the correct entitlements to do so.

SystemExtensions framework

As a spiritual successor to kernel extensions, the SystemExtensions framework provides developers the ability to enhance the functionality of the operating system. As stated in WWDC2019’s System Extension presentation, kernel extensions have some problems when developing and deploying. Because the kernel environment is so unforgiving for mistakes, kernel code and kernel extension code must effectively be perfect. Any bugs can cause the entire machine to kernel panic and require a restart.

In addition, any issues that require a debugger become even more difficult as debugging the kernel requires two machines and special hardware or software to enter a debug session. These problems are remedied with the introduction of System Extensions in macOS 10.15. Because System Extensions run in usermode, developers can now leverage modern development tools to triage and debug issues that crop up, and a System Extension crashing will not cause a kernel panic. A sample System Extension project from Apple can be found here, which demonstrates how a macOS app can package, load, and communicate with a Network Extension.

When it comes to developing and deploying a System Extension, there are several hoops that developers must jump through. The first hoop is the entitlements required to load the System Extension. Next up is the location of installation and structure of the application (although this can be worked around with systemextensionsctl developer [on|off]). And finally, having the end user click through the proper prompts to allow the System Extension to load.

APIs

There are several notable APIs that are used when developing and loading System Extensions:

Entitlements and capabilities

In order to develop applications with the ability to load System Extensions, there are entitlements and capabilities that must be enabled. Developers must enable the following capability in their apps in order to install system extensions.

This option can be found in the App ID configuration section under the “Certificates, Identifiers & Profiles” tab.

The next step is to add the com.apple.developer.system-extension.instal entitlement to your application, which will have a System Extension packaged with it. Once these steps are complete, your app should be able to install and load System Extensions that are developed with the same Team ID. System Extensions themselves may need additional entitlements depending on what other things they need to do. If the System Extension needs to use the EndpointSecurity Framework, the System Extension must in turn have the com.apple.developer.endpoint-security.client entitlement.

If the System Extension needs to use the NetworkExtension framework, then the System Extension and application that’s hosting the System Extension must have the com.apple.developer.networking.networkextension entitlement along with the features that the System Extension will be implementing. Possible features are:

In addition to the entitlement, the capability must be enabled on the developer account.

The sample code provided by Apple shows how to use a Content Filter feature as a part of a System Extension, which uses the Network Extension framework. The sample code attached with this blog will also show how to use the Content Filter as well.

App structure

The next caveat to loading and installing a System Extension is where the System Extension must be placed and the general structure of the application that is hosting the System Extension. As stated in WWDC2019, System Extensions must be bundled within an application, and in turn this application must be installed under /Applications/. There is a further requirement in that the System Extension must fall under the following app structure:

Installation

During the installation and loading of the System Extension, there will be multiple prompts to the user regarding loading the System Extension. After the application that is hosting the System Extension has been installed under /Applications/ and the application attempts to load the System Extension for the first time, the user will be prompted.

If the user wishes to continue with the loading of the System Extension, they can go to their Security & Privacy preferences pane to allow the System Extension to continue loading.

If the System Extension is making use of the NetworkExtension framework, there will be an additional prompt to the user.

Finally, once the user has clicked through all the prompts, the Network Extension is allowed to begin its work.

Gotchas

While developing your application to use the new SystemExtension framework or the new EndpointSecurity framework, you may run into some hiccups. Here are a couple.

EndpointSecurity client randomly getting killed

Attempts to perform blocking operations directly in the handler will result in the EndpointSecurity client being killed when there are too many system events at once.

// Define our block handler

es_handler_block_t handler = ^void(es_client_t *_Nonnull,
const es_message_t *_Nonnull p_msg)
{
    // Look up the event handler for a given message type
    if (p_msg->event_type == ES_EVENT_TYPE_AUTH_EXEC)
    {
        // Handle the message
        HandleExecAuth(p_msg);
    }
    else
    {
        // Log an error
    }
};
// ES_EVENT_TYPE_AUTH_EXEC handler
static void HandleExecAuth(const es_message_t* p_msg)
{
    __block dispatch_semaphore_t sema = dispatch_semaphore_create(0);
    uint64_t responseMachTime = mach_absolute_time();
    uint64_t responseMachDeadlineDelta = p_msg->deadline - responseMachTime;
    // Make sure we have time to do analysis and respond
    if (responseMachDeadlineDelta >= (10 * NSEC_PER_SEC))
    {
        // Perform analysis work on a separate thread
        dispatch_async(dispatch_get_global_queue(QOS_CLASS_DEFAULT,
    0), ^{
            // Perform analysis that may have blocking operations.
         // If there are too many operations on the system while
         // analysis is ongoing, the EndpointSecurity subsystem may
         // kill your client.
            sleep(1);
            // Analysis complete, wake up the waiting thread
            (void)dispatch_semaphore_signal(sema);
        });
        // Wait for the analysis thread to signal us, otherwise wait until 1 second before the deadline
        long result = dispatch_semaphore_wait(sema, dispatch_time(DISPATCH_TIME_NOW, responseMachDeadlineDelta - (1 * NSEC_PER_SEC)));
        if (result != 0)
        {
            std::cout << "timed out performing analysis" << std::endl;
        }
    }
    else
    {
        std::cout << "Skipping analysis because we don't have enough time to perform it" << std::endl;
    }
      // This can be ES_AUTH_RESULT_DENY as well depending on what
      // analysis has returned
        es_respond_auth_result(g_client, p_msg, ES_AUTH_RESULT_ALLOW, false);
}

By attempting to perform analysis on a separate thread while the handler thread waits for a result or timeout, no other events are being dequeued from the event callback dispatch queue, which at some point fills up the queue and results in the EndpointSecurity client being killed.

The correct thing to do is to use es_copy_message() to copy the message, return from the event handler immediately, and perform deadline management and analysis on a separate thread.

// ES_EVENT_TYPE_AUTH_EXEC handler
static void HandleExecAuth(const es_message_t* p_msg)
{
    uint64_t responseMachTime = mach_absolute_time();
    uint64_t responseMachDeadlineDelta = p_msg->deadline - responseMachTime;
    // Make sure we have time to do analysis and respond
    if (responseMachDeadlineDelta >= (10 * NSEC_PER_SEC))
    {
        __block es_message_t* pCopiedMessage = es_copy_message(p_msg);
        // Perform analysis work and deadline management
        // on a separate thread
        dispatch_async(dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0), ^{
            __block dispatch_semaphore_t sema = dispatch_semaphore_create(0);
            dispatch_async(
dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0),
 {
                // Perform analysis that may have blocking operations.
                sleep(1);
                // Analysis complete, wake up the waiting thread
                (void)dispatch_semaphore_signal(sema);
            });
            // Wait for the analysis thread to signal us, ‘
 // otherwise wait until 1 second before the deadline
            long result = dispatch_semaphore_wait(sema, dispatch_time(DISPATCH_TIME_NOW, responseMachDeadlineDelta - (1 * NSEC_PER_SEC)));
            if (result != 0)
            {
                std::cout << "timed out performing analysis" << std::endl;
            }
            // This can be ES_AUTH_RESULT_DENY as well depending on
            // what your analysis has returned
            es_respond_auth_result(g_client, pCopiedMessage,
ES_AUTH_RESULT_ALLOW, false);
            es_free_message(pCopiedMessage);
            pCopiedMessage = nullptr;
        });
    }
    else
    {
        std::cout << "Skipping analysis because we don't have enough time to perform it" << std::endl;
    }
    // Return without responding to the original message,
    // it will be responded to in another thread
}

Launch daemons can load System Extensions, but cannot unload them

One of the more bizarre quirks of the SystemExtension framework is that launch daemons that are packaged in an application installed under /Applications/ have the ability to load System Extensions that are packaged with the application. The proper prompts will show up for the user to authorize the System Extension to load — however, when the launch daemon that has previously loaded the System Extension attempts to unload the same System Extension, deactivationRequestForExtension will fail with OSSystemExtensionErrorDomain error 13. Under further investigation, the following error is revealed: OSSystemExtensionErrorAuthorizationRequired. This is odd since the launch daemon had authorization to load, but not to unload. Looking further in the Console for possible hints reveals some more information:

default 14:56:56.581780-0700 opendirectoryd getpwuid failed with result Not Found
error 14:56:56.583919-0700 authd Fatal: interaction not allowed (session has no ui access) (engine 237)
default 14:56:56.583939-0700 authd Failed to authorize right 'com.apple.system-extensions.admin' by client '/Applications/MyApp.app' [40455] for authorization created by '/Applications/MyApp.app' [40455] (3,0) (-60007) (engine 237)
default 14:56:56.584357-0700 MyApp failed to create authref: -60007

Although the information is not particularly useful as to exactly what we can do to remedy the situation, by finally reaching out directly to Apple, they’ve addressed the situation in the following manner:

Wrapping up

The new EndpointSecurity framework and the new SystemExtension framework provides an incredible amount of modernized functionality for threat detection on macOS. Kernel extensions long provided the capabilities needed to extend system functionality to allow third party security vendors the ability to detect and stop threats. However, this all came at the cost of potential system instability and old/outdated development tools. With the introduction of these new frameworks, security vendors will have a smoother development time, less buggy code (due to the ability to use modern debugging tools), and less catastrophic consequences if their code happens to have a bug in it. Overall, these new frameworks are a welcomed addition to the security toolset.