Controlling services with a GUI using unmanaged memory IPC

Running a security program as a service has some great benefits:

• Its start and stop events, along with any errors, are automatically logged as system events.
• It is trivial to make the process run under the local system account.
• Failure recovery policies can be defined and managed by the operating system itself.

However, the biggest problem is that services cannot communicate with the desktop window manager and therefore cannot directly own any window handle. All control of the service must be done through other mechanisms. Whilst a skilled user might be able to interact with the service control manager directly using the command line, it is not user friendly and provides a limited experience. To really provide a useful user interfact, the service must accept some form of inter-process communication (IPC) that accepts incoming commands and parameters and returns status messages.

In this project, I have decided to use unmanaged memory IPC. This is a form of IPC in which blocks of memory in the service and GUI frontend application are allocated and then used to communicate between the two programs. The initial setup of this method requires a handshake in which the address of a memory block can be passed to the other process. To do so we can send a custom command to the service using the service control manager.

The following is a simple example of how unmanaged memory IPC can be set up:

1. Client program uses VirtualAlloc to allocate two blocks of memory - one for parameters, one for return values. It fills the memory blocks with the pattern 0x1337D00D. The memory pages for the return value block are marked as PAGE_READWRITE. The memory pages for the parameters block are marked as PAGE_READONLY. Giving execute permissions would be bad for security.
2. Client program sends an AETHER_CLIENT_PID message to the service.
3. Service begins waiting for the process ID of the client program.
4. Client program sends its process ID to the service as a message.
5. Service stores the process ID.
6. Client program sends an AETHER_ADDR_PARAM message to the service.
7. Service begins waiting for parameter memory address.
8. Client program encodes the address of the parameter memory block as an integer and sends it to the service as a message.
9. Service stores the memory address.
10. Client program sends an AETHER_ADDR_RETN message to the service.
11. Service begins waiting for return value memory address.
12. Client program encodes the address of the return value memory block as an integer and sends it to the service as a message.
13. Service stores the memory address.
14. Client program sends an AETHER_IPC_TEST message to the service.
15. Service uses OpenProcess to get a handle to the client program's process, then uses ReadProcessMemory to check that all buffers read 0x1337D00D. It then uses WriteProcessMemory to fill the memory blocks with 0x00000000.
16. Client program waits 100ms and checks the memory blocks to make sure the pattern was written correctly.
17. If either test (15 or 16) failed, throw an error message.

At this point, the client program can write parameters into the parameters memory block (it alters the protection constant to PAGE_READWRITE temporaraly) and send a service control message to the service asking it to perform an operation. The service can then read the parameter using ReadProcessMemory. It performs some initial parameter checking, and returns a status code back to the return value message block to state whether or not the task has started. If everything is OK, it then performs the operation, and returns any necessary return values to that block.

To prevent problems such as overlap (e.g. a return value for one operation being sent when a return value for another is expected) each operation is designated a unique identification number. Furthermore, the code that manages IPC is designed such that only one operation can be communicated about at one point in time. To do this, I will use the Mutex class.

Parameters and return values will be encoded as binary serialized structures (using the BinaryFormatter class) with a prepended length value. In order to provide a literally identical structure for both client and service code, the shared structures will be placed in a separate library.