1 Introduction
Hardware virtualization is the de facto enabling technology for cloud computing which allows customers to rent computing resources via VMs. Applications are deployed in specialized operating system (OS) images called virtual appliances that contain a standard OS kernel and other necessary software components [1]. Each VM is a self-contained virtual computer that provides strong isolation between each other in multi-tenant cloud platforms. Generally, VMs are often specialized to run a single task, such as a Web server, a database, and scale out by cloning VMs through image templates.
Although traditional VMs provide strong isolation by running general-purpose guest OSes, they often suffer from high performance/storage overhead in terms of large memory consumption, long booting latency, and virtualization cost [2–4]. OS virtualization technologies such as containers [5] allow multiple instances share the same OS kernel as the host. Thus, containers can be more efficient than VMs, but do not provide as good isolation as VMs. Recently, unikernels have emerged as a lightweight way to deploy cloud services. Unikernels are application-specialised, single-address-space machine images constructed by using library operating systems [2]. The entire software stack of a unikernel includes OS kernel, necessary system libraries, language runtime, and application source code. They are compiled into a lightweight, single-purpose application that can directly run on a standard hypervisor. Unikernels can offer near bare-metal application performance, strong isolation, minimal memory footprint, and extremely low startup latency. Fig.1 shows the architectures of VMs, containers, and unikernels. In general, unikernels can offer VM-like strong isolation, and are even more lightweight and efficient than containers.
Due to these promising characteristics of unikernels, they have been increasingly studied for network function virtualization [6–8], micro-services [9], IoT applications [7, 10], and serverless computing [11, 12]. In these scenarios, cloud computing platforms can efficiently deploy hundreds of unikernel-based VMs in a single physical server, and thus main memory becomes the primary bottleneck resource for high-density application deployment. Recently, NVM technologies such as PCM [13] and 3D XPoint [14] have become increasingly popular in cloud data centers because they can offer extremely large memory capacity at a low expense. However, since they have relatively lower performance and limited write endurance compared to DRAM [15], it is more practical to use NVMs combining with DRAM to fully exploit their advantages [16]. Although there have been many studies on heterogeneous memory management in the OS layer, there still remain many challenges to utilize NVMs for unikernel-based VMs [17].
First, the existing memory management system for VMs was not designed for single-address-space unikernels, and requires redesign to efficiently utilize heterogeneous DRAM/NVM memories. Currently, heterogeneous memories are not visible to guest OSes since hypervisors and VMs are initially designed to use a single type of memory. Modern OSes have already supported heterogeneous memories by using the non-uniform memory access (NUMA) architecture or different namespaces, and VMs have a opportunity to use DRAM and NVM by mapping them to different NUMA nodes or namespaces. Unfortunately, most unikernels do not support the NUMA architecture and namespaces, making the visibility of heterogeneous memories to unikernels difficult.
Second, unikernels usually suffer from high performance penalty due to two-layer address translations in the virtualization environment. A translation lookaside buffer (TLB) miss in a guest OS leads to 24 memory references due to two-layer page table walking, and the performance overhead may even exceed 50% of total execution time of applications [18–21]. Although previous studies have shown that using large pages (or hugepages) can significantly reduce the address translation overhead, they often preclude lightweight memory management such as fine-grained memory allocation, page migration, and memory de-duplication. It is challenging to bridge this fundamental conflict in unikernels.
In this paper, we present UCat, the first work to support heterogeneous memory management for unikernels. We enable heterogeneous memory allocation in unikernels by mapping two memory regions to the host-managed DRAM and NVM namespaces, respectively. UCat exploits large pages to reduce the cost of two-layer address translations in virtualization environments, and leverages slab memory allocation to reduce memory waste at the application layer. Overall, we make the following contributions:
● We manage heterogeneous memories in unikernels by mapping different memory regions in unikernels to different host-managed namespaces. This simple-yet-efficient way exposes the memory heterogeneity to unikernels, and thus supports heterogeneous memory allocation in unikernels.
● We design a multi-grained memory allocation mechanism in unikernels by taking the advantage of both large pages and slab memory allocation. This mechanism not only reduces the cost of two-layer address translations in a virtualization environment, but also reduces memory waste due to internal memory fragmentation.
● We implement UCat based on a popular unikernel–OSv. Experimental results show that our mechanisms reduce unikernels’ memory consumption by 50% and TLB miss rate by 41%, and improve the throughput of real-world benchmarks such as memslap and YCSB by up to 18.5% and 14.8%, respectively.
The rest of this paper is organized as follows. We present the background and motivation in Sections 2 and 3, respectively. Section 4 describe our address mapping mechanism for supporting heterogeneous memories in unikernels, and multi-grained memory management mechanism by integrating large pages and slab memory allocation. Section 5 present the performance evaluation of UCat. We present related work in Section 6 and conclude in Section 7.
2 Background
In this section, we present the necessary background about unikernels and address translations in virtualization environments.
2.1 Unikernel applications and cloud
A unikernel is an application-specialized VM image. It is constructed by linking an application and parts of LibOS required by the application at compile time. A unikernel can be executed as a VM on top of a hypervisor such as Xen, VMware ESXi, and KVM. Unikernels are characterized as specialization and single-address-space. That implies a unikernel is constructed only for one application, and the application and the kernel share a single memory address space.
This design principle brings many benefits. First, unikernels are very lightweight in terms of image size and runtime memory footprint because they only contain the application and its necessary parts of LibOS. Second, unikernels are highly isolated because hypervisors such as Xen and VMWare ESXi provide strong isolation between VMs on top of a hypervisor. Third, unikernels offer better security because they expose small attack surface. The absence of system calls and shells blinds attackers to subvert unikernels. Moreover, unikernels are written in statically-typed languages that provide memory safety guarantee. For example, MirageOS [2] is written in OCaml language. Fourth, unikernels are executed efficiently because unikernels are single-address-space and there is no context switches between the user application and the kernel in unikernels. Overall, these promising features make unikernels an ideal option for deploying micro-services and serverless applications in cloud platforms. They not only improve VM density for cloud providers [8], but also are of great benefit to cloud users for cost reduction.
However, it may be quite difficult to port legacy applications from traditional software stack to unikernels [8, 22]. First, unikernels are often constructed by linking application source code with libOS at compile time. However, some applications’ source code may be not available due to proprietary limitations. Second, because different unikernels are written in different programming languages, it may be difficult to refactor legacy applications in other languages. Third, unikernels usually only support limited OS features and libraries to achieve lightweight deployment [3, 23]. If legacy applications require certain system features and libraries that are not supported by the unikernel, the application should be modified or refactored.
Although unikernels have many promising features, the difficulty of porting legacy applications impede their wide application in the industry. One solution is to provide binary compatibility. For example, OSv [3], HermiTux [4], and X-Containers [24] provide the Linux binary compatibility and allow the Linux binaries to support unikernels. Binary compatibility can significant reduce the burden of unikernel application developers. As a result, we implement UCat based on a popular unikernel–OSv.
2.2 Native address translation
Virtual memory is an important concept and abstraction in modern computer systems. It brings many benefits: the ability of sharing memory between processes, enhanced security due to memory isolation, providing more memory than the available physical memory space (memory overcommitment), etc. Most virtual memory management systems use page tables to translate virtual address (VA) to physical address (PA). The address translation process is usually called page table walking. The cost of address translation is mainly determined by the levels of page tables. For x86-64 architectures, the page tables have four levels. Fig.2 illustrates the native page table walking in x86-64 architectures.
Modern CPUs usually speed up address translations by caching the most recently accessed page tables in the page table walking (PTW) cache. In addition, to avoid frequent page table walks, TLB has been widely used to store the recent VA-to-PA translations for future use. It has become a common component of the memory management unit (MMU). When 4 KB pages are used, a TLB miss leads to four memory references in a VA-to-PA translation. Thus, page table walking usually has a significant impact on application performance if the TLB miss rate is high.
2.3 Address translations in virtualization environments
In a virtualization environment, the address translation from VA to PA is divided into two stages. In the first stage, the guest OS performs a page table walk that translates guest virtual address (GVA) into guest physical address (GPA). In the second stage, the hypervisor further translates the GPA into host physical address (HPA). There are mainly two approaches to achieve the two-layer address translation: shadow page table (SPT) and hardware-assisted virtualization address translation.
Fig.3 illustrates the work process of SPT. When the SPT is used, the hypervisor maintains a separate shadow page table for each process in a VM. The SPT can directly translate a GVA into a HPA efficiently. However, when the guest page table is updated, the SPT should be modified at the same time. Therefore, the modification of the guest page table needs to be tracked by the hypervisor. To monitor the modification of guest page tables, the guest page table is set as read-only. As a result, any modification made by the guest OS would trigger a page fault and is trapped into the hypervisor. This procedure is also called VM-exit, which is rather costly for VM-resident applications. The hypervisor then modifies the guest page table and updates GVA-to-HPA mapping in the SPT. Although the SPT provides a direct mapping of GVA to HPA, it also has some limitations. First, the hypervisor needs to maintain a separate SPT for each process in a VM, resulting in non-trivial memory consumption. Second, frequent VM-exits also incur significant performance overhead. Third, the synchronization of guest page tables and SPTs also wastes a lot of CPU resource.
To improve the performance of address translations, Intel and AMD proposed Extended Page Table (EPT) [25] and Nested Page Tables (NPT) [26], respectively. As these two technologies are similar, in the following we take Intel's EPT as an example to illustrate the process of address translations. The EPT eliminates the dependency between the GVA-to-GPA translation and the GPA-to-HPA translation. If a page fault occurs in the guest OS, it can be handled directly by the guest OS without a VM-exit. When the GVA is translated into GPA by the guest OS, the EPT then translates GPA to HPA. This process is similar to the native x86-64 address translation. Since only one EPT is needed for a VM, the memory consumption due to address translation is greatly reduced.
Fig.4 depicts the two-dimensional EPT-supported page table walk in virtualization environments. When both the guest OS and the host OS are mapped using 4 KB pages, each level of translation in the guest page table requires an EPT walk which requires four memory references. Thus, a TLB miss in a VM leads to total (num_of_ page_dir + num_of_ page) × accesses_ per_ EPT_walk − final_ page_ access memory references for the GPA-to-HPA translation. Thus, total memory references are required for accessing a page with 4 KB upon a TLB miss. Large pages (such as 2 MB pages) can be exploited to speed up the address translation in virtualization environments. As shown in Fig.2, using 2 MB pages in a bare-metal machine can reduce one memory reference. When 2 MB pages are used in both the guest OS and the host OS, total memory references are required. Thus, memory references are reduced in the two-dimensional page table walks when the page size is changed from 4 KB to 2 MB. As a result, using 2 MB large pages can significantly reduce the number of memory references for a GPA-to-HPA translation [25, 27].
3 Motivation
Unikernels are lightweight virtualization technologies with smaller image size, better security, shorter startup time, and higher performance. Because of these advantages, unikernels are more promising for deploying microservices in cloud platforms than traditional VMs. First, because unikernels only contain necessary libraries for a single application, the image size and runtime memory footprint are very small. As shown in Tab.1, the image size of a Web server built with unikernel is only 3.5 MB, while the image size built with a container is almost 3 times larger than that of unikernel. As for VMs, their images generally are hundreds of MB and even more larger. Second, the startup time of unikernels are very short. This advantage is particularly useful for fast service deployment and failure recovery. Fig.5 shows the startup time of unikernels compared with a Docker container and a native Linux-based VM. The boot time of a native Linux is hundreds of times longer than the unikernel. The boot time of dockers is also almost 10 times as long as that of unikernels. Third, because unikernels consume less resource, more unikernels can be deployed on a single physical server, and thus significantly improve the cost-efficiency of cloud applications.
Tab.1 The image size of dockers and unikernels |
| Docker | Unikernel | |
|---|---|---|
| DNS Server | 9.82 MB | 3.3 MB |
| Web Server | 10.1 MB | 3.5 MB |
OSv is a typical unikernel designed for cloud platforms, enabling easy deployment and management of micro-services and serverless applications. However, there still remain challenges to use OSv in heterogeneous memory systems. First, since OSv is not aware of the heterogeneity of the underlying heterogeneous memory resources, it does not support heterogeneous memory allocation. To the best of our knowledge, none of unikernels support heterogeneous memory allocation. This is a major obstruct for deploying unikernel-based VMs in heterogeneous memory systems.
Second, OSv also suffers high performance overhead of the two-layer address translations in the virtualization environment, because OSv also uses four-level page tables to translate the GVA into the GPA. Although large pages can be used in OSv to speed up address translation, large pages often impede lightweight memory management such as fine-grained memory allocation/migration/sharing, resulting in high latency of large page allocations, costly large page migration in heterogeneous memory systems, and internal fragmentation. When the memory resource is overallocated, using large pages can lead to costly page swapping and significant system performance degradation. That is the root cause that most commercial Databases such as mongoDB [28] recommend to disable transparent huge pages (THP) for better performance. Therefore, there still remains challenges of using large pages to reduce the cost of address translation for OSv-based VMs.
Third, the OSv’s memory allocation mechanism cannot adapt to diversifying memory requirement of cloud applications, especially for memory requests with different sizes. The OSv’s memory allocation mechanism mainly includes two stages. In the first stage (pre-SMP enabled), OSv allocates memory directly from the free_ page_ranges. In the second stage, the memory is allocated according to the data size. If the data size is less than or equal to 1024 bytes, the memory is allocated from the malloc_ pools. If the data size is larger than 1024 bytes and less than 4 KB, an entire 4 KB page is allocated. If the data size is larger than 4 KB, the memory is allocated from the free_ page_ranges. Memory requests with different sizes often result in high cost of address translation and a waste of memory resource. For example, an entire 4 KB page is allocated to a request of 1030 bytes, resulting in a waste of nearly 3 KB memory. This implies that there is still vast room to improve the utilization of memory resource in OSv through fine-grained memory allocation.
To address the above problems, we present UCat, a heterogeneous memory management mechanism for a typical unikernel–OSv. First, we develop an address mapping mechanism to support heterogeneous memory allocation in OSv, so that OSv can perceive the underlying heterogeneous memories. Second, we implement a heterogeneous memory management mechanism combining large pages and slab allocation. We use large pages to reduce the cost of address translation, and leverage slabs to support fine-grained memory allocation to reduce the memory consumption of OSv.
4 Design and implementation
In this section, we first give an overview of UCat, and then present the technical details of the address mapping mechanism and the multi-grained memory allocation mechanism. Finally, we integrate these two mechanisms into the open-source OSv.
4.1 Architecture overview
Fig.6 depicts the architecture of UCat. Intel Optane DC Persistent Memory Module (PMM) can be used in two operation modes: Memory Mode and App Direct Mode. We use the App Direct Mode so that the Optane DC PMMs and DRAM are visible to the host OS.
Although DRAM and Intel Optane Persistent Memory can be placed in different NUMA nodes in a physical host, NUMA accesses may degrade the performance of applications [14], especially for unikernels that usually run only one single program in the VM. Since Linux can manage NVM in a contiguous physical memory space through a pmem namespace, it is potential to manage NVM and DRAM through different namespaces separately, as shown in Fig.6. In this way, the unikernel can simply distinguish NVM and DRAM by mapping two continuous memory regions to the DRAM and NVM namespaces managed by the host server.
To use heterogeneous memories, OSv should be aware of the capacity and region of DRAM and NVM. In UCat, each OSv divides its main memory space into a DRAM region and a NVM region, and then informs the memory boundary (called heterogeneous_memory_boundary) to the KVM hypervisor via quick emulator (QEMU). When KVM obtains the memory boundary, it can map the OSv’s DRAM and NVM address space to the corresponding namespaces. As shown in Fig.6, the DRAM region in OSv is mapped to namespace 0, and the NVM region is mapped to namespace 1. Upon a page fault, the page fault address (fault_addr) is stored in the CR2 register. By comparing the fault_addr and the heterogeneous_memory_boundary, KVM can determine the type of memory required by OSv and apply for memory from namespace 0 or namespace 1 accordingly. In addition, to reduce the two-layer address translation overhead and the internal memory fragmentation, we combine large pages and different slab classes to manage the heterogeneous memories in a multi-grained manner.
4.2 Cooperative address space mapping
OSv needs to perceive the heterogeneity of the underlying physical memory to allocate DRAM and NVM in OSv. A simple way is to manage the physical DRAM and NVM in two namespaces separately, and then the OSv can map two memory regions to the underlying different namespaces in the host.
To support heterogeneous memory allocation in OSv, we propose a simple-yet-efficient mechanism called front-end/back-end cooperative address space mapping, as shown in Fig.7. The front-end and back-end refer to the OSv-based VM and the underlying hypervisor (QEMU/KVM), respectively. We map the DRAM region in OSv to namesapce 0, and the NVM region to namesapce 1. To achieve this goal, we modify the OSv startup module and the OS initialization module to support heterogeneous memory allocation during the OSv booting. Specifically, the OSv startup module should be aware of the DRAM and NVM sizes. These two parameters are stored in a special location of the OSv image before the VM is started.
First, we construct a QEMU command including the DRAM and NVM sizes to start a VM. QEMU parses the two memory regions assigned to the VM and creates a memory boundary parameter, which is stored in the VM-related meta-data structure. At this time, the KVM hypervisor can obtain the memory boundary between the DRAM region and the NVM region in OSv, and then maps the two memory regions to different namesapces.
Second, when the VM starts, the OSv initialization module parses the parameters of DRAM and NVM sizes from the OSv image, and stores the memory boundary as a global variable in the VM at runtime. Therefore, the memory management subsystem of OSv can recognize the type of memory requested by the application. At this time, both the front-end (OSv) and the back-end (QEMU/KVM) is aware of the memory boundary between DRAM and NVM regions. For each memory request, KVM can allocate different physical memories to upper VMs according to the memory boundary exposed by each OSv. As shown in Fig.7, if the virtual memory requested by OSv locates in the DRAM region, KVM allocates DRAM from namesapce 0. Otherwise, KVM allocates NVM from namesapce 1.
With the collaboration between front-end and back-end, we establish the memory address mapping between OSv and KVM in a simple yet efficient way. Our mechanism is also applicable to other unikernels to support heterogeneous memories.
4.3 Multi-grained memory allocation
Since OSv still uses four-level page tables for address translation, it still suffers the high cost of the two-layer address translation. On the other hand, OSv only supports a limited number of page sizes, and thus may waste a considerable amount of memory resource. To mitigate internal memory fragmentation, a simple approach is to split a large page into 4 KB pages when the utilization of the large page is low. In contrast, when the number of continuous 4 KB pages exceeds a given threshold, they can be merged into a large page to reduce the cost of address translation of these small pages. Although large pages splitting and promotion [18, 27, 29, 30] have been long explored in the past decades, unfortunately there is not a real implementation in modern computers because of its high complexity in software/hardware.
Our multi-grained memory allocation mechanism takes the advantage of both large pages and slab allocation to speed up the GVA-to-HPA address translation while still retaining a low degree of internal memory fragmentation. Since there are two-layer address translations in a virtualization environment, a TLB miss can result in 24 memory references during the two-layer page table walking when both VMs and the host OS all use 4 KB pages. In UCat, we use 2 MB large pages to reduce the cost of two-layer address translations. On the one hand, large pages can significantly improve the TLB coverage, and thus reduce the TLB miss rate. On the other hand, using large pages can further reduce the number of memory references from 24 to 15 (37.5% reduction) upon a TLB miss in OSv.
Moreover, to reduce the waste of memory resource, we exploit slab allocation to support fine-grained memory allocation. Specifically, we logically split 2 MB large pages into different slab classes ( ) to satisfy multi-grained memory requests. As shown in Fig.8, for the slab class t, a 2 MB large page can be split into multiple slabs of bytes. In the beginning of each slab class, there is a slab header recording the address offset of each slab. Generally, a 2 MB large page can satisfy memory requests, where t denotes the size of a slab, and h denotes the space consumed by the slab header. We note that the address translation of these slabs still relies on TLBs and page tables of large pages. When the large page address of a slab is translated, the slab allocator should first consult the slab header to get the address offset of the requested slab, and then access the data. In this way, UCat can manage memory in fine-grained slab classes while still benefiting the performance advantages of large pages for address translation.
We modify the memory allocation strategy in OSv to support fine-grained memory allocation. When a memory request is greater than or equal to 2 MB, UCat directly allocates memory from the free_ page_ranges. For memory requests less than 2 MB, we exploit the slab allocator to find the best-fit slab class, and then return the address offset of the available slab to the object. Since UCat initializes different slab classes for allocation when a OSv-based VM is created, the slab allocator can always find a slab class that best fits for the memory request with the least memory waste. If there is a free slab available in a slab class, the memory allocation can be completed quickly. Otherwise, Ucat should first apply for a large page from the free_ page_ranges, and then append it to an existing slab class. On the other hand, if one slab class contains a large amount of free slabs for a long time, this slab class can be reclaimed and reassigned to another slab class with few free slabs.
4.4 Work flow of memory allocation
Since there is significant performance gap between DRAM and NVM, we bind the memory used by the OSv kernel to DRAM for high performance. Because applications may have a very large memory footprint, we can allocate DRAM and NVM alternately to applications according to a pre-defined ratio. Since a unikernel is built by compiling the source code of the application with the LibOS together, the programmer can specify the ratio in advance of the compiling. However, the value of the ratio is usually hard to determine in practice because the memory usage of an application may dynamically change at runtime. Thus, we simply allocate DRAM and NVM to the application alternately according to the DRAM-to-NVM ratio in the unikernel. The guest kernel still uses the vanilla memory allocation mechanism in OSv, while the application uses our multi-grained memory allocation mechanism for higher performance and less memory waste according to the DRAM-to-NVM ratio in the unikernel. Fig.9 depicts the execution flow of our memory allocation mechanism in UCat. For the application, if the requested memory can be offered by a slab class, a free slab is returned immediately, and thus the latency of slab allocation is very low. However, if the requested slab class is not available or there is no free space, the slab allocator should first determines which kinds of memory should be allocated according to the memory boundary, and then apply for a 2 MB large page. At last, it constructs the slab-related data structure for allocation. This process usually leads to a rather long tail latency of memory allocation.
5 Evaluation
In this section, we first verify the effectiveness of our address mapping mechanism, and then evaluate the performance of multi-grained memory allocation mechanism using real-world workloads.
5.1 Methodology
Experiment setup In our experiments, the host server is equipped with dual-socket Intel Xeon Gold 6230 2.10 GHz 20-core processors, 128 GB of DRAM, and 1 TB of Intel Optane DC Persist Memory Module (DCPMM). We organize DRAM and DCPMM in two namespaces using ndctl and daxctl tools [31, 32], and use Optane DCPMMs in App Direct Mode. The host server runs Ubuntu 19.10, the KVM hypervisor with Linux kernel version 5.1.1, and QEMU 2.11.50. The guest VM is a unikernel running modified OSv 0.54.
We compare our multi-grained memory allocation mechanism with the OSv memory management mechanism. If not specified otherwise, they are all deployed on the host with the same settings and run Memcached [33], which is a high-performance, distributed in-memory key-value object caching system for accelerating database queries.
Real-world workloads We use memslap [34] and YCSB [35] benchmarks to evaluate the performance of Memcached deployed in OSv. Memslap is a load generation and benchmark tool for Memcached. It can generate different workloads with configurable parameters such as threads, concurrency, and execution time. If not specified in the benchmark, we use the default parameter settings. For example, the ratio of gets to sets is 9:1, and the task window size of each concurrency is 10K. YCSB is a general-purpose performance benchmark tool, which can evaluate the performance of various non-relational databases. YCSB includes six types of workloads representing different application scenarios, as shown in Tab.2.
Tab.2 Description of YCSB workloads |
| Workload | Description | Ratio of operations |
|---|---|---|
| A | Update heavy | 50% reads and 50% writes |
| B | Read mostly | 95% reads and 5% writes |
| C | Read only | 100% reads |
| D | Read latest workload | 95% reads and 5% inserts |
| E | Range-query intensive | 95% scans and 5% inserts |
| F | Read-modify-write | 50% reads and 50% read-modify-writes |
Micro-benchmarks We develop a micro-benchmark to evaluate the performance of OSv and UCat in detail. The workload of this application is to allocate memory at first, update the memory, and release the memory finally. We can configure the number of memory allocations and the size of memory requested to generate different memory access intensity.
5.2 Validation of the address mapping mechanism
To validate the effectiveness of our front-end/back-end address mapping mechanism, we place the data in Memcached in DRAM and NVM with different ratios, and run memslap benchmark to evaluate the performance. We also evaluate the performance in DRAM-only and NVM-only scenarios for reference. We use UDP protocol for network transmission. We set the number of threads to 8, the concurrency to 128, and total execution time to 30 seconds. Fig.10 shows the transactions per second (TPS) under different DRAM-to-NVM allocation ratios, all normalized to the NVM-only configuration. When the proportion of data distributed in DRAM increases (from 1:16 to 1:1), the throughput of OSv becomes higher accordingly because the performance of DRAM is much higher than NVM. These results demonstrates that the heterogeneous memory resources are visible to the application, and our address mapping mechanism can map the virtual DRAM/NVM regions to different host namespaces effectively.
5.3 Performance of the multi-grained memory allocation
In the following, we conduct a set of experiments to evaluate the performance of our multi-grained memory allocation mechanism.
5.3.1 Execution time reduction
We compare UCat to OSv with the address mapping mechanism enabled (called OSv-plus) under different DRAM-to-NVM allocation ratios. For each experiment, we run the micro-benchmark to allocate/update/free the memory in different locations for 1 million times. At first, we compare UCat with OSv-plus using 4 KB pages. Fig.11 shows that our multi-grained memory management mechanism can significantly reduce the execution time by 38.8% on average compared with OSV-plus under the same DRAM-to-NVM allocation ratio. The reason is that using large pages can significantly reduce the TLB miss rate and the number of memory accesses due to page table walking. On the other hand, compared with OSv’s memory allocation mechanism, our multi-grained memory allocation mechanism also reduces the latency of memory allocation because we have more free slabs available in each slab class for allocation, and do not allocate memory from the free_ page_ranges frequently. We also compare UCat with OSv-plus using 2 MB large pages. We find that UCat show similar performance with OSv-plus for all cases because the two memory allocation schemes can fully exploit the benefit of large pages for accelerating address translations. However, OSv-plus using only 2 MB large pages wastes a large amount of memory space, as shown in the following section. These results demonstrate the efficiency of our multi-grained memory management mechanism.
5.3.2 Memory saving
In this section, we construct a micro-benchmark to illustrate the effectiveness of our memory management mechanism in memory saving. The micro-benchmark allocates 1300 B memory blocks for 5.12 million times. For memory requests larger than 1 KB but smaller than or equal to 4 KB, OSv’ memory allocator directly allocates an entire 4 KB page. Therefore, for each memory request, OSv will consume 4KB memory. Consequently, 5.12 million memory requests need 19.5 GB memory. In our experiments, OSv actually consumes total 19.7 GB memory including 0.2 GB memory used for the kernel and application codes. In contrast, UCat only needs 9.8 GB memory to run the application. Ideally, if we do not consider the memory internal fragmentation, only 6.20 GB memory is required for the application. Tab.3 shows the memory consumed and wasted by OSv and UCat compared with the ideal condition. Our mechanism significantly reduces the memory consumption by 50% compared with OSv, and reduces the memory waste by (13.5 GB vs. 3.6 GB). The reason is that UCat can split large pages into more fine-grained sizes and return the slab that best fits the memory request, and thus significantly reduce the memory consumption.
Tab.3 Memory consumed and wasted compared with the ideal condition |
| Mechanism | Memory consumed | Memory wasted |
|---|---|---|
| OSv | 19.7 GB | 13.5 GB |
| UCat | 9.8 GB | 3.6 GB |
5.3.3 TLB miss rate
We run memslap and YCSB to evaluate the TLB miss rate of Memcached in OSv. We exploit the performance profiling tool–perf in the host Linux to count the TLB miss rate of OSv. For memslap, we set the number of threads to 4, and the concurrency to 256, and the total execution time to 30 seconds. For YCSB, we use workload A and set the field length to 128. Fig.12 shows that UCat reduces the TLB miss rate of memslap and YCSB by 41% and 28% compared with OSv, respectively. Because one TLB entry can cover a much wider address space when using large pages, and thus significantly reduces the TLB miss rates of applications.
5.3.4 Memslap evaluation
To compare the memory management mechanism of UCat with that of OSv, we run the memslap benchmark using both TCP and UDP communication protocols. For fairness, we run both UCat and the vanilla OSv only using DRAM. We set different combinations of parameters in our experiments. Fig.13 shows the TPS for different values of thread and concurrency, all normalized to OSv at the same settings. For example, the numbers “4+64” in the X-axis represent that the number of threads and concurrency are 6 and 64, respectively. For the UDP communication, UCat improves TPS by up to 18.5%, and 15.4% on average. The throughput for the TCP communication is a little lower than the UDP communication when the number of threads becomes larger. Overall, the workload throughput of UCat is always better than that of OSv for each configuration. The performance improvement is mainly attributed to using large pages for address translations. They not only reduce the TLB miss rate, but also reduce the total memory references due to page table walking in virtualization environments.
5.3.5 YCSB evaluation
We also evaluate the performance of UCat and OSv by running six YCSB benchmark with different field-length parameters. The data requests of workload A, B, C, E, and F all follow a zipfian distribution, while workload D follows a latest distribution. Similarly, we run both UCat and the vanilla OSv only using DRAM for fairness. Fig.14 shows the throughput improved by UCat, all relative to the OSv’s vanilla memory management mechanism for the same field-length setting. UCat improves the workload throughput by up to 14.9%. For different KV sizes reflected by the field-length parameter, the performance of different workloads is similar. This implies that our multi-grained memory management mechanism is able to fully take the advantage of large pages for accelerating address translations, regardless of the sizes of memory allocations.
6 Related work
In recent years, there have emerged a few unikernel projects. Rumprun [36], OSv [3], and HermiTux [4] mainly focus on the compatibility for legacy applications. Rumprun is based on the NetBSD rump kernel and can run existing unmodified POSIX compliant software as a unikernel. OSv is designed for cloud applications. It is able to run unmodified Linux applications, however, the application source code should be recompiled as a relocatable shared object. HermiTux is also able to run unmodified Linux binaries as unikernels. ClickOS [6] is a high-performance software middle-box particularly designed for network function virtualization. However, these unikernels all require applications’ source code available. Otherwise, the applications are unable to recompile and re-link to these unikernels.
Some unikernels only support some special programming languages. MirageOS [2] is the first unikernel written in OCaml. It is designed for secure, high-performance network applications. HaLVM [37] is a port of the Glasgow Haskell compiler tools suite that enables Haskell programmers to develop unikernel-based virtual machines. Clive [38] is a unikernel implemented in Golang language, and applications in Clive should be compiled with a special compiler. These unikernel projects do not support most popular programming languages such as C/C++/java, and thus limit their applicability for cloud application deployment.
There have been a few recent studies on heterogeneous memory management for virtualization environments. HMvisor [39] exploits a NUMA abstraction to construct two virtual NUMA nodes in VMs, and dynamically maps the virtual NUMA nodes to the physical NUMA nodes managed by the KVM hypervisor. In this way, KVM exposes the underlying heterogeneous memory resources to up-layer VMs in a simple-yet-efficient way. Unfortunately, it is very difficult to support the NUMA mechanism in OSv. HeteroOS [40] exploit VM-level memory access behaviors to guide data placement on DRAM/NVM heterogeneous memories in virtualization environments. RAMinate [41] and HeteroVisor [42] support dynamic heterogeneous memory management for VMs. They achieve page migration in the hypervisor layer, without exposing the memory heterogeneity to VMs. However, they need to suspend VMs during page migrations because VMs are not aware of the on-the-fly pages. The above studies all focus on supporting heterogeneous memory management for VMs. To the best of our knowledge, there has been little work on heterogeneous memory management for unikernels. UCat is the first to support heterogeneous memories and multi-grained memory allocation in unikernels.
7 Conclusion and future work
In this paper, we present UCat, a heterogeneous memory management system for unikernels. We implement UCat based on a popular unikernel–OSv. First, we propose front-end/back-end cooperative address mapping to expose the host memory heterogeneity to unikernels. Second, we implement a multi-grained memory allocation mechanism that takes the advantage of both large pages and slab allocations. It not only reduces the cost of two-layer address translation in virtualization environments, but also reduces memory waste due to internal memory fragmentation. Our experimental results demonstrates the effectiveness and efficiency of our mechanisms for heterogeneous memory management in unikernels. The proposed mechanisms are also applicable to other virtualized appliance such as VMs, containers.
In the future, we will optimize our prototype to support more flexible slab memory allocations, allowing the size of slabs and the number of slab classes adapt to application's memory access behaviors. Moreover, we will enable fast in-memory checkpointing (zero-copy) for unikernels by exploiting the non-volatility of persistent memory. This functionality for unikernels may incite new research opportunities such as Function-as-a-Service (FaaS) for the future cloud computing paradigm.