SmartOS, One of Best Live Operating Systems for Virtualization

by Davoud Teimouri · Published 12/02/2023 · Updated 10/06/2023

SmartOS is a live OS. What does it mean? SmartOS always boot from removable media and network. There is no need to use disk capacity for itself.

SmartOS is a lightweight operating system which made for virtualization based on OpenSolaris and Illumos. It combines bhyve and KVM virtualization.

On This Post:

SmartOS and Virtualization

SmartOS is a specialized Type 1 Hypervisor platform based on illumos. It supports two types of virtualization:

OS Virtual Machines (Zones): A light-weight virtualization solution offering a complete and secure userland environment on a single global kernel, offering true bare metal performance and all the features illumos has, namely dynamic introspection via DTrace
Hardware Virtual Machines (KVM, Bhyve): A full virtualization solution for running a variety of guest OS’s including Linux, Windows, *BSD, Plan9 and more

SmartOS is a “live OS”, it is always booted via PXE, ISO, or USB Key and runs entirely from memory, allowing the local disks to be used entirely for hosting virtual machines without wasting disks for the root OS. This architecture has a variety of advantages including increased security, no need for patching, fast upgrades and recovery.

Virtualization in SmartOS builds on top of the foundational illumos technologies inherited from OpenSolaris, namely:

ZFS for storage virtualization
Crossbow (dladm) for network virtualization
Zones for virtualization and containment
DTrace for introspection
SMF for service management
RBAC/BSM for auditing and role based security
And more

Hypervisor on USB Drive

However, SmartOS is typically “installed” by downloading and copying the OS image onto a USB key and then booting that key. On the first boot a configuration utility will configure your base networking, allow you to set the root password, and allow you to select which disks to use to create the ZFS Zpool which will provide persistent storage.

When you log into SmartOS you will enter the “global zone”. From here you can download VM Images using the imgadm tool, which are pre-configured OS and KVM virtual machines. You can then use the vmadm tool to create and manage both OS and hardware virtual machines.

An important aspect of SmartOS is that both OS (Zones) and hardware virtual machines are both built on Zones technology. In the case of OS virtualization, the guest virtual machine is provided with a complete userland environment on which to run applications directly. In the case of HVM virtualization, the qemu or bhyve process will run within a stripped down Zone. This offers a variety of advantages for administration, including a common method for managing resource controls, network interfaces, and administration. It also provides HVM guests with an additional layer of security and isolation not offered by other virtualization platforms.

Finally, instances are described in JSON. Both administrative tools, imgadm and vmadm, accept and return all data in JSON format. This provides a simple, consistent, and programmatic interface for creating and managing VM’s.

SmartOS is a fundamental component of the Triton Data Center (Triton) product.

SmartOS Virtualization Technologies

Its virtualization technologies are different from other virtualization technologies.

OS virtualization is great for applications that can run natively in SmartOS, but many of us have legacy apps that must run in Windows or Linux, eg for Windows, Active Directory, Windows video encoding. What can we do for them? Joyent uses hosted virtualization for its virtual machine solution.

Remember that when you lose the Host OS, you lose any valuable capabilities it provided. In the case of Vmware, this came down to device drivers. Since they could embed these in the hypervisor, bare-metal HW virtualization made more sense.

SmartOS provides two types of hardware virtualization: KVM and Bhyve. Each provides hardware emulation and is a VMM (virtual machine monitor). Notice that it is not a strict hypervisor layer. Each HVM is independent — there is one per Guest OS instead of one supporting several VMs. This is because it has combined both the VM and the VMM.

SmartOS Zones

Inspired by FreeBSD jails, the fundamental technology behind a SmartMachine is the concept of zones. A zone is a virtualized instance of SmartOS that behaves like an isolated system even when functioning along side other zones on the same machine. Each zone on a system shares a pool of resources and the single operating system kernel. However, zones are never aware of other zones on the system and are process secure. A zone is similar to a virtual machine, but is distinct in that it shares the base system kernel, whereas each virtual machine runs its own OS kernel. Zones are an inherent part of the operating system and impose no additional overhead. Each process that runs includes the zone ID as an attribute. Thus, zones scale and perform better than virtual machines since there no additional kernel or layering involved.

Bhyve

SmartOS includes the Bhyve virtual machine manager, originally ported from FreeBSD. Bhyve supports most Intel and AMD processors for running hardware based virtual machines.

It offers a number of advantages over KVM, among them:

Better tracking of, and integration wth, upstream FreeBSD
Higher performance for CPU, and I/O operations (including disk and network I/O).
Lower overhead, resulting in lower host CPU utilization while guests are idle.

Bhyve is fully supported and production ready.