Understanding vTopology in vSphere 8: A Deep Dive into NUMA and vNUMA Management

by Davoud Teimouri · Published 06/12/2024 · Updated 05/12/2024

With its state-of-the-art innovations, VMware vSphere continues to lead the virtualization ecosystem, and vSphere 8 brings major improvements to vTopology, a framework designed to optimize NUMA (Non-Uniform Memory Access) and vNUMA (Virtual Non-Uniform Memory Access) configurations for modern workloads. Effectively managing NUMA and vNUMA is essential in virtualized environments to maximize performance for CPU- and memory-intensive applications. In this extensive blog post, we’ll examine the development of vTopology in vSphere 8, compare it to previous iterations, and offer practical insights into optimizing NUMA and vNUMA configurations. Let’s also highlight important features, best practices, and real-world scenarios to help you better understand.

On This Post:

What is vTopology in vSphere?

vTopology is a virtualization framework in vSphere designed to map virtual machine (VM) resources such as CPU and memory effectively to the underlying physical hardware. Its primary goal is to:

Align the virtual NUMA topology (vNUMA) with the physical NUMA topology.
Enhance workload performance by reducing latency and ensuring memory locality.
Provide administrators with granular control over NUMA-aware applications, such as databases, big data analytics platforms, and enterprise ERP systems.

Overview of NUMA and vNUMA Concepts

Before diving into vTopology in vSphere 8, let’s revisit NUMA and vNUMA concepts:

NUMA (Non-Uniform Memory Access)

NUMA architecture divides a physical server into multiple NUMA nodes, each comprising a subset of the server’s processors and memory. This structure reduces memory access latency for processes bound to the same NUMA node.

vNUMA (Virtual NUMA)

When large VMs (typically with more than 8 vCPUs) are created, vSphere emulates NUMA nodes within the VM to optimize resource allocation. vNUMA is critical for:

NUMA-aware applications: Applications that can take advantage of NUMA locality to boost performance.
Resource-heavy workloads: Ensuring that CPU and memory access patterns are efficient.

In earlier versions of vSphere, configuring and optimizing vNUMA required a solid understanding of NUMA topology and manual tuning for specific workloads.

vTopology in vSphere 8: Key Enhancements

Dynamic vNUMA Adjustment

In vSphere 8, dynamic vNUMA adjustment enables automatic reconfiguration of vNUMA topology based on VM resource changes. For example, if you hot-add CPUs or memory, the vNUMA topology updates dynamically to match the new configuration.

Enhanced NUMA-Aware Scheduling

The vSphere Distributed Resource Scheduler (DRS) in vSphere 8 has been improved to be more NUMA-aware. It considers memory locality and CPU utilization when placing workloads across hosts.

Support for Complex Workloads

Modern workloads, such as machine learning, artificial intelligence (AI), and containerized applications, often have non-linear resource demands. The enhanced vTopology in vSphere 8 accommodates these complexities by providing:

Better mapping of vNUMA nodes to physical NUMA nodes.
Advanced tuning options for latency-sensitive applications.

NUMA and CXL Integration

With Compute Express Link (CXL) emerging as a standard for memory expansion, vSphere 8 integrates vTopology capabilities to manage CXL memory pools effectively. This ensures future proofing for workloads requiring disaggregated memory.

Comparison with Earlier vSphere Versions

vSphere 6.x

Static vNUMA Configuration: Administrators had to configure vNUMA manually, which was challenging for dynamic environments.
Limited Hot-Add Support: Adding vCPUs or memory to a VM often required a reboot to realign vNUMA.
Basic NUMA-Aware Scheduling: DRS had basic NUMA-awareness but struggled with highly dynamic workloads.

vSphere 7.x

Improved NUMA Scheduling: vSphere 7 introduced better integration of NUMA-awareness into DRS.
Partial Dynamic Adjustments: While vNUMA could adapt to some resource changes, limitations remained for hot-added resources.
Enhanced Monitoring: Administrators gained improved visibility into NUMA and vNUMA configurations through the vSphere Client.

vSphere 8

Fully Dynamic vNUMA: No manual intervention needed for resource changes.
Advanced NUMA Placement: Better handling of complex workloads, such as those in hybrid and multi-cloud environments.
Unified Management: vSphere 8 integrates NUMA and vNUMA management into the enhanced vSphere Client for easier configuration and monitoring.

Real-World Use Cases for vTopology Optimization

Database Performance Optimization
- NUMA-aware databases like Oracle and SQL Server benefit from vTopology improvements, ensuring efficient CPU and memory usage.
AI/ML Workloads
- Machine learning frameworks, such as TensorFlow and PyTorch, leverage optimized NUMA configurations for faster training and inference.
High-Performance Computing (HPC)
- HPC workloads with strict latency requirements thrive with vSphere 8’s NUMA-aware scheduling.

Best Practices for NUMA and vNUMA Management

Understand Your Workload
- Determine if your application is NUMA-aware and how it accesses CPU and memory.
Right-Size VMs
- Avoid creating oversized VMs that span multiple NUMA nodes unnecessarily, as this can lead to performance penalties.
Leverage Dynamic Features
- Use the dynamic vNUMA adjustments in vSphere 8 for environments with frequently changing resource demands.
Monitor Performance
- Regularly monitor CPU and memory performance to ensure optimal NUMA alignment using tools like esxtop or the vSphere Client.
Avoid Overcommitment
- Avoid overcommitting resources on NUMA nodes, especially for latency-sensitive applications.

Best Practices for Configuring Sockets and Cores in vSphere

Configuring virtual machine (VM) sockets and cores efficiently is critical for achieving optimal performance, particularly in environments with NUMA and vNUMA considerations. With the advancements in vTopology in vSphere 8, the approach to socket and core configuration has evolved, offering better alignment with NUMA nodes and enabling administrators to maximize workload performance.

Why Sockets and Cores Configuration Matters

The way you configure sockets and cores affects how VMs interact with the underlying physical hardware. Key factors include:

NUMA Node Mapping: Ensures memory locality and reduces latency.
CPU Scheduling: Impacts how the hypervisor allocates CPU resources.
Licensing: Many software licenses are based on the number of sockets, so configuration affects costs.

vSphere 8: Sockets and Cores with vTopology

With vTopology in vSphere 8, administrators have a simplified way to ensure VM configurations align optimally with NUMA nodes, thanks to dynamic vNUMA adjustments and enhanced NUMA-aware scheduling.

Best Practices with vTopology in vSphere 8:

Match Virtual Sockets to Physical NUMA Nodes
- Ensure the number of virtual sockets aligns with the number of physical NUMA nodes.
- Example: On a host with 2 NUMA nodes, configure the VM with 2 sockets if the workload is NUMA-aware.
Leverage Dynamic vNUMA
- When using vTopology, dynamic vNUMA automatically adjusts the vNUMA topology to match changes in CPU and memory resources. This eliminates the need for manual reconfiguration during hot-add operations.
Avoid Overloading NUMA Nodes
- Allocate vCPUs and memory to avoid spanning NUMA nodes unnecessarily.
- Use monitoring tools (e.g., esxtop) to verify NUMA node alignment.
Utilize High-Performance Mode
- Set VMs to High Performance power policy for latency-sensitive workloads, ensuring efficient CPU and memory usage.

Configuring Sockets and Cores Without vTopology in vSphere 8

If vTopology is not utilized, or you are in a more static environment, follow these practices:

Pre-Calculate NUMA Mapping
- Ensure the total vCPUs do not exceed the capacity of a single NUMA node unless the application benefits from spanning multiple nodes.
- Example: On a host with 8 cores per NUMA node, configure the VM with up to 8 vCPUs (1 socket x 8 cores) for optimal locality.
Static vNUMA Configuration
- For static workloads, manually set the vNUMA topology to align with the physical NUMA nodes. This is particularly useful for NUMA-aware applications like databases.
Avoid Over-Configuring Cores per Socket
- Configure a balance of sockets and cores that matches the application’s threading behavior and the underlying hardware topology.

Sockets and Cores in Earlier vSphere Versions (6.x and 7.x)

In earlier versions of vSphere, administrators had limited tools for managing sockets and cores, requiring more manual effort.

Best Practices for vSphere 6.x:

Static vNUMA Alignment
- vNUMA was not dynamic; configure virtual sockets to align manually with physical NUMA nodes.
Limit Core Density
- Avoid creating high core-per-socket densities (e.g., 1 socket x 16 cores) unless required by application licensing.
Use NUMA-Aware Scheduling
- Ensure workloads are NUMA-aware to benefit from memory locality and reduced latency.

Best Practices for vSphere 7.x:

Improved NUMA Scheduling
- vSphere 7 introduced better NUMA-aware scheduling, but dynamic vNUMA adjustments were limited.
Monitor NUMA Alignment
- Use vSphere Client to verify NUMA alignment during VM placement and resource adjustments.
Right-Size VM Configurations
- Right-sizing VMs to fit within NUMA boundaries was critical for performance. Use tools like esxtop for NUMA node monitoring.

Example Configurations for Small, Medium, and Large VMs with Different vSphere Versions and Sockets/Cores Configurations

Let’s explore practical examples for small, medium, and large virtual machines (VMs) using the server setup provided. The physical server has two sockets, each with 26 physical cores and 52 logical cores, and 512 GB of memory. Below, we’ll configure and optimize the small, medium, and large VMs under different vSphere versions, considering vTopology, NUMA, and vNUMA management.

Physical Server Setup:

2 Sockets
26 Physical Cores per Socket (52 Logical Cores total)
512 GB Memory

1. Small VM Configuration (6 Cores, 24 GB Memory)

vSphere 8 with vTopology

VM Configuration:
- vCPUs: 6 vCPUs (1 socket, 6 cores)
- Memory: 24 GB
Best Practices:
- vSphere 8’s dynamic vNUMA adjustment ensures that the VM’s 6 vCPUs and 24 GB of memory are allocated to the same NUMA node, reducing memory latency.
- The VM will fit within the NUMA node of one socket, so no NUMA spanning occurs, ensuring efficient resource utilization.

vSphere 8 without vTopology

VM Configuration:
- vCPUs: 6 vCPUs (1 socket, 6 cores)
- Memory: 24 GB
Best Practices:
- Even without vTopology, ensure that the small VM fits entirely within a single NUMA node (1 socket x 6 cores).
- Assign memory to match the same NUMA node, aligning memory and CPU for optimized performance.

vSphere 7.x

VM Configuration:
- vCPUs: 6 vCPUs (1 socket, 6 cores)
- Memory: 24 GB
Best Practices:
- Manually configure the VM to avoid NUMA spanning, aligning 6 vCPUs with the available cores on one socket and memory within the same NUMA node.
- vNUMA configuration is static in vSphere 7, so make sure no additional cores are added beyond the 6 vCPUs.

vSphere 6.x

VM Configuration:
- vCPUs: 6 vCPUs (1 socket, 6 cores)
- Memory: 24 GB
Best Practices:
- Manually configure vNUMA to ensure the 6 vCPUs stay on a single NUMA node, and avoid spanning across multiple nodes.
- Set vNUMA topology manually to align with physical NUMA nodes.

2. Medium VM Configuration (26 Cores, 128 GB Memory)

vSphere 8 with vTopology

VM Configuration:
- vCPUs: 26 vCPUs (1 socket, 26 cores)
- Memory: 128 GB
Best Practices:
- vSphere 8’s dynamic vNUMA will automatically map the VM’s 26 vCPUs and 128 GB of memory to a single NUMA node of the physical server (1 socket x 26 cores).
- This configuration ensures optimal resource utilization, avoiding any NUMA spanning and ensuring memory locality.

vSphere 8 without vTopology

VM Configuration:
- vCPUs: 26 vCPUs (1 socket, 26 cores)
- Memory: 128 GB
Best Practices:
- Even without vTopology, ensure the VM fits entirely within one NUMA node (1 socket x 26 cores), aligning memory and CPU on the same NUMA node.
- This ensures that the VM does not span NUMA nodes, which would result in inefficient memory access and increased latency.

vSphere 7.x

VM Configuration:
- vCPUs: 26 vCPUs (1 socket, 26 cores)
- Memory: 128 GB
Best Practices:
- Manually configure the vNUMA topology to align the 26 vCPUs with the physical NUMA node.
- The VM should fit within a single NUMA node (1 socket x 26 cores). This is particularly important since vSphere 7 still requires manual tuning to ensure NUMA alignment.

vSphere 6.x

VM Configuration:
- vCPUs: 26 vCPUs (1 socket, 26 cores)
- Memory: 128 GB
Best Practices:
- Similar to vSphere 7, you will need to manually configure vNUMA to prevent spanning across NUMA nodes.
- The VM should fit entirely within one NUMA node to optimize performance.

Large VM Configuration (48 Cores, 256 GB Memory)

vSphere 8 with vTopology

VM Configuration:
- vCPUs: 48 vCPUs (2 sockets, 24 cores per socket)
- Memory: 256 GB
Best Practices:
- With vTopology, the VM’s 48 vCPUs will be dynamically adjusted across both physical NUMA nodes, each NUMA node receiving 24 vCPUs and 128 GB of memory.
- Dynamic vNUMA adjustment ensures efficient mapping of vNUMA nodes to physical NUMA nodes, improving CPU and memory locality without manual intervention.

vSphere 8 without vTopology

VM Configuration:
- vCPUs: 48 vCPUs (2 sockets, 24 cores per socket)
- Memory: 256 GB
Best Practices:
- Without vTopology, ensure that 48 vCPUs are distributed across 2 NUMA nodes (2 sockets x 24 cores). Each socket should be aligned with the VM’s vNUMA configuration to maximize memory locality and avoid NUMA spanning.
- In this scenario, static vNUMA mapping would be essential to ensure optimal performance, ensuring no CPU or memory is accessed across NUMA boundaries.

vSphere 7.x

VM Configuration:
- vCPUs: 48 vCPUs (2 sockets, 24 cores per socket)
- Memory: 256 GB
Best Practices:
- Similar to vSphere 8 without vTopology, you will need to manually configure the VM’s 48 vCPUs and 256 GB memory to ensure proper NUMA alignment.
- Allocate 24 vCPUs and 128 GB memory to each NUMA node, ensuring no memory or CPU is stretched across the two nodes.

vSphere 6.x

VM Configuration:
- vCPUs: 48 vCPUs (2 sockets, 24 cores per socket)
- Memory: 256 GB
Best Practices:
- In vSphere 6.x, you will need to configure the VM manually to ensure that the 48 vCPUs are distributed across the 2 physical NUMA nodes, with each socket allocated 24 vCPUs and 128 GB of memory.
- Ensure the manual vNUMA mapping aligns with the physical hardware, as vSphere 6.x lacks dynamic adjustments.

These examples illustrate how to configure small, medium, and large VMs effectively for different vSphere versions and NUMA configurations. The key takeaway is that, with vTopology in vSphere 8, dynamic adjustments and automatic vNUMA mapping make it much easier to optimize resource allocation without manual intervention. However, in earlier vSphere versions (6.x and 7.x), a more hands-on approach is required to ensure that virtual machines are aligned with NUMA boundaries for maximum performance.

By following these best practices and using the proper socket and core configurations for each VM size, administrators can ensure their virtualized workloads are optimized for CPU and memory performance across different vSphere environments.

Let me know if the configurations are not correct or you have bigger virtual machines or physical hosts and you have to choose different configurations for achieve best performance.

Common Challenges and How to Overcome Them

Challenge: NUMA Node Spanning

Solution: Limit NUMA node spanning unless the application explicitly benefits from it. Use vSphere Client to configure vNUMA boundaries effectively.

Challenge: Underutilization of NUMA Nodes

Solution: Right-size VMs and monitor node usage to distribute workloads evenly across NUMA nodes.

Future Outlook of vTopology in vSphere

As virtualization technologies evolve, the importance of NUMA and vNUMA optimization will grow, especially with emerging trends like CXL, persistent memory, and disaggregated architectures. VMware’s continued enhancements to vTopology ensure that vSphere remains a robust platform for modern workloads.

FAQs on NUMA, vNUMA, and vTopology

Q1: What is the key advantage of vTopology in vSphere 8?

A: The dynamic adjustment of vNUMA topology ensures seamless performance optimization without manual intervention.

Q2: Can vNUMA benefit non-NUMA-aware applications?

A: While NUMA-aware applications benefit the most, efficient resource allocation through vNUMA can indirectly improve performance for other workloads.

Conclusion

vTopology in vSphere 8 represents a leap forward in managing NUMA and vNUMA, offering dynamic capabilities and advanced features for modern workloads. By understanding its enhancements and adopting best practices, administrators can maximize resource utilization, reduce latency, and enhance application performance in virtualized environments.

Stay tuned for more deep dives into VMware vSphere and other cutting-edge virtualization technologies! If you have questions or insights about vTopology, leave a comment below.