Featured

Sheaves

Sheaves were added to the kernel.

Sheaves are a new layer for caching objects via per-cpu arrays.

To understand Sheaves, you need to understand a bit about how the kernel allocates memory:

(round trip through memory lane LOL)

SLAB: Used for small objects (most likely, smaller than a page)

Slab works by taking one page or more and dividing it into (smaller) object slots. On request, a slot in the "slab" can be used to fulfill the allocation request.

Historically, there was multiple Slab implementations: Slab (the original), Slob (for embedded devices) and Slub (Latest).

Now, only the Slub implementation remains, so when you ear Slab and Slub, they are being used interchangeably.

Slub was added to the kernel in 2.6.22 (2007).^[14] It brought new features like proper NUMA policy support, and it fixed some of the issues of the original Slab that got a 5-10% performance increase along with a reduction 50% of Slab cache being used by the system.^[15]

Lets get back to the subject: Sheaves are seeking to solve the relative un-optimized nature of Slub under some workload.

Slub has this issue where, on a system with a lot of cores, there might be considerable contentions around the locks needed by the implementation.

I'm not about to go into a full Slub breakdown but the bulk of the issue is that Slub will often require you to hold a lock that is shared across multiple core.

Examples for things that might get you to require a lock:

Freeing 1 object from a slab.
Adding 1 object to a slab.
Needing another slab.

emphasis on the might...^[16]

Sheaves address this by reducing the need for a lock to a minimum and taking advantage of preallocation.

So, for my fellow non native-english folks, Sheaves are bundles of stalks,of plant.

Now lets get into how they work:

Take a CPU trying to get a object of 256B, it will be able to get an object from the core specific sheave.

This sheave has a set numbers of object within it.

Whenever it is empty (doesn't have any other object to give) The core should have access to a backup sheave from where it can source more objects.

If, at any time, an object is freed, it will be put back into the sheave.

For everything that we talked about, there was no need to have a lock, thus, no need to block/wait any other core.

If both sheaves on the core are empty, you can request another sheaves from a barn. This barn is shared across multiple core thus requires a lock to access.

Once the new sheave is retrieve, no lock is required, and allocations resumes locally.

Here i also want to highlight the preallocation feature. If you know the average number of object used for a multicore process along with the worst case scenario number, you could make sure to have enough storage for the average on the on core sheaves, and enough for worst case in the barn.

Big gains could be seen if it is implement at larger scale through the kernel.

Some benchmarks were done showing significant gain along with some regression.^[17]

For now, sheaves are an Opt-In feature, so... we will see where it goes!^[18]

Bonwick's Magazines^[50]

---

AMD [[AMD/Terms#Secure Encrypted Virtualization (SEV)|SEV-SNP]]'s CipherText Hiding

Support for AMD SEV-SNP's CipherText Hiding.

https://relocatevote.org/

Support for KVM x86 CET virtualization.

Control-flow Enforcement Technology (CET) is comprised off 2 features: Shadow Stacks (SHSTK) and Indirect Branch Tracking (IBT). They can be used to provide Control-flow integrity (CFI) to the guests.

The goal here is to prevent an attacker from redirect the control-flow to an unauthorized target.

Shadow Stacks (SHSTK) is a feature that creates a copy of the stack and, when returning, compares the running return address on top of the stack with the copy kept in the shadow stack. This feature is implemented by both Intel and AMD

Indirect Branch Tracking (IBT) is a feature that requires a special ENDBRANCH instruction at the target of an Indirect Branch (Think jmp %rax). If the target of an Indirect Branch doesn't have the instruction, the CPU will generate an exception (#CP fault). This feature is only implemented by Intel and will be disabled on AMD. ^[31]

CPU

X86

Intel

Retpoline Optimization for FineIBT (Intel)^[1]

Retpoline when added to assembly should look like :

cs call __x86_indirect_thunk_r11

This will be replaced by:

call *r11
nop3

The nop3 exist to keep alignment, if cs call __x86_indirect_thunk_r11 creates a 5-byte wide hole, it needs to be filled even if call *r11 only takes 2 byte. The nop3 will fill the 3 missing byte but will also take 1 CPU cycle to execute.

This can be optimized by using another method called CS Padding. Using the CS Segment Override Prefix (0x2E) we can fill these missing byte without wasting any cycle:

cs cs cs call *r11

There is just one problem with this, Not all CPU can process the CS Prefix without wasting cycle (Like Atom/E-cores). Worst than that, it is longer than a nop.

The middle ground found was to allow up to 3 CS Prefix. If more are needed, a nop will be used in order to minimize the tax on ill-equipped CPU.^[2]^[3]

The Intel Trust Domain Extensions (TDX) now supports Kexec.

Kexec is a mechanism of the kernel to load and boot a new kernel from a currently running one. This allows us to skip the bootloader phase and hardware initialization phase.

Kexec didn't work with TDX as the memory encryption hardware didn't respect cache coherency (the ability to keep a consistent value along multiple cache). This would allow it to write back old cachelines on top of data in the new kernel.

Changes were made to make sure this couldn't be an issue, but Kexec will still be unable to boot a TDX enable kernel (only to boot from it). Excluded from this fix are early Intel Xeon Sapphire Rapids processors (2023) as they have a bug preventing the fix from being effective.^[4]

Fix for Intel Xeon 6 PCIE Performance Issues.

Xeon 6 have an issue where if a PCIE is bifurcated (split) into a 2x, it becomes sensible some of its configurations.(will have perf issues if not set correctly) The fix configs:

Extended Tag => Disabled

Allows the PCIE device to handle more transaction at a time, from 32 (Disabled) to 256 (Enabled).

10-Bit Tag Requester => Disabled

Same but instead of 8-Bit we got 10, giving us 1024 transaction. Some work was done in 2021 to try to add the feature to the kernel but was never merged.^[6]

Max_Read_Request_Size (MRRS) => 0 (128B)

The size of the request.

https://cdrdv2.intel.com/v1/dl/getContent/837176

The fix now disallows any incorrect configuration.^[7]

-Removed support for FRED on some pre-production CPUs ^[13]

-Microcode baseline updated ^[23]

-Intel power slide ^[24]

AMD

New AMD Versal TRNG (TrueRNG) Driver. Used in the AMD Versal Adaptive SoCs which is a weird mix of CPU and FPGA so a somewhat programmable silicon. These chips are possible because of AMD's acquisition of Xilinx in 2022 which specialized in these type of chips.^[8] (used for networking/AI...)

We also have Error Detection And Correction (EDAC) for AMD Versal NET SoCs.^[9]

Still in Error Detection And Correction (EDAC), AMD multiple model to their Family 26 line which gives us a look into the future for AMD, We find code for 16-channel of memory^[11], which will allow more performance and capacity. Not long after, Intel confirmed that they would cancel their Diamond Rapids CPU (2026) with 8-channel, leaving only the 16-channel option.^[10]

Good old AMD Bulldozer (2011) sees a patch!

for the young folks: Back in the early 2010s, AMD was struggling, they were behind in every way to Intel and were bleeding out cash. The decided to go on a full redesign and they came up with the Bulldozer. Now it was a failure and it took AMD to the brink of bankruptcy... but that's all before they struck gold with Ryzen and became the comeback story we know and love.

Now for the patch: Bulldozer added a few instruction sets:

ABM, XOP, FMA4 and F16C

Remember in 6.16 when "X86_NATIVE_CPU" was added to compile the kernel with "-march=native" to optimize to the running CPU? It turns out the XOP instruction set isn't handled correctly by the kernel and would result in a bunch of errors when compiling using the old CPU.

The support for decoding the instruction was added and this fixed the problem.^[12]

Loongarch

Support for the Loongson Security Engine chip. The chip allows offloading random number generation (RNG), trusted platform module (TPM2) and more...^[5]

Risc-V

Support for MIPS vendor extensions. By MIPS vendor extensions, i'm talking about instructions added to a RISC-V CPU that is TradeMarked by MIPS (the corp). So not old instructions, but for a new CPU made by MIPS, the P8700.^[22]

Other

cpa_flush(), The code for flushing the translation lookaside buffer (TLB) as part of the Change Page Attribute (CPA), will now use the flush_kernel_range() function directly. This will allow cpa_flush() to access optimized instructions like INVLPGB or RAR. (which gave great improvement in their implementation, see benchmark^[19]) ^[20]

Attack Vector Controls (added last release) Can Now Manage VMSCAPE Mitigation (which is another branch predictor issue XD). it is with the guest-to-host attack vectors.^[21]

Crypto

Optimization using the new 2-way interleaved SHA-256 hashing support

SHA-256 is a serialized algorithm, It works by processing 64-Bytes blocks, that requires the 32-bit output of the prior block.

Modern CPUs have the capability of processing multiple hash at once in parallel, but there is no way for us to apply multiple execution unit to one single hash.

The optimization comes from recognizing that we have more than one hash to process, and to feed the execution unit a second hash while waiting for the first one to finish.

This optimization yields throughput gains of up to 98% on X86^[25] and 70% on ARM^[26].

It is currently applied to fs-verity (a feature to check the integrity of a read-only file) and a patch for dm-verity (which is fs-verity at the block device level) is scheduled for next release.

Important to note that the patch only extend for up to 2 execution units, some CPU can have more!^[27]

VM

Support for AMD Secure AVIC.

Before getting the the new feature we need to understand a bit about how interrupt are handled in the CPU.

For, a long time now, CPUs use x2APIC to store and manage interrupts. (APIC stands for: Advanced Programmable Interrupt Controller). This is true for VMs as well where the host will emulate an x2APIC for the guest.

This is slow, so AMD created the Advanced Virtual Interrupt Controller (AVIC) which brings hardware acceleration to VMs. (KVM now enables AVIC by default for Zen 4 and newer, so this is still getting some work)...

Secure AVIC seeks to be the next evolution, providing a solution for confidential VMs, inline with AMD's expansion of features linked to Secure Encrypted Virtualization (SEV-SNP).

The Secure AVIC instance is owned and managed by the guest, which prevents a lot of possible attack pathway (like Interrupt Injection) while still being hardware accelerated.^[32]

This has been in the works for some time (2023) there was a presentation at the Linux Plumbers Conference about it. ^[33]

https://www.youtube.com/watch?v=0LGqYAS2u5w

While we are talking about new SEV-SNP features, lets talk about the new Secure TSC support.^[35](if this sound familiar, it is because we covered it with 6.14 but it ultimately removed from the final release)...

Detection of FreeBSD's Bhyve Hypervisor. Bhyve detection used to be only useful for printing 'Hypervisor detected: Bhyve' at boot time, but now as Bhyve rolls out support for more than 255 vCPUs^[29] and detecting the 15-bit MSI is needed to have proper support.^[30]

While we are talking about more than 255 vCPUs, a bug affecting AMD EPYC-Genoa KVM/QEMU guests was fixed. This bug would misread topology information for 255 vCPUs or more.^[28]

KVM on RISC-V now supports the SBI FWFT extension, Zicbop and bfloat16 extensions for guests/VMs. We covered theses instructions when they were added to the kernel with 6.16.^[35]

Virtual Function I/O (VFIO) was saved from removal from the kernel. 2 Devs (Mostafa Saleh and Pranjal Shrivastava) took up the role of Maintainer for the vfio-platform driver. Support for the NVIDIA GB300 was added along with more small fixes. ^[36]

-The Microsoft Hyper-V code for Linux 6.18 has now unified its guest entry code for both KVM and Microsoft Hyper-V (MSHV).

-A new feature for Hyper-V users is Linux 6.18 now supporting Kexec and Kdump from Azure Confidential VMs (CVMs)

-Microsoft has also now formally deprecated its Hyper-V frame-buffer "hyperv_fb" driver in favor of the Hyper-V Direct Rendering Manager (DRM) driver.^[34]

Graphic

Tyr as a new Rust DRM driver for Arm Mali GPUs and eventually aims to be a replacement to the Panthor driver.^[43]

The code that was added to the kernel is only a skeleton, but the devs behind it have managed to run Gnome and SuperTuxKart on a testing build.^[44]

-The new Rocket accelerator driver for the NPU found on newer Rockchip SoCs.

-better AMD GCN 1.0 support in the AMDGPU driver.^[37]

-Intel SLPC power_profile support.^[38]

-Intel Wildcat Lake display enabling.^[39]

-Intel is back to working on the Habana Labs accelerator driver.^[40]

-Nouveau now defaults to using the NVIDIA GSP firmware.^[41]

-Panthor DRM driver support for the Arm Mali G710, G510, G310, Gx15, Gx20, Gx25 GPUs.

-Various other DRM driver improvements.^[42]

FS

NEW DM-PCACHE aims to be used in front of slower block devices by providing this high-throughput, low-latency cache. When compared to the other options in the kernel (BCache and DM-writecache) we can see a higher throughput than BCache while maintaining the low latency of DM-writecache.^[46]

-Linear RAID "md-linear" To Support Atomic Writes^[49]

-Fix Lockups When Systemd Units Read Lots Of Files ^[48]

-Kbuild Brings An Optimization For gen_init_cpio On Btrfs Or XFS^[47]

-Block Code Introduces Lockless Bitmap For Software RAID^[45]