FRED is a New feature for Event Delivery (Event meaning Interrupts, Exceptions, Syscalls).^[3]

added with Linux 6.9 (2024)

some issues with pre-production hardware

With Linux 7.1, FRED is now enabled by default for compatible CPUs.

For now, this means only Intel Panther Lake but we will soon see Intel Diamond Rapids and EVEN AMD Zen 6 because yes, FRED which initially was an Intel exclusive, will be brought to AMD.

You can thank The x86 Ecosystem Advisory Group for that, which contains Linus BTW.

FRED seeks to fix the issues with the current way of handling Interrupts, Exceptions and Syscalls because, on X86, they are broken:^[6]

Before getting in FRED we must understand how the previous system works:

Since the very start of X86, back when we were talking about 16-bit CPUs, We have been using the IDT to handle Interrupts. The IDT is a table, so an array of pointer, that is used to determine the address of the handler for a specific interrupt.

Some are hardware enforced and others can be changed by the software. https://wiki.osdev.org/Interrupt_Descriptor_Table

Lets go step by step into what happens when an interrupt is executed:

Now, quick reminder, The stack is the #1 way for the CPU to keep data. So pushing into the stack adds data to memory while popping out of the stack removes from memory.

Here is the information that is pushed onto the stack:

This structure is called the IRET frame, because it is used by the IRET instructions to return from the interrupt.

Before going forward, you need to understand that under a 64-bit machine running 64-bit software, A lot of the registers being saved are useless. For example, the only use of the SS Register is to prevent the CPU from running a Kernel-Mode Stack with User-Mode. This leaves most of the bits being saved to be wasted space.

Now if that doesn't sound bad, lets take the example of a Syscall.

Syscalls would work in a similar way but they don't have the advantage of the CPU constructing the IRET frame for it. This means the kernel, had to save, Not only the registers that it fears loosing but also the IRET frame in software.

This is very slow but was the only way to keep compatibility, keep in mind the same idea exist since X86's creation.

FRED works by changing the behavior of the CPU when it comes to Events:

Lets do the same exercise we did with IDT but with the new FRED feature.

A Syscall is executed:

This is the time where you might say: "Maple, Besides the padding and the Event Data..... Nothing changed????"

But this is where the Intel Engineer are being very smart.

Remember the space that was taken by useless information in the saved registers? We now use that space to store addition information.

FRED being a feature that is only available for IA_32e Long Mode (64-bit) makes it so we can assume most of the bits within the SS and CS registers. We also see a new concept appear:

Stack levels are the solution that FRED uses in order to make sure that we should be running in Kernel-mode or User-Mode.

Keep in mind that Stack Level doesn't equal in any way to Privilege level (ring 0...3).

Once an interrupt is executed, FRED will look inside of a new register called IA32_FRED_STKLVLS.

This register is to be considered an array of 32, 2-bit Stack Levels linked to the first 32 interrupt number.

(Keep in mind that FRED recycles the same Numbers for the interrupt within IDT).

So when executing an interrupt, it will check if it needs to create a new stack based on if the stack level within the IA32_FRED_STKLVLS is higher than they current stack level.

If that is the case, FRED will use the corresponding Pointer within the 4 new Registers:

IA32_FRED_RSP0, IA32_FRED_RSP1, IA32_FRED_RSP2, and IA32_FRED_RSP3.

These registers contain a pointer to a free space in memory to start a new Stack.

Another change that comes with FRED is the new instructions ERETU and ERETS which returns either to User-Mode or Kernel-Mode and removes some of the steps that the kernel would usually have to do itself in these transitions.

FRED comes with one of the most MASSIVE performance gain we have seen so far, with some task like Flexible IO Tester getting an almost 2X performance gain and PostgreSQL Having massive gains as well. I think that the most surprising thing is how universal the gain is. With a normal change in feature, we would see some losers and some winners where ,here ,we are winning across the board and in a pretty impressive way. ^[5]

NTFS has always been the cause of headache for linux users.

The NTFS3 Driver brought by Paragon Software back in Linux 5.15 (Oct 2021) was a major step up.

Considering that the NTFS Driver we had before was Read-Only and that.... it was very finicky for anyone who actually used it, NTFS3 brought Modernity and writes to our kernel.

Not everything was perfect, for along time Konstantin The maintainer of NTFS3 was... Not doing his job to the point that there was report made by Kari Argillander ^[13] Who essentially asked if he could take maintainer-ship of the subsystem. The subsystem didn't switch hands, but one man made it his mission to save the Kernel from bad NTFS support.

Namjae Jeon, Who at the time raised his hand to become co-maintainer of NTFS3, Saw that the situation wasn't feasable. He than proceed, for the next 4 Years, to Upgrade the Original NTFS Driver, added Write support, and more and more feature until today. In 7.1 The NTFS driver from Namjae is more complete (if we use XFSTests as the standard) than the NTFS3 Paragon version.

With all of that said, pushing to the kernel wasn't easy for our Hero. ^[14]

His own tree didn't even run...

There is a lot of nuance in how the kernel works, one of them is how one should prepare a new feature to be merged in the kernel.

The steps:

Here the first step is very important, the goal of having a stable tree, even if outdated, is to be able to show your code in working condition. Working on the latest version and updating it while making your feature is like working on quicksand.

Passing the change off to Linux-Next (Mark Brown) allows us to show the feature being merge with the latest version, it will show which part may fail in modern condition.

It then becomes trivial for Linus to merge the final version as the change he has to do on his end should be... None or very small.

It is the job of the person accepting the merge to make the code work under their tree. This means that Mark (For Linux-Next) and Linus (For Linux) will help and make sure that the new code work with their own.

With the additional knowledge granted from the dictator himself, Namjae took 24H and repackaged his changes to be acceptable for the kernel.^[15]

Testing will be needed in order to know if this new driver is any good, I will for sure talk about it again.^[16]

32-bit ARM now has RealTime support, it now joins ARM64, Loooongarch, Risc-V and X86 in the RT compatible crowd.^[9]

PowerPC is still in the works for RT. ^[10]

Very Experimental support for protectedKVM (pKVM).

pKVM is an attempt at isolating the guess VM by using Exception Levels (EL).

Arm64 has 3 Exception Level.

Technically there also is EL3 that is ARM Trusted Firmware but not important for today.

pKVM seeks to use these levels to split memory between guests and the host.

This allows more security but it isn't meant as a replacement for more complex solutions like ARM's Confidential Compute Architecture (CCA).

pKVM requires you to trust whoever configured the kernel while CCA only relies on the Silicon.

The feature can be rolled out to older CPUs (ARMv8.1+)(2011) and it is less intensive than CCA.

A lot of this work is being push for android which can make good use of the feature.^[20]

Workaround for hardware bug with ARM C1-Pro CPUs. The bug would allow to invalidate memory that would still be in the process of being accessed by the SME (Scalable Matrix Extensions).

The workaround involves the use of an Data Synchronisation Barrier (DSB) via Inter-Processor Interrupt (IPI). This only applies to the affected CPUs that have SME enabled.^[19]

Removing I486 and ELAN support. Follow source for good explainer but:^[1]

also Linus stated that there also was additional deletion:

> independently starting to remove some really

old networking hardware support, and removing some SoC support that

never went anywhere. ^[2]

The push for more deletion comes with the rise of AI tools that detect error and potential bugs with old unmaintained code. The cost of maintaining these mostly unused parts of the kernel is too great, so it was decided that removal was the best option. A big example would be the Integrated Services Digital Network (ISDN), which we can quickly describe as VoIP before VoIP, was removed in its entirety. This is no big lost, a select number of people are still using this subsystem and are actively fading it out from their hardware, i highly doubt anyone using this even has an up to date kernel... outside of that, mostly network driver and amateur radio drivers were removed. ^[36]

This cutting spree resulted in a 138 thousand line cleanup, One of the biggest deletion count i have seen in a single merge request! ^[37]

Along with these removal we also have seen the removal of Russia's Baikal CPUs.

Baikal CPUs were Russia's Home grown CPU solution. As a result of their war against Ukraine, the Russian company was unable to get their chips. (sanctions prevented TSMC to sell anything to them.) and T-Platforms, the parent company of Baikal Electronics, entered bankruptcy in October 2022 and was sold for parts.^[11]

X86 Now handles restart like most Architectures.

Until now, the X86 code didn't call the do_kernel_restart() function, which is responsible for triggering the custom restart handlers. This wasn't an issue as X86 hardware didn't need such handlers. New hardware, the Intel Lightning Mountain required a custom restart handler which made this change necessary.^[21]

Intel LASS is now considered complete, the Restrictions around EFI services and vsyscall have been removed. The feature that we covered when it was added in 6.19

essentially enforces in hardware the split between User-Space and Kernel-Space memory. without LASS, It is possible to get access to privileged information from the page walk. LASS makes this impossible by preventing the page walk from happening if we don't have the required permission.

Won't get deeper into it, we talked about it before. It is a good thing to see completed and it is one of the features that i would like to see on AMD, Perhaps the X86 ecosystem advisory group can work on that (; .^[24]

AMD's P-State Driver (The Driver that controls power) has a big update with a couple of Features:

This new feature will allow the administrator/user-space to set Per-Core Floors frequency that the CPU will take into account when pressured by throttling events (Like overheating or power limitations). This is exposed using the new "floor_freq" and "floor_count" sysfs attributes.

This feature seeks to modify the EPP values based on if the Power supply of the laptop is plugged in or not. Remember EPP values can have drastic implications on the performance or power consumption of a laptop. The Fine tuning of these EPP values is where a lot of performance gain can be seen. Dynamic EPP can be enabled and disabled via new boot time options "amd_dynamic_epp". In the RC5 they removed the Kconfig option to disable it.^[7]

Before, you would have to select 1 of 4 modes to manage your EPP value, you can now set it directly to the value of your choice (between 0-255).^[8]

HIGHMEM Support for LoongArch, this features makes sense for 32-bit as the total memory that the kernel can use without the feature is 512MB, leaving at most 256MB to the Userspace.^[17] With HIGHMEM, you are now able to manage up to 2.25GB.^[18]

Big improvement to the Intel QAT Crypto driver, Remember, The intel QuickAssist (QAT) accelerators are chips that can handle compression and Encryption algorithms in place of the CPU. This frees the CPU to do something else while QAT processes the data.

Now keep in mind that it is not a magic bullet, in 6.17 we learned that for small chunks of data, QAT can be way slower than letting the CPU handle it.

In this release, QAT Gen 4 and 5 got Zstd Compression support (Not decompression)

_CPUs compatible with QAT Gen 4 start with Sapphire Rapids, which launched in January 2023. accelerator cards do exist._

We also have full support for QAT Gen 6 For Both Zstd Compression and Decompression.

QAT Gen 6 also now have an Anti-Rollback feature that prevents downgrade of the QAT Firmware which could be used to exploit a security vulnerability.

_QAT Gen 6 should come with the upcomming Diamond Rapids CPU_^[12]

ARM64 gets an optimized CRC64 NVMe algorithm with NEON Polynomial Multiply Long (PMULL), How does it work? no clue, but it yeilds a 6x performance gain so good on them!^[28]

exFAT

VFS

XFS, F2FS and EXT4 were all calm this release, Mostly small optimization, bug fixes and surely laying the groundwork to be the title of a future kernel review. ^[31]

BTRFS was a bit more active with moving the shutdown IOCTL support out of experimental, it was added in 6.19. ^[33]

JFS now has more robust data integrity checking.^[32]

HFS is still looking good, more work was pushed this released with a lot of the patch related to bugs found by Syzbot which fuzzes (send unexpected input) main kernel branches. I'm surprised that we see that much activity on what was a dead FS a few releases ago.^[35]

The T10 PI code was reworked (T10 PI is used for to generate and verify integrity information).^[29] It is now faster and easier to use with the different Filesystems. it was hooked into XFS and it shows no change to write performance and 15% gain in read performance.^[30]

other than that IO_URING got some improvement and MD, a subsystem for raid, fixed a couple of potential deadlock.^[34]

ISDN is not the only network subsystem that was removed from this release,

the UDP-Lite code was removed as well, now it was deprecated since 2023 and called for removal in 2025, safe to say that it's removal won't make any issue.

UDP-Lite was UDP but without the checksum, this made sense with VoIP as Corrupted data is better than no data in the Audio and video world. The standard UDP protocol would simply discard all packets that don't match the checksum. This mad a lot of sense in 2007 when it was added to linux 2.6.20 but isn't needed anymore. UDP-Lite was shown to be a hotspot for bugs, so its removal an easy choice, especially when considering the 3% speedup that comes with the cleanup of the UDP code.^[38]

IPv6 cannot be compiled as a module anymore.

To understand why this was needed, lets understand the change between a module and built-in.

When compiling built-in, the code loads with the kernel, it is the kernel. This means that at no point can another subsystem fail to access it.

When compiling as a module, there is additional check that other subsystems need to do in order to confirm if the module is loaded.

TLDR, modules will = indirect calls, Built-in = direct function calls

This check can affect performance, and considering that we are talking about something as important as IPv6, i'm surprised it wasn't done before.

This is a win for security and performance, i'm sure the devs for core networking and BPF are happy about that one (; ^[39]

-(better 2014 GPU support) AMD Valve Deliver Better Kaveri / Kabini APU Experience^[4]

SCHED_EXT Brings some good change with first the groundwork for Cgroup Sub-scheduler, this would allow sched_ext to use scheduler optimized for specific task based on the Cgroup. This could allow us to have a scheduler running for a DB system with the knowledge specific to DB performance (Like queues and locks) and another scheduler for a Game (that would keep track of the rendering deadline) to be on the same system, at the same time. ^[22]

Along with that, SCHED_EXT also made a change where a task should run.

Before, IF

It would then look for a CPU within the same Last Level Cache (LLC, commonly L3 cache) to execute the task.

The latest update adds another option before resorting to the Last Level Cache which check the SMT sibling of the previous core.(SMT here refers to the technology where you have 2 thread per physical core, hyper-threading for Intel.)

The SMT Sibling shares most cache with the previous core, making it faster than resorting to a core that shares LLC.

On large SMT systems, it appears to boost throughput by 2-3%!^[23]

Test have shown that Workqueues could spend a lot of CPU cycles waiting for a spin lock. Until now, Workqueues could divide the task into multiple pool, the issue here being that this pool would be generated at the Last Level Cache (LLC) level. It is common for CPUs with high core count to share the same LLC for example Nvidia Grace 72CPUs, 1 LLC... This can lead to contention on the lock for the pool which isn't good! The new WQ_AFFN_CACHE_SHARD seeks to avoid this by dividing the CPUs within the same LLC if there is more than 8 core. ^[25]

Benchmark on the Nvidia Grace showed an improvement of around 5X the throughput with the patch.^[26]

Earlier, we were talking about the removal of X86 32-bit code but this isn't the only change in 32-bit land. The Inode->i_ino field now changed from an unsigned long to a u64. this means that instead of being 32-bits on 32-bit system and 64-bits on 64-bit, It will now always be 64-bits. This means that the Inode identifier is now universal and doesn't require weird hacks that hashes the 64-bit long into the u32 hole.

This change is another change that makes it clear, 64-bit is the default size for the Linux kernel. This doesn't mean that we will see 32bit go away, even if we remove X86_32, we still need it for ARM32, and the interest seems to still be there, Loongarch32 is an example of that.^{[27] [42]}

Audio support for the Steam Deck OLED was fixed after 2 years of being broken... for those that are interested.^[40]

When comparing on an AMD Threadripper 9980X, a 64 core 128thread CPU, using Linux 6.6, 6.12, 6.18 and 7.1 we see that 7.1 performs around 13% faster than 6.6, a kernel from late 2023.

When 7.1 isn't first, it is close 2nd to 6.18. This is good news, it shows a good trajectory for the future of the kernel, a kernel where features doesn't mean being slower.^[41]

Linux 7.1-rc1

Linux 7.1-rc2

Linux 7.1-rc3

Linux 7.1-rc4

> So just to make it really clear: if you found a bug

using AI tools, the chances are somebody else found it too. If you

actually want to add value, read the documentation, create a patch

too, and add some real value on top of what the AI did. Don't be the

drive-by "send a random report with no real understanding" kind of

person. Ok?

Linux 7.1-rc5

>this is too big, and this is the heads-up that I'll be

pushing back on pointless pull requests with fixes that just aren't

that important. And yes, several of these series were triggered by AI

code review.

Linux 7.1-rc6

Linux 7.1-rc7

Linux 7.1

https://www.phoronix.com/review/linux-71-features-changes