Meltdown and Spectre bring a world of hurt to the cloud

thunderstorm

Meltdown and Spectre represent a new class of security problem and the cloud — based as it is  on virtual machines — is uniquely vulnerable.

The good news is patches are either already in place or coming soon to all major cloud platforms and server operating systems. The bad news is that your data may still be at risk of theft. And even if your data is safe, your performance is likely to suffer.

These two security problems are present in essentially all modern chips. They work in similar ways.

Meltdown, which has been patched on most operating systems running on Intel processors, breaks down the wall between user applications and an operating system’s memory allocations. Attackers can potentially exploit it to spy on the memory of other programs and operating systems.

Spectre breaks down the barriers between different applications. It could trick applications into accessing arbitrary program (but not kernel memory locations. Spectre is harder to exploit than Meltdown, but it is also harder to mitigate. For now, there are no general-purpose Spectre patches.

All that said, these are fundamentally new security holes found in any system using modern processors and shared memory. While attacks using these methods won’t be easy, we really don’t know how well we’ll be able to protect ourselves against potential attacks.

So, can you think of a technology that relies on the hottest new CPUs and gigantic amounts of shared memory? I can. It’s called the cloud.

In particular, cloud providers which use Intel CPUs and Xen paravirtualization for virtualization may be at risk. In addition, containers that share one kernel, such as Docker, LXC, and OpenVZ are all potential victims.

As SANS security expert Jake Williams put it, a major Meltdown attack vector “may target kernel addresses that are shared between the container and host kernel in many paravirtualization instances (e.g. Xen) and kernel sandboxes (e.g. Docker). It is possible that attackers may leak data from outside the container, leading to a hypervisor escape.”

If rogue processors can escape from a hypervisor, we’re all in a world of hurt. The top cloud providers knew this was coming. So, we know that Amazon Web Services (AWS), Google Cloud, and Microsoft Azure are patching their systems as fast as possible. We also know that Rackspace is on top of Meltdown and Spectre.

AWS reports Windows Server 2008R2 and 2012R2 patches are not available through Windows Update, but an organization can update them manually. AWS also reports there’s no word yet from Microsoft on patch availability for Server 2003, 2008 SP 2, or 2012 RTM.

Even where there are fixes, there’s another problem. If you haven’t seen it yet, you soon  will — poor performance with the deployed patches.

Red Hat‘s Meltdown/Spectre performance benchmarks (from after the Linux Meltdown patches have been deployed) discovered the following performance problems:

  • Measureable: 8 percent to 19 percent — Highly cached random memory with buffered I/O, OLTP database workloads, and benchmarks with high kernel-to-user space transitions are impacted between 8 percent to 19 percent. Examples include OLTP Workloads (tpc), sysbench, pgbench, netperf (< 256 byte), and fio (random I/O to NvME).
  • Modest: 3 percent to 7 percent — Database analytics, Decision Support System (DSS), and Java VMs are impacted less than the “Measurable” category. These applications may have significant sequential disk or network traffic, but kernel/device drivers are able to aggregate requests to moderate the level of kernel-to-user transitions. Examples include SPECjbb2005, Queries/Hour, and overall analytic timing (sec).
  • Small: 2 percent to 5 percent — HPC (High Performance Computing) CPU-intensive workloads are affected the least, with only 2 percent to 5 percent performance impact, because jobs run mostly in user space and are scheduled using cpu-pinning or numa-control. Examples include Linpack NxN on x86 and SPECcpu2006.
  • Minimal: Linux accelerator technologies that generally bypass the kernel in favor of user direct access are the least affected, with less than 2 percent overhead measured. Examples tested include DPDK (VsPERF at 64 byte) and OpenOnload (STAC-N). Userspace accesses to VDSO like get-time-of-day are not impacted. We expect similar minimal impact for other offloads.

You can expect to see similar problems with Windows servers. There is no way to patch this performance problem–even with chip microcode–with any of today’s processors.

Google released a migration called “Retpoline,” a binary modification technique that protects against “branch target injection” including a methodology to mitigate performance issues, that has already been included in LLVM and GCC complier updates.

If that works, great, but we can be certain that patches will cause significant speed problems on some workloads. In the coming weeks, beside double-checking your security, you must watch your system loads like a hawk. Your CPU allotments and allocations may have been great last week but, with the patches in your services, they may be starved for processing power this week.

RELATED LINKS

Enterprise security lessons for Meltdown and Spectre defence

Cloud migration: Making the right choices up front stacks the odds in your favour

Trackbacks

  1. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  2. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  3. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  4. […] using Linux, macOS, or Windows, won’t see many of a opening hit. The genuine pain from Meltdown and Spectre will be felt on a cloud with a server, not on a […]

    Like

  5. […] or Home windows, will never see a great deal of a performance strike. The actual discomfort from Meltdown and Spectre will be felt on the cloud with the server, not on the Personal […]

    Like

  6. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  7. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  8. […] macOS, or Home windows, will never see considerably of a efficiency hit. The serious soreness from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  9. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  10. […] running Linux, macOS, or Windows, won’t see much of a performance hit. The real pain from Meltdown and Spectre will be felt on the cloud with the server, not on the […]

    Like

  11. […] is that all about? I found little comment on this anywhere online except an article that said AWS have reported the same problem with non automatically delivered patches through […]

    Like

  12. […] Meltdown and Spectre bring a world of hurt to the cloud […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: