This is the second unikernel article this week I think, and the second that really undersells the benefits of having a full OS. Let's just swap out a couple of words:
> With a unikernel, the [observability] surface is much smaller. If the functionality isn't in the application (the operating system), then the [SRE] is screwed. There's no next hop.
Tools such as `strace`, eBPF, `lsof`, `ss`, etc. have tremendous value. The article mentions some nice things for observability ("structured logging, OTel") but so much gets thrown away.
In general, the more specific tool you have, the more tailored to the goal approach you may have, with all features optimized for that goal. If you need those tools from the full OS, you might have their specialized versions in a unikernel.
In a type 1 hypervisor running a JVM on top, you would use Java Flight Recorder.
The tools exist, and in a container world no one should be running those tools directly anyway, it is all about rootless immutable containers, ideally without OS layers for deployment.
Not just that. Unikernel sounds ok if all your linux server does is run one service that doesn't need anything but tcp/ip, and whose users don't need extra resources. But being able to deploy a new binary and other files through any means (ssh, pipelines, git), having localized log files, etc., are more than a convenience. Sure, I could log on another server, run a db on another server, and deploy html/css/js from yet another server, but would that make life easier? And this is just a really simple case. If you have multiple services, you'll need multiple servers with a uni-kernel approach, or build some monster thing. It's horses for courses.
Welcome to cloud computing, where software is cattle, not pets.
There isn't a Linux server taking care of a couple of services.
There are services deployed as Kubernetes pods, or serverless language runtimes across a cluster abstraction, running directly on top of type 1 hypervisors, or minimal kernel images enough to power distroless container images.
Let me know when your local OS does multi-machine scheduling across heterogeneous hardware and application constraints, networking, observability, service accounts and a finer grained permissions system, in a way that’s far less confusing and piecemeal.
Also, this argument only works if you just broaden the definition of “OS” to “anything that runs that isn’t your apps”.
maybe we _should_ be thinking about building proper distributed operating systems instead of loose collections of services. we really missed an opportunity to rethink the OS interface for 'the cloud'. or rather we kind of wasted alot time and maybe we should get our shit together and do it properly. no matter how much we chant 'cattle not pets' everyone is still installing individual nodes and logging in and puttering around.
Agreed! There were many promising distributed operating systems, most got stuck at the research phase however. But the cloud pretty much killed them all.
Not that many people keep large distributed workloads anymore, and SLURM-like schedulers on top of regular operating systems were deemed good enough for supercomputers.
not always. Asci-red had a distributed version of OSF/1. this wasn't a great design because it was built on distributed shared memory, so it had fault tolerance and performance problems. the MTA had a shared memory monolithic BSD, but that was almost a forced choice.
The Cray T3D had a really nice distributed services model (really kind of a distributed exokernel) that would be worth emulating
on some machines there was an interactive ganged run facility like 'mpirun'. I think even this distinction where there are 'login' or 'service' nodes and anonymous runners would be at least a little structure. I guess you could argue that k8s is this, but it kind of falls short and seems to require a lot of continuous propping up.
I worked on a single system image project that didn't ever get shipped. it turns out that a syscall proxy layer isn't that difficult to implement, and that leaves you with an undisturbed user space.
anyways, there are a whole host of design options that don't look like k8s or slurm. and having spent some time recently on non-hyperscaler clusters, that bottom level of bringup (provisioning, networking management, reporting, fault tolerance, distributed filesystems) is really a nightmare. I think we lost a lot of traction there by raising generations exclusively in someone else's operational context.
Yep. We say cattle not pets, but even cattle often need personal attention for one reason or another. It doesn’t mean you’ll never need that option in your back pocket
if you didn't have a multi-user operating system running on every node that basically demands this, because of its complexity and fragility, and you had the necessary introspection to look at memory usage and open sockets, and disk bandwidth, and everything else you might need to mange those processes, then the utility of your escape hatch - which also is complicated and fragile, becomes negative.
> Tools such as `strace`, eBPF, `lsof`, `ss`, etc. have tremendous value. The article mentions some nice things for observability ("structured logging, OTel") but so much gets thrown away.
Trivial to implement. I have a full custom (from scratch written) debugging/benchmarking implementation written both for Linux and two of my custom kernels, both of which are REALLY unique when it comes to architecture. My custom debugger also supports numerous inline scripting languages.
It's also not like you are stuck with a full OS when you are running Linux. You can remove features from the kernel you don't need, and you can configure a bare-bones busybox to be your userspace, or just run your application directly as the init process.
Take this with a grain of salt. I haven't run unikernels in prod, so it's all a figment of my imagination.
> Tools such as `strace`, eBPF, `lsof`, `ss`, etc. have tremendous value.
But doesn't their value lie in getting you access to the core of the OS that you wouldn't be able to touch otherwise? What if your network stack was a library you could instrument just like you do with the rest of your app code?
Also, I don't need a tool to tell me I'm being punished by the IO or CPU scheduler if I am the only one using the CPU and IO :)
If your app, is the only one running, and the whole kernel is a library, you can simply have said library instrumented with as much observability as you need, directly and you now no longer need to worry about which processes are using cpu and IO because…it’s all you.
General purpose OSses are a tax, one that people pay because it's cheaper than rolling your own.
Has the math changed? Maybe. I remember the days before multitasking and protected memory. Things were faster, but more fragile. But these days there are so many things happening that they are, on some dimensions, just as fragile.
The real cost is going to be hardening, and maybe LLMs can do that as well.
It has, a long time ago when doing serveless became fashionable, and too many realised they didn't want to manage Kubernetes clusters by hand.
So you get a type 1 hypervisor, which technically is like a micro kernel, which then boots whatever else on top including language runtimes, or chiseled containers.
Unikernels ideas aren't much different from some embedded deployment scenarios, with the application, language runtime and OS all linked together into the device firmware.
It has. In fact the current ensemble of OSs/kernels (Linux/BSDs/Windows/macOS) are pretty much DoA. They have far too much legacy decisions holding them back and the SOLE reason they were used was because rolling your own kernel _per project_ took too much time. With agentic development this has changed, anyone can now roll their own kernel that is a) faster and more optimized b) more secure c) more obscure than the current offerings [important because it makes reverse engineering harder].
I've been tempted on a few occasions. Mind you, if you're stuck in the cloud for whatever reason you have an OS under the hood anyway, but at $WORK we own the rest of the stack anyway, and it wouldn't be that much more work to hook into the virtio devices directly. It might even be less work than everything else we do to work around the kernel.
I've had limited adventures in embedded software and as part of that became a fan of Zephyr (http://www.zephyrproject.org) and wonder about its potential application as a unikernel.
On the minus side: I think porting "traditional" software - say Nginx - to it will be agonising. And I can't make any guarantees about its performance in the same way that Alpine Linux container images are not necessarily fast.
But there's an entire toolchain, debugging story, drivers etc. for something that compiles to tiny and starts up basically instantaneously. Surely that's got to be useful for something?
This is really fun to see again. I first played with baremetal around ~2013 when I was a university student. I really enjoyed message passing between small computers with the network API where you just got to write ethernet frames, no messing with any higher protocols.
sign in with google on "create free account" gave a 400 error.
I might be interested in this because that is cheap, but what exactly are the networking rules/costs? ip4/6, bandwidth, egress, throughput, connection limits?
If you have serious attack surface minimisation goals you need to embrace FOGAs. LLM coded unikernels have way too much bloat.
The Zynq UltraScale+ parts pair a hard ARM CPU with FPGA fabric. You can progressively isolate fast path and high security components in logic. LLMs are getting quite good at using synthesis tools.
But they're still, as far as I've seen, terrible at debugging HDL. There's just too many interconnected signals. Blocking vs nonblocking assignments and scoping are very hard for it. But I have yet to try opus 5.5 on it.
Agreed. You have to give them device-in-the-loop with trace or a decent simulator. Presumably the big EDA companies are working on tooling integration and bespoke models, and they have huge libraries of designs to draw on.
Yeah I have yet to get it to close the loop. I have some stuff theorized but everything else ends up in the way. Last time it got stuck going to the open internet because obviously I'm not letting it do that but it wouldn't stop trying. I just gave up after that. But once I can close the loop, I have big plans.
> I think dependent types are the winner for next-generation languages. Anything that lets you codify more into the type system is more back pressure.
This is one reason I'm so bullish on Lean4 as a general programming language
It's now essentially proven that expressive, strong type systems like those of Rust present an advantage for LLM assisted coding
Lean's type system, in which types are first class, fully programmable objects takes it so much further. Particularly when it comes to embracing dependant types
Yeah that's generally the argument for using a managed runtime (like Ocaml in MirageOS's case, or I could see Go or even the JVM fitting here) for these kinds of things. Running a VM which has no pointers, memory access etc primitives, and is garbage collected etc direct on "metal" gives more peace of mind about that sort of thing.
You could make the argument that Rust w/ its memory safety is a candidate, but w/ Rust it's still entirely possible and fairly easy to break out of that. And Rust/Cargo applications have a habit of using a bazillion third party deps that you then need to keep a close eye on.
I’ve always thought there should be a module-level safety property in the type system such that unsafe would be a capability granted by the user’s code.
this question always comes up. I personally think the status quo is pretty weak here. I have certainly added a gdb stub to a unikernel before, but that apparently given that it was never used it wasn't the right answer.
people always talk about losing the excellent debugging facilities that linux gives you. if a service crashes in a big cloud environment what is that you do now that works so well. do you think its intractable to implement that in a single-process kernel?
I don't know if anything like this has been built, but I'd imagine you could have the hypervisor capture stack traces / log buffers / core dumps when a unikernel VM crashes.
I'm not a security researcher but I know that VMs have been the tool of choice to step through malware execution for at least the past 15 years. I recall coming across ida extensions for it.
The big thing for me is the liveness is completely gone or at least can be. In a regular box I can at least poke around at the facilities even if the process dies. I don’t think it’s impossible I just think it’s a hard problem I don’t see emphasized enough.
This is also why I'm not sure the unikernel buys you much security. If you corrupt arbitrary memory, you can execute arbitrary machine code. Doesn't matter what the unikernel exposes.
In general, sure. MirageOS is in OCaml though, where you have a memory-safe language with a small C runtime. Corrupting arbitrary memory is much less likely.
All situations have their own constraint, but I'll play devil's advocate. These are not strong opinions of mine:
Use Rust and the chances of an application overflow drop off a cliff. The debug-ability also increases the surface for bugs themselves. The majority of cases we're talking about chips that can comfortably be hardware-simulated. And/or all you need is some way for a dev machine to edit memory and an AI can work through the night to explore a configuration space much larger than you would ever have.
I generally agree Unikernels are cool as I did back in 2010 when I first came across the idea. But the idea that RCE are contained because there is no shell and no interpreter does not hold up. If they can run code on the machine they can run a compiler in your unikernel and device drivers and whatever else they will need. Granted it's much harder to do all these things through the straw of a RCE bug, but the argument that because the tools are not there they can't do damage is wrong their malicious code was not there either but they found a way to bring it in.
Sorry but the security "win" of unikernels are not binary.
Yes, in theory you have a smaller attack surface, in practice its not exactly true.
Yes, you throw away a lot of tools that you don't need, which means that sideways traversal is much much harder, it doesn't actually stop your front door being smashed down.
You are also on the hook for detecting and updating any ported library. This sounds easy right? just look for the CVEs and then re-build the image?
Ok but what version of the ported library is in your unikernel? in the porting did it have the same bug? Did the LLM cock up the porting, did
Basically you are on your own. So saying "unikernels! you're secure now" is at best misleading.
The argument is that LLMs are making unikernels easier to work with, and having a whole OS stack leaves a big attack surface, as demonstrated by the scads of LPEs and other vulns we've been seeing in Linux over the past year+, right?
But you know what else is easier with LLMs? Properly hardening your Linux system. "Patch every week" is the recommendation from upstream because the kernel security team has to consider every possible deployment configuration and environment. If you implement all the KSPP recommendations, use a config and kernel command line that makes sense for your actual application, and use a properly configured MAC LSM, you would not have been affected by any of the big blockbuster vulns of the last year.
So if you can tell an LLM to port a library in a loop and feel confident in the results, surely you can feel equally confident telling the LLM to harden your Linux environment.
n.b.: I am not endorsing the idea that just throwing LLMs at security problems is actually a good answer, I'm just saying that the case the article makes isn't actually comparing like for like.
Totally agree with this article. With AI, I think now is the perfect time to consider where Unikernels might fit into your architecture, and how you can leverage them to minimize your attack surface.
I’ve started looking at them myself, and have been comparing them to mature VMMs like Firecracker, and asking myself where each piece of my stack might be best run.
Virtual machines and containers are no longer an effective isolation mechanism when AI is involved. VM breakouts are becoming trivial. So everyone should be considering how to bake better security into the their runtimes.
Most human written code (even when written by experts) is quite bad. Really bad actually. If you have a compliant agent (Daybreak/abliterated or otherwise) you can just provide them with a list of most likely VM escape routes and there's a ~100% chance they'll succeed unless the code has been explicitly pruned by an LLM prior.
They are not. When discovered, they are still big news. With few exceptions, businesses (and other institutions) continue to treat the cloud providers' VM-as-a-service offerings as secure. AI can aid the defenders too.
> So everyone should be considering how to bake better security into the their runtimes.
If VM isolation is not trustworthy, my first concern would be whether the other VMs on the physical host machine might do something malicious and break into my VM.
A natural r&d project would be a high frequency oms or something like an nyse bid/ask/match book manager.
These apps tend to do kernel bypass anyway for net i/o after hard scrubbing the os to remove as many interrupts and other superfluous jitter the oms does not need.
However, I believe the low level kernel bypass code (eg dpdk, libfabric) depend on Linux to talk to the hw in a disciplined way.
Anybody know better?
A decently faster oms this way could be a practical alternative to fpgas we sometimes see in that space. (hw would be co-located of course)
Reducing attack surface is definitely a plus but it is nowhere close to the number one security benefit of running unikernels.
That's why I never really liked talking about "reducing attack surface" that much because folk inevitably turn to lines and code, which while reducing is good, just simply doesn't communicate what the biggest problem truly is.
Vuln exploitation is the number one entry point for data breaches and os command injection is the number one CWE in CISA Kev from last year.
System intrusion was repeated something like 64 times in last year's DBIR.
The operating system itself is literally the problem as it's inherently meant to run many different programs whereas unikernels only run one.
Honestly, I think there is a lot of similarities between unikernels, at least the one I'm active with and qubes. The big difference to me is that qubes is more desktop/consumer focused and nanos (the one I'm involved with) is more server-focused but the same sort of ideology/principles gets exposed - just at varying levels and because of the end environment they get architected differently.
I’m completely ignorant about this and likely just missing the point but: isn’t the point of an OS to not have to (vibe)code filesystems and networking by hand every time you need them? Also, how does a unikernel cooperate with other applications? Would they all live in separate networked unikernels managed by a hypervisor? And, if they all (vibe)coded their own fs and network wouldn’t that introduce subtle bugs and inconsistencies that would eventually bring the whole thing down and lead you back to the need for shared primitives in the first place?
Maybe a good halfway point is to still have a unikernel but the libraries for common stuff like networking or fs are already written according to a standard and plug and play and reusable across applications?
You don't build the networking or filesystem by hand every time you need them. You pull them from reviewed and maintained library code from a repository. That's how MirageOS etc work. Someone else has written e.g. TCP/IP -- hopefully well -- and you link against it.
The point that makes this different from an OS is that there's no shared service, no syscalls to do that stuff -- just subroutine calls -- and no timesharing (except at the hypervisor level). It's just a single runtime running on hypervisor direct against the virtualized hardware.
Because, yeah, all this work has happened in hardware in the last 30 years to make that possible, but we still treat the operating system as the best unit of resource sharing. When in fact it's kind of a jack of all trades master of none. If you're booting a whole linux kernel -- with its giant framework built to co-tenant a pile of applications and users -- just to run one process, there's definitely something to look at there. Even if the unikernel space itself is relatively immature.
You don't need to emphasize the "vibe coded every time you need them" thing. That's not what anybody was being talked about in the article. The "vibe coding" piece is pointing out that the missing pieces in the ecosystem can be more rapidly filled in now because of agentic tooling.
> running on hypervisor direct against the virtualized hardware.
Well, here is your actual OS then. You named it "hypervisor", and changed the system interface, but it's still there, revirtualizing the access to hardware and ensuring separation of different users/applications/unikernels.
Does it though? And I'd really like to see how you'll revirtualize e.g. an NVIDIA GPU (notoriously context-full piece of hardware), while allowing transparent shared access to it from different applications. Heck, you'll run into trouble with most of PCI-connected devices except for the most primitive once.
yea, "virtualizing" an NVIDIA GPU is not truly feasible right now from what I see.
I do know that AMD has done more in this space. When I worked at Google there were (other) people on the Stadia team doing this. And some open source bits out there that I've seen, including from AMD.
Unfortunately, my real world work... and most real inference etc work others are doing... is on CUDA/NVIDIA.
This isn't too dissimilar to DOS programs. DOS also gave you a filesystem, memory management, a shell a process loder, driver interface, but it was all in real mode, and you were free to replace these components with your own as you pleased.
Or if you're more familiar with embedded, like a BSP or RTOS
Arbitrating access to the hardware between multiple tenants still gonna be a PITA (as it was during the DOS days), especially if you want to expose actual hardware, not some generic VirtIO devices. Even with the latter, the hypervisor would have to do quite a bit of the resource management and interface translation.
Unless you're literally running a single application on a dedicated piece of hardware in which case... yeah, sure, you can do the OS-as-a-library schtick. But you could do it 10 years ago, too.
Yep. Except every application runs in a down-and-dirty virtual machine instead of a nice clean process, so they get to experience the joy that is hardware booting and get to configure virtual device buffers instead of calling send(). Progress.
Over time, what prevents the reviewed-and-maintained-library-code from developing creeping featuritis and consequently evolving the additional attack surface that unikernels currently lack?
Unikernels simply redraw abstraction boundaries. You can still use libraries and third-party code to implement common primitives. In fact, that's exactly what many unikernel frameworks give you (unikraft, hermit, mirage)
Are their abstraction boundaries protected by virtual memory and syscall interface such the submodules of a program cannot corrupt parts of memory that belong to others? Can I restart subtasks without affecting others? A traditional OS gives me all of that with relatively minimal overhead.
With a capability-based microkernel you get to assign ownership and resources to a very granular level that increases robustness against crashes and protection against attacks to the core system policy. A unikernel can give all of that but I feel like then it will be worse to program against rather than a microkernel.
So overall I don't see any advantage of unikernels. They don't help you write more correct programs IMO and the performance gains are limited.
I guess the others have made these points, but since unikernels run in VMs with virtual hardware, I guess a virtual ethernet card can have an interface that's easy to program for, making the driver trivial. You're not talking to real hardware in any case.
Similarly, a filesystem's just a data structure that happens to be on disc. You can mmap your harddrive and let the host hypervisor handle swap, preemption etc.
It's similar to a process contract, but with slightly different boundaries
The thing I don't get: We'll now run those N unikernel things in VMs with a hypervisor underneath. How is that different from running static binaries in a locked-down OS, i.e., with a traditional kernel? You can surely break out of a unikernel by "popping the kvm" as the article mentions. How is that more secure? In principle we are running the same things, just call them differently (app becomes unikernel+app, kernel becomes hypervisor).
It's conceptually equivalent but the details differ. The program has access to more abilities and in terms of common conventions the security boundary is much more robust. If nothing else I'd argue it's a superior abstraction with which to make the delineation.
Both have weaknesses, but with an OS, you have certain bottlenecks - like your app memory is paged and has its own memory space, data structures are often duplicated in user and kernel memory, syscalls have cost, which is why io_uring is a thing in Linux.
This is all unnecessary overhead and complexity, as the OS doesn't really mediate hardware access in this case.
But you're right on some points, it's not really more secure, but there's a hell of a lot less complexity where things can go wrong.
It is not any different. Actually, it is worse because a VM is a much worse environment to run a application since you need that unikernel stub as well as the application.
The only advantage is that cloud services provide VM tenants so you are already stuck with a VM substrate and you want to minimize layers from that point on.
However, as a practical matter, the design of the Linux kernel is grossly insecure and basically impossible to make even remotely safe as evidenced by the unending sprawl of LPEs. Hypervisors are generally designed and configured in a way that is only mostly insecure instead of grossly insecure; a bad is better than terrible situation.
However, there is nothing stopping you from designing a kernel in a similar way and providing shared tenant processes with similar guarantees. You just need to port your applications to the new operating system, but you would also need to port them to a unikernel anyways if you wanted to go that route.
The only free action is lumping your applications with the full OS and dumping it onto a hypervisor. Anything else has porting work which is why the multi-tenant VM model is advantageous at all in the first place.
For that matter, the actual Linux kernel need not be large at all. It's really the user space stuff that takes up the bulk of what people think of as the largess of Linux.
Still, there's a whole pile of stuff there that many applications don't need. If you truly can run the entire of the OCaml runtime and standard libraries etc without it, and you're happy writing your apps in OCaml... MirageOS has always felt winning-ish to me as a general idea.
And to TFA's point, now with LLMs the relative exotic nature of OCaml as a platform need not be a huge hindrance. I like the language, but have only dipped my toes in. Definitely tempting to try writing apps on it w/ MirageOS now. Though I doubt I'd convince anybody to pay me to do that...
I was involved a bit in Unikernel Linux (https://github.com/unikernelLinux/) which is basically a Linux kernel where you can run a single application linked with the kernel. It can run baremetal or as a guest under KVM. It's both the best and worst of all worlds.
A lot of comments here about observability. Well UKL solves this nicely because you can run regular userspace processes in the unikernel, thus you can (optionally of course) run a shell, perf, and anything else you need to observe what the machine is doing.
I saw a post that said roughly "I thought computing was doomed to be 'Linux, Python, and HTTP' for the rest of my life. Now we're free". That's it. Let it sink in. We're free. We can make the computer do whatever we want.
I've been using OCaml for a couple of years now but haven't dug into Mirage yet. It is super cool. Loved the anecdote about Yaron. Vibecoding in OCaml feels like a superpower. Adding Oxcaml and having your whole company on that must feel very good.
It may be time to revisit cosmopolitan (https://github.com/jart/cosmopolitan) and redbean (redbean.dev - although the tls certs are expired and it seems scrubbed off of github).
The two choices offered are a false delimma. The MILS kernels that came before seL4, like INTEGRITY RTOS and LynxSecure, had middleware to make deplpyments easier. The open-source solutions mixed standalone apps with user-mode VM's for Linux compatibility. OKL4 let one use drivers across them.
Looking at what's out there, I think GenodeOS should be mentioned for application-specific systems. They can even build some of their stuff on seL4 if you want.
I'll also add there's been unikernel projects that aren't in exotic languages. Even C++ (IncludeOS?). There's probably one in Rust by now. If not, it could probably build on Redox's components.
I’ve been thinking about this but I’m not sold on any of the arguments presented
I was a swing vote and you lost my support in the first debate
What I care most about is benchmarks. I see AI rewriting its harnesses for the hardware it’s on and getting faster tokens/sec, and I can see that as the future. I think, okay we can go deeper
Nothing in this article addressed results, its just breaking every known convention in the scariest and most inconvenient way possible while claiming that the attack surface is smaller when its actually unknown with now limited debugging capability
I could overlook that if I got more access to RAM this way and wildly improved performance
Intuitively, I think I have some appreciation for how much silicon has been poured into virtualization. Its always seemed like a "yeah but you could avoid those costs by not doing that" but I'm more receptive to the idea that these might in some cases be really good ways to get some of the isolation workloads demand with the hardware helping us out, these days. There's so much securing for vm's, and maybe it's just easier than trying to secure in userlands: let the hardware help.
Long time interest in microvm's but they felt heavier weight than I wanted. Now it feels like maybe they might actually in some regards in some ways be lighter weight than managing workloads yourself in userland. Maybe. I dunno. Interesting times, i'm open to it.
One of the problems is there's only a small certain percentage of scenarios/applications that benefit from very short start times.
I'd wager most of the services running on the interwebs are web servers, database servers, inference servers etc that don't change over their lifetime really. They're just doing the same thing all day long every day.
If you're doing stuff like what I'm doing for work right now, which is, yeah, multiplexing potentially oodles of user-submitted jobs, and those jobs are best expressed as distinct images or containers, then yes, managing start times is absolutely imperative in improving utilization/occupancy and therefore reducing costs.
But I'm not convinced that's a typical scenario, not typical enough to drive enough time and money investment in this space maybe?
Also there ain't currently no real "hypervisor" for the (NVIDIA) GPU. Not practically anyways. And that's arguably where we need it the most. Or at least I do, for Day Job(tm).
So unikernels and microvms may have to lean on other arguments for adoption: security and simplicity-to-reason-about might be those...
> I do wonder what kind of wins we're going to see from unikernels.
I come from security so I'm more disposed towards the benefits thereof, however, being a person that has had a commercial and open source unikernel presence for a while now - "ease of use" is probably the largest selling point that I see from most of our user base. There is nothing out there that gives you the performance, security and cost benefits from shipping your unikernels to AWS, GCP, etc. in seconds. Not lambda, not cloudflare, not any of the millions of microvm providers. You quote netlify right here (who actually don't use unikernels per-se but small little linux vms (big difference!)) but why would you even use them when you can do it cheaper by shipping straight to ec2?
I was thinking that we don't need programs to share CPUs anymore now that each machine has so many of them. We could have an OS which binds each program to one CPU. When it runs out of CPUs, it suggests to close an existing program. Then programs don't need to pay context-switching cost.
A typical desktop OS runs ~100 processes before you get to run anything. Even if you would delegate one core to system stuff, most programs are multi-threaded, so you are going to run out of cores quickly.
It's a shame a multiple physical CPU machine never took off for workstations. It would have been awesome if you could delegate a real weaker CPU (physical die) for OS background tasks and then had a stronger CPU for userspace workloads.
This is the second unikernel article this week I think, and the second that really undersells the benefits of having a full OS. Let's just swap out a couple of words:
> With a unikernel, the [observability] surface is much smaller. If the functionality isn't in the application (the operating system), then the [SRE] is screwed. There's no next hop.
Tools such as `strace`, eBPF, `lsof`, `ss`, etc. have tremendous value. The article mentions some nice things for observability ("structured logging, OTel") but so much gets thrown away.
In general, the more specific tool you have, the more tailored to the goal approach you may have, with all features optimized for that goal. If you need those tools from the full OS, you might have their specialized versions in a unikernel.
In a type 1 hypervisor running a JVM on top, you would use Java Flight Recorder.
The tools exist, and in a container world no one should be running those tools directly anyway, it is all about rootless immutable containers, ideally without OS layers for deployment.
Not just that. Unikernel sounds ok if all your linux server does is run one service that doesn't need anything but tcp/ip, and whose users don't need extra resources. But being able to deploy a new binary and other files through any means (ssh, pipelines, git), having localized log files, etc., are more than a convenience. Sure, I could log on another server, run a db on another server, and deploy html/css/js from yet another server, but would that make life easier? And this is just a really simple case. If you have multiple services, you'll need multiple servers with a uni-kernel approach, or build some monster thing. It's horses for courses.
Welcome to cloud computing, where software is cattle, not pets.
There isn't a Linux server taking care of a couple of services.
There are services deployed as Kubernetes pods, or serverless language runtimes across a cluster abstraction, running directly on top of type 1 hypervisors, or minimal kernel images enough to power distroless container images.
And before you know it you've built a very complex, and not very secure, operating system.
Depends on the skills on managing EC2, AKS,...
Let me know when your local OS does multi-machine scheduling across heterogeneous hardware and application constraints, networking, observability, service accounts and a finer grained permissions system, in a way that’s far less confusing and piecemeal.
Also, this argument only works if you just broaden the definition of “OS” to “anything that runs that isn’t your apps”.
> Let me know when your local OS does ...
Sounds like Plan 9. Which would put that around 30ish years ago. Mainframe OSes can probably make similar claims even earlier.
maybe we _should_ be thinking about building proper distributed operating systems instead of loose collections of services. we really missed an opportunity to rethink the OS interface for 'the cloud'. or rather we kind of wasted alot time and maybe we should get our shit together and do it properly. no matter how much we chant 'cattle not pets' everyone is still installing individual nodes and logging in and puttering around.
Agreed! There were many promising distributed operating systems, most got stuck at the research phase however. But the cloud pretty much killed them all.
Not that many people keep large distributed workloads anymore, and SLURM-like schedulers on top of regular operating systems were deemed good enough for supercomputers.
not always. Asci-red had a distributed version of OSF/1. this wasn't a great design because it was built on distributed shared memory, so it had fault tolerance and performance problems. the MTA had a shared memory monolithic BSD, but that was almost a forced choice.
The Cray T3D had a really nice distributed services model (really kind of a distributed exokernel) that would be worth emulating
on some machines there was an interactive ganged run facility like 'mpirun'. I think even this distinction where there are 'login' or 'service' nodes and anonymous runners would be at least a little structure. I guess you could argue that k8s is this, but it kind of falls short and seems to require a lot of continuous propping up.
I worked on a single system image project that didn't ever get shipped. it turns out that a syscall proxy layer isn't that difficult to implement, and that leaves you with an undisturbed user space.
anyways, there are a whole host of design options that don't look like k8s or slurm. and having spent some time recently on non-hyperscaler clusters, that bottom level of bringup (provisioning, networking management, reporting, fault tolerance, distributed filesystems) is really a nightmare. I think we lost a lot of traction there by raising generations exclusively in someone else's operational context.
Yep. We say cattle not pets, but even cattle often need personal attention for one reason or another. It doesn’t mean you’ll never need that option in your back pocket
if you didn't have a multi-user operating system running on every node that basically demands this, because of its complexity and fragility, and you had the necessary introspection to look at memory usage and open sockets, and disk bandwidth, and everything else you might need to mange those processes, then the utility of your escape hatch - which also is complicated and fragile, becomes negative.
> Tools such as `strace`, eBPF, `lsof`, `ss`, etc. have tremendous value. The article mentions some nice things for observability ("structured logging, OTel") but so much gets thrown away.
Trivial to implement. I have a full custom (from scratch written) debugging/benchmarking implementation written both for Linux and two of my custom kernels, both of which are REALLY unique when it comes to architecture. My custom debugger also supports numerous inline scripting languages.
It's also not like you are stuck with a full OS when you are running Linux. You can remove features from the kernel you don't need, and you can configure a bare-bones busybox to be your userspace, or just run your application directly as the init process.
Take this with a grain of salt. I haven't run unikernels in prod, so it's all a figment of my imagination.
> Tools such as `strace`, eBPF, `lsof`, `ss`, etc. have tremendous value.
But doesn't their value lie in getting you access to the core of the OS that you wouldn't be able to touch otherwise? What if your network stack was a library you could instrument just like you do with the rest of your app code?
Also, I don't need a tool to tell me I'm being punished by the IO or CPU scheduler if I am the only one using the CPU and IO :)
If your app, is the only one running, and the whole kernel is a library, you can simply have said library instrumented with as much observability as you need, directly and you now no longer need to worry about which processes are using cpu and IO because…it’s all you.
General purpose OSses are a tax, one that people pay because it's cheaper than rolling your own.
Has the math changed? Maybe. I remember the days before multitasking and protected memory. Things were faster, but more fragile. But these days there are so many things happening that they are, on some dimensions, just as fragile.
The real cost is going to be hardening, and maybe LLMs can do that as well.
It has, a long time ago when doing serveless became fashionable, and too many realised they didn't want to manage Kubernetes clusters by hand.
So you get a type 1 hypervisor, which technically is like a micro kernel, which then boots whatever else on top including language runtimes, or chiseled containers.
Unikernels ideas aren't much different from some embedded deployment scenarios, with the application, language runtime and OS all linked together into the device firmware.
Naturally on desktop computing is another matter.
It has. In fact the current ensemble of OSs/kernels (Linux/BSDs/Windows/macOS) are pretty much DoA. They have far too much legacy decisions holding them back and the SOLE reason they were used was because rolling your own kernel _per project_ took too much time. With agentic development this has changed, anyone can now roll their own kernel that is a) faster and more optimized b) more secure c) more obscure than the current offerings [important because it makes reverse engineering harder].
I've been tempted on a few occasions. Mind you, if you're stuck in the cloud for whatever reason you have an OS under the hood anyway, but at $WORK we own the rest of the stack anyway, and it wouldn't be that much more work to hook into the virtio devices directly. It might even be less work than everything else we do to work around the kernel.
I've had limited adventures in embedded software and as part of that became a fan of Zephyr (http://www.zephyrproject.org) and wonder about its potential application as a unikernel.
On the plus side: it's a unikernel. You compile it and your application together and get a binary. It has support for running on virtual hardware (https://docs.zephyrproject.org/latest/hardware/virtualizatio...) and virtio is the new bios, right?
On the minus side: I think porting "traditional" software - say Nginx - to it will be agonising. And I can't make any guarantees about its performance in the same way that Alpine Linux container images are not necessarily fast.
But there's an entire toolchain, debugging story, drivers etc. for something that compiles to tiny and starts up basically instantaneously. Surely that's got to be useful for something?
> think porting "traditional" software - say Nginx - to it will be agonising
I think this is basically a solved problem with enough agents and enough compute now
It’s good to see this! I’m offering a cloud service to host unikernels based on my BareMetal kernel.
~$0.005 CAD/h for a small network-enabled VM with 4MiB of RAM and no disk.
https://baremetal.returninfinity.com
This is really fun to see again. I first played with baremetal around ~2013 when I was a university student. I really enjoyed message passing between small computers with the network API where you just got to write ethernet frames, no messing with any higher protocols.
Is this kernel the same one still?
I’m glad you remember it! I’ve been plugging away at it for quite some time now.
The kernel is mainly the same - just modifications for running under Firecracker.
I wonder if tailscale node can live there.
Potentially! It really depends on building it to run on BareMetal.
sign in with google on "create free account" gave a 400 error.
I might be interested in this because that is cheap, but what exactly are the networking rules/costs? ip4/6, bandwidth, egress, throughput, connection limits?
Can you try it again? I see some valid logins via Google.
None yet - this is a beta so bandwidth pricing will come later.
If you have serious attack surface minimisation goals you need to embrace FOGAs. LLM coded unikernels have way too much bloat.
The Zynq UltraScale+ parts pair a hard ARM CPU with FPGA fabric. You can progressively isolate fast path and high security components in logic. LLMs are getting quite good at using synthesis tools.
https://www.amd.com/en/products/system-on-modules/kria/k26/k...
But they're still, as far as I've seen, terrible at debugging HDL. There's just too many interconnected signals. Blocking vs nonblocking assignments and scoping are very hard for it. But I have yet to try opus 5.5 on it.
Agreed. You have to give them device-in-the-loop with trace or a decent simulator. Presumably the big EDA companies are working on tooling integration and bespoke models, and they have huge libraries of designs to draw on.
Yeah I have yet to get it to close the loop. I have some stuff theorized but everything else ends up in the way. Last time it got stuck going to the open internet because obviously I'm not letting it do that but it wouldn't stop trying. I just gave up after that. But once I can close the loop, I have big plans.
> I think dependent types are the winner for next-generation languages. Anything that lets you codify more into the type system is more back pressure.
This is one reason I'm so bullish on Lean4 as a general programming language
It's now essentially proven that expressive, strong type systems like those of Rust present an advantage for LLM assisted coding
Lean's type system, in which types are first class, fully programmable objects takes it so much further. Particularly when it comes to embracing dependant types
As mentioned before on this topic. What about debug ability? An application overflow now corrupts part of the network stack.
In an oxide episode there were some mentions of reading off data lines but I just don’t think that’s practical.
The reduced attack service is cool but not at the expense of my visibility and liveness of the system
Yeah that's generally the argument for using a managed runtime (like Ocaml in MirageOS's case, or I could see Go or even the JVM fitting here) for these kinds of things. Running a VM which has no pointers, memory access etc primitives, and is garbage collected etc direct on "metal" gives more peace of mind about that sort of thing.
You could make the argument that Rust w/ its memory safety is a candidate, but w/ Rust it's still entirely possible and fairly easy to break out of that. And Rust/Cargo applications have a habit of using a bazillion third party deps that you then need to keep a close eye on.
I’ve always thought there should be a module-level safety property in the type system such that unsafe would be a capability granted by the user’s code.
this question always comes up. I personally think the status quo is pretty weak here. I have certainly added a gdb stub to a unikernel before, but that apparently given that it was never used it wasn't the right answer.
people always talk about losing the excellent debugging facilities that linux gives you. if a service crashes in a big cloud environment what is that you do now that works so well. do you think its intractable to implement that in a single-process kernel?
I don't know if anything like this has been built, but I'd imagine you could have the hypervisor capture stack traces / log buffers / core dumps when a unikernel VM crashes.
I'm not a security researcher but I know that VMs have been the tool of choice to step through malware execution for at least the past 15 years. I recall coming across ida extensions for it.
The big thing for me is the liveness is completely gone or at least can be. In a regular box I can at least poke around at the facilities even if the process dies. I don’t think it’s impossible I just think it’s a hard problem I don’t see emphasized enough.
This is also why I'm not sure the unikernel buys you much security. If you corrupt arbitrary memory, you can execute arbitrary machine code. Doesn't matter what the unikernel exposes.
In general, sure. MirageOS is in OCaml though, where you have a memory-safe language with a small C runtime. Corrupting arbitrary memory is much less likely.
All situations have their own constraint, but I'll play devil's advocate. These are not strong opinions of mine:
Use Rust and the chances of an application overflow drop off a cliff. The debug-ability also increases the surface for bugs themselves. The majority of cases we're talking about chips that can comfortably be hardware-simulated. And/or all you need is some way for a dev machine to edit memory and an AI can work through the night to explore a configuration space much larger than you would ever have.
I generally agree Unikernels are cool as I did back in 2010 when I first came across the idea. But the idea that RCE are contained because there is no shell and no interpreter does not hold up. If they can run code on the machine they can run a compiler in your unikernel and device drivers and whatever else they will need. Granted it's much harder to do all these things through the straw of a RCE bug, but the argument that because the tools are not there they can't do damage is wrong their malicious code was not there either but they found a way to bring it in.
Sorry but the security "win" of unikernels are not binary.
Yes, in theory you have a smaller attack surface, in practice its not exactly true.
Yes, you throw away a lot of tools that you don't need, which means that sideways traversal is much much harder, it doesn't actually stop your front door being smashed down.
You are also on the hook for detecting and updating any ported library. This sounds easy right? just look for the CVEs and then re-build the image?
Ok but what version of the ported library is in your unikernel? in the porting did it have the same bug? Did the LLM cock up the porting, did
Basically you are on your own. So saying "unikernels! you're secure now" is at best misleading.
The argument is that LLMs are making unikernels easier to work with, and having a whole OS stack leaves a big attack surface, as demonstrated by the scads of LPEs and other vulns we've been seeing in Linux over the past year+, right?
But you know what else is easier with LLMs? Properly hardening your Linux system. "Patch every week" is the recommendation from upstream because the kernel security team has to consider every possible deployment configuration and environment. If you implement all the KSPP recommendations, use a config and kernel command line that makes sense for your actual application, and use a properly configured MAC LSM, you would not have been affected by any of the big blockbuster vulns of the last year.
So if you can tell an LLM to port a library in a loop and feel confident in the results, surely you can feel equally confident telling the LLM to harden your Linux environment.
n.b.: I am not endorsing the idea that just throwing LLMs at security problems is actually a good answer, I'm just saying that the case the article makes isn't actually comparing like for like.
Totally agree with this article. With AI, I think now is the perfect time to consider where Unikernels might fit into your architecture, and how you can leverage them to minimize your attack surface.
I’ve started looking at them myself, and have been comparing them to mature VMMs like Firecracker, and asking myself where each piece of my stack might be best run.
Virtual machines and containers are no longer an effective isolation mechanism when AI is involved. VM breakouts are becoming trivial. So everyone should be considering how to bake better security into the their runtimes.
>VM breakouts are becoming trivial.
How exactly?
Most human written code (even when written by experts) is quite bad. Really bad actually. If you have a compliant agent (Daybreak/abliterated or otherwise) you can just provide them with a list of most likely VM escape routes and there's a ~100% chance they'll succeed unless the code has been explicitly pruned by an LLM prior.
> VM breakouts are becoming trivial
They are not. When discovered, they are still big news. With few exceptions, businesses (and other institutions) continue to treat the cloud providers' VM-as-a-service offerings as secure. AI can aid the defenders too.
> So everyone should be considering how to bake better security into the their runtimes.
If VM isolation is not trustworthy, my first concern would be whether the other VMs on the physical host machine might do something malicious and break into my VM.
A natural r&d project would be a high frequency oms or something like an nyse bid/ask/match book manager.
These apps tend to do kernel bypass anyway for net i/o after hard scrubbing the os to remove as many interrupts and other superfluous jitter the oms does not need.
However, I believe the low level kernel bypass code (eg dpdk, libfabric) depend on Linux to talk to the hw in a disciplined way.
Anybody know better?
A decently faster oms this way could be a practical alternative to fpgas we sometimes see in that space. (hw would be co-located of course)
Reducing attack surface is definitely a plus but it is nowhere close to the number one security benefit of running unikernels.
That's why I never really liked talking about "reducing attack surface" that much because folk inevitably turn to lines and code, which while reducing is good, just simply doesn't communicate what the biggest problem truly is.
Vuln exploitation is the number one entry point for data breaches and os command injection is the number one CWE in CISA Kev from last year.
System intrusion was repeated something like 64 times in last year's DBIR.
The operating system itself is literally the problem as it's inherently meant to run many different programs whereas unikernels only run one.
> The operating system itself is literally the problem as it's inherently meant to run many different programs whereas unikernels only run one.
Unless you rely on security through compartmentalization. See: https://qubes-os.org
Honestly, I think there is a lot of similarities between unikernels, at least the one I'm active with and qubes. The big difference to me is that qubes is more desktop/consumer focused and nanos (the one I'm involved with) is more server-focused but the same sort of ideology/principles gets exposed - just at varying levels and because of the end environment they get architected differently.
I’m completely ignorant about this and likely just missing the point but: isn’t the point of an OS to not have to (vibe)code filesystems and networking by hand every time you need them? Also, how does a unikernel cooperate with other applications? Would they all live in separate networked unikernels managed by a hypervisor? And, if they all (vibe)coded their own fs and network wouldn’t that introduce subtle bugs and inconsistencies that would eventually bring the whole thing down and lead you back to the need for shared primitives in the first place?
Maybe a good halfway point is to still have a unikernel but the libraries for common stuff like networking or fs are already written according to a standard and plug and play and reusable across applications?
You don't build the networking or filesystem by hand every time you need them. You pull them from reviewed and maintained library code from a repository. That's how MirageOS etc work. Someone else has written e.g. TCP/IP -- hopefully well -- and you link against it.
The point that makes this different from an OS is that there's no shared service, no syscalls to do that stuff -- just subroutine calls -- and no timesharing (except at the hypervisor level). It's just a single runtime running on hypervisor direct against the virtualized hardware.
Because, yeah, all this work has happened in hardware in the last 30 years to make that possible, but we still treat the operating system as the best unit of resource sharing. When in fact it's kind of a jack of all trades master of none. If you're booting a whole linux kernel -- with its giant framework built to co-tenant a pile of applications and users -- just to run one process, there's definitely something to look at there. Even if the unikernel space itself is relatively immature.
You don't need to emphasize the "vibe coded every time you need them" thing. That's not what anybody was being talked about in the article. The "vibe coding" piece is pointing out that the missing pieces in the ecosystem can be more rapidly filled in now because of agentic tooling.
> running on hypervisor direct against the virtualized hardware.
Well, here is your actual OS then. You named it "hypervisor", and changed the system interface, but it's still there, revirtualizing the access to hardware and ensuring separation of different users/applications/unikernels.
I mean, sure we can put labels wherever we want. I don't really care. MirageOS even has "OS" in its name.
The point is all the other things: total number of lines of code, permissions and security, memory management, drivers, process mgmt, it all changes.
Does it though? And I'd really like to see how you'll revirtualize e.g. an NVIDIA GPU (notoriously context-full piece of hardware), while allowing transparent shared access to it from different applications. Heck, you'll run into trouble with most of PCI-connected devices except for the most primitive once.
yea, "virtualizing" an NVIDIA GPU is not truly feasible right now from what I see.
I do know that AMD has done more in this space. When I worked at Google there were (other) people on the Stadia team doing this. And some open source bits out there that I've seen, including from AMD.
Unfortunately, my real world work... and most real inference etc work others are doing... is on CUDA/NVIDIA.
This isn't too dissimilar to DOS programs. DOS also gave you a filesystem, memory management, a shell a process loder, driver interface, but it was all in real mode, and you were free to replace these components with your own as you pleased.
Or if you're more familiar with embedded, like a BSP or RTOS
Arbitrating access to the hardware between multiple tenants still gonna be a PITA (as it was during the DOS days), especially if you want to expose actual hardware, not some generic VirtIO devices. Even with the latter, the hypervisor would have to do quite a bit of the resource management and interface translation.
Unless you're literally running a single application on a dedicated piece of hardware in which case... yeah, sure, you can do the OS-as-a-library schtick. But you could do it 10 years ago, too.
That is taken care by hypervisors.
Yep. Except every application runs in a down-and-dirty virtual machine instead of a nice clean process, so they get to experience the joy that is hardware booting and get to configure virtual device buffers instead of calling send(). Progress.
Over time, what prevents the reviewed-and-maintained-library-code from developing creeping featuritis and consequently evolving the additional attack surface that unikernels currently lack?
Unikernels simply redraw abstraction boundaries. You can still use libraries and third-party code to implement common primitives. In fact, that's exactly what many unikernel frameworks give you (unikraft, hermit, mirage)
Are their abstraction boundaries protected by virtual memory and syscall interface such the submodules of a program cannot corrupt parts of memory that belong to others? Can I restart subtasks without affecting others? A traditional OS gives me all of that with relatively minimal overhead.
With a capability-based microkernel you get to assign ownership and resources to a very granular level that increases robustness against crashes and protection against attacks to the core system policy. A unikernel can give all of that but I feel like then it will be worse to program against rather than a microkernel.
So overall I don't see any advantage of unikernels. They don't help you write more correct programs IMO and the performance gains are limited.
I guess the others have made these points, but since unikernels run in VMs with virtual hardware, I guess a virtual ethernet card can have an interface that's easy to program for, making the driver trivial. You're not talking to real hardware in any case.
Similarly, a filesystem's just a data structure that happens to be on disc. You can mmap your harddrive and let the host hypervisor handle swap, preemption etc.
It's similar to a process contract, but with slightly different boundaries
The thing I don't get: We'll now run those N unikernel things in VMs with a hypervisor underneath. How is that different from running static binaries in a locked-down OS, i.e., with a traditional kernel? You can surely break out of a unikernel by "popping the kvm" as the article mentions. How is that more secure? In principle we are running the same things, just call them differently (app becomes unikernel+app, kernel becomes hypervisor).
It's conceptually equivalent but the details differ. The program has access to more abilities and in terms of common conventions the security boundary is much more robust. If nothing else I'd argue it's a superior abstraction with which to make the delineation.
Both have weaknesses, but with an OS, you have certain bottlenecks - like your app memory is paged and has its own memory space, data structures are often duplicated in user and kernel memory, syscalls have cost, which is why io_uring is a thing in Linux.
This is all unnecessary overhead and complexity, as the OS doesn't really mediate hardware access in this case.
But you're right on some points, it's not really more secure, but there's a hell of a lot less complexity where things can go wrong.
It is not any different. Actually, it is worse because a VM is a much worse environment to run a application since you need that unikernel stub as well as the application.
The only advantage is that cloud services provide VM tenants so you are already stuck with a VM substrate and you want to minimize layers from that point on.
However, as a practical matter, the design of the Linux kernel is grossly insecure and basically impossible to make even remotely safe as evidenced by the unending sprawl of LPEs. Hypervisors are generally designed and configured in a way that is only mostly insecure instead of grossly insecure; a bad is better than terrible situation.
However, there is nothing stopping you from designing a kernel in a similar way and providing shared tenant processes with similar guarantees. You just need to port your applications to the new operating system, but you would also need to port them to a unikernel anyways if you wanted to go that route.
The only free action is lumping your applications with the full OS and dumping it onto a hypervisor. Anything else has porting work which is why the multi-tenant VM model is advantageous at all in the first place.
Unikernels are on the smaller side of the spectrum; Linux on the larger side, with a lot of room in the middle...
For that matter, the actual Linux kernel need not be large at all. It's really the user space stuff that takes up the bulk of what people think of as the largess of Linux.
Still, there's a whole pile of stuff there that many applications don't need. If you truly can run the entire of the OCaml runtime and standard libraries etc without it, and you're happy writing your apps in OCaml... MirageOS has always felt winning-ish to me as a general idea.
And to TFA's point, now with LLMs the relative exotic nature of OCaml as a platform need not be a huge hindrance. I like the language, but have only dipped my toes in. Definitely tempting to try writing apps on it w/ MirageOS now. Though I doubt I'd convince anybody to pay me to do that...
I was involved a bit in Unikernel Linux (https://github.com/unikernelLinux/) which is basically a Linux kernel where you can run a single application linked with the kernel. It can run baremetal or as a guest under KVM. It's both the best and worst of all worlds.
A lot of comments here about observability. Well UKL solves this nicely because you can run regular userspace processes in the unikernel, thus you can (optionally of course) run a shell, perf, and anything else you need to observe what the machine is doing.
This is exactly right.
I saw a post that said roughly "I thought computing was doomed to be 'Linux, Python, and HTTP' for the rest of my life. Now we're free". That's it. Let it sink in. We're free. We can make the computer do whatever we want.
Cast off the yokes back-compat and history
I would vouch they are already here for some of us, and they are easy actually.
Type 1 hypervisors running language runtimes on top, for serverless computing, aren't much different from the unikernels vision.
I've been using OCaml for a couple of years now but haven't dug into Mirage yet. It is super cool. Loved the anecdote about Yaron. Vibecoding in OCaml feels like a superpower. Adding Oxcaml and having your whole company on that must feel very good.
The only time you didn't lie was when mentioning seL4.
You're Linus arguing against Tanenbaum. Even if you win, you still lose.
i'm sorry what? eli5?
This was a famous debate between Tanenbaum and Torvalds on the pros/cons of micro vs monolithic kernel OSes.
https://en.wikipedia.org/wiki/Tanenbaum%E2%80%93Torvalds_deb...
It may be time to revisit cosmopolitan (https://github.com/jart/cosmopolitan) and redbean (redbean.dev - although the tls certs are expired and it seems scrubbed off of github).
The two choices offered are a false delimma. The MILS kernels that came before seL4, like INTEGRITY RTOS and LynxSecure, had middleware to make deplpyments easier. The open-source solutions mixed standalone apps with user-mode VM's for Linux compatibility. OKL4 let one use drivers across them.
Looking at what's out there, I think GenodeOS should be mentioned for application-specific systems. They can even build some of their stuff on seL4 if you want.
https://genode.org/
I'll also add there's been unikernel projects that aren't in exotic languages. Even C++ (IncludeOS?). There's probably one in Rust by now. If not, it could probably build on Redox's components.
I’ve been thinking about this but I’m not sold on any of the arguments presented
I was a swing vote and you lost my support in the first debate
What I care most about is benchmarks. I see AI rewriting its harnesses for the hardware it’s on and getting faster tokens/sec, and I can see that as the future. I think, okay we can go deeper
Nothing in this article addressed results, its just breaking every known convention in the scariest and most inconvenient way possible while claiming that the attack surface is smaller when its actually unknown with now limited debugging capability
I could overlook that if I got more access to RAM this way and wildly improved performance
I do wonder what kind of wins we're going to see from unikernels.
I used to regard V8 Isolates as a best possible sort of technology, with userlands juggling lots of processes.
Seeing netlify & unikraft switch to microvm's and have such a huge speed up was a bit of an awakening for me. Those are really fast start times! https://www.netlify.com/blog/edge-functions-firecracker-micr... https://unikraft.com/customer-stories/edge-functions-netlify...
Intuitively, I think I have some appreciation for how much silicon has been poured into virtualization. Its always seemed like a "yeah but you could avoid those costs by not doing that" but I'm more receptive to the idea that these might in some cases be really good ways to get some of the isolation workloads demand with the hardware helping us out, these days. There's so much securing for vm's, and maybe it's just easier than trying to secure in userlands: let the hardware help.
Long time interest in microvm's but they felt heavier weight than I wanted. Now it feels like maybe they might actually in some regards in some ways be lighter weight than managing workloads yourself in userland. Maybe. I dunno. Interesting times, i'm open to it.
Edit: I just chatted with Astra Pro some about these ideas, if anyone wants some sense material to chew on. https://chatgpt.com/share/6aca924c-8e64-83ea-a5c3-aae08b2cdf...
One of the problems is there's only a small certain percentage of scenarios/applications that benefit from very short start times.
I'd wager most of the services running on the interwebs are web servers, database servers, inference servers etc that don't change over their lifetime really. They're just doing the same thing all day long every day.
If you're doing stuff like what I'm doing for work right now, which is, yeah, multiplexing potentially oodles of user-submitted jobs, and those jobs are best expressed as distinct images or containers, then yes, managing start times is absolutely imperative in improving utilization/occupancy and therefore reducing costs.
But I'm not convinced that's a typical scenario, not typical enough to drive enough time and money investment in this space maybe?
Also there ain't currently no real "hypervisor" for the (NVIDIA) GPU. Not practically anyways. And that's arguably where we need it the most. Or at least I do, for Day Job(tm).
So unikernels and microvms may have to lean on other arguments for adoption: security and simplicity-to-reason-about might be those...
Agreed. The vast majority of prod workloads are persistent services that we want running 24/7.
> I do wonder what kind of wins we're going to see from unikernels.
I come from security so I'm more disposed towards the benefits thereof, however, being a person that has had a commercial and open source unikernel presence for a while now - "ease of use" is probably the largest selling point that I see from most of our user base. There is nothing out there that gives you the performance, security and cost benefits from shipping your unikernels to AWS, GCP, etc. in seconds. Not lambda, not cloudflare, not any of the millions of microvm providers. You quote netlify right here (who actually don't use unikernels per-se but small little linux vms (big difference!)) but why would you even use them when you can do it cheaper by shipping straight to ec2?
That's what unikernels allow you to do.
I was thinking that we don't need programs to share CPUs anymore now that each machine has so many of them. We could have an OS which binds each program to one CPU. When it runs out of CPUs, it suggests to close an existing program. Then programs don't need to pay context-switching cost.
it would be per process. We don’t have that many cores on our machines, check “ps”.
A typical desktop OS runs ~100 processes before you get to run anything. Even if you would delegate one core to system stuff, most programs are multi-threaded, so you are going to run out of cores quickly.
Yeah but the OS I'm speaking of would not be a typical OS.
Is it a good thing to have so many processes that the user doesn't understand and often isn't even aware of?
If it's a security concern now, think of how bad of a problem it will be after AI is writing most programs that people use.
It's a shame a multiple physical CPU machine never took off for workstations. It would have been awesome if you could delegate a real weaker CPU (physical die) for OS background tasks and then had a stronger CPU for userspace workloads.