Ha, add me to the list. I built one with libvirt and cloud-init too, mostly focused on network containment (private/host networks are blocked) and VM hardening. There's also a check command that runs nmap from inside the VM to verify nothing private is reachable.
Heh, yeah also very similar. I don't strip devices like you are - I actually want the memory balloon functionality for example.
My working threat model for my projects is less about "malicious agent trys to escape VM" and more "well meaning agent got confused and screwed up" - the added protection of a VM is desirable, and it's useful for stuff like nested virtualization (agent working on improving the VM orchestration), makes containers inside the sandbox straightforward
I hadn't seen the stuff about filtering spoofed packets so I'll probably add that.
One thing I'm struggling with at the moment which I may as well ask here.
Many of my projects use pnpm, which ideally uses hard links from a shared store of dependencies.
One of my goals is to reduce the disk overhead of VM based isolation, so I'd like a shared pnpm store across projects (and the same for maven, etc as well)
With https://virtio-fs.gitlab.io/ mounts, each mount inside the VM is a separate FS, and I can't hard link across them. I've tried playing around with ideas like bind mounting the directory structure from the host in a folder to have a single virtio-fs mount inside the VM - but the bind mounts still can't be hard linked across it seems.
I'm basically wondering if I'm missing something or just trying to do something that isn't possible.
All the host folders are on the same FS - but I don't want to organise the host system around the VM mounting needs, and I also don't want to mount extraneous directories into the VM (eg: simple solution could be just mount ~/projects but kinda self defeating)
Add me to the list, although I use podman containers (https://github.com/evertheylen/probox). I find that the hard part is not restricting the container/VM, but to make it "easy" to do things in the container that require something protected. How will you commit? Where does your agent run, and can it access github? How will you deploy? How will you SSH to a server?
Happy to hear what other people are thinking. It seems like putting an intercepting HTTP proxy is getting popular in other projects.
It feels like a fundamental problem. The more you restrict the agents, the less useful they become. The less you restrict them, the more dangerous they become.
It makes sense, but a good system where you can say "give this VM access to repo X, branch Y, and server Z for the next 4 hours" without leaking any secrets into the VM would be a major improvement already.
On the git access part, I don't let my agent VMs have write access to GitHub currently. I do give them dedicated users on a self hosted gitea instance, and basically manage the blast radius through branch protection etc.
I think at a high level I'm trying to treat them as junior/intern employees where I give them enough permission to do their role, but not enough rope to hang themselves.
I see. But then I have to debug some failing CI pipeline and the agent is quite good at going through the logs, so I give in and `gh auth login` anyway...
I see nothing this brings over smolvm machines. I did not use lima / krunkit outside colima but in colima it was just slow, had outdated devx and was unreliable compared to controlling every detail with an embedded vm library.
This is something that could run on top of smolvm, if the authors chose it, rather than Lima/QEMU. It both sets up virtual machines and what goes into them, letting you chooose toolchains and coding agents. smolvm doesn't provide dev tool setup within a vm, though it does point to images that already have toolchains and agents installed.
I have built my agent harness around smolvm and it has been working wonderfully.
I have seen a single bug up to now (iirc you cannot take a snapshot from a machine you rehydrate from a pack) but found a workaround.
I think the only thing I would love a solution for is the ability to bind credentials even when you use tailscale. You cannot right now because the traffic goes through the tailscale interface.
I tried to develop an electron app this way; ubuntu VM (using qemu/multipass) on macos as host with Intellij IDEA. Couldnt make it work properly. Sure, I can install all the toolchain in the VM, and Intellij would recognize it (they have a “remote dev” setup), but for instance for electron apps you cannot just run a headless vm, and there is no convinient way to make a change in your mac, hot reload in the VM, and interact with the electron app in your mac.
One of the fake recruiters contacted me on LikedIN for a Solidity dev role, with an assignment repo that came as a zip in DM. Everything looked fine since Finder only showed .sol files, but Finder doesn't show .git and other hidden folders by default. However .git was there, and it had a hook that ran eval() on a remote address on commit/push.
As I read it, .git protection stops the VM from changing hooks, and --clone leaves the hook behind since a clone doesn't copy hooks. However, the hook is still in the folder on the host. Does devvm inspect and warn when a folder already has hooks before you start?
This looks quite similar to docker sandbox (sbx). It uses Lima for VMs under the hood. Windows advertised as not supported (Lima is missing features on Windows).
Like Docker Sandbox this meets what I think should be our standards in the AI age: kernel isolation and secret shielding in addition to network and disk isolation.
I have been working on a similar project because I haven't seen one before that to me didn't have security design deficiencies. This project looks to have made good design decisions around security.
The things that are design choices for these tools are
* agent integration
* docker image composition
* are vms more persistent or throw away
* managing configuration
It does look like it has some design decisions that are optimal for convenience but might lead to inflexibility. It assumes 1 VM per directory. It is trying to auto-detect what someone needs in an image and provide well integrated images and tools out of the box but I don't see how to just provide my own image.
I missed the one directory part, I'd definitely recommend making that more flexible. With my equivalent of this I often mount multiple repositories into a single VM, two examples:
- a bunch of gitops repos: terraform mono repo, argocd-esk conf repo, grafana provisioning repo, etc.
- frontend + backend repo (tbh may merge these to a mono-repo, but here we are)
I think the auto detection is interesting, but tbh with my solution I've been happy with just install all the tools regardless of what a specific project needs - with a COW base layer this is cheap.
I just made my own as an excuse to practice my podman skills. It's saved me countless times over and over. It doesn't have git installed at all and doesn't even mount .git so I feel safe nothing can be FUBAR'd.
It's probably a fine project, but the instructions are like "trust this thing, give us your login" , which still gets alarm bells ringing in my head pretty fast.
systemd is not the only thing using linux namespaces. For example, the suggested bubblewrap uses namespaces to allow isolated networks (--unshare-net) and changing rootfs (--bind /mnt/foo /)
Damn, I've been building something very similar the past few weeks - haven't quite gotten to a public release yet.
Mines based on libvirt and uses cloud-init for creating the images, with shared golden images across projects.
Skimming your docs it wasn't totally clear to me if your images are built per project, or COW from shared toolchains.
Looks like you have a few features in there that I might take some inspiration from though, eg the proxying stuff.
Funny how many of us are trying to solve the same problem at the moment.
Ha, add me to the list. I built one with libvirt and cloud-init too, mostly focused on network containment (private/host networks are blocked) and VM hardening. There's also a check command that runs nmap from inside the VM to verify nothing private is reachable.
https://github.com/ertug/kvm-sandbox
Heh, yeah also very similar. I don't strip devices like you are - I actually want the memory balloon functionality for example.
My working threat model for my projects is less about "malicious agent trys to escape VM" and more "well meaning agent got confused and screwed up" - the added protection of a VM is desirable, and it's useful for stuff like nested virtualization (agent working on improving the VM orchestration), makes containers inside the sandbox straightforward
I hadn't seen the stuff about filtering spoofed packets so I'll probably add that.
One thing I'm struggling with at the moment which I may as well ask here.
Many of my projects use pnpm, which ideally uses hard links from a shared store of dependencies.
One of my goals is to reduce the disk overhead of VM based isolation, so I'd like a shared pnpm store across projects (and the same for maven, etc as well)
With https://virtio-fs.gitlab.io/ mounts, each mount inside the VM is a separate FS, and I can't hard link across them. I've tried playing around with ideas like bind mounting the directory structure from the host in a folder to have a single virtio-fs mount inside the VM - but the bind mounts still can't be hard linked across it seems.
I'm basically wondering if I'm missing something or just trying to do something that isn't possible.
All the host folders are on the same FS - but I don't want to organise the host system around the VM mounting needs, and I also don't want to mount extraneous directories into the VM (eg: simple solution could be just mount ~/projects but kinda self defeating)
(Fedora workstation host)
Add me to the list, although I use podman containers (https://github.com/evertheylen/probox). I find that the hard part is not restricting the container/VM, but to make it "easy" to do things in the container that require something protected. How will you commit? Where does your agent run, and can it access github? How will you deploy? How will you SSH to a server?
Happy to hear what other people are thinking. It seems like putting an intercepting HTTP proxy is getting popular in other projects.
It feels like a fundamental problem. The more you restrict the agents, the less useful they become. The less you restrict them, the more dangerous they become.
It makes sense, but a good system where you can say "give this VM access to repo X, branch Y, and server Z for the next 4 hours" without leaking any secrets into the VM would be a major improvement already.
On the git access part, I don't let my agent VMs have write access to GitHub currently. I do give them dedicated users on a self hosted gitea instance, and basically manage the blast radius through branch protection etc.
I think at a high level I'm trying to treat them as junior/intern employees where I give them enough permission to do their role, but not enough rope to hang themselves.
I see. But then I have to debug some failing CI pipeline and the agent is quite good at going through the logs, so I give in and `gh auth login` anyway...
Somebody shared this in another thread: https://pleasedonotescape.com/
I guess they can add all the sibling comments to the list, as well as mine: https://github.com/AdrianVollmer/Agent-Sandbox
(Python, Podman, Squid for network filtering)
I see nothing this brings over smolvm machines. I did not use lima / krunkit outside colima but in colima it was just slow, had outdated devx and was unreliable compared to controlling every detail with an embedded vm library.
This is something that could run on top of smolvm, if the authors chose it, rather than Lima/QEMU. It both sets up virtual machines and what goes into them, letting you chooose toolchains and coding agents. smolvm doesn't provide dev tool setup within a vm, though it does point to images that already have toolchains and agents installed.
smol supports docker so i can use any docker based toolchain setup i want and that is not tied to a new niche project
I also moved to smolvm.
That project is underrated.
I have built my agent harness around smolvm and it has been working wonderfully.
I have seen a single bug up to now (iirc you cannot take a snapshot from a machine you rehydrate from a pack) but found a workaround.
I think the only thing I would love a solution for is the ability to bind credentials even when you use tailscale. You cannot right now because the traffic goes through the tailscale interface.
Anyway. smolvm is simply amazing.
I tried to develop an electron app this way; ubuntu VM (using qemu/multipass) on macos as host with Intellij IDEA. Couldnt make it work properly. Sure, I can install all the toolchain in the VM, and Intellij would recognize it (they have a “remote dev” setup), but for instance for electron apps you cannot just run a headless vm, and there is no convinient way to make a change in your mac, hot reload in the VM, and interact with the electron app in your mac.
One of the fake recruiters contacted me on LikedIN for a Solidity dev role, with an assignment repo that came as a zip in DM. Everything looked fine since Finder only showed .sol files, but Finder doesn't show .git and other hidden folders by default. However .git was there, and it had a hook that ran eval() on a remote address on commit/push.
As I read it, .git protection stops the VM from changing hooks, and --clone leaves the hook behind since a clone doesn't copy hooks. However, the hook is still in the folder on the host. Does devvm inspect and warn when a folder already has hooks before you start?
This looks quite similar to docker sandbox (sbx). It uses Lima for VMs under the hood. Windows advertised as not supported (Lima is missing features on Windows).
Like Docker Sandbox this meets what I think should be our standards in the AI age: kernel isolation and secret shielding in addition to network and disk isolation.
I have been working on a similar project because I haven't seen one before that to me didn't have security design deficiencies. This project looks to have made good design decisions around security.
The things that are design choices for these tools are
It does look like it has some design decisions that are optimal for convenience but might lead to inflexibility. It assumes 1 VM per directory. It is trying to auto-detect what someone needs in an image and provide well integrated images and tools out of the box but I don't see how to just provide my own image.
I missed the one directory part, I'd definitely recommend making that more flexible. With my equivalent of this I often mount multiple repositories into a single VM, two examples:
- a bunch of gitops repos: terraform mono repo, argocd-esk conf repo, grafana provisioning repo, etc.
- frontend + backend repo (tbh may merge these to a mono-repo, but here we are)
I think the auto detection is interesting, but tbh with my solution I've been happy with just install all the tools regardless of what a specific project needs - with a COW base layer this is cheap.
I just made my own as an excuse to practice my podman skills. It's saved me countless times over and over. It doesn't have git installed at all and doesn't even mount .git so I feel safe nothing can be FUBAR'd.
How does it compare to existing similar products?
`sbx`, safehouse and the like
It's probably a fine project, but the instructions are like "trust this thing, give us your login" , which still gets alarm bells ringing in my head pretty fast.
Should probably add it to the long list of projects here: https://pleasedonotescape.com/
Cool, but I am not installing a curl script, make it a package through managers.
thanks but i will just use systemd with linux namespaces.
That's a layer of protection, but it still exposes your real Linux kernel to the sandbox, and that's more CVEs than I'd consider ideal.
There are yet thousands of these so-called "sandbox" projects that exists. This one sounds rather really wasteful on resources?
Bubblewrap would just be a better alternative for this.
bwrap has a lot of sharp edges that can bite you, if you're not aware of them.
bubblewrap is linux only and offers a fraction of isolation of a microvm
right chain of tought, but systemd is the only to truly leverage linux namespaces capabilities, allowing private networks, other rootfs, etc
systemd is not the only thing using linux namespaces. For example, the suggested bubblewrap uses namespaces to allow isolated networks (--unshare-net) and changing rootfs (--bind /mnt/foo /)