False. Fil-C secures C and C++. It’s more comprehensively safe than Rust (Fil-C has no escape hatches). And it’s compatible enough with C/C++ that you can think of it as an alternate clang target.
Fil-C is impressive and neat, but it does add a runtime to enforce memory safety which has a (in most cases acceptable) cost. That's a reasonable strategy, Java and many other langs took this approach. In research, languages like Dala are applying this approach to safe concurrency.
Rust attempts to enforce its guarantees statically which has the advantage of no runtime overhead but the disadvantage of no runtime knowledge.
> but in practice fails, because of pervasive use of `unsafe`.
Yes, in `unsafe` code typically dynamic checks or careful manual review is needed. However, most code is not `unsafe` and `unsafe` code is wrapped in safe APIs.
I'm aware C already has a runtime, this adds to it.
> Yes, in `unsafe` code typically dynamic checks or careful manual review is needed. However, most code is not `unsafe` and `unsafe` code is wrapped in safe APIs.
Those are the excuses I heard from C++ programmers for years.
Memory safety is about guarantees enforced by the compiler. `unsafe` isn't that.
The stuff Fil-C adds is on the same footing as `unsafe` code in Rust- its implementation isn't checked, but its surface area is designed so that (if the implementation is correct) the rest of the program can't break it.
Whether the amount and quality of this kind of code is comparable between the two approaches depends on the specific programs you're writing. Static checking, which can also be applied in more fine-grained ways to parts of the runtime (or its moral equivalent) is an interesting approach, depending on your goals.
> The stuff Fil-C adds is on the same footing as `unsafe` code in Rust- its implementation isn't checked, but its surface area is designed so that (if the implementation is correct) the rest of the program can't break it.
It’s not the same.
The Fil-C runtime is the same runtime in every client of Fil-C. It’s a single common trusted compute base and there’s no reason for it to grow.
On the other hand Rust programmers use unsafe all over the place, not just in some core libraries.
Yeah, that's what I meant by "depends on the specific programs you're writing." Confining unsafe Rust to core libraries is totally something people do.
I mean, again, yeah. I specifically compared the safe API/unsafe implementation aspect, not who writes the unsafe implementation.
To me the interesting thing about Rust's approach is precisely this ability to compose unrelated pieces of trusted code. The type system and dynamic semantics are set up so that things don't just devolve into a yolo-C-style free-for-all when you combine two internally-unsafe APIs: if they are safe independently, they are automatically safe together as well.
The set of internally-unsafe APIs you choose to compose is a separate question on top of that. Maybe Rust, or its ecosystem, or its users, are too lax about this, but I'm not really trying to have that argument. Like I mentioned in my initial comment, I find this interesting even if you just apply it within a single trusted runtime.
There’s no reason to believe that one program is inherently representative. sudo-rs eschews dependencies and so is likely to be higher than most programs.
Furthermore, 170 uses in a 200 line program vs a one million line program are very different. I don’t know off hand how big sudo-rs is.
Even in embedded OS kernels, it’s often around 1%-5% of code. Many programs have no direct unsafe code at all.
> Fil-C achieves this using a combination of concurrent garbage collection and invisible capabilities (each pointer in memory has a corresponding capability, not visible to the C address space)
In almost all uses of C and C++, the language already has a runtime. In the Gnu universe, it's the combination of libgcc, the loader, the various crt entrypoints, and libc. In the Apple version, it's libcompiler_rt and libSystem.
Fil-C certainly adds more to the runtime, but it's not like there was no runtime before.
It makes it a lot less performant and there is no avoiding or mitigating that downside. C++ is often selected as a language instead of safer options for its unusual performance characteristics even among systems languages in practice.
Fil-C is not a replacement for C++ generally, that oversells it. It might be a replacement for some C++ software without stringent performance requirements or a rigorously performance-engineered architecture. There is a lot of this software, often legacy.
Fil-C’s perf sucks on some workloads. And it doesn’t suck on others.
Extreme examples to give you an idea:
- xzutils had about 1.2x overhead. So slower but totally usable.
- no noticeable overhead in shells, systems utilities, ssh, curl, etc. But that’s because they’re IO bound.
- 4x or sometimes maybe even higher overheads for things like JS engines, CPython, Lua, Tcl, etc. Also OpenSSL perf tests are around 4x I think.
But you’re on thin ice if you say that this is a reason why Fil-C will fail. So much of Fil-C’s overhead is due to known issues that I will fix eventually, like the function call ABI (which is hella cheesy right now because I just wanted to have something that works and haven’t actually made it good yet).
> It makes it a lot less performant and there is no avoiding or mitigating that downside.
You can’t possibly know that.
> C++ is often selected as a language instead of safer options for its unusual performance characteristics even among systems languages in practice.
Is that why sudo, bash, coreutils, and ssh are written in C?
Of course not.
C and C++ are often chosen because they make systems programming possible at all due to their direct access to syscall ABI.
> Fil-C is not a replacement for C++ generally, that oversells it.
I have made no such claim.
Fil-C means you cannot claim - as TFA claims - that it’s impossible to make C and C++ safe. You have to now hedge that claim with additional caveats about performance. And even then you’re on thin ice since the top perf problems in Fil-C are due to immaturity of its implementation (like the fact that linking is hella cheesy and the ABI is even cheesier).
> It might be a replacement for some C++ software without stringent performance requirements or a rigorously performance-engineered architecture. There is a lot of this software, often legacy.
It’s the opposite in my experience. For example, xzutils and simdutf have super lower overhead in Fil-C. In the case of SIMD code it’s because using SIMD amortizes Fil-C’s overheads.
> C and C++ are often chosen because they make systems programming possible at all due to their direct access to syscall ABI.
Surely Fil-C cannot provide direct access to syscalls without violating the safety guarantee. There must be something ensuring that what the kernel interprets as a pointer is actually a valid pointer.
> Fil-C means you cannot claim - as TFA claims - that it’s impossible to make C and C++ safe. You have to now hedge that claim with additional caveats about performance. And even then you’re on thin ice since the top perf problems in Fil-C are due to immaturity of its implementation (like the fact that linking is hella cheesy and the ABI is even cheesier).
The world of compilers is littered with corpses of projects that spent years claiming faster performance was right around the corner.
I believe you can make it faster, but how much faster? We'll see.
I think these types of compatibility layers will be a great option moving forward for legacy software. But I have a hard time seeing the case for using Fil-C for new code: all the known disadvantages of C and C++, now combined with performance closer to Java than Rust (if not worse), and high difficulty interoperating with other native code (normally C and C++'s strength!), in exchange for marginal safety improvements over Rust (minus Rust's more general safety culture).
edit: I feel bad writing such a dismissive comment, but it's hard to avoid reacting that way when I see unrealistically rosy portrayals of projects.
> Surely Fil-C cannot provide direct access to syscalls without violating the safety guarantee. There must be something ensuring that what the kernel interprets as a pointer is actually a valid pointer.
This is exactly what Fil-C does.
> all the known disadvantages of C and C++
The main disadvantage of C and C++ is unsafety and fil-C comprehensively fixes that.
> edit: I feel bad writing such a dismissive comment, but it's hard to avoid reacting that way when I see unrealistically rosy portrayals of projects.
How is my portrayal unrealistically rosy?
Even the fact that you know what the current perf costs are is the result of me being brutally honest about its perf.
Okay, I just checked. It does not. I wrote: "There must be something ensuring that what the kernel interprets as a pointer is actually a valid pointer." And sure enough, your runtime manually wraps each Linux syscall to do exactly that:
ioctl is even harder because some ioctls take pointers to structs that themselves contain pointers; on Linux that includes v4l2 and mmc. It looks like you don't handle that properly, judging by:
My point is: having to go through wrapper functions is not what I'd call "direct" access to "ABI". (Also, the wrappers don’t even wrap the syscall ABI directly; they wrap the libc ABI that in turn wraps syscalls.)
You might object that the wrappers are thin enough that they still count as direct. While that's a matter of definitions, I think my previous comment made it clear what _I_ meant when I questioned "direct", given my followup sentence about "actually a valid pointer".
But beyond quibbles about who meant what, this lack of directness matters because it implicates the portability of your approach.
At least as currently implemented, you rely on compiling almost everything (even libc) inside the sandbox, while having a ‘narrow waist’ of syscall wrappers mediating access between the sandbox and the outside world. That should work for most use cases on Linux, where there's already an assumption that different processes can have completely different library stacks, and static linking is common. Even if you want to do a GUI application you should be able to recompile the entire GTK or Qt stack inside the sandbox, and it doesn’t matter if other apps are using different versions or GTK or Qt.
But what about other operating systems? For server and CLI software you can probably still get away with exposing a small syscall/libc API, similar to Cosmopolitan (though that will still require significant effort for each OS). But for GUIs and platform integration more broadly, you’re expected to use the platform-provided libraries that live in your address space. They are usually proprietary, and even when they’re not, the system isn’t designed for multiple versions of the libraries to coexist.
I know I’m not telling you anything you don’t already know. But it’s an important point, because aside from performance, the _other_ big reason that Rust relies on user-written unsafe code is for FFI. If anyone can write their own FFI bindings, as opposed to making all FFI bindings live in a centralized runtime, then it becomes more feasible to scale the mammoth task of writing safe wrappers for all those ABIs. Your approach explicitly rejects user-written unsafe code, so I don’t know how you can possibly end up with reasonable OS library coverage.
Now sure, you didn’t claim anything about GUIs or portability. Perhaps this is more on topic for my previous comment’s point about “high difficulty interoperating with other native code” which you didn’t rebut. But it’s also relevant to “direct access to syscall ABI”, because if there _were_ some way to provide direct access to syscall ABI while remaining memory-safe, then the same approach would probably extend to other system ABIs. For example, a fully CHERI-aware system actually would allow for that. It’s an unfair comparison, because CHERI assumes cooperation from both the hardware and the OS, while you’re trying to run on existing hardware and OSes. I have no idea if we’ll ever see CHERI in general purpose systems. But in exchange, CHERI achieves something that’s otherwise impossible: combining direct system access and memory safety. And I originally read your comment as claiming to do the impossible.
It’s true that the Fil-C approach focuses on having a memory safe userland with no exceptions, which means no FFI to unsafe code (aside from the Fil-C runtime itself, which is just syscalls wrappers and a small handful of other things).
I have considered what FFI to native could look like, and have so far rejected it because it hurts the “no unsafe code” purism.
You’re right that this limits me to those OSes where the syscalls themselves are an adequate ABI. Linux isn’t the only such OS. It just happens to be the only OS Fil-C supports right now.
One of these days, a project will catch on that's vastly simpler than any memory solution today, yet solves all the same problems, and more robustly too, just like how it took humanity thousands of years to realize how to use levers to build complex machines. The solution is probably sitting right under our noses. I'm not sure it's your project (maybe it is) but I bet this will happen.
That’s a really great attitude! And I think you’re right!
I think in addition to possibly being the solution to safety for someone, Fil-C is helping to elucidate what memory safe systems programming could look like and that might lead to someone building something even better
Fil-C is impressive and neat, but it does add a runtime to enforce memory safety which has a (in most cases acceptable) cost. That's a reasonable strategy, Java and many other langs took this approach. In research, languages like Dala are applying this approach to safe concurrency.
Rust attempts to enforce its guarantees statically which has the advantage of no runtime overhead but the disadvantage of no runtime knowledge.
Rust attempts to enforce guarantees statically, but in practice fails, because of pervasive use of `unsafe`.
Fil-C doesn't "add a runtime". C already has a runtime (loader, crt, compiler runtime, libc, etc)
> but in practice fails, because of pervasive use of `unsafe`.
Yes, in `unsafe` code typically dynamic checks or careful manual review is needed. However, most code is not `unsafe` and `unsafe` code is wrapped in safe APIs.
I'm aware C already has a runtime, this adds to it.
> Yes, in `unsafe` code typically dynamic checks or careful manual review is needed. However, most code is not `unsafe` and `unsafe` code is wrapped in safe APIs.
Those are the excuses I heard from C++ programmers for years.
Memory safety is about guarantees enforced by the compiler. `unsafe` isn't that.
The stuff Fil-C adds is on the same footing as `unsafe` code in Rust- its implementation isn't checked, but its surface area is designed so that (if the implementation is correct) the rest of the program can't break it.
Whether the amount and quality of this kind of code is comparable between the two approaches depends on the specific programs you're writing. Static checking, which can also be applied in more fine-grained ways to parts of the runtime (or its moral equivalent) is an interesting approach, depending on your goals.
> The stuff Fil-C adds is on the same footing as `unsafe` code in Rust- its implementation isn't checked, but its surface area is designed so that (if the implementation is correct) the rest of the program can't break it.
It’s not the same.
The Fil-C runtime is the same runtime in every client of Fil-C. It’s a single common trusted compute base and there’s no reason for it to grow.
On the other hand Rust programmers use unsafe all over the place, not just in some core libraries.
Yeah, that's what I meant by "depends on the specific programs you're writing." Confining unsafe Rust to core libraries is totally something people do.
You're equating a core runtime that doesn't grow with libraries written by anyone.
There's no world in which a Fil-C user would write unsafe code. That's not a thing you can do in Fil-C.
Rust users write unsafe code a lot and the language allows it and encourages it even.
I mean, again, yeah. I specifically compared the safe API/unsafe implementation aspect, not who writes the unsafe implementation.
To me the interesting thing about Rust's approach is precisely this ability to compose unrelated pieces of trusted code. The type system and dynamic semantics are set up so that things don't just devolve into a yolo-C-style free-for-all when you combine two internally-unsafe APIs: if they are safe independently, they are automatically safe together as well.
The set of internally-unsafe APIs you choose to compose is a separate question on top of that. Maybe Rust, or its ecosystem, or its users, are too lax about this, but I'm not really trying to have that argument. Like I mentioned in my initial comment, I find this interesting even if you just apply it within a single trusted runtime.
It’s not about who writes it.
The important question is: is this a per-program recurring cost, or a per-language-implementation fixed cost.
That’s unsafe is a recurring cost.
Fil-C’s runtime is a fixed cost.
Yeah, you're still responding to something I'm not saying, and not saying anything I'm trying to argue with.
I wrote "who" as shorthand for "the language implementation vs the individual programs."
> Rust users write unsafe code a lot
This isn't the case.
Over 170 uses of unsafe in sudo-rs. That’s just one example.
That’s “a lot” in my book.
There’s no reason to believe that one program is inherently representative. sudo-rs eschews dependencies and so is likely to be higher than most programs.
Furthermore, 170 uses in a 200 line program vs a one million line program are very different. I don’t know off hand how big sudo-rs is.
Even in embedded OS kernels, it’s often around 1%-5% of code. Many programs have no direct unsafe code at all.
> It's more comprehensively safe than Rust
Yeah. By adding a runtime.
> Fil-C achieves this using a combination of concurrent garbage collection and invisible capabilities (each pointer in memory has a corresponding capability, not visible to the C address space)
https://github.com/pizlonator/llvm-project-deluge/tree/delug...
> Yeah. By adding a runtime.
So? That doesn't make it any less safe or useful.
In almost all uses of C and C++, the language already has a runtime. In the Gnu universe, it's the combination of libgcc, the loader, the various crt entrypoints, and libc. In the Apple version, it's libcompiler_rt and libSystem.
Fil-C certainly adds more to the runtime, but it's not like there was no runtime before.
It makes it a lot less performant and there is no avoiding or mitigating that downside. C++ is often selected as a language instead of safer options for its unusual performance characteristics even among systems languages in practice.
Fil-C is not a replacement for C++ generally, that oversells it. It might be a replacement for some C++ software without stringent performance requirements or a rigorously performance-engineered architecture. There is a lot of this software, often legacy.
> a lot less performant
Is this just you speculating? How much is "a lot"? Where's the data? Let's get some benchmarks!
I mean bro isn’t totally wrong.
Fil-C’s perf sucks on some workloads. And it doesn’t suck on others.
Extreme examples to give you an idea:
- xzutils had about 1.2x overhead. So slower but totally usable.
- no noticeable overhead in shells, systems utilities, ssh, curl, etc. But that’s because they’re IO bound.
- 4x or sometimes maybe even higher overheads for things like JS engines, CPython, Lua, Tcl, etc. Also OpenSSL perf tests are around 4x I think.
But you’re on thin ice if you say that this is a reason why Fil-C will fail. So much of Fil-C’s overhead is due to known issues that I will fix eventually, like the function call ABI (which is hella cheesy right now because I just wanted to have something that works and haven’t actually made it good yet).
> It makes it a lot less performant and there is no avoiding or mitigating that downside.
You can’t possibly know that.
> C++ is often selected as a language instead of safer options for its unusual performance characteristics even among systems languages in practice.
Is that why sudo, bash, coreutils, and ssh are written in C?
Of course not.
C and C++ are often chosen because they make systems programming possible at all due to their direct access to syscall ABI.
> Fil-C is not a replacement for C++ generally, that oversells it.
I have made no such claim.
Fil-C means you cannot claim - as TFA claims - that it’s impossible to make C and C++ safe. You have to now hedge that claim with additional caveats about performance. And even then you’re on thin ice since the top perf problems in Fil-C are due to immaturity of its implementation (like the fact that linking is hella cheesy and the ABI is even cheesier).
> It might be a replacement for some C++ software without stringent performance requirements or a rigorously performance-engineered architecture. There is a lot of this software, often legacy.
It’s the opposite in my experience. For example, xzutils and simdutf have super lower overhead in Fil-C. In the case of SIMD code it’s because using SIMD amortizes Fil-C’s overheads.
> C and C++ are often chosen because they make systems programming possible at all due to their direct access to syscall ABI.
Surely Fil-C cannot provide direct access to syscalls without violating the safety guarantee. There must be something ensuring that what the kernel interprets as a pointer is actually a valid pointer.
> Fil-C means you cannot claim - as TFA claims - that it’s impossible to make C and C++ safe. You have to now hedge that claim with additional caveats about performance. And even then you’re on thin ice since the top perf problems in Fil-C are due to immaturity of its implementation (like the fact that linking is hella cheesy and the ABI is even cheesier).
The world of compilers is littered with corpses of projects that spent years claiming faster performance was right around the corner.
I believe you can make it faster, but how much faster? We'll see.
I think these types of compatibility layers will be a great option moving forward for legacy software. But I have a hard time seeing the case for using Fil-C for new code: all the known disadvantages of C and C++, now combined with performance closer to Java than Rust (if not worse), and high difficulty interoperating with other native code (normally C and C++'s strength!), in exchange for marginal safety improvements over Rust (minus Rust's more general safety culture).
edit: I feel bad writing such a dismissive comment, but it's hard to avoid reacting that way when I see unrealistically rosy portrayals of projects.
> Surely Fil-C cannot provide direct access to syscalls without violating the safety guarantee. There must be something ensuring that what the kernel interprets as a pointer is actually a valid pointer.
This is exactly what Fil-C does.
> all the known disadvantages of C and C++
The main disadvantage of C and C++ is unsafety and fil-C comprehensively fixes that.
> edit: I feel bad writing such a dismissive comment, but it's hard to avoid reacting that way when I see unrealistically rosy portrayals of projects.
How is my portrayal unrealistically rosy?
Even the fact that you know what the current perf costs are is the result of me being brutally honest about its perf.
I suspect something else is going on.
> This is exactly what Fil-C does.
Okay, I just checked. It does not. I wrote: "There must be something ensuring that what the kernel interprets as a pointer is actually a valid pointer." And sure enough, your runtime manually wraps each Linux syscall to do exactly that:
https://github.com/pizlonator/llvm-project-deluge/blob/6804d...
For harder cases like fcntl, where arguments can be either pointers or integers, you have to enumerate the possible fcntl arguments:
https://github.com/pizlonator/llvm-project-deluge/blob/6804d...
ioctl is even harder because some ioctls take pointers to structs that themselves contain pointers; on Linux that includes v4l2 and mmc. It looks like you don't handle that properly, judging by:
https://github.com/pizlonator/llvm-project-deluge/blob/6804d...
--
My point is: having to go through wrapper functions is not what I'd call "direct" access to "ABI". (Also, the wrappers don’t even wrap the syscall ABI directly; they wrap the libc ABI that in turn wraps syscalls.)
You might object that the wrappers are thin enough that they still count as direct. While that's a matter of definitions, I think my previous comment made it clear what _I_ meant when I questioned "direct", given my followup sentence about "actually a valid pointer".
But beyond quibbles about who meant what, this lack of directness matters because it implicates the portability of your approach.
At least as currently implemented, you rely on compiling almost everything (even libc) inside the sandbox, while having a ‘narrow waist’ of syscall wrappers mediating access between the sandbox and the outside world. That should work for most use cases on Linux, where there's already an assumption that different processes can have completely different library stacks, and static linking is common. Even if you want to do a GUI application you should be able to recompile the entire GTK or Qt stack inside the sandbox, and it doesn’t matter if other apps are using different versions or GTK or Qt.
But what about other operating systems? For server and CLI software you can probably still get away with exposing a small syscall/libc API, similar to Cosmopolitan (though that will still require significant effort for each OS). But for GUIs and platform integration more broadly, you’re expected to use the platform-provided libraries that live in your address space. They are usually proprietary, and even when they’re not, the system isn’t designed for multiple versions of the libraries to coexist.
I know I’m not telling you anything you don’t already know. But it’s an important point, because aside from performance, the _other_ big reason that Rust relies on user-written unsafe code is for FFI. If anyone can write their own FFI bindings, as opposed to making all FFI bindings live in a centralized runtime, then it becomes more feasible to scale the mammoth task of writing safe wrappers for all those ABIs. Your approach explicitly rejects user-written unsafe code, so I don’t know how you can possibly end up with reasonable OS library coverage.
Now sure, you didn’t claim anything about GUIs or portability. Perhaps this is more on topic for my previous comment’s point about “high difficulty interoperating with other native code” which you didn’t rebut. But it’s also relevant to “direct access to syscall ABI”, because if there _were_ some way to provide direct access to syscall ABI while remaining memory-safe, then the same approach would probably extend to other system ABIs. For example, a fully CHERI-aware system actually would allow for that. It’s an unfair comparison, because CHERI assumes cooperation from both the hardware and the OS, while you’re trying to run on existing hardware and OSes. I have no idea if we’ll ever see CHERI in general purpose systems. But in exchange, CHERI achieves something that’s otherwise impossible: combining direct system access and memory safety. And I originally read your comment as claiming to do the impossible.
It’s true that the Fil-C approach focuses on having a memory safe userland with no exceptions, which means no FFI to unsafe code (aside from the Fil-C runtime itself, which is just syscalls wrappers and a small handful of other things).
I have considered what FFI to native could look like, and have so far rejected it because it hurts the “no unsafe code” purism.
You’re right that this limits me to those OSes where the syscalls themselves are an adequate ABI. Linux isn’t the only such OS. It just happens to be the only OS Fil-C supports right now.
I love this shameless self-promotion. ;)
Fil-C is in the cards for my next project.
Thank you for considering it :-)
Hit me up if you have questions or issues. I’m easy to find
One of these days, a project will catch on that's vastly simpler than any memory solution today, yet solves all the same problems, and more robustly too, just like how it took humanity thousands of years to realize how to use levers to build complex machines. The solution is probably sitting right under our noses. I'm not sure it's your project (maybe it is) but I bet this will happen.
That’s a really great attitude! And I think you’re right!
I think in addition to possibly being the solution to safety for someone, Fil-C is helping to elucidate what memory safe systems programming could look like and that might lead to someone building something even better