Types are puzzles. A good Rustacean will make sure that the pieces fit to make the picture.
That's why in crates where I need to make sure certain functions are called in order, I use a Ticket<T>, where one function returns a Ticket<Func1Done> with the output and the other has to consume it as an input.
The typestate pattern is a specialization of making only valid states representable
That sounds like a Wrapper<Func1Payload>, not a Ticket<Func1Call> that will become an extra parameter of Func2 whose only purpose is to prove to Func2 that you called Func1.
And you wanted to make sure that func2 and func3 can only be called after func1 has been called.
A wrapper on the output of func1 here would be awkward because then you return Wrapper<Func1Done>(bar0). But func2 does not even need a bar0 and neither does func3.
So the solution is to return (bar0, Wrapper<Func1Done>) from func1 where
I think this is a good argument for the Ticket, however, for the case where these three functions are generically useful, not just used in this specified order, I would write a specific function, just copy-paste of the bodies capturing the required ordering as an implementation detail.
Obviously if you are operating in a wide, concurrent async system then the Ticket and separate function calls is the better mechanism for the ordering.
That's fair but then you have to make your args a struct for this bespoke purpose; an anti pattern.
Also, many other functions can depend on the ticket from func_1. So making the ticket separate and generic on the process is the right (imo) solution here.
Why should this be such a bad anti pattern? Sure, a function might not need to work on the entire data model, but with pass by reference, does it matter that much? I dont see big negatives by using struct args, possibly wrapped in some typestate.
On the other hand, doesnt seprating args and typestate defeat the purpose? Since they can now be constructed separately.
It's because I use Ticket<T> in situations where I need to remember (force other users to use) a sequence of functions that take arguments not necessarily constructed by others.
//One could also place all the data in a giant struct and move that across all functions but that eventually leads to struct bloat unless we use an explicit state machine, in which case type state is better
I’ve been experimenting with Rust’s type state pattern—I’m trying to build something that builds an inventory of some object storage prefix (recording the version and size of each object in the prefix), but the pattern seemed so cumbersome. The goal was to avoid committing to a particular I/O color (sync vs async) and to have a testable no_std core, but I have so much less confidence in the typestate version compared to the “define traits for I/O and build an imperative loop around it”. I’m curious if anyone has suggestions (I realize it’s probably difficult to help without access to source code).
I’m probably doing it wrong, but when there’s a state with multiple transitions out, I can either model it as distinct methods per transition in which case the caller needs to know how to transition between states or I can have the caller pass an enum in which moves the branch into the state machine at the expense of an enum and a match statement. It’s also like 10x the code. Again, I’m very open to the possibility that I’m doing something wrong. Curious how you would model a state machine for (1) reserving the right to do the inventory (2) querying the next page of results (based on a cursor) and (3) recording the page information and the next cursor.
The article also mentions that typestates can be cumbersome:
> Typestate improves code faultlessness and testability, but comes at the cost of more boilerplate code and can degrade readability.
I have noticed this in my own code. `Ticket` with an internal variable tracking the state makes using it simpler. I just have to store one object in my struct `struct MyData { ticket: Ticket }` and call `ticket` methods in the correct order.
Typestate `Ticket<T>` is not as simple. I have to wrap it in my own enum: `enum TicketState { Ticket1(Ticket<Func1Done>), Ticket2(Ticket<Func2Done>), }` to store in my struct: `struct MyData { ticket: TicketState }`. Then every time I call `ticket` methods, I must extract the correct variant value first. That degrades readability and creates extra run-time cost.
I know this wasn’t the crux of your post, but do you find that you primarily look at types as puzzles in a majority of your code? I have a fundamentally different view and find other perspectives interesting when thinking about language design.
As a separate point, I think this is an excellent example of making invalid states unrepresentable.
If you were to design Ikea furniture, you'd make pieces that only fit in to the total configuration the correct way.
Types provide that same phenomenon in programming imo. At the end of the day we are shoveling and playing with bytes so we need to provide handles to these processes which make sure that we can't fit a "square peg into a round hole"
I view types as "the thing that prevents programmers and agents alike from writing bad and incorrect code in the future". They're a way of encoding invariants that doesn't require the programmer to worry about accidentally breaking them (because breaking the invariant will cause a compiler error).
Types are not the only way to encode such invariants. tests (and to a smaller extend lints and agent hooks) are other such mechanisms.
On this beat, I think people really under-appreciate the value of tests that check the structure of code to verify some general property, instead of checking the behavior of particular code paths.
At work, we have an internal system where we use a specific type to pass certain information around. It is extremely easy to construct an (empty) instance of that type wherever it is needed, but that is almost always wrong, you actually have to do the work and figure out how to get a real instance from somewhere. To make matters worse, whether the instance is empty or not doesn't matter in development, but matters a lot in production.
Because agents are lazy, they tend to construct empty instances whenever they feel like it, and there was no immediate feedback mechanism that could tell them it was wrong. It's not something you can easily encode in a type for example. I therefore build one (imperfectly, based on ruff lints), but it solved the problem entirely.
I think this is similar to what Axum does to make sure your router has its state declared before you try to instantiate it. Are there other examples in popular libraries?
> where I need to make sure certain functions are called in order
The ticket approach is a neat way to handle this, but I’ve always felt that functions needing to be called in a specific order is usually a bad code smell.
I’m sure there are times it’s unavoidable or maybe even the cleanest approach, but I don’t think I’ve encountered one in my career. When are you finding you need to do this?
I use Typestates and Newtypes extensively. The metric that shows Typestate and Newtypes are beneficial is: How many method calls or parameters can be called / used that compile but are not valid use cases. You want to minimise this number. I love having a type state where I can only make 1 or 2 method calls because the state enforces there are only a few parsing / validation / transition methods available. And there is only one valid way to supply the parameters, I cannot use the strings in the wrong location. I only wish we had named parameters like ObjC.
From a language design perspective I go back and forth on named params. I think the only conclusion I’ve reached is that I am not in favor of them being optional, but I think that is more a concern for implementation of the language and less about how it effects users.
How do you find the feature useful in this instance, I can’t quite picture how that works for typestate pattern functions.
This is not really ST. This is analogous to eating at an old school restaurant.
You can't just walk in to the food service counter and say "give me a burger"; you need to first get a ticket from the cashier proving that you've ordered a burger and then provide that ticket to the guy at the counter.
If you are asking in the context of Idris 2.0 (the current version), ST is not really related.
However, if you mean ST in Idris 1.0, there is a definite correlation. The mechanism that ST used for enabling local mutations was very similar to the mechanism that the typestate pattern in Rust is using. ST was a framework for formalizing State Machines in dependent types which is the mechanism TFA is analyzing.
Types are puzzles. A good Rustacean will make sure that the pieces fit to make the picture.
That's why in crates where I need to make sure certain functions are called in order, I use a Ticket<T>, where one function returns a Ticket<Func1Done> with the output and the other has to consume it as an input.
The typestate pattern is a specialization of making only valid states representable
One of my favorite useful patterns
why not just create a wrapper type for the payload that is returned by func1 and func2 takes it as a parameter?
Because you may want to share certain behaviors between the two wrapper types via generic impl
That sounds like a Wrapper<Func1Payload>, not a Ticket<Func1Call> that will become an extra parameter of Func2 whose only purpose is to prove to Func2 that you called Func1.
Maybe I misunderstood something.
Okay let's say you had three functions
func1(foo_0) -> bar0
func2(foo_1, foo_2) -> bar1
func3(foo_3, foo_3) -> bar2
And you wanted to make sure that func2 and func3 can only be called after func1 has been called.
A wrapper on the output of func1 here would be awkward because then you return Wrapper<Func1Done>(bar0). But func2 does not even need a bar0 and neither does func3.
So the solution is to return (bar0, Wrapper<Func1Done>) from func1 where
struct Wrapper<T>(//cheating ())
I think this is a good argument for the Ticket, however, for the case where these three functions are generically useful, not just used in this specified order, I would write a specific function, just copy-paste of the bodies capturing the required ordering as an implementation detail.
Obviously if you are operating in a wide, concurrent async system then the Ticket and separate function calls is the better mechanism for the ordering.
That's fair but then you have to make your args a struct for this bespoke purpose; an anti pattern.
Also, many other functions can depend on the ticket from func_1. So making the ticket separate and generic on the process is the right (imo) solution here.
Why should this be such a bad anti pattern? Sure, a function might not need to work on the entire data model, but with pass by reference, does it matter that much? I dont see big negatives by using struct args, possibly wrapped in some typestate.
On the other hand, doesnt seprating args and typestate defeat the purpose? Since they can now be constructed separately.
It's because I use Ticket<T> in situations where I need to remember (force other users to use) a sequence of functions that take arguments not necessarily constructed by others.
async fn write_buffer(buf: &mut [u8]) -> Ticket<BufferWritten>
//Best that it it's own function for readability
async fn complex_counter_logic(ctr: Arc<AtomicUsize>, ticket: Ticket<BufferWritten>) -> Ticket<ComplexCounterLogic>
//One could also place all the data in a giant struct and move that across all functions but that eventually leads to struct bloat unless we use an explicit state machine, in which case type state is better
I’ve been experimenting with Rust’s type state pattern—I’m trying to build something that builds an inventory of some object storage prefix (recording the version and size of each object in the prefix), but the pattern seemed so cumbersome. The goal was to avoid committing to a particular I/O color (sync vs async) and to have a testable no_std core, but I have so much less confidence in the typestate version compared to the “define traits for I/O and build an imperative loop around it”. I’m curious if anyone has suggestions (I realize it’s probably difficult to help without access to source code).
Why is it cumbersome?
I’m probably doing it wrong, but when there’s a state with multiple transitions out, I can either model it as distinct methods per transition in which case the caller needs to know how to transition between states or I can have the caller pass an enum in which moves the branch into the state machine at the expense of an enum and a match statement. It’s also like 10x the code. Again, I’m very open to the possibility that I’m doing something wrong. Curious how you would model a state machine for (1) reserving the right to do the inventory (2) querying the next page of results (based on a cursor) and (3) recording the page information and the next cursor.
In the type state pattern you would have something like this
pub trait ValidState {}
struct StateMachine<'a, T>
where
{
untyped: &'a mut UntypedStateMachine,
_marker: PhantomData<T>
}
fn reserve_right<'a>(state: StateMachine<'a, Begin>) -> StateMachine<'a, Reserved>
fn query<'a>(state: StateMachine<'a, Reserved>) -> StateMachine<'a, Queried>
fn record<'a>(state: StateMachine<'a, Queried>) -> StateMachine<'a, Recorded>
The article also mentions that typestates can be cumbersome:
> Typestate improves code faultlessness and testability, but comes at the cost of more boilerplate code and can degrade readability.
I have noticed this in my own code. `Ticket` with an internal variable tracking the state makes using it simpler. I just have to store one object in my struct `struct MyData { ticket: Ticket }` and call `ticket` methods in the correct order.
Typestate `Ticket<T>` is not as simple. I have to wrap it in my own enum: `enum TicketState { Ticket1(Ticket<Func1Done>), Ticket2(Ticket<Func2Done>), }` to store in my struct: `struct MyData { ticket: TicketState }`. Then every time I call `ticket` methods, I must extract the correct variant value first. That degrades readability and creates extra run-time cost.
You don't need the enum? You just require Ticket<T_0> as a function argument.
It's really not that cumbersome, it's like two extra lines of code...
I know this wasn’t the crux of your post, but do you find that you primarily look at types as puzzles in a majority of your code? I have a fundamentally different view and find other perspectives interesting when thinking about language design.
As a separate point, I think this is an excellent example of making invalid states unrepresentable.
Types are puzzles in a good way.
If you were to design Ikea furniture, you'd make pieces that only fit in to the total configuration the correct way.
Types provide that same phenomenon in programming imo. At the end of the day we are shoveling and playing with bytes so we need to provide handles to these processes which make sure that we can't fit a "square peg into a round hole"
I view types as "the thing that prevents programmers and agents alike from writing bad and incorrect code in the future". They're a way of encoding invariants that doesn't require the programmer to worry about accidentally breaking them (because breaking the invariant will cause a compiler error).
Types are not the only way to encode such invariants. tests (and to a smaller extend lints and agent hooks) are other such mechanisms.
On this beat, I think people really under-appreciate the value of tests that check the structure of code to verify some general property, instead of checking the behavior of particular code paths.
At work, we have an internal system where we use a specific type to pass certain information around. It is extremely easy to construct an (empty) instance of that type wherever it is needed, but that is almost always wrong, you actually have to do the work and figure out how to get a real instance from somewhere. To make matters worse, whether the instance is empty or not doesn't matter in development, but matters a lot in production.
Because agents are lazy, they tend to construct empty instances whenever they feel like it, and there was no immediate feedback mechanism that could tell them it was wrong. It's not something you can easily encode in a type for example. I therefore build one (imperfectly, based on ruff lints), but it solved the problem entirely.
I think this is similar to what Axum does to make sure your router has its state declared before you try to instantiate it. Are there other examples in popular libraries?
> where I need to make sure certain functions are called in order
The ticket approach is a neat way to handle this, but I’ve always felt that functions needing to be called in a specific order is usually a bad code smell.
I’m sure there are times it’s unavoidable or maybe even the cleanest approach, but I don’t think I’ve encountered one in my career. When are you finding you need to do this?
> I’ve always felt that functions needing to be called in a specific order is usually a bad code smell
Can you give an example where a different design eliminates the need for the ticket pattern?
This was a talk at the FUNARCH workshop at this year’s ICFP.
Here’s the livestream: https://www.youtube.com/live/c0pw1iVs_Q0?is=hwm2xa4cZOcqF5tW
Well post the individual talks in the following days!
Talk starts as 6:01:10
I use Typestates and Newtypes extensively. The metric that shows Typestate and Newtypes are beneficial is: How many method calls or parameters can be called / used that compile but are not valid use cases. You want to minimise this number. I love having a type state where I can only make 1 or 2 method calls because the state enforces there are only a few parsing / validation / transition methods available. And there is only one valid way to supply the parameters, I cannot use the strings in the wrong location. I only wish we had named parameters like ObjC.
From a language design perspective I go back and forth on named params. I think the only conclusion I’ve reached is that I am not in favor of them being optional, but I think that is more a concern for implementation of the language and less about how it effects users.
How do you find the feature useful in this instance, I can’t quite picture how that works for typestate pattern functions.
Check 'bon' crate for named parameters built on newtype.
pdf: https://dl.acm.org/doi/pdf/10.1145/3830438.3830958
You deserve a medal for that.
Maybe I am missing something but where is the entire source code?
Could someone compare this to ST in Idris?
Yes, I believe it should be possible for someone to do that.
This is not really ST. This is analogous to eating at an old school restaurant.
You can't just walk in to the food service counter and say "give me a burger"; you need to first get a ticket from the cashier proving that you've ordered a burger and then provide that ticket to the guy at the counter.
That's literally the type state pattern
If you are asking in the context of Idris 2.0 (the current version), ST is not really related.
However, if you mean ST in Idris 1.0, there is a definite correlation. The mechanism that ST used for enabling local mutations was very similar to the mechanism that the typestate pattern in Rust is using. ST was a framework for formalizing State Machines in dependent types which is the mechanism TFA is analyzing.