Show HN: jevals – replacing LLM judges with typed Jev decisions github.com 47 points by gbayomi 3 days ago
adityamishra241 1 day ago Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type? gbayomi 1 day ago It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!
gbayomi 1 day ago It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!
jacksun788 1 day ago Curious — how's this compare to laya? gbayomi 1 day ago It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc
gbayomi 1 day ago It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc
This is going to be a hot use case.
Agreed
Interesting idea. How do you handle cases where the decision depends on context that isn't captured by the typed decision type?
It’s very dependent on the underlying model right now, but it performs well for the current versions of jev/keV/laya!
Curious — how's this compare to laya?
It can also run with laya. The idea is to have an easy way to run common evals with decision models and easily switch the “backend” from jev/kev/laya etc