OP's was leaky slop from day one [0][1], as is his article [2]
It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]
[0]: Author fails to grasp the concept of virtual environments https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment...
[1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): https://huggingface.co/DeepMostInnovations/sales-conversion-... https://huggingface.co/DeepMostInnovations/sales-conversion-...
[2]: 100% of this post is AI-generated https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8f...
> not to mention the egregious target leakage
I was curious about this so I skimmed the paper [0]:
> SalesRLAgent achieved 96.7% accuracy, outperforming the best commercial alternative by 23.7 percentage points and the best LLM approach by 34.7 percentage points.
For a fuzzy natural language task like this, this magnitude of improvement should already set off alarm bells (Though i admit I'm not even sure what accuracy is even measured here, and the paper doesn't help either). Also, "best LLM" here refers to GPT-4 (at the time of upload, the public already had access to GPT-o3 and). I would have loved to contextualize the performance by looking at model size, but the paper is frustratingly devoid of detail in that regard:
> The core of SalesRLAgent is a reinforcement learning architecture consisting of: • A state encoder network that processes Azure OpenAI embeddings and features • A policy network that estimates conversion probability based on the current state • A value network that estimates the expected cumulative reward • A meta-learning module that assesses prediction confi dence
Also:
> Beyond technical metrics, we evaluated SalesRLAgent in real-world sales environments through A/B testing. [...] After 90 days across 217 representatives and 12,433 con versations, we observed: • 43.2% increase in conversion rate for the test group
This would be a pretty huge result but the fact that this is just shoved into a single paragrpah with no further discussion on methodology, baselines and setup makes me very suspicious.
[0] https://arxiv.org/abs/2503.23303
For anyone else verifying, the target leakage appears to be as follows:
1)`outcome` is part of `metrics` at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
2) `metrics` goes into `ConversationState` at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
3) `metrics` (including `outcome`) makes its way into `ConversationState.state_vector` at https://huggingface.co/DeepMostInnovations/sales-conversion-..., and is returned from environment `step()` and `reset()` functions at https://huggingface.co/DeepMostInnovations/sales-conversion-... and https://huggingface.co/DeepMostInnovations/sales-conversion-...
4) model ingests `state_vector` as input at https://huggingface.co/DeepMostInnovations/sales-conversion-...