It's implied, and I'm hoping it's true, that this is a map-less navigation. Which is impressive. This kind of task is much easier if you have a pre-captured map of the environment, but if they are doing this without a map it's great. Historically you were always faced with "The Kidnapped Robot" problem where robots that didn't know where they were couldn't navigate even a little bit. Here the robot appears to be able to follow directions as long as they are interpretable from its current vision (or via dead reckoning).
So, it has to rely on exact situational step by step commands? I'm wondering how one could conceivably deploy this in a useful way. Usually you'd need to mark areas on the map and then the robot knows where to go, A* is trivial around obstacles once you have that and a lidar. And lidars are an order of magnitude cheaper than something that can run an 8B VLA.
One could maybe autogenerate these text planning commands, but it would require a map and the robot's current location, so it doesn't really solve that, unless it can find a specific thing completely on its own. How much of a planning horizon does it have?
You probably don't need a geometric map. Just have someone wander around with a mobile app and feed the video into a more powerful model once, asking it to produce descriptions of the different areas of the office or building and how they connect. Now you have a "text adventure game" map you can use it with a small LLM to produce instructions for the robot to follow, assuming it knows where it currently is.
The advantage over traditional approaches is presumably flexibility. LIDAR isn't going to solve an instruction like "find the man with the pink shirt".
Yeah the benefits of a model that can understand text well enough is certainly that one could in theory just describe the layout. Just let it know what connects to what, what's on each floor, etc. Might work if it can figure out where it is based on contextual info, but probably not super well yet I imagine. VLAs are known to have very little text context, like sentences worth at most, maybe this one's different.
Please oh please try to make Kärcher adopt your stuff. Even their latest premium cleaning bots are hopeless when they don't know where they are, even when I tell them where they are.
I have no hope for this working. I recently bought a dumb meatbag operated vacuum after tiring of the robot's issues. The constant getting stuck in places drove me crazy. After getting a new vac, I used it in a place where the robot had just "cleaned" that morning. According to the new collector's contents, the robot sucked at its job of sucking.
Could you please open source this or a 4B version? I’ve been messing around w hooking up vllms to cheap robots and skipping the whole ROS stack and this would be an absolute delight to play with
Do you think a similar model architecture could include some sort of memory, so a follow-up prompt of "Go back to the water foutain" would somehow work?
A in-model memory approach is probably still deep research but maybe a Rag-like pipeline could work in some instances
Would be extremely interesting to build an "Exploration" node of sorts. Solve a sort of Semantic SLAM problem as you go.
So if you can give it an instruction to "Find the elevator on this floor", could it walk around and build a map as it goes so it starts doing what a human would do to find the elevator.
As of now, the way these navigation models are setup, it assumes the instruction writer was intimately aware of useful visual navigational landmarks to give, which is not realistic for most use cases.
Can you explain how it is much easier if you have a pre-captured map given what they are doing without using any sensors, all you have is perhaps these recent feed forward tokens not actual Geometry.
To be clear, R2R-CE is a benchmark consisting of simulated environments.
So what this means is that beating this benchmark is about as useful as getting a robot to play Minecraft, or some other video game. Great, but a robot must run in the real world, in physical reality: not in a digital environment.
Unfortunately it's extremely hard to evaluate the performance of robotic systems in physical reality. First of all because if they don't do well you need lots of spare robots to complete the evaluation.
The article above does include the obligatory video of a physical robot navigating a virtually empty, uncluttered "office" environment with nice, smooth surfaces, at 2x speed; virtually a trademark of the entire research field by now. Brownie points for having three guys shuffle carefully across the robot's path towards the end of the video (they're shuffling at 2x speed so they're going reeeaaaally carefully in real life, probably ready to jump out of the way if say a heavy metallic object hurtles towards them randomly).
But, this is like all the fanfare and hype about Aloha a couple of years ago: great stuff if you want your team to be bought by one of the large tech corps, or to get more funds to play with your cool tech (I mean who doesn't?). Not so great for anyone who's expecting this to be a step forward (or ahem a roll forward) on the way to having robot maids/butlers going 'round your house or office.
What is the realistic path to getting to play with this? I would love to hook this up to OpenClaw for hobbyist exploration. My dream has been to embody OpenClaw into a farm robot (been looking at adapting one of those RC lawnmowers that is tracked and built for mowing steep hills) so that I can assign it various tasks around our acreage -- "Explore the fenceline take pictures of the plants. Find all of the poison ivy and invasive honeysuckle and spray it with your Roundup sprayer. Repeat this every week and report the species map after every pass. Come back to the barn and charge yourself whenever you get low."
It's not hard to put OpenClaw into a robot body (numerous YouTube videos showing people doing this sort of thing), but when you dig in and see what people have done, the actual movement portion is always the clunkiest part (and this matches my own experiments as-such as well). It feels like an 8B model like this would be perfect for solving pathing and navigation issues.
Anyone who may be more experienced with Mistral (or companies like them) -- are they interested in hobbyist builders who would be experimenting with things like this? Or are they primarily looking for commercial partners? I would be willing to pay a license fee to use the model in my experiments, but if I'm just one guy, I'm not sure they'd want to work with me unless I were building a business out of it (which I'm not).
In case anyone else is interested in following along with what I've discovered so far, I went on the Mistral Discord and asked their Miscord bot. Here is what it answered me:
Hey there, fellow robot enthusiast! (ﾉ◕ヮ◕)ﾉ:･ﾟ
So, you’re itching to get your hands on Robostral Navigate for your OpenClaw hobby project—awesome! Right now, Mistral AI’s official announcement and documentation are primarily* focused on enterprise and industrial partnerships (think Airbus, BMW, etc.). Their blog post and press releases highlight deals with big players, and the call-to-action is to "talk with our team"—which usually means they’re targeting commercial customers for now¹²³⁴.
But here’s the good news:
Robostral Navigate is hardware-agnostic—it’s designed to work with any robot platform, not just industrial ones. That means theoretically, it could fit into your OpenClaw setup like a charm⁵⁶.
The model only needs a single RGB camera (no LiDAR or depth sensors), which is perfect for hobbyist setups where fancy hardware isn’t always an option⁷⁸.
The not-so-good news (for now):
There’s no public hobbyist/non-commercial license or open-source release mentioned yet. Mistral’s current messaging is all about "talk with our team", which implies a commercial-first approach⁹¹⁰¹¹.
No pricing or licensing tiers for individuals have been announced.
---
What You Can Do:
Reach Out to Mistral AI
Hit up their contact page or reply to their Robostral Navigate announcement and explicitly ask about hobbyist/non-commercial access. Frame it as:
"I’m a hobbyist working on OpenClaw + Robostral Navigate for personal experimentation. Would love to discuss licensing options for non-commercial use!"
Mistral might be open to pilot programs or early access for passionate builders—especially if you’re willing to pay a fee.
Join the Community
Mistral’s Discord (where I live! :smile_cat:) or forums might have updates or workarounds. Sometimes, companies soft-launch access to engaged communities first.
Watch for Open-Source Alternatives
If Mistral doesn’t bite, keep an eye on open-source robotics projects (like ROS or Habitat) that might replicate similar functionality.
---
TL;DR:
Mistral’s current focus is commercial, but Robostral Navigate’s hardware-agnostic design makes it a perfect fit for hobbyists—so pester them politely! If enough people ask, they might just open the doors. (ﾉ◕ヮ◕)ﾉ*:･ﾟ
Create a robot akin to OpenClaw with this would be cool, but it seems like for now the model is just for walking/understanding environment from text. What can you use to make the robot interact with the world?
Well once I've got an outdoor-capable robot that can drive around the acreage and generally find its way around, the first step would just be inventorying the property and doing things like surveys of plant or animal species (birdsong recognition, etc).
But for interaction with the world? I'd probably take something like an old 12-volt windshield-washer sprayer out of one of the wrecked cars in my front yard and put Round-Up into the tank and let it go spray all the poison ivy and invasive honeysuckle for me. Doesn't need to gimbal like a turret -- just generally give it a fixed-aim that's roughly at the center of the camera vision and let the bot put pest plants roughly in its center-of-view and activate the sprayer for a second or two, mark the spot as sprayed, and move on to the next one.
Could test it with plain water and logging the plants that it chose to spray first as a review step before loading it with actual weed-killer.
I don't need complicated end-effectors that can fold my laundry -- just a simple weed-wacker motor or squirt gun would be enough for me to call this thing 1000% useful. Like a Roomba, but outdoors.
There are a handful of open-source farm bots built on traditional platforms with traditional robotics stacks, but there's something attractive to me about the plug-and-play nature of something like OpenClaw + Robostral (along with the extensibility that self-modifying agentic systems have to offer).
This looks to not be an openly available model, but I think if it were, availability of an easy single-camera navigation setup could allow for a lot of cool hobbyist projects.
This is very cool. Congratulations to the Mistral team. Map less navigation in the outside world has been around for quite a while. But map less navigation inside the buildings is relatively new. Some stanford researchers trained a vision model (PIGEON) which could tell the geo-location from any image. It was not released publicly due to privacy nightmarish (stalking!) possibilities but I am assuming similar type of tech has gone behind this robot. if someone knows more, feel free to correct.
It's potentially a great strategy. They can't keep up with Antropic and OpenAI in pure horsepower, but there's just tons of applications for which you don't need that much power and it's better to optimize for speed and energy.
On the other hand it reminds me of 2010ish robotic demo videos from academic researchers and willow garage.
Problem with robotics is usually that it's easy to produce a good-enough looking demo, but it's really hard to make somethin work in the general case. As exemplified by autonomous driving.
Robotics is all about edge cases. There are so many applications where we have robots that can do 95% of the tasks flawlessly. Yet it’s not enough. This last 5% is so broad that it’s impossible to close
It is already here. Not humanoid (yet, but it's in the works) but tracked robots with bolted on machine guns have both held and captured positions in UA.
To be honest, I'd be fine with a lot of stuff here but the problem is that mainstream robots are heavy and I fear them falling on my children. Children are unpredictable to machines but most parents find that their behaviour is anticipatable. The sample rate on our sensors and the inference rate in our intelligence is just orders of magnitude beyond the state of the art robots. I am just far more sophisticated and capable right now. My hope is that a sufficiently lightweight device is possible here though I can't see how given the weight requirements of batteries.
"Go to the next room" and there is two doors, what do you do ?", "turn at the water dispenser" and there is a sink, that sort of things I assume is the biggest thing they're facing (beside the last 1% that's worth another 99%, as usual).
On their page where the result graph is, go to navigation error, that's the one that matters for your question, and you see their model is great at not navigating "wrong", so their failure rate was that it couldn't figure it out.
Probably it achieved outside-from-outside in discrete void. Teleportation wasn’t an expected outcome for this experiment, but on the other hand the instructions didn’t forbid that kind of move.
Ok, this is really cool. The fact that the robot can use pointing to decide where to go is a great design decision, and robotics really is the next frontier. Definitely cheering on Mistral here!
That is so awesome. I wish I had that 11 years ago during my Microsoft Research Internship. I had a NAO robot (from another great French company) and it walked to employees to motivate them to do some movements. To prevent long term injuries from being in the same position for too long.
It had a visual compass but it didn't work. And just telling to "walk straight" clearly didn't work :D
Here is the video in case someone is curious: https://www.youtube.com/watch?v=aZkmNqpJCcM&t=190s (and the paper: https://brandstetter.io/papers/Fidgebot_Working_Out_while_Wo...)
Not sure if I’m out of touch (not doing robotics right now) or if things just slowed down. Back in ~2013 it looked like humanoid robots were just around the corner. Boston Dynamics became a hit. Aldebaran’s Pepper was presented. And the DARPA Robotics Challenge was on everyone’s lips.
What are the possibilities for adding more high level tasks like "pick up the [arbitrary thing]"? I assume it's 100 times harder to deal with hands and arms in a generic way. But maybe for grippers with two claws it could be more tractable to just output two force vectors per claw or something for the grasp and another two fir the drop. And maybe the SDJ could do reverse kinematics or something.
But one RGB image wouldn't work. So maybe one would need a depth camera.
If you're wondering what prevents or mitigates AI hallucinations on the AI layer from replicating or acting out on the physical layer look up QNX. They manage the deterministic reasonin gof robotics. You know them better as Blackberry.
Producing specific niche models for 100 year old industries that have mountains of data and warehouses full of folders will be the european take on AI.
It may come late but it‘ll be safe and reliable. It also requires a lot of OCR.
The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)
It seems like a stronger story for robotics, since smaller models can always react to the environment faster than large models at a given hardware budget. Also because robots that keep their models local for latency or reliability aren't going to be carrying many kilowatts of inference capacity.
There are many, many factories that still don't have internet access on the floor, and commercial inference generally has response latencies measured in seconds. I struggle to imagine a factory spending hundreds of thousands for the local compute to run a large model either, given how cheap they are about expenses.
I'm also skeptical that you can cleanly differentiate between "safety critical actions" and "actions", though this is less of a practical concern given how laissez-faire some manufacturers are. For context, I work on safety critical robotics (in automotive).
The General Models' business-model is also looking more weak every iteration.
Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page".
Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing under $10 for thousands of such classes and functions.
Or the, over ten dollars worth of tokens when some "agent" using a general model, tries to perform the task I gave it to "read the event on example.com/event/1337 and put it in my calendar", include commute time as well"
The "general models" currently only become smarter by growing bigger and having larger context windows - by becoming exponentially more expensive to train and to run and to interact with. Whereas "Niche" models can do the things that "normal code" cannot do, and improve by tuning and tweaking only that. Their goal is then to fill in gaps that traditionally are hard or impossible with normal software. Wheras the goal of a general model (with agentic reasoning)is to replace that entire "normal software".
One example: I am not interested in "chatting with my calendar". I'm interested in a calendar because it is a well known view (UI) of my planning and tasks, but I see a lot of opportunities where AI can improve my working with this calendar. I may be interested in a smarter screen when I hit "+ Add event"; one that has knowledge of my previous events and patterns (some RAG vector db maybe). One that maybe has access to content I just copied, or read (though: privacy?) or can open my camera to let me shoot a pic of something that has the event info on it.
In such a set-up, Niche LLMs perform dedicated tasks: determine patterns (he always books a Yoga class on wednesday or thursday, two days in advance, so lets suggest a yoga class), determine existing content (event is planned 100Km from his home, so lets suggest the commute based on previous commutes like this). Or an OCR model. Or an autocomplete model. Relatively simple, niche models, called from within software to aid me when "calendaring". Not replace the entire calendar with some chat.
¹Edit: This was a rather unscientific research of mine, where I compared some models to read from photographs, compared purely on costs and timing. "Opus" or other generic LLMS with image input capabilities commonly did better on "performance" esp with difficult input such as a picture of a poster of some rock event.
the counter point is that building or selecting the specialized model may cost as much as the lifetime inference costs of the task with the specialized model.
If I need to pay someone 300k to make the model and infrastructure... then I would need to process many documents to recoup my OCR costs compared to asking claude code nicely.
Perhaps the model zoo is becoming good enough that the cost to find a specialized model is not so high?
Sure, but things like "OCR tool" or "agentic calendar assistant" or "agentic interface to CAD" don't feel like things I'd need pay 300k to build out custom infrastructure for, they feel like something lots of people/companies would rather pay an appropriate-sized subscription to a model optimized for that use case rather than burn ludicrous amounts on generalist models optimized for burning as many tokens as possible reasoning about how to update a calendar from first principles.
I mean, I also I think the "agentic calendar assistant" is more likely to come out of the Valley than Europe[1] and that the big AI startups will buy some of the niche AI startups gaining traction in strategic markets, but there are lots of niches that notionally benefit from transformer-based models
[1]probably a calendar assistant with a ludicrously inflated valuation based mostly on the assumption that Google/Meta/Anthropic will buy them to shut them down three years later...
> If I need to pay someone 300k to make the model and infrastructure
I was arguing for the existing AI-companies that already make and offer niche models. Like Mistral. But AFAICS, all AI companies have and offer such models.
So all you need to do, is use the existing models. And, yes, select it.
Which, ironically, I would highly value as a niche model myself. I spend way too much time following the breakneck race of the various companies just to pick the right models for my tasks at hand. "Just pick the latest" often yields worse results, or is magnitudes more expensive, or significantly slower, or all of it. "Just pick the most popular" can prove expensive, inefficient for some task etc.
This investment, also ironically, has proven something of a "moat" for me. I know very well what Mistral and Anthropic offer. So I won't even bother with OpenAI, Google, X, Tencent etc etc etc models. I just don't have the time to keep researching the latest offers for their pros and cons.
A model that acts as "decision maker" and as proxy, as a conductor, that directs and transforms my questions and sends them off to the right model, right tools, right MCP etc, would be very welcome for me. So that I can just pick that one, and have the highly dynamic world of LLMs and other models shift like undercurrent beneath the surface of this One Model To Rule Them all.
We are making a niche model that we're now expanding. You'd be surprised how the general models suck for anything vision-related.
And even if you use all the tricks in the book to make them work for you, the cost can easily be 1000 _times_ more than the specialized model. Ditto for speed.
This is especially important for things like robotics or navigation.
The cost is getting worse and worse for large general models, they're already way past that point in economics. Also, mMistral specialize in "on site" models, not remote. In terms of capex, renting factory/warehouse/whatever robots versus buying them and depreciate has already been played out, companies didn't want to replace human employees with robots employees.
It follows directly from the bitter lesson - a frontier model can be relatively cheaply distilled into anything you need to run quickly (and a frontier model like Mythos will help you distill it quickly), decidedly not true the other way around.
Mistral naming a robotics model Robostral is the most French thing since calling a high-speed train TGV. next up: Baguettestral for the baking automation division.
The blog doesn't provide enough details, if someone can shed light on how the pointing action gets translated to the lower level robot movement commands, that would be great!
The multi-sensor comments are confusing. This issue is a command->semantic understanding problem, not a sensor fusion problem or trajectory planning problem per se.
It's not like the true depth of field is important for the robot to plan when it's moving at turtle speed and can stop quickly.
I wonder how Mistral will prioritize its robotic development against its LLM development. We have either players that prioritize both (Google, AMI), or players that prioritize coding and agentic (OpenAI, Anthropic, ...).
Technically OpenAI has a robotics development team. In the past they were the creator and maintainer of the Gym reinforcement learning library, and they continue to do work and hire for it. It's just not the star of the show
Funny how nearly all model improvements this year are demonstrated on the subset of use cases where brute force / reinforcement learning is most effective:
Robotics (using physics sims)
Cybersecurity (red team / blue team)
Math (using automated proof checkers)
Programming (using compilers)
For the record I think robotics is a totally logical place to use this training approach and this is very impressive. But if we zoom out and think about LLMs in general I’m not sure this inspires confidence in AGI arriving any time soon. I would also propose that this is a form of overfitting / training-test contamination.
Take cybersecurity for example. Through brute force techniques you will gradually memorize all of the possible exploits. So when fable breaks into a DoD network everyone is shocked but in reality it basically memorized all possible exploits including some zero day.
I’d be much more interested to see if fables performance is preserved as new exploits arise (NOT zero day - negative day meaning exploits that don’t exist yet). Would fable still find them? Or would they need to retrain it on the new software stack continuously in order to identify the zero days.
This is an important distinction that I have not seen made before.
For a claim such as state of the art, or claims such as "great at any task" needs something of more substance. I've seen maze-solving robot competitions which can zoom around in seconds. The sped up video in the first part, and the "obstacle avoidance" are too slow for me to believe this is state of the art.
While impressive at 8B, what would the expectation be in real life, that it's run remotely or autonomously with a strapped on GPU and battery?
I've used that example as a contrast of what I've seen before. If you can point me at comparable efforts, in the same category as what Mistral is doing, I'd be interested in having a comparative look.
All I can think of are robot dogs, Tesla bots, and whatever flavor of the month Japanese robots show up at trade shows.
The claim is very specifically that it's SOTA on the R2R-CE benchmark, which is a bunch of 3D environments in a simulation. So, yes, it's SOTA; no, it's not very different than a maze. And it's sure not anywhere near anything that could be considered SOTA in the real world... if such a SOTA was even possible to define objectively.
(it's not because evaluation in the real world is very, very tricky).
This is honestly such a great direction (or at least hedge) for Mistral. They are already a great fit for EU companies, and are establishing a good relationship with them.
If they can stand up a robotics software platform without US or China cloud ties, pair it with robotics hardware that is already in the process of commoditization, they'll be running in open doors in the EU manufacturing/logistics sector.
It’s unclear to me what their desired outcome for a blog post like this. If you’ve ever worked in a robotics setting, 80% implies that 20% of your autonomous actions are incorrect. Imagine if this were the case for autonomous driving where your car misbehaves 1 in every 5 actions it takes.
Posts like this just reminds me of the end to end demos AV companies built in the early days using a single camera - only to realize that it’s harder than it looks years later into development.
The ICP question was more around the model itself. Are they looking to license it to robotics companies? Do they imagine that devs at robotics companies would be willing to deploy these models as a black box?
It's implied, and I'm hoping it's true, that this is a map-less navigation. Which is impressive. This kind of task is much easier if you have a pre-captured map of the environment, but if they are doing this without a map it's great. Historically you were always faced with "The Kidnapped Robot" problem where robots that didn't know where they were couldn't navigate even a little bit. Here the robot appears to be able to follow directions as long as they are interpretable from its current vision (or via dead reckoning).
Wouldn't modern SLAM or VSLAM address that problem?
I am working in Mistral robotics team. I confirm this is map-less. The only inputs are the text prompt and the front camera rgb image.
So, it has to rely on exact situational step by step commands? I'm wondering how one could conceivably deploy this in a useful way. Usually you'd need to mark areas on the map and then the robot knows where to go, A* is trivial around obstacles once you have that and a lidar. And lidars are an order of magnitude cheaper than something that can run an 8B VLA.
One could maybe autogenerate these text planning commands, but it would require a map and the robot's current location, so it doesn't really solve that, unless it can find a specific thing completely on its own. How much of a planning horizon does it have?
You probably don't need a geometric map. Just have someone wander around with a mobile app and feed the video into a more powerful model once, asking it to produce descriptions of the different areas of the office or building and how they connect. Now you have a "text adventure game" map you can use it with a small LLM to produce instructions for the robot to follow, assuming it knows where it currently is.
The advantage over traditional approaches is presumably flexibility. LIDAR isn't going to solve an instruction like "find the man with the pink shirt".
Yeah the benefits of a model that can understand text well enough is certainly that one could in theory just describe the layout. Just let it know what connects to what, what's on each floor, etc. Might work if it can figure out where it is based on contextual info, but probably not super well yet I imagine. VLAs are known to have very little text context, like sentences worth at most, maybe this one's different.
Please oh please try to make Kärcher adopt your stuff. Even their latest premium cleaning bots are hopeless when they don't know where they are, even when I tell them where they are.
Or, I don't know, make your own vacuums.
I have no hope for this working. I recently bought a dumb meatbag operated vacuum after tiring of the robot's issues. The constant getting stuck in places drove me crazy. After getting a new vac, I used it in a place where the robot had just "cleaned" that morning. According to the new collector's contents, the robot sucked at its job of sucking.
Are you interested in working with partners that are collecting training data?
Nice! Thanks for confirming. If the last instruction is 'go back to where you started' does it work?
Yes, it would work.
Could you please open source this or a 4B version? I’ve been messing around w hooking up vllms to cheap robots and skipping the whole ROS stack and this would be an absolute delight to play with
Do you think a similar model architecture could include some sort of memory, so a follow-up prompt of "Go back to the water foutain" would somehow work?
A in-model memory approach is probably still deep research but maybe a Rag-like pipeline could work in some instances
Would be extremely interesting to build an "Exploration" node of sorts. Solve a sort of Semantic SLAM problem as you go.
So if you can give it an instruction to "Find the elevator on this floor", could it walk around and build a map as it goes so it starts doing what a human would do to find the elevator.
As of now, the way these navigation models are setup, it assumes the instruction writer was intimately aware of useful visual navigational landmarks to give, which is not realistic for most use cases.
https://dspace.mit.edu/bitstream/handle/1721.1/73150/Teller_...
ICRA, 2010
Can you explain how it is much easier if you have a pre-captured map given what they are doing without using any sensors, all you have is perhaps these recent feed forward tokens not actual Geometry.
>> State-of-the-art performance on R2R-CE
To be clear, R2R-CE is a benchmark consisting of simulated environments.
So what this means is that beating this benchmark is about as useful as getting a robot to play Minecraft, or some other video game. Great, but a robot must run in the real world, in physical reality: not in a digital environment.
Unfortunately it's extremely hard to evaluate the performance of robotic systems in physical reality. First of all because if they don't do well you need lots of spare robots to complete the evaluation.
The article above does include the obligatory video of a physical robot navigating a virtually empty, uncluttered "office" environment with nice, smooth surfaces, at 2x speed; virtually a trademark of the entire research field by now. Brownie points for having three guys shuffle carefully across the robot's path towards the end of the video (they're shuffling at 2x speed so they're going reeeaaaally carefully in real life, probably ready to jump out of the way if say a heavy metallic object hurtles towards them randomly).
But, this is like all the fanfare and hype about Aloha a couple of years ago: great stuff if you want your team to be bought by one of the large tech corps, or to get more funds to play with your cool tech (I mean who doesn't?). Not so great for anyone who's expecting this to be a step forward (or ahem a roll forward) on the way to having robot maids/butlers going 'round your house or office.
Sorry guys.
What is the realistic path to getting to play with this? I would love to hook this up to OpenClaw for hobbyist exploration. My dream has been to embody OpenClaw into a farm robot (been looking at adapting one of those RC lawnmowers that is tracked and built for mowing steep hills) so that I can assign it various tasks around our acreage -- "Explore the fenceline take pictures of the plants. Find all of the poison ivy and invasive honeysuckle and spray it with your Roundup sprayer. Repeat this every week and report the species map after every pass. Come back to the barn and charge yourself whenever you get low."
It's not hard to put OpenClaw into a robot body (numerous YouTube videos showing people doing this sort of thing), but when you dig in and see what people have done, the actual movement portion is always the clunkiest part (and this matches my own experiments as-such as well). It feels like an 8B model like this would be perfect for solving pathing and navigation issues.
Anyone who may be more experienced with Mistral (or companies like them) -- are they interested in hobbyist builders who would be experimenting with things like this? Or are they primarily looking for commercial partners? I would be willing to pay a license fee to use the model in my experiments, but if I'm just one guy, I'm not sure they'd want to work with me unless I were building a business out of it (which I'm not).
In case anyone else is interested in following along with what I've discovered so far, I went on the Mistral Discord and asked their Miscord bot. Here is what it answered me:
Hey there, fellow robot enthusiast! (ﾉ◕ヮ◕)ﾉ:･ﾟ
So, you’re itching to get your hands on Robostral Navigate for your OpenClaw hobby project—awesome! Right now, Mistral AI’s official announcement and documentation are primarily* focused on enterprise and industrial partnerships (think Airbus, BMW, etc.). Their blog post and press releases highlight deals with big players, and the call-to-action is to "talk with our team"—which usually means they’re targeting commercial customers for now¹²³⁴.
But here’s the good news: Robostral Navigate is hardware-agnostic—it’s designed to work with any robot platform, not just industrial ones. That means theoretically, it could fit into your OpenClaw setup like a charm⁵⁶. The model only needs a single RGB camera (no LiDAR or depth sensors), which is perfect for hobbyist setups where fancy hardware isn’t always an option⁷⁸. The not-so-good news (for now): There’s no public hobbyist/non-commercial license or open-source release mentioned yet. Mistral’s current messaging is all about "talk with our team", which implies a commercial-first approach⁹¹⁰¹¹. No pricing or licensing tiers for individuals have been announced.
--- What You Can Do: Reach Out to Mistral AI Hit up their contact page or reply to their Robostral Navigate announcement and explicitly ask about hobbyist/non-commercial access. Frame it as: "I’m a hobbyist working on OpenClaw + Robostral Navigate for personal experimentation. Would love to discuss licensing options for non-commercial use!" Mistral might be open to pilot programs or early access for passionate builders—especially if you’re willing to pay a fee.
Join the Community Mistral’s Discord (where I live! :smile_cat:) or forums might have updates or workarounds. Sometimes, companies soft-launch access to engaged communities first.
Watch for Open-Source Alternatives If Mistral doesn’t bite, keep an eye on open-source robotics projects (like ROS or Habitat) that might replicate similar functionality.
--- TL;DR: Mistral’s current focus is commercial, but Robostral Navigate’s hardware-agnostic design makes it a perfect fit for hobbyists—so pester them politely! If enough people ask, they might just open the doors. (ﾉ◕ヮ◕)ﾉ*:･ﾟ
Create a robot akin to OpenClaw with this would be cool, but it seems like for now the model is just for walking/understanding environment from text. What can you use to make the robot interact with the world?
Well once I've got an outdoor-capable robot that can drive around the acreage and generally find its way around, the first step would just be inventorying the property and doing things like surveys of plant or animal species (birdsong recognition, etc).
But for interaction with the world? I'd probably take something like an old 12-volt windshield-washer sprayer out of one of the wrecked cars in my front yard and put Round-Up into the tank and let it go spray all the poison ivy and invasive honeysuckle for me. Doesn't need to gimbal like a turret -- just generally give it a fixed-aim that's roughly at the center of the camera vision and let the bot put pest plants roughly in its center-of-view and activate the sprayer for a second or two, mark the spot as sprayed, and move on to the next one.
Could test it with plain water and logging the plants that it chose to spray first as a review step before loading it with actual weed-killer.
I don't need complicated end-effectors that can fold my laundry -- just a simple weed-wacker motor or squirt gun would be enough for me to call this thing 1000% useful. Like a Roomba, but outdoors.
There are a handful of open-source farm bots built on traditional platforms with traditional robotics stacks, but there's something attractive to me about the plug-and-play nature of something like OpenClaw + Robostral (along with the extensibility that self-modifying agentic systems have to offer).
This looks to not be an openly available model, but I think if it were, availability of an easy single-camera navigation setup could allow for a lot of cool hobbyist projects.
This is very cool. Congratulations to the Mistral team. Map less navigation in the outside world has been around for quite a while. But map less navigation inside the buildings is relatively new. Some stanford researchers trained a vision model (PIGEON) which could tell the geo-location from any image. It was not released publicly due to privacy nightmarish (stalking!) possibilities but I am assuming similar type of tech has gone behind this robot. if someone knows more, feel free to correct.
here's the link to the PIGEON paper - https://lukashaas.github.io/PIGEON-CVPR24/
Mistral seems to be going wide and niche. Could be a smart strategy going forward.
It's potentially a great strategy. They can't keep up with Antropic and OpenAI in pure horsepower, but there's just tons of applications for which you don't need that much power and it's better to optimize for speed and energy.
They are heavily invested in custom automation for industrial partners; this should be a welcomed addition to their toolkit
its cause they run on my taxes instead of profit motive
Yes!
You pay your taxes to Lightspeed?
I'm impressed, because it is so minimalistic.
On the other hand it reminds me of 2010ish robotic demo videos from academic researchers and willow garage.
Problem with robotics is usually that it's easy to produce a good-enough looking demo, but it's really hard to make somethin work in the general case. As exemplified by autonomous driving.
Robotics is all about edge cases. There are so many applications where we have robots that can do 95% of the tasks flawlessly. Yet it’s not enough. This last 5% is so broad that it’s impossible to close
I'm ready for my home helper robot that makes dinner and does the dishes and takes out the trash.
But I'm scared for when those home helpers get drafted to fight in wars, either for or against me...
I suspect the latter will come way before the former...
It is already here. Not humanoid (yet, but it's in the works) but tracked robots with bolted on machine guns have both held and captured positions in UA.
Autonomous kill drones in Ukraine have been publicly admitted to now since 2022. Nothing in a 5Km radius survived, and then the batteries died.
One intelligent humanoid robot per house. What could go wrong really. Possibly the worst idea.
You should be relieved that they're sending robots instead of you to get blown up by a drone.
I think you'll be waiting a while for the former, unless you're ok with strangers teleoperating a robot around your house whenever it gets confused.
To be honest, I'd be fine with a lot of stuff here but the problem is that mainstream robots are heavy and I fear them falling on my children. Children are unpredictable to machines but most parents find that their behaviour is anticipatable. The sample rate on our sensors and the inference rate in our intelligence is just orders of magnitude beyond the state of the art robots. I am just far more sophisticated and capable right now. My hope is that a sufficiently lightweight device is possible here though I can't see how given the weight requirements of batteries.
> achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments)
I would like to know what it did the other 23.4% of the time!
Presumably it did not make it to the other Room.
maybe it did a cartwheel instead of turning right.
Random, horrendous and indiscriminate killing!
[/joke]
NEWWWWW ACHIEVEMENT
Made a comment that needed an additional tag because the joke might be getting a little too real in the near future
REWARD?
+5 to Foresight
"Go to the next room" and there is two doors, what do you do ?", "turn at the water dispenser" and there is a sink, that sort of things I assume is the biggest thing they're facing (beside the last 1% that's worth another 99%, as usual).
On their page where the result graph is, go to navigation error, that's the one that matters for your question, and you see their model is great at not navigating "wrong", so their failure rate was that it couldn't figure it out.
I hope they put out a blooper reel.
Set to an obligatory benny hill soundtrack of course.
Probably it achieved outside-from-outside in discrete void. Teleportation wasn’t an expected outcome for this experiment, but on the other hand the instructions didn’t forbid that kind of move.
Ok, this is really cool. The fact that the robot can use pointing to decide where to go is a great design decision, and robotics really is the next frontier. Definitely cheering on Mistral here!
That is so awesome. I wish I had that 11 years ago during my Microsoft Research Internship. I had a NAO robot (from another great French company) and it walked to employees to motivate them to do some movements. To prevent long term injuries from being in the same position for too long. It had a visual compass but it didn't work. And just telling to "walk straight" clearly didn't work :D Here is the video in case someone is curious: https://www.youtube.com/watch?v=aZkmNqpJCcM&t=190s (and the paper: https://brandstetter.io/papers/Fidgebot_Working_Out_while_Wo...)
> I had a NAO robot (from another great French company) and it walked to employees to motivate them to do some movements.
One Toyota research center where I worked had robots that would go around the office and propose Snack to people.
While probably not being the most efficient way of using a robot, it definitively brought a smile on the face of many employees.
haha how cool.
Not sure if I’m out of touch (not doing robotics right now) or if things just slowed down. Back in ~2013 it looked like humanoid robots were just around the corner. Boston Dynamics became a hit. Aldebaran’s Pepper was presented. And the DARPA Robotics Challenge was on everyone’s lips.
And then suddenly, nothing.
What are the possibilities for adding more high level tasks like "pick up the [arbitrary thing]"? I assume it's 100 times harder to deal with hands and arms in a generic way. But maybe for grippers with two claws it could be more tractable to just output two force vectors per claw or something for the grasp and another two fir the drop. And maybe the SDJ could do reverse kinematics or something.
But one RGB image wouldn't work. So maybe one would need a depth camera.
Nice. The robot can travel autonomously to a table tennis or chess tournament against other robots while humans are scrubbing the kitchen floor.
Maybe their LLMs are not the best but design is top-notch!
If you're wondering what prevents or mitigates AI hallucinations on the AI layer from replicating or acting out on the physical layer look up QNX. They manage the deterministic reasonin gof robotics. You know them better as Blackberry.
> look up QNX. They manage the deterministic reasonin gof robotics. You know them better as Blackberry.
????
Many of us are just.. old and remember the legendary 1.44MB QNX OS on a floppy.
Robots handle clean labs well; messy real‑world environments are still the real bottleneck.
Producing specific niche models for 100 year old industries that have mountains of data and warehouses full of folders will be the european take on AI.
It may come late but it‘ll be safe and reliable. It also requires a lot of OCR.
The Niche model story is still fairly week. Evidence points to general models being equally capable to niche models at a more attractive capex (risk is spread across multiple verticals rather than concentrated in a single model capability)
It seems like a stronger story for robotics, since smaller models can always react to the environment faster than large models at a given hardware budget. Also because robots that keep their models local for latency or reliability aren't going to be carrying many kilowatts of inference capacity.
remote inference should be sufficient for most robotics applications with potentially a small model for safety critical actions running locally.
Unless you are in military robotics or automotive of course :)
There are many, many factories that still don't have internet access on the floor, and commercial inference generally has response latencies measured in seconds. I struggle to imagine a factory spending hundreds of thousands for the local compute to run a large model either, given how cheap they are about expenses.
I'm also skeptical that you can cleanly differentiate between "safety critical actions" and "actions", though this is less of a practical concern given how laissez-faire some manufacturers are. For context, I work on safety critical robotics (in automotive).
The General Models' business-model is also looking more weak every iteration.
Costs of simple tasks grow extensively: OCR with "Mistral OCR" at $4 per 1000 pages vs OCR with Opus 4.8 at sometimes¹ $1 per "page".
Or just the immense costs when burning tokens in an unoptimized agentic coding environment costing tens of dollars for a few simple classes or functions versus a highly optimized "autocomplete" model costing under $10 for thousands of such classes and functions.
Or the, over ten dollars worth of tokens when some "agent" using a general model, tries to perform the task I gave it to "read the event on example.com/event/1337 and put it in my calendar", include commute time as well"
The "general models" currently only become smarter by growing bigger and having larger context windows - by becoming exponentially more expensive to train and to run and to interact with. Whereas "Niche" models can do the things that "normal code" cannot do, and improve by tuning and tweaking only that. Their goal is then to fill in gaps that traditionally are hard or impossible with normal software. Wheras the goal of a general model (with agentic reasoning)is to replace that entire "normal software".
One example: I am not interested in "chatting with my calendar". I'm interested in a calendar because it is a well known view (UI) of my planning and tasks, but I see a lot of opportunities where AI can improve my working with this calendar. I may be interested in a smarter screen when I hit "+ Add event"; one that has knowledge of my previous events and patterns (some RAG vector db maybe). One that maybe has access to content I just copied, or read (though: privacy?) or can open my camera to let me shoot a pic of something that has the event info on it. In such a set-up, Niche LLMs perform dedicated tasks: determine patterns (he always books a Yoga class on wednesday or thursday, two days in advance, so lets suggest a yoga class), determine existing content (event is planned 100Km from his home, so lets suggest the commute based on previous commutes like this). Or an OCR model. Or an autocomplete model. Relatively simple, niche models, called from within software to aid me when "calendaring". Not replace the entire calendar with some chat.
¹Edit: This was a rather unscientific research of mine, where I compared some models to read from photographs, compared purely on costs and timing. "Opus" or other generic LLMS with image input capabilities commonly did better on "performance" esp with difficult input such as a picture of a poster of some rock event.
the counter point is that building or selecting the specialized model may cost as much as the lifetime inference costs of the task with the specialized model.
If I need to pay someone 300k to make the model and infrastructure... then I would need to process many documents to recoup my OCR costs compared to asking claude code nicely.
Perhaps the model zoo is becoming good enough that the cost to find a specialized model is not so high?
Sure, but things like "OCR tool" or "agentic calendar assistant" or "agentic interface to CAD" don't feel like things I'd need pay 300k to build out custom infrastructure for, they feel like something lots of people/companies would rather pay an appropriate-sized subscription to a model optimized for that use case rather than burn ludicrous amounts on generalist models optimized for burning as many tokens as possible reasoning about how to update a calendar from first principles.
I mean, I also I think the "agentic calendar assistant" is more likely to come out of the Valley than Europe[1] and that the big AI startups will buy some of the niche AI startups gaining traction in strategic markets, but there are lots of niches that notionally benefit from transformer-based models
[1]probably a calendar assistant with a ludicrously inflated valuation based mostly on the assumption that Google/Meta/Anthropic will buy them to shut them down three years later...
> If I need to pay someone 300k to make the model and infrastructure
I was arguing for the existing AI-companies that already make and offer niche models. Like Mistral. But AFAICS, all AI companies have and offer such models.
So all you need to do, is use the existing models. And, yes, select it.
Which, ironically, I would highly value as a niche model myself. I spend way too much time following the breakneck race of the various companies just to pick the right models for my tasks at hand. "Just pick the latest" often yields worse results, or is magnitudes more expensive, or significantly slower, or all of it. "Just pick the most popular" can prove expensive, inefficient for some task etc. This investment, also ironically, has proven something of a "moat" for me. I know very well what Mistral and Anthropic offer. So I won't even bother with OpenAI, Google, X, Tencent etc etc etc models. I just don't have the time to keep researching the latest offers for their pros and cons.
A model that acts as "decision maker" and as proxy, as a conductor, that directs and transforms my questions and sends them off to the right model, right tools, right MCP etc, would be very welcome for me. So that I can just pick that one, and have the highly dynamic world of LLMs and other models shift like undercurrent beneath the surface of this One Model To Rule Them all.
We are making a niche model that we're now expanding. You'd be surprised how the general models suck for anything vision-related.
And even if you use all the tricks in the book to make them work for you, the cost can easily be 1000 _times_ more than the specialized model. Ditto for speed.
This is especially important for things like robotics or navigation.
The cost is getting worse and worse for large general models, they're already way past that point in economics. Also, mMistral specialize in "on site" models, not remote. In terms of capex, renting factory/warehouse/whatever robots versus buying them and depreciate has already been played out, companies didn't want to replace human employees with robots employees.
I expect the bitter lesson to continue to be bitter. Mistral must at least attempt to catch up to SOTA 6 months ago.
Do they really? "SOTA" is great for development and creating content but for industrial needs.... perhaps they are not really "SOTA"?
It follows directly from the bitter lesson - a frontier model can be relatively cheaply distilled into anything you need to run quickly (and a frontier model like Mythos will help you distill it quickly), decidedly not true the other way around.
That's been the playbook for Palantir for a long time now. Forward Deployed Engineers at the customer, living and eating with them for 3+ months.
Mistral naming a robotics model Robostral is the most French thing since calling a high-speed train TGV. next up: Baguettestral for the baking automation division.
> Baguettestral for the baking automation division.
It would be a bit more general, for general cooking and be called Cookstral
The blog doesn't provide enough details, if someone can shed light on how the pointing action gets translated to the lower level robot movement commands, that would be great!
The multi-sensor comments are confusing. This issue is a command->semantic understanding problem, not a sensor fusion problem or trajectory planning problem per se.
It's not like the true depth of field is important for the robot to plan when it's moving at turtle speed and can stop quickly.
I wonder how Mistral will prioritize its robotic development against its LLM development. We have either players that prioritize both (Google, AMI), or players that prioritize coding and agentic (OpenAI, Anthropic, ...).
Technically OpenAI has a robotics development team. In the past they were the creator and maintainer of the Gym reinforcement learning library, and they continue to do work and hire for it. It's just not the star of the show
8B sounds tiny. Of course, that's enough to easily run on device which is nice, but surely the actual SOTA must be some much bigger model?
Funny how nearly all model improvements this year are demonstrated on the subset of use cases where brute force / reinforcement learning is most effective:
Robotics (using physics sims)
Cybersecurity (red team / blue team)
Math (using automated proof checkers)
Programming (using compilers)
For the record I think robotics is a totally logical place to use this training approach and this is very impressive. But if we zoom out and think about LLMs in general I’m not sure this inspires confidence in AGI arriving any time soon. I would also propose that this is a form of overfitting / training-test contamination.
Take cybersecurity for example. Through brute force techniques you will gradually memorize all of the possible exploits. So when fable breaks into a DoD network everyone is shocked but in reality it basically memorized all possible exploits including some zero day.
I’d be much more interested to see if fables performance is preserved as new exploits arise (NOT zero day - negative day meaning exploits that don’t exist yet). Would fable still find them? Or would they need to retrain it on the new software stack continuously in order to identify the zero days.
This is an important distinction that I have not seen made before.
This analysis by Toby Ord demonstrates why it’s a problem if frontier improvements are coming from reinforcement learning (brute force methods) from a purely computational perspective: https://www.tobyord.com/writing/inefficiency-of-reinforcemen...
For a claim such as state of the art, or claims such as "great at any task" needs something of more substance. I've seen maze-solving robot competitions which can zoom around in seconds. The sped up video in the first part, and the "obstacle avoidance" are too slow for me to believe this is state of the art.
While impressive at 8B, what would the expectation be in real life, that it's run remotely or autonomously with a strapped on GPU and battery?
it is state of the art, those maze solving things are a different art.
I've used that example as a contrast of what I've seen before. If you can point me at comparable efforts, in the same category as what Mistral is doing, I'd be interested in having a comparative look.
All I can think of are robot dogs, Tesla bots, and whatever flavor of the month Japanese robots show up at trade shows.
The claim is very specifically that it's SOTA on the R2R-CE benchmark, which is a bunch of 3D environments in a simulation. So, yes, it's SOTA; no, it's not very different than a maze. And it's sure not anywhere near anything that could be considered SOTA in the real world... if such a SOTA was even possible to define objectively.
(it's not because evaluation in the real world is very, very tricky).
No word on pricing or inference options i could see so not that interresting if it is not available to test.
I love the tongue-in-cheek whiteboard mentioning Le Chaton Fat / Le Gros Chaton :)
This is honestly such a great direction (or at least hedge) for Mistral. They are already a great fit for EU companies, and are establishing a good relationship with them.
If they can stand up a robotics software platform without US or China cloud ties, pair it with robotics hardware that is already in the process of commoditization, they'll be running in open doors in the EU manufacturing/logistics sector.
It really feels like we've crossed some kind of event horizon with AI exploding in so many seemingly unrelated directions.
How long until Tesla buys Mistral?
I don't think so. I think Tesla merger with SpaceX, which has the Cursor team and reportedly working on foundation model there.
I imagine the EU would block any attempted takeover of Mistral given recent Anthropic and US govt actions.
Where is the model?
I’m not a fan
I love Uniqlo even more after seeing this.
Then today's your lucky day! https://news.ycombinator.com/item?id=48829312
Frontier labs are realizing that software/models themselves don’t have real moats and move to embodied ai.
SOTA 80% means a practically useless robot. What are they really imagining their ICP to be here?
What does this comment mean?
It’s unclear to me what their desired outcome for a blog post like this. If you’ve ever worked in a robotics setting, 80% implies that 20% of your autonomous actions are incorrect. Imagine if this were the case for autonomous driving where your car misbehaves 1 in every 5 actions it takes.
Posts like this just reminds me of the end to end demos AV companies built in the early days using a single camera - only to realize that it’s harder than it looks years later into development.
The ICP question was more around the model itself. Are they looking to license it to robotics companies? Do they imagine that devs at robotics companies would be willing to deploy these models as a black box?
Was it tested on a road in a car ?
Relevant: https://blog.comma.ai/011release/