by tezza 5 days ago

I review LLMS, and 6.1 sol is fantastic.

What kind of stuff were you throwing at it and where was it letting you down?

So what have __I__ been doing with it, then?

I’ve got 6.1-sol to take an Amiga driving game binary and add networked car driving into a web enabled Amiga emulator… that’s interpreting machine code, Amiga architecture and then making meaningful changes.

So yeah, 6.1-sol is good.

Kurtz79 4 days ago

Yeah, I think people saying "model X sucks" should really specify their use case, otherwise is really not useful to other people. I have been using it for personal projects (mostly game development) and for data analysis and automation at work, and it has been fine. I switched to Claude/Opus as my main driver after 5.5 (keeping an OpenAI Plus sub) and it has also been fine, a smidge better, maybe.

People working on large codebases, and on live, production software or harder problems might have a different experience.

Speaking as a fellow Amiga enthusiast, I had it fix a visual glitch to a game of my childhood that I initially thought it was an emulator issue but it turns out it was in the original game.

It took Sol 6.1 just 10 minutes to idenfify it, fix it and deliver a working patched image.

alastairr 4 days ago

My experience with personal / greenfield projects has been good.

Where it falls down is on the boring stuff - writing tight PRs for features / changes in large production codebases (data area in my case). Without a lot of handholding, tendency is towards 'bolting on' new code resulting in overcomplexity & missing obvious opprtunities for simplificatation.

  • tezza 4 days ago

    Thanks for sharing. I do extensive direct comparisons, and use both Anthropic and OpenAI tools in anger on large professional codebases.

    I think you are correct, OpenAI tools need more handholding.

    However, I think that is actually intentional from OpenAI... they stick to what you the POH ask them and deliver what you ask. They embellish less. Anthropic tools by contrast embellish a lot, and they're very good too.

    Perhaps start explicitly asking the OpenAI tools to embellish or think outside the box. That may unlock that experience and results you prefer. If you already did this, would you mind sharing more detail on what effect that had (or not) for your workload?