Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat.
Just writing this down so I can be praised/mocked in 5 years.
I'm also of this opinion, but also that it doesn't have to be Apple (but they are well positioned). What I've seen with running local models on my 48GB M4 MBP is really impressive - it's not the same level as hosted stuff, but it's better than what I was using a year or two ago.
I suspect we'll see a hybrid before an all or nothing. Local models for computer control or delegating, online models for things that need strong reasoning, planning, and knowledge access. Again, I'd be more than happy to be wrong. I just see models growing faster than the hardware can.
Apple has failed to live up to the Steve Jobs era and initial iPhone hype. It's not just "AI experiences", it's computing in general. Maybe the consumer sector is just dead/dying. Maybe consumers are just running out of cash because filthy VCs are destroying communities and forcing the 99% into poverty.
The one thing that is marginally exciting: the Apple SoC or M series chips.
It's unfortunate they are locked behind crappy macOS and other proprietary apple crap.
Unsurprising. Apple seriously thought the iPad would replace computers and usher in a "post-PC" word during their "What is a computer?" ad campaign era. Now they are sticking phone chips in laptop chassis.
For the general consumer, they were basically right though. Most people don't use laptops except for work. The primary computing device is the phone, and phones have basically become become mini-ipads in form factor since that ad aired.
> Apple seriously thought the iPad would replace computers
...for some users. See their "Mac is a truck" analogy.
And it has. My parents haven't owned a Windows or Mac machine in six years, since they got rid of the one I gifted to them a decade ago. Its all iPad and iPhone.
You should not buy a fully loaded Mac Studio for AI unless you absolutely NEED macOS. You will be wasting so much electricity idling on prefill while your GPU pulls 150-250w from the wall.
Buy an Nvidia Spark, then whatever cheap Mac you want to use as a thin client. There's no reason to force Apple Silicon's round peg into a square hole like AI inference.
Outperforms doing what? Inference is not a homogeneous workload, memory bandwidth correlates to decode speed and layer swapping but not necessarily inference speed overall.
The other half of that equation is latency, predicated on prefill performance which needs a powerful GPU and ideally ALU-level optimization to build larger KV caches quickly. Even the M5 gets smoked in this department, the M5 Max has a 50% longer TTFT on Qwen's 27b dense model at only 16k of context, which is a pretty typical starting context to use for agentic editing in normal apps like OpenCode/Claude Code: https://raw.githubusercontent.com/Osmantic/MMBT-Messy-Model-...
For agentic, 50-256k token on-device coding sessions, the Spark will be faster and consume less power running larger models. Without an external GPU (which Apple doesn't support), Apple Silicon will always be bottlenecked during prefill. Apple's failure to address this with their GPU architecture is a big reason why Apple Silicon viewed as a waste of time and money for professional datacenter deployment.
For agentic work, you just cache the prefill kv cache of the relevant system prompts. TTFT is a little slow (10s of seconds, oh no!) the first time you boot up a new harness.
I keep hearing people make this claim that TTFT is a problem, and… it just isn’t, if you’re running oMLX.
Folks in my camp keep saying this, and folks in your camp keep beating a drum we tell you isn’t resonating. Not sure why I keep bothering to argue; you can’t buy a high RAM Mac Studio like mine anymore.
They're not usable for deployment. They're perfectly fine for "enthusiast" low-end usage with 10-30B models, but the same goes for almost every dGPU made in the last 10 years. Your Mac Studio cannot run frontier LLMs at an interactive speed, even Apple has given up on using it as an inference backbone.
I'm perfectly happy with Apple not becoming an "everything we do is AI-centric" business.
I'm fatigued by it all at this point. It's streamlining the interesting and fun parts out of my job (by practical necessity of use there), and if I used it half as much outside of work I'm sure it'd do the same there too.
I know public opinion polling supports that, but the parts of my social circle which are outside of tech seem to be, at worst, apathetic (and at best enthusiastic, though that's not a big fraction).
That said, I think it's a good thing that this sentiment is coming to the forefront.
I was actually surprised to hear my brother-in-law deride LLMs as being useless for areas he has expertise in when I visited for the 4th of July.
He was complaining that he would ask how to perform a certain repair on a car, and the LLMs he tried (ChatGPT & Grok) would give him a long involved process and he'd ask why not do it this simpler way and it would say, oh you're right! He just found it gave bad advice and realized (rightly) that in areas he has less expertise in he has no way to judge how good the outputs are.
This is from a guy who loves tech, historically worshipped Elon, loves his Tesla, and (rightfully again) didn't buy into SpaceX because he thought it was overvalued.
In the past when I visited for holidays he was liable to have a positive outlook on LLMs and their utility. Seems telling that he's starting to see the cracks.
Yep. In my areas of expertise I can easily catch it coming up with wrong information, bad calculations etc. So laypeople are probably being led astray quite often.
This is easily the biggest problem with the current models. The models are just way too eager to please / say yes to the point that the models are happy to lie/make shit up if it means it can say yes.
> It's streamlining the interesting and fun parts out of my job
Interesting. For me it's streamlining the tedious and attentionally taxing parts of my work tasks. I love solving problems, I don't particularly love shaving yaks.
I don't mind the subtle ML integrations that they have put in the photos app: plant ID, recognizing faces, removing background, OCR text search (even for handwriting!), etc.
You’re listing every feature I’d like to remove; Meanwhile Android users have great holiday pictures at the bottom of the Pyramids because they can remove the people from their pictures.
Apple is doing something very different. Their AI experience for end users definitely has been a little behind.
Apple Silicon, however, has been quite unique for the last 4-6 years and it's increasing overlap with LLMS.
The model/chip optimizations are definitely improvements, the thing that is really standing out the past 2 years is how much the open source model community has been making possible, especially when you know a group of use cases.
Here’s the two main reasons why local inference won’t compete any time soon with the cloud:
1. Most useful LLM work is done in parallel. A Mac Mini can run one LLM inference thread at a time. The cloud can spool up dozens and spread that inference across efficiently batched operations over a fleet of hardware.
2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally. But the advantages of running >5x the token throughput per thread can’t be overstated. Add in the multi-threading advantage and it’s a knock-out punch for local LLMs.
Local inference has a role: if you’re working with extremely private matters or you want an uncapped model that will talk dirty or generate NSFW photos, local is the only option. I think Apple and others will continue to also run a lot of useful workloads locally such as text editing suggestions, speech to text, text to speech, and image manipulation. As local hardware improves, these capabilities will get better too.
But, for most LLM work, the cloud will continue to dominate for a long time to come, if not forever.
You can always buy multiple Macs. I think Apple's great differentiator could be making frontier class models local. I you want more threads, buy more Macs.
I don't want to run any workflows on someone else's computers.
As far as I know, there isn’t an interface like nvlink that allows these macs to work in tandem; they would just send their data over ethernet, maybe thunderbolt/usb-c?
> A Mac Mini can run one LLM inference thread at a time.
That’s not accurate. With MLX, at least, parallel inference is both possible and useful. Model serving tools like LM Studio and oMLX support parallel generation with continuous batching, and the total throughput increases with it.
You are not wrong, but the practical reality of local hardware is to be batch-constrained in comparison with a multi-user inference cloud. You will never be able to compete cost effectively in your home lab with a cloud that has >100M end users streaming millions of inference requests per second across a gigantic fleet of machines.
Can I run a few inferences in parallel on my Mac Mini? Yes. But put 1,000 Mac Minis in a datacenter serving 1,000 copies of myself? That's going to be more efficient.
Oh certainly, a local machine can't parallelize like the cloud can. Does anyone think that, though? What it can do is provide good-enough inference to support common business needs, without creating unnecessary data risks.
Why on earth would you need to do that for most things? Not everyone wants to be the next Coreweave. It sounds like you're killing a fly with a cannon to me.
There is so much happening in that scene, where tokens/sec double or 10x
So I could see the same hardware doing 20 tokens/sec on a large model suddenly doing 200 tokens/sec in the future, a better device in the future doing 500 tokens/sec, while having vision models baked in, audio models etc
Users wont consciously switch to local, they will just have it and use it
and a toyota camry can’t compete with a lamborghini but the camry vastly outsells them.
you’re far far far more likely to see a camry or equivalent in an americans driveway than you are a super car.
you’re also more likely to see an enthusiast with a corvette or equivalent muscle car spending way more than it’s worth to tinker on that car in their garage than you are a super car.
I guess what I'm doing is not considered that useful then? I usually only have zero, one, or occasionally two things actively doing inference at a time, be it claude code sessions or one of the chatgpt/claude web interfaces, and i bet that's true for like 95% of people using llms. And anyway i bet even the hardcore people using a bunch of parallel agents would appreciate having access to local, private inference for some things.
You're obviously right though that cloud inference isn't going away anytime soon
What about when claude code starts spawning sub-agents? it's pretty good about doing that now even if a user doesn't request it. Does this could as parallel inference?
I haven't seen it do it by itself that much so far. Even then, a lot of the time is spent waiting for some shell command to complete, which isn't inference. I guess being able to do parallel inference is like having multiple cores in a cpu, you can get by without it by just sharing that single core, and having it just gives you a performance boost under certain circumstances
Interesting. For me it seems to be much more common that even basic requests start spinning up sub-agents. But I also routinely ask it to do this so maybe it's stored this as a preference somewhere deep in its bowels.
And yes, I agree. I find the experience better less for speed and more for context management. But it's far from necessary.
There is common wisdom: during gold rush, sell shovels.
Apple's shovel (ahem, Mac mini) is the highest quality.with Companies burning money left, right and center, Apple can dispense with advertising altogether
In 2010 one of the standard configurations for the Mac Pro was $4,999. Once you customised ram, storage, peripherals and software it could easily end up above $15,000, or $23k today accounting for inflation. Apple hardware is one thing that has actually got cheaper over time.
Or they could use that same amount of memory to ship 64x Macbook Neos, and probably make higher margins off the hardware volume.
Those Macbook Neo users would be very reliant on Apple intelligence, enough maybe to pay for a service with it. I think Apple's much happier going this path.
> Or they could use that same amount of memory to ship 64x Macbook Neos, and probably make higher margins off the hardware volume
If it's an "or," absolutely. But if it's an or, they should be prioritising Macbooks over the Mac Mini Doug Brooks is discussing.
When we breach the "and" of memory supply sufficient to allow for more Mac minis (and Mac Studios), I think it would make sense to consider relaunching Xserve (with new branding, of course) as a consumer/small business product.
Memory supply isn't what held back XServe. We wouldn't need XServe if Apple treated the Mac like a regular computer and supported usable, first-class headless workflows and eGPUs.
The writing has been on the wall since 2019. Apple doesn't like the old way of computing, their goal is to expand the ecosystem by prioritizing install-base and then pushing first-party service offerings like they did with the iPhone. And like they did with the iPhone, Apple is great at ignoring power users to focus on features that make them more money.
You may be waiting a few decades for this type of product, memory supply be damned.
Apple sure doesn't act like it. The Mac is still a minority market share of PCs, and their entrants into spaces like AR do nothing to compete with incumbents.
Now that the Mac Pro is depreciated, Apple's plan to pivot to service offerings seems set in stone. That's the "want it all" attitude they've adopted with the App Store.
It's really not. Apple's phone margins have been as high as 30-40% per-unit, it's likely that they make at least ~$80-150 per Macbook Neo sold.
At the $150 mark (which is probably accurate factoring in lifetime service spend), that's a $10,000 minimum return on the 64x Macbook Neos. Apple can charge that type of premium on consumer hardware, but they're in no position to command $10,000 margins on professional hardware. They're not Nvidia, Apple has always been LARPing as an HPC vendor.
But surely you understand how your preferred customer is the less profitable one?
Apple won't subsidize these low-margin enthusiast products with the profits made from services and higher-margin hardware. Tim Cook would much rather ship the 64 Macs, and get ~15-20 school-age kids hooked on Apple One or the App Store for the rest of their life. There's understandably not much patience for catering to people that want to opt-out of the Apple Intelligence service ecosystem, effectively leeching off of more successful products. The volume and opportunity cost kills the concept in the cradle.
I think so, too, and I think it'll end up being a race between Apple & NVIDIA (or NVIDIA partners) to see who realizes this first. It would probably be easier for Apple to do it because it wouldn't require a form factor adjustment [over the Mac Studio they already have]. That said, NVIDIA already offers chipsets for both the lower end (DGX Spark with Vera + GB10, at roughly the $4500 price point) and higher end (DGX Station with Vera + GB300, for $85-100k). The DGX Station is equivalent to ~5-6 RTX6000 GPUs attached to a mid-range CPU server, but far more than most individual developers would want or need. I've heard through the grapevine that NVIDIA's received consistent feedback that they need something like a "GB20" that slots above the Spark/GB10 and can simultaneously run larger models for inference while hosting a dev environment on the same box. You can daisy-chain Sparks just like you can daisy-chain Mac Minis, but you're still constrained on model performance based on what a single device can accommodate.
Form factor is the easy part - both Nvidia and Apple are experienced SOC designers.
The hard part is the GPU architecture. Apple Silicon was designed with a laser focus on raster efficiency (similar to AMD's GPUs) which makes a lot of sense for highly mobile hardware, but is a crippling mistake for high-performance compute. Apple's largest Ultra chips are hamstrung with SOC-tier GPU performance, their highest-end desktops are outperformed by Nvidia's laptop offerings. Apple has to find a way to scale upwards without imposing too much architectural strain on their cheaper hardware like the iPhone and Macbook. Nvidia has already solved this issue; full CUDA compute stacks are usable on extremely cheap GPUs like the Nintendo Switch's Tegra SOC, or the Mac Mini-sized Jetson boards.
In terms of "who needs to redesign more to address the market", Apple has a lot of technical debt to unearth before they catch up to Nvidia. And if they do catch up, Nvidia will still support Linux and other differentiating features that Apple refuses to implement. It definitely feels like Nvidia is closer to a winner with the Spark than Apple is with the Mini or Studio.
I don’t think we should use current prices as landmarks for large scale demand. That Studio’s current prices is inflated because of a (presumably) short term supply crunch, not because the average user is willing to pay $24k for a home AI inference device.
It assumes that RAM remains supply constrained and that none of the existing RAM contracts are cut short.
But Meta and xAI putting A TON of AI compute onto the market. OpenAI and Anthropic are raising the costs of inference (by reducing how much inference users get via subscriptions). And we haven’t seen Oracle / CoreWeave struggle to pay their debts yet, but they will be selling assets once they get close to that point.
Unless the raw materials have an inherent limit on mining/production due to the amount present on the planet, why should or would companies not ramp up to eventually meet demand?
Edit: Okay, this doesn’t mean that that’s actually possible in the short-term, so I think you’re right. But that means as the silver lining, in the medium term horizon there’ll be enough supply again? :’)
Thanks for the link (and underlying thoughts), I really hadn’t considered that.
So essentially, due to technological progress and other factors inducing price collapses (or at least cycles), you can’t start stockpiling insane amounts of finished-product semiconductor, which means you can’t scale production at current technology levels to infinity either?
I heard that China was spinning up DDR5 (but not HBM?) production in the next couple of years, with the hope of outcompeting Korea and Taiwan in the mid to long term.
In two to four years, the Chinese will have at least half the memory market worldwide, and once in, they will continue forward and not look back.
I also believe there would be one or two tech companies that will get into memory by taking it in-house to make sure that they won’t have this problem again in the future.
For existing producers expanding capacity would be a risky move. But it's the perfect time for any newcomers to enter the market. Low yields and worse product don't matter as much right now, and by the time the market cools down you have everything dialed in and can compete on even ground
> it's the perfect time for any newcomers to enter the market
This is a good hypothesis. Curious if anyone has data on the failure rates of new entrants in semiconductors based on how frothy it was on founding.
On one hand, more demand makes selling easier. On the other hand, a shortage makes your input costs (consumable and capital) pricier.
EDIT: It seems like the 2 to 3 year lead time and a crowding effect from new entrants historically made booting up a fab into a boom a bad bet [1]. (The article argues, convincingly, that this time may be different.)
If demand doesn't fall down or current manufacturers supply go up, somebody (presumably in China) will spin up fabs. Apple wanted to use blacklisted Chinese RAM already.
Ebay and Amazon are flooded with it. Especially if you are looking for anything prior to DDR5. DDR2 and DDR3 are especially flooded with weird brands you've never heard of before.
Unfortunately its not so cheap anymore as everyone ramped prices up of course.
Last year I could still get 32GB of DDR4 for under $60 from chinese brands.
DDR5 is still mostly made with DUV (remember Intel 14+++++++++?), and even though manufacturers have slowly been moving a few layers to EUV the advantage is at the margin. Lack of EUV at scale will not prevent China from ramping useful RAM into this market.
Chinese fabs might not be so tied with red tape and regulation upon regulation (which is a funny reversal, in terms of "communism vs capitalism" bureucracy/inefficiency cold war thinking)
>1.) China is not communist, even remotely so. China is fascist in every sense of the word.
Except in the actual historical sense. They appear to enjoy all sorts of freedoms, increased prosperity, even have elections at different levels but under a single party system. Which is not necessarily that different than a effectively two party system.
>2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.
Now that China is more effective, "it's easy because they're authoritarian". Before the argument was "authoritarianism can never be as effective as free-market democracy".
>3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."
It's real world capitalism, not some fantasy some guy imagined removing all warts.
I think the most ironic fact of the 21st century is that there are less than 20,000 naturalized citizens in China. Western leftists don't really have a good explanation for that one and it definitely leans into the fascist characterization.
If the increased demand is not short term, production capacity will eventually increase. In the meantime, the logistics disruptions and industrial material shortages and energy inflation will disappear as soon as the wars disrupting them stop, which should bring prices down.
If demand and prices keep rising without production capacity being built fast enough, there will likely eventually be a rush leading to overinvestment and price crashes, but there are too many other factors involved; state investment for security, international politics and trade relations, the possibility of an AI bubble burst, etc.
There are wars coming. The prices are not going down.
We are in a bubble which will be burst the moment the world starts retaliating against the US' 20+ year history of supporting genocide and committing war crimes unabated.
The underlying demand for massive RAM builds is coming from AI hyperscalers.
There are lots of signals that the sector has been overinvested and that corporate customers are pulling back on spending as the cost of the APIs is revealed.
Once the hyperscalers start struggling to bay their debts (it will happen, just a question of time), there will be a supply glut.
So the only question is: do we share the same definition of “short term”.
By the way, they cut the supply because they run out of RAM. They still sell the 96 GB model. Everything else is gone, no stock, they cannot manufacture them. Even minis are 48GB max!
Funnily the current high end Mac Studio are not suited for current LLMs. M3 Ultra is "quite an old" chip for AI, despite its bandwidth. The issue for running local models (especially LLMs), you need few things to align really well: 1) compute power (affecting PP) 2) VRAM capacity (affecting model size you can load) 3) Bandwidth (somewhat affecting decoding speed).
The issue with the M3 chip is the compute performance, as it doesn't fit well the transformer architecture. This changed with the M5 (apple baked their own matmul into the chip), which would significantly speed up PP (and video/image generation btw), making the M5 Ultra significantly faster than the M3 Ultra and in practice much more usable. You can try to load Kimi or GLM on M3 Ultra, but it's not usable. Now the M5 Ultra is not out yet, but undoubtedly it will be a superior offering, and shilling 15k on 512GB version is actually reasonable (if it's every priced remotely around that tag).
> cannot imagine a personal computing usage which can justify a 10k machine
For me, the privacy pitch wins. I have a friend visiting, however, who spends like $2,400 with Anthropic every year. That's a solid ROI even if the thing becomes obsolete after a couple years. (I'm still on my 2020 MacBook Pro. I love it and will be sad when I have to replace it.)
> That's a solid ROI even if the thing becomes obsolete after a couple years.
How can that be a solid return on investment? There's no model you can run locally to have frontier model level performance. Also who spends 2.4k yearly for personal AI usage, like what's the usecase? If your friend is spending that money for his business then it's not personal computing.
> There's no model you can run locally to have frontier model level performance
I'm betting he doesn't need a frontier model. Sonnet, today, is likely good for 80% of his tasks, which largely involve repretitive, tedious work.
> Also who spends 2.4k yearly for personal AI usage, like what's the usecase? If your friend is spending that money for his business then it's not personal computing
I have a friend who is a teacher and spends that much yearly for ChatGPT and Claude. It saves him many hours a year in time assembling documents and creating posters, flyers, and personalized learning materials for his students.
In addition to privacy I’d like to be able to burn as many tokens as the hardware will let me 24/7 without getting a surprise bill at the end of the month. I don’t care if it is slower that the cloud, I’m not in a hurry.
Apple is a premium brand with high brand loyalty. Do you not think even 1 billionaire would want something like that? Even to just say that they bought it. Apple could sell things at a price point much more than $10k.
There's only so many billionaires and at Apple's scale you would not offer such a product for public sale even if you do custom builds for the rich and famous.
Apple makes product lines with assembly lines, its not a hand fab or custom build type of place.
At the end of the day, it’s just an Apple computer, not a Ferrari or an Aston Martin. I hardly think an Apple computer can be considered as a luxury item, unless they release it as a limited edition
I just went to Apple.com and specced out the top of the line 14" Macbook Pro, and the price is $9,849.00. Well over 10K with California sales tax.
I could absolutely justify having that machine for the actual work I do, and I'm not even doing any of the really hard AI things. If I were, I might prefer a Thelio Mega workstation from System76, which is $90,383.00 fully loaded.
Alas I can not afford a 10K personal computer right now, which is not the same thing.
> I cannot imagine a personal computing usage which can justify a 10k machine.
If Apple released a machine that would let me run, say, DeepSeek V4 Pro or GLM 5.2 locally at 100 tok/s for $10k, I might hurt myself running to get my credit card.
But then, I'm also posting this sitting in my truck while my family attends an event, with a Vision Pro on my face so I can monitor 7 Claude Code sessions without constantly switching screens on my laptop.
$10,000 divided by five divided by 12 is only $166.66 a month without interest over 5 years a Mac M2 Ultra or Mac M3 Ultra are easily usable computers over five years, so are most of the medium to high-end laptops that Apple sells for one person or locally in a small company.
It’s not necessarily out of reach or unusable over the course of time, the thing I keep hearing over and over is how usable many people find Mac’s very useable over the course of time, obviously software support plays a big part of that.
On top of that, AI models are getting more useful despite their smaller size. The future is a personal computer future, not a mainframe computer one. Yes, larger computers are useful, but most people will not be using that larger computers, which will be relegated to universities and larger companies.
Apples on stage use cases for their hardware and software makes me wonder if they actually use computers over there, or what a "job" at apple entails.
I am unsure that apple themselves understand why their hardware (top end & bottom end) has been so successful, without this understanding leaning into these use cases isn't really going to be possible.
Obviously they are playing 12d chess. They stopped selling high memory machines, they stopped selling pro machines. They are the king of local Ai compute, definitely not stumbling backwards into a product category they didn't know existed.
With their apple finger right there on the pulse, they are going hard on the VR/AR glasses (following the lead of the visionary CEO of facebook), cars and folding phones. By the end of the year (tm) we 100% will have all the features that were showcased and demonstrated 2 releases ago.
Sadly, that is an outdated PoV. It has probably not been valid, since last century.
It's just that Apple isn't really focused on software development professionals, and it's still fashionable to throw shade on them, so we hear a lot of kvetching about it, in communities like this.
I dunno. It's not my bailiwick. I do know lots of pro editors use Macs, but I think they use DaVinci Resolve (not Final Cut Pro). I'm interested in finding out what you use for it.
I've been using Macs for all kinds of stuff, since 1986, so I can definitely state they get work done.
But I still strongly believe that Apple hates pro users because they don't make as much money and because they get in the way of serving laymen. The Aperture fiasco, the Final Cut Saga, the Xcode war of attrition and the never ending chain of failures with MacPro - all suggest that I'm right.
Good on ya. I'm not interested in fighting about this stuff. I've had people hating on me for using Apple since the 1980s. It gets a bit old. Sort of like high school popularity contests.
All companies have deficiencies in the tech world. However, I would rather be in Apple’s position today, in comparison to Intel or Microsoft, particularly over the last 20 years.
And I definitely feel the pain with Aperture, not making it to Apple Silicon.
I feel that it does, but I’m also a dev. I used to run a multi platform shop for years, and have a pretty good idea of what kind of support various companies give.
That’s just excuses for example how does a relatively small company like Black Magic Design With one of the best programs in its class manage to support three operating systems MacOS, Windows OS and Linux OS simultaneously?
There are many larger software companies that can support all three operating systems, they just make up excuses/reasons as to why they can’t perform over the years.
Ages ago, back when the Macs would come out, my co-workers and I would take a bit of time to configure the most expensive possible configuration --- time was, it was pretty easy to hit six figures, but over time, that has gradually come down.
I asked an Apple (via a sales rep who visited our company to showcase in internal iPad healthcare app) to please do this for iCloud when iCloud Drive was in-development. We would have easily paid $50,000 for a rack-able Mac Pro you could point "managed" devices at.
How do you design XServe being tied to IBM/Intel the way you want to make it, and why would you, when you’re having problems with Intel, why? would you go any further with Intel hardware wise? Makes no sense. The next four years will be interesting with an engineer CEO in charge.
Apple knows the market demand for this type of device.
You may have paid $50,000 for it, but you’re only one customer. At Apple scale they need to focus their finite resources on the products that serve the largest market demand.
$50,000 rack mount servers are not a large demand.
From a historic standpoint though Apple came back from near-death because they differentiated by focusing on the consumer first.
While other companies were recycling the same beige boxes meant to be tucked under desks for home use, Apple came out with products in candy colors.
While Microsoft was rolling out new business process tool SKUs, Apple came out with GarageBand and bundled it in for free.
Apple is not the company prioritizing going after a Fortune 500 company to replace their fleet with Macs. So their focus isn't going to be to design products and features to try to get that deal closed.
If I had the capital I’d make an household inference appliance.
No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.
Even the appliance faceplate would be a display showing stats like an old HiFi stereo.
Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory
Fairly sure most iGPUs these days are zero-copy and can dynamically allocate memory so what does "unified memory" mean to you exactly? A wider bus would be nice but it's not exactly a groundbreaking new invention.
> Unified memory in Linux creates a single address space accessible to both the CPU and GPU, eliminating the need to manually copy data between system RAM and video memory. It is enabled via NVIDIA's CUDA, AMD's ROCm/HIP, or generic kernel-level Heterogeneous Memory Management (HMM).
So it does exist and is available for platforms that matter.
Unfortunately their chatbot, while amazingly fast, doesn't know anything about the company running it.
Anyway I wouldn't mind an ASIC running a diffusion language model locally. Even if eventually it would become dated. Beats outsourcing all that to a company that's running on VC money which in the future might either perish or worse - dominate the market and charge whatever they wish.
Yes. Solar thermal heaters on the roof are common in Florida and other parts of the south. Some people also use heat recovery devices attached to the AC condenser. Further north I've only seen natural gas heating (e.g. in very rich NYC exurbs). The amount of shade over the pool has a big effect.
I'm keeping an eye on Tenstorrent for this. Pricing seems like its going to end up being in between a super memory dense unified memory platform, and a purpose built GPU.
Definitely on the edge of what would make sense at home, but its interesting.
What I’m not sure to understand is that if you want to just run Claude code or openclaw type software with llm apis or subscriptions (and not run local models) to benefit from a local file system and always-on capability for ‘second brain’ type of workflows, I guess you don’t need a Mac mini but can run it on a raspberry pi or an old laptop ? Does anyone have experience with that ?
Yup, for openclaw and APIs you dont need a big PC. I run something lightweight on the RPI4 8gb. Many people run local LLMs which is where a mac is useful. Frankly I dont think you can beat the value of an openrouter subscription and API calls.
> Many people run local LLMs which is where a mac is useful
Unless you go for the very expensive options, most of the Mac Minis really aren't suitable for running local LLMs, they're painfully slow with prefill/processing input, and the models you are able to run don't handle long context very well, which these sort of long-running agents perform very differently with when you can.
I'll agree with your latter point, hard to beat the value of using something like OpenRouter or similar remote inference.
Even with local models, you can run the agent software and the inference workload on different hosts, which is what I'm doing at home. Beefy server responsible for inference, tiny VM on other server is running the actual agent software + RPC + bridges and what not.
Why not go direct to the source instead of paying an extra 5.5%? Seems like it'd be trivial to have AI wire up connections to your preferred inference providers and save yourself some money over time.
If you're referring to the markup charged by OpenRouter, you can use harnesses like OpenClaw/Hermes without it and go direct like you're saying. If you're talking about actually "routing", then you don't get that out of the box. However, the popular use of those harnesses doesn't often use the smart routing approach with a single agent. Instead, the approach is to create multiple agents, each with a role and a model tailored to that role by cost and functionality.
If you or anyone else don’t mind, I have a have a question or 2.
I use Claude Pro ($20/m) as a glorified search engine (no ads/SEO) plus simple hobbyist dev things (shell scripts, managing my Mac, apps etc.
I also use it for tasks like - “search the web for top ten selling EVs, put them in a table” and then iterate - pivot tables, charts, additional research”. It could be cars, it could be broccoli. Code Work has facilities to streamline this type of work, but I usually drop into the CLI.
How much if any functionality would I need to recreate if I switch to OpenRouter and would be match my costs with the API approach. I don’t want any cost overruns. With Codex or Claude, if I run of tokens, no big deal, I can wait.
Pretty much. Running a local harness calling an llm via APIs doesn't necessarily take a lot of resources. But whatever tasks you want that agent to do via tool calling will be limited by the resources of the machine it runs on if you run those locally, so that's what should inform your choice of specs in this case
...or any cheap VPS? I now do most of "second brain" things via pi harness with Opencode Go subscription, and it costs me like 20 bucks a year, with added benefit of "you can have tmux and open session realtime on whatever device".
A Pi running macOS more or less. Not dissing it though. Killer machine for those who don’t need a lot of power locally. Also a great kiosk for some things.
With how apple seemed to be caught by surprise when it came to Macbook Neo demand, I'm not sure they have the quantities of SoC's around to handle the demand a Mini Neo could drive. Especially if they could do it for $299.
I will go out on a limb and say that's not going to be an Apple product, period. It doesn't fit anywhere in the value envelope.
The relevant questions here are: will the person using this machine also conceivably be wearing a pair of $549 AirPod Max? Or a $399 base Apple Watch? Does that person expect to pay more or less for their largest-screen computing device than their headphones?
Framing that way points toward a $350 price point being a laptop for young children (younger than Apple Watch age, so lower elementary). That's a whole different software experience beyond just the hardware.
It's anecdotal but the kind of people I know that bought Mac Minis for this purpose are what I'd call "light techies." They definitely know how to use an iPhone or a Mac but would struggle on the CLI of a Linux box.
Anyone who wanted the OpenClaw use case that is comfortable with Linux probably already has several Linux machines (including a few Raspberry Pis) on-hand.
> Is it a lack of knowledge from the users or do they really value iMessage integration that much?
My understanding is that the barrier to entry to using iMessage makes iMessage a LOT more secure from spam. If you want to do mass iMessages you have to register as a business with Apple, go through all sorts of checks and attestations, etc.
At any rate, iMessages are a lot more trustworthy than SMS. So being able to spam people via iMessage is very desirable. I recall a few months ago a guy posting his little spam-iMessage-as-a-Service product here on HN. You could build your little iMessage spam army using a bunch of Mac Minis...
I do exactly this. You can run the whole thing on a PI. I have actually installed asahi Linux on my Mac and I connect to it remotely so you can be sure I will never upgrade my Mac again because it’s already overbuilt.
Correct. You do benefit from some headroom for things like launching browsers etc but refurbs or mini PCs (with at least 16gb; ideally 32gb of memory) from the likes of Minisforum or GMKTec work well enough if you're wanting to spend a little bit of money.
Yeah I find the Mac mini trend is kind of baffling.
It seems like it's driven either by 1) people hearing Macs are good for AI, buying one, and using Claude for inference, not realizing that you interact with the anthropic API from an internet connected hair dryer. Or 2) people want their agents to have blue bubbles.
I find it hard to believe that enough normal people are doing on device inference is driving Mac Mini's out of stock. And even if they were the Mac mini is not actually a very good platform for it.
One aspect you're missing is that people running a claw type agent thing need to run it on a Mac to automate software in the Apple ecosystem.
Neo-Siri in iOS 27 removes the need for a lot of this, but before then, if you want to ask a robot about information that is stored in Apple notes, or to send an iMessage, a Mac mini is your only practical option.
The demand is not coming from 'normal Apple customers' it's coming from people who want a machine that can run local AI.
It has nothing to do with Macs being especially good at AI. It has everything to do with being one of the last 'cheap' devices being sold with that much unified RAM.
There are two angles to this. One is that if you want to integrate your agent setup into the Apple ecosystem you need a network connected Mac running 24/7.
The second is that the puck is heading towards local models. The people running their own 'Claws are usually experimenting running their own services either to save money or to explore the future where 95% of requests are handled on device.
You don’t need a Mac Mini just for that, but they’re fairly inexpensive (or were anyway) and quality is very good. The people who buy them may never use all the performance available, but they’re more interested in convenience than getting the cheapest thing possible.
You can sort of justify it by assuming it will last a long time and they’ll use it for other things, too.
Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind.
Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.
For some models like the popular coding and chat models, things move faster. For things like images, voice, sound etc they definitely lag a long way behind.
How is it a pain exactly? It’s just learning and only takes a day or two to get up to speed. We seem to have forgotten that for the past fifty years doing all kinds of tasks on computers has been tedious and involved and time consuming to even get working. My first computer had 48kb of RAM and to play a game you had to load it off cassette for five minutes. That was annoying. Having LM Studio download a model and load so you can chat or attach an agent it is effortless and easy in comparison.
In general I just don't like not using a cross platform architecture of some kind.
MLX is fine, but the cross platform alternatives (ONNX) are terrible, GGUF often lacks and doesn't have the "easy convert" that some of the commenters below say MLX has.
It’s not for the AI inference, it’s for the tool calls and desktop GUI app workloads and browser. There aren’t any on-device models capable enough of real work that can run on lower end Mac Minis. But for running a few browsers and GUI apps, you’re much better off buying a Mac Mini than paying for a more expensive and worse-performing container in the cloud. Browsers were not designed to run in Linux containers but they run optimally on baremetal desktop OSes. An M4 Mac Mini beats the single core performance of any VM you might rent in the cloud, in terms of raw compute per dollar (Geekbench scores).
> Apple's Mac mini and Mac Studio have become the machines of choice for running AI agents, according to Doug Brooks, Apple's senior product manager of Apple silicon.
This is mostly an US phenomenon, no Mac mini nor Mac Studio around here.
Only Thinkpads and Macbooks laptops talking to hyperscalers.
Around where? They're pretty popular for it in the UK right now given our obscene energy pricing as they end up being one of the best low power options for local llm. If you're not in the local llm space you obviously wouldn't see it. It's like saying Tennis isnt popular around here then admitting you dont frequent a tennis court so wouldn't even know.
Being a Mac switcher since 2003 I am as much of a fanboy as anybody else but this quote from the article caught my attention, and smells like PR.
> Many AI tools are also Mac-first or Mac-only
I fail to recall AI tools Mac-only general purpose AI or agentic tools. Most of the claws, harnesses, studios and inference engines seem to be multiplatform. You can say you can run then in a Mac with a nicer UI wrapper or whatever, but "Mac-first" or "Mac-only"?
oMLX[1] is the only one that comes to mind but it's not exactly unique to mac, it just runs MLX models and provides a nice gui. It does have the whole paged SSD KV cache thing, not sure if thats working on other platforms.
My guess is the OS. People who want a server often enough want to choose the OS, Apple wants to supply the OS and the hardware together so they're not blamed every time the two turn out to be incompatible, as happened the other way round in the 90s.
Target audience - B2B. There are multiple videos of Steve Jobs saying that he hates B2B, because the people using the devices are not the ones making the purchasing decision. It is pretty much against Apples DNA, and all their B2B they have today is a means for them to sell more B2C.
Because they killed the market, no one would now buy a macOS server, when Linux distributions, and to a lesser extent FreeBSD, own the server room.
They would even sell less than Windows Server licenses.
By the way, they are down the same path with the workstation market, now that they only top level answer is the Mac Studio.
Workstation market wants flexible towers that they can customise to their own liking and special use cases.
The main reason Swift exists for Linux, is that app developers need to have servers somewhere, and if they want to share Swift code with the backend, well it isn't going to be on macOS Server.
whos going to buy one? You cant trust them not to kill it within a few years and cease all software updates AND make it impossible to install a different OS to keep it going. Until they stop being dicks about what you can do with the hardware you own it's a non starter.
Apple itself was a major user of Xserve, Apples needs for cloud compute are massive and growing, and Apple could probably rent Xserve as a cloud to justify the cost and sell it to privacy focused consumers and businesses.
You think Apple is a saint, but they have a completely locked down mobile computing/phone platform. Why the trust in them doing what is best for the user?
Oh please the neural engine is mostly useless for LLMs. Siri in iOS 27 is laughably pathetic and slow compared to GPT Live DESPITE sending personal context to their (attested) cloud to execute anything but the most basic queries. Still years behind.
I think the relevant comparison is the developer beta, which has access to the Gemini-powered Siri that will roll out publicly later this year. From the reviews I've seen, Apple won't be "years behind" (which surely they were) for long.
I did say iOS 27. I am using the new Siri. It’s better, but extremely unreliable (half of all requests fail) and slow. It should be like a Codex running on my phone with the ability to chain skills (intents) to execute a task, but it’s too crappy for that.
And the voice is still a poor text to speech model, very far behind GPT live.
Apple could dominate this niche if they decided, for a while until prices fall, to eat some margin and bump up RAM in high end models. Couple that with a new M series chip with even faster AI performance.
It’s not a huge niche but it’s an influential one. They’d get the engineers and CXOs of AI ventures and a lot of academics and hobbyists.
For the platform it would keep them cemented as the high end vendor. In the long term it would position them to take advantage of any software or training breakthroughs that deliver frontier model performance at that scale.
Except that Mac ultra M3 they talk about is now only being sold in the 96 GB configuration. It’s no longer being sold in larger Ram configurations by Apple Apple because of the global RAM shortage. And you can not add memory after purchase because it’s integrated/soldered on.
> And you can not add memory after purchase because it’s integrated/soldered on.
For as much as I dont like this aspect of modern computing, I understand why it is done from a technical perspective. Power, heat, and performance are all "better" when ram is on the motherboard vs in a "stick".
> "The speed of AI development right now is just crazy," Brooks said. "I can't imagine where we're going to be a year from now, three months from now, or even a month from now," he added.
I don't think I'm taking this out of context when I say this is unintentionally correct. Apple still doesn't know what to do about AI.
Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.
Everyone else is using it as a content scraper and praying nobody will step in to end the piracy/fraud.
The new Siri isnt that exciting at least on my iPhone 15 Pro Max. I know it's beta but it's sluggish and often says try again. I watched many videos on Youtube saying its amazing but maybe not so much on older phones? Also, I need a Siri I can talk in an unfettered manner from my lock screen while Im driving without having to unlock the screen. Probably a big ask for Siri to know my voice via voice fingerprint allowing unfettered access from my lock screen.
Others running the beta now on newer iPhones and enjoying it more so?
I use it on my 15 pro max. It's a major improvement, but I still don't use siri much. It's gone from murderously infuriating to passably usable. I still use gpt for any sort of exploratory conversations. I haven't bothered trying to use siri for that because to me that's not what siri is for.
Although given how effed up the voice for chatgpt is now with the latest updates I might talk with siri more.
Because I use carplay in tandem with my phone where the map is on the carplay screen and turn by turn directions are on my phone, it's always unlocked so I haven't run into whatever lock screen issue you brought up.
> The new Siri isnt that exciting at least on my iPhone 15 Pro Max
On my Pro 16 it has its ups and downs - I still can't get it to "play my running playlist on shuffle" whilst running (this is the only thing I used Siri for before the beta and it would improve my life immeasurably if it worked). But it responds to things like "how long will it take to drive to the AirBnb booking in my inbox", and "when is X playing a concert in Y - add a calendar entry with details" perfectly.
This is a beta and I have hopes, but I can imagine it will run better on a 17 and later
> Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.
This is... a view.
Maybe I live in a strange sphere of strange ("normie"-ish) people, but the people around me are for sure using AI. Mostly chatgpt to be fair. They use it to compare products that they intend to buy, identify plants in nature, create travel plans, find interesting places to visit nearby, give movie suggestions based on what they have previously enjoyed and so on and so forth. AI is becoming a very integrated part of their reality. To "google" something and digging through the search results manually is very rapidly being replaced by asking chatgpt, for better or worse.
Chatgpt is so damn good for cooking it is unbelievable. It will learn your family's tastes over time, you can tell it what pantry staples you keep is stock, and you can take any recipe you find online and ask for stuff like "find a way to make this preparable in 45 minutes instead of 2 hours, what trade offs will I be making?"
I agree! I think it's because there's so much cooking material in its training data. I wonder what proportion of the internet is food blogs and recipes... Probably a lot!
This is true but I can do the same on free ChatGPT without even logging in. I wouldn't pay $5 a month for that functionality, much less $20 (or >$1000 to be able to run it at home).
SOTA AI for "serious" work is in a different position, used by fewer people but with big pockets and sometimes a pathological dependence on it.
Nobody does what to do, they are just throwing trillions at it betting they’ll figure out.
If they don’t they’ll be screwed, if they do Apple will quickly catch up with a better and more refined product, as they’ve always done.
When you don't limit on-device AI to "can be used to run coding model", Mac minis are great. My M4/16gb been working on a long term research project using Qwen 2.5 8b for months now. The performance is good enough for processing a lot of small text prompts.
Now just imagine they'd kept making the Mac Pro and enabled compute offload to GPUs. Or even just passthrough to Linux VMs. Would've been quite the AI machine.
The issue with the Apple is that they didn’t really develop any competitive local AI machine. Their strategy/marketing falls flat when you ask them how exactly they implement AI: they buy it from Google cloud. In the future local AI may become a thing but that’s 4-5 years away. I count the 2q-4q and atrocious performance as “local ai” only for the enthusiasts crowd not for people doing competitive work.
> "people often want a system that's under their control, isolated from their primary machine, and capable of running 24 hours a day, seven days a week," said Brooks. "A Mac mini is an amazing system for that," he added.
These execs are so out of touch they believe Apple hardware to be "a system that's under their control", how does it come to this? Besides, a VM without bi-directional sharing of data gives you pretty much the exact same thing.
Did hundreds/thousands of developers really go out there and bought Mac Minis just because one prominent technology semi-celebrity happens to have used a Mac Mini for the development of their thing? Seems bananas people would spend hundreds on monies on something they barely grasp how it works.
The remote access story for macOS is absolute sadness, without Jump Desktop there would be zero performant ways to access that “system under my control”.
And all of that because Tim Apple fears any feature that could mean people could have less than one iDevice per person.
I'm curious what you mean. I have been accessing Macs remotely over SSH and VNC for like 20 years and it's always been easy and as performant as the network would allow.
The built in high rez screen sharing between Macs works well too. Through Tailscale I’ve accessed my main Mac from both the opposite coast of the US and from the other side of the Pacific and it works great.
Host resolution automatically matches that of client, image quality is great, framerate is decent, latency is minimal. The host creates virtual screens for the connection so connected screens don’t light up and the machine remains locked to anybody accessing it physically too, which is a nice privacy assurance.
Ok, I didn’t want to take the bait but this one’s just too much.
> “He also described a shift toward running AI locally rather than in the cloud – a move motivated by privacy, security, and the rising cost of inference as agents consume more tokens.”
Classic Apple. No more just beating the “security and privacy” drum, now its “tokens are expensive!”
<neanderthal voice/> Cloud scary. Cloud expensive. Mac good. Buy Mac!
> “He also singled out what he calls ‘transparent AI’ on iPhone and iPad, referring to features scattered throughout the operating system and third-party apps that work quietly without announcing themselves as AI.”
<neanderthal voice/> Apple use AI, Apple just not say it. Apple smart, not lagging behind industry! Buy iPhone!
How about you invest in developing your own models, correctly? And provide a secure and private inference cloud service on your fancy Apple silicon? And integrate that into your platform so Siri gets smarter without you farming queries out to Google Gemini? Bill me for it in iCloud+ I’ll probably pay for those tokens.
But why should apple invest in developing their own models? Why would it be correct?
Or phrase it in a very similar ask, why don't they invest in power plants? The model space is truly crowded, what do they gain or recover suppose they are SOTA? Across the Pacific they are pumping out free models that are only 6-12 months behind. What business sense does it make for Apple to develop their own models?
They don’t believe in this model and never had. You sound like the people who screamed that Apple was dying because they were not making a netbook style Mac in 2009.
Apple is the only big tech company with a non existent financial exposure to the current capex bubble.
Let big dogs bark at the moon. They are the loud ones, at least until the moon implodes.
I’m not seeing how it’s bad that a company is pushing in the direction of user hardware ownership. Of course it’s self-serving, that’s what companies do, but with most of the rest of the industry increasingly leaning in the direction of eliminating powerful general purpose computers in favor of thin dumb clients with useful compute being gated by subscriptions, it’s nice to see some dissent.
AI features not being constantly shoved in my face and just selectively silently integrated where it’s most useful is preferred to what the rest of the industry has been doing, too. I think most of us are pretty sick of AI getting tacked onto things that don’t need it and then given prominent promotion and UI positioning, potentially at the cost of features we actually use.
They could be doing more, sure, but directionally this all seems fine?
Apple has totally failed to deliver interesting AI experiences so far ... and I still think they're going to be the dominant provider of AI in 5 years. We're just one or two advances in chips / models / both away from being able to run very good local models for free on mid-tier Apple devices. The privacy, cost, and latency story there will be too much for OpenAI/Anthropic/Google to beat.
Just writing this down so I can be praised/mocked in 5 years.
I'm also of this opinion, but also that it doesn't have to be Apple (but they are well positioned). What I've seen with running local models on my 48GB M4 MBP is really impressive - it's not the same level as hosted stuff, but it's better than what I was using a year or two ago.
I suspect we'll see a hybrid before an all or nothing. Local models for computer control or delegating, online models for things that need strong reasoning, planning, and knowledge access. Again, I'd be more than happy to be wrong. I just see models growing faster than the hardware can.
Apple has failed to live up to the Steve Jobs era and initial iPhone hype. It's not just "AI experiences", it's computing in general. Maybe the consumer sector is just dead/dying. Maybe consumers are just running out of cash because filthy VCs are destroying communities and forcing the 99% into poverty.
The one thing that is marginally exciting: the Apple SoC or M series chips.
It's unfortunate they are locked behind crappy macOS and other proprietary apple crap.
> it's computing in general.
Unsurprising. Apple seriously thought the iPad would replace computers and usher in a "post-PC" word during their "What is a computer?" ad campaign era. Now they are sticking phone chips in laptop chassis.
For the general consumer, they were basically right though. Most people don't use laptops except for work. The primary computing device is the phone, and phones have basically become become mini-ipads in form factor since that ad aired.
> Apple seriously thought the iPad would replace computers
...for some users. See their "Mac is a truck" analogy.
And it has. My parents haven't owned a Windows or Mac machine in six years, since they got rid of the one I gifted to them a decade ago. Its all iPad and iPhone.
I wrote: "we should all be buying a fully loaded Mac Studio (128GB of ram, 20 CPU cores, a lot of GPU and Neural cores.)" April, 2023
We are both late and early.
https://news.ycombinator.com/item?id=35527692
You should not buy a fully loaded Mac Studio for AI unless you absolutely NEED macOS. You will be wasting so much electricity idling on prefill while your GPU pulls 150-250w from the wall.
Buy an Nvidia Spark, then whatever cheap Mac you want to use as a thin client. There's no reason to force Apple Silicon's round peg into a square hole like AI inference.
Benchmarking that I've seen shows that the M5 Max outperforms the DGX Spark, e.g. https://www.reddit.com/r/LocalLLaMA/comments/1tfzsd6/m5_vs_d... or https://www.reddit.com/r/LocalLLaMA/comments/1tr7hzw/psa/. Seems to me that Apple is doing pretty well with local AI inference.
Outperforms doing what? Inference is not a homogeneous workload, memory bandwidth correlates to decode speed and layer swapping but not necessarily inference speed overall.
The other half of that equation is latency, predicated on prefill performance which needs a powerful GPU and ideally ALU-level optimization to build larger KV caches quickly. Even the M5 gets smoked in this department, the M5 Max has a 50% longer TTFT on Qwen's 27b dense model at only 16k of context, which is a pretty typical starting context to use for agentic editing in normal apps like OpenCode/Claude Code: https://raw.githubusercontent.com/Osmantic/MMBT-Messy-Model-...
For agentic, 50-256k token on-device coding sessions, the Spark will be faster and consume less power running larger models. Without an external GPU (which Apple doesn't support), Apple Silicon will always be bottlenecked during prefill. Apple's failure to address this with their GPU architecture is a big reason why Apple Silicon viewed as a waste of time and money for professional datacenter deployment.
For agentic work, you just cache the prefill kv cache of the relevant system prompts. TTFT is a little slow (10s of seconds, oh no!) the first time you boot up a new harness.
I keep hearing people make this claim that TTFT is a problem, and… it just isn’t, if you’re running oMLX.
Folks in my camp keep saying this, and folks in your camp keep beating a drum we tell you isn’t resonating. Not sure why I keep bothering to argue; you can’t buy a high RAM Mac Studio like mine anymore.
It's really not that hard. If Apple Silicon was competitive for inference, then Apple would not be shelving them: https://9to5mac.com/2026/03/02/some-apple-ai-servers-are-rep...
They're not usable for deployment. They're perfectly fine for "enthusiast" low-end usage with 10-30B models, but the same goes for almost every dGPU made in the last 10 years. Your Mac Studio cannot run frontier LLMs at an interactive speed, even Apple has given up on using it as an inference backbone.
"Apple has totally failed to deliver interesting AI experiences so far ..."
You think this is a mistake...
https://www.crikey.com.au/2025/01/08/apple-new-artificial-in...
Of course. Do you think this was on purpose? All part of Apple's brilliant master plan?
I'm perfectly happy with Apple not becoming an "everything we do is AI-centric" business.
I'm fatigued by it all at this point. It's streamlining the interesting and fun parts out of my job (by practical necessity of use there), and if I used it half as much outside of work I'm sure it'd do the same there too.
>I'm fatigued by it all at this point.
This is the prevailing opinion of people even outside of tech.
I know public opinion polling supports that, but the parts of my social circle which are outside of tech seem to be, at worst, apathetic (and at best enthusiastic, though that's not a big fraction).
That said, I think it's a good thing that this sentiment is coming to the forefront.
I was actually surprised to hear my brother-in-law deride LLMs as being useless for areas he has expertise in when I visited for the 4th of July.
He was complaining that he would ask how to perform a certain repair on a car, and the LLMs he tried (ChatGPT & Grok) would give him a long involved process and he'd ask why not do it this simpler way and it would say, oh you're right! He just found it gave bad advice and realized (rightly) that in areas he has less expertise in he has no way to judge how good the outputs are.
This is from a guy who loves tech, historically worshipped Elon, loves his Tesla, and (rightfully again) didn't buy into SpaceX because he thought it was overvalued.
In the past when I visited for holidays he was liable to have a positive outlook on LLMs and their utility. Seems telling that he's starting to see the cracks.
Yep. In my areas of expertise I can easily catch it coming up with wrong information, bad calculations etc. So laypeople are probably being led astray quite often.
> it would say, oh you're right!
This is easily the biggest problem with the current models. The models are just way too eager to please / say yes to the point that the models are happy to lie/make shit up if it means it can say yes.
> It's streamlining the interesting and fun parts out of my job
Interesting. For me it's streamlining the tedious and attentionally taxing parts of my work tasks. I love solving problems, I don't particularly love shaving yaks.
Did you mean 5 months? :)
I don't mind the subtle ML integrations that they have put in the photos app: plant ID, recognizing faces, removing background, OCR text search (even for handwriting!), etc.
I don't like when I'm trying to copy a photo online and it copies some piece of text in the photo instead
You’re listing every feature I’d like to remove; Meanwhile Android users have great holiday pictures at the bottom of the Pyramids because they can remove the people from their pictures.
I'm not sure that's what this article is about.
Apple is doing something very different. Their AI experience for end users definitely has been a little behind.
Apple Silicon, however, has been quite unique for the last 4-6 years and it's increasing overlap with LLMS.
The model/chip optimizations are definitely improvements, the thing that is really standing out the past 2 years is how much the open source model community has been making possible, especially when you know a group of use cases.
Here’s the two main reasons why local inference won’t compete any time soon with the cloud:
1. Most useful LLM work is done in parallel. A Mac Mini can run one LLM inference thread at a time. The cloud can spool up dozens and spread that inference across efficiently batched operations over a fleet of hardware.
2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally. But the advantages of running >5x the token throughput per thread can’t be overstated. Add in the multi-threading advantage and it’s a knock-out punch for local LLMs.
Local inference has a role: if you’re working with extremely private matters or you want an uncapped model that will talk dirty or generate NSFW photos, local is the only option. I think Apple and others will continue to also run a lot of useful workloads locally such as text editing suggestions, speech to text, text to speech, and image manipulation. As local hardware improves, these capabilities will get better too.
But, for most LLM work, the cloud will continue to dominate for a long time to come, if not forever.
You can always buy multiple Macs. I think Apple's great differentiator could be making frontier class models local. I you want more threads, buy more Macs.
I don't want to run any workflows on someone else's computers.
As far as I know, there isn’t an interface like nvlink that allows these macs to work in tandem; they would just send their data over ethernet, maybe thunderbolt/usb-c?
Why yes you can! https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-stu...
> A Mac Mini can run one LLM inference thread at a time.
That’s not accurate. With MLX, at least, parallel inference is both possible and useful. Model serving tools like LM Studio and oMLX support parallel generation with continuous batching, and the total throughput increases with it.
You are not wrong, but the practical reality of local hardware is to be batch-constrained in comparison with a multi-user inference cloud. You will never be able to compete cost effectively in your home lab with a cloud that has >100M end users streaming millions of inference requests per second across a gigantic fleet of machines.
Can I run a few inferences in parallel on my Mac Mini? Yes. But put 1,000 Mac Minis in a datacenter serving 1,000 copies of myself? That's going to be more efficient.
Oh certainly, a local machine can't parallelize like the cloud can. Does anyone think that, though? What it can do is provide good-enough inference to support common business needs, without creating unnecessary data risks.
Why on earth would you need to do that for most things? Not everyone wants to be the next Coreweave. It sounds like you're killing a fly with a cannon to me.
I think local inference will be fast enough
There is so much happening in that scene, where tokens/sec double or 10x
So I could see the same hardware doing 20 tokens/sec on a large model suddenly doing 200 tokens/sec in the future, a better device in the future doing 500 tokens/sec, while having vision models baked in, audio models etc
Users wont consciously switch to local, they will just have it and use it
> 2. Faster inference hardware such as the chips from Cerebras and Groq cannot be run locally
Well just because they don't sell them. Doesn't mean that will be the case forever.
You lost it at when you said forever…
and a toyota camry can’t compete with a lamborghini but the camry vastly outsells them.
you’re far far far more likely to see a camry or equivalent in an americans driveway than you are a super car.
you’re also more likely to see an enthusiast with a corvette or equivalent muscle car spending way more than it’s worth to tinker on that car in their garage than you are a super car.
> Most useful LLM work is done in parallel
I guess what I'm doing is not considered that useful then? I usually only have zero, one, or occasionally two things actively doing inference at a time, be it claude code sessions or one of the chatgpt/claude web interfaces, and i bet that's true for like 95% of people using llms. And anyway i bet even the hardcore people using a bunch of parallel agents would appreciate having access to local, private inference for some things.
You're obviously right though that cloud inference isn't going away anytime soon
What about when claude code starts spawning sub-agents? it's pretty good about doing that now even if a user doesn't request it. Does this could as parallel inference?
I haven't seen it do it by itself that much so far. Even then, a lot of the time is spent waiting for some shell command to complete, which isn't inference. I guess being able to do parallel inference is like having multiple cores in a cpu, you can get by without it by just sharing that single core, and having it just gives you a performance boost under certain circumstances
Interesting. For me it seems to be much more common that even basic requests start spinning up sub-agents. But I also routinely ask it to do this so maybe it's stored this as a preference somewhere deep in its bowels.
And yes, I agree. I find the experience better less for speed and more for context management. But it's far from necessary.
There is common wisdom: during gold rush, sell shovels.
Apple's shovel (ahem, Mac mini) is the highest quality.with Companies burning money left, right and center, Apple can dispense with advertising altogether
Its a shame RAM prices are getting in the way of Apple. The 512 GB RAM Mac Studios would have been worthwhile.
Apple could probably sell a machine starting at $10,000 if they architected it as the sole place one’s Private Cloud Compute [1] ran.
It would need a path to a $2,500 machine, I think. But this is a niche I don’t think another consumer-facing brand could do like Apple.
[1] https://security.apple.com/blog/private-cloud-compute/
The out-of-stock $6000 M3 Ultra Mac Studios with 800GB+ memory bandwidth are going for $24,000 on eBay, so yes definitely
Let me clarify, I think Apple could sell a device at the scale Apple sells at around the $10 to 25 thousand price point.
Like, take out the price sheets for the Apple Car. Then sell me an AI tower at those price points.
In 2010 one of the standard configurations for the Mac Pro was $4,999. Once you customised ram, storage, peripherals and software it could easily end up above $15,000, or $23k today accounting for inflation. Apple hardware is one thing that has actually got cheaper over time.
https://www.macworld.com/article/209019/macguide2010.html
I’m still disappointed they didn’t make a Mac Pro with 4 or 8 or 16 or 32 ultra M chips for something insane
Or they could use that same amount of memory to ship 64x Macbook Neos, and probably make higher margins off the hardware volume.
Those Macbook Neo users would be very reliant on Apple intelligence, enough maybe to pay for a service with it. I think Apple's much happier going this path.
> Or they could use that same amount of memory to ship 64x Macbook Neos, and probably make higher margins off the hardware volume
If it's an "or," absolutely. But if it's an or, they should be prioritising Macbooks over the Mac Mini Doug Brooks is discussing.
When we breach the "and" of memory supply sufficient to allow for more Mac minis (and Mac Studios), I think it would make sense to consider relaunching Xserve (with new branding, of course) as a consumer/small business product.
If we reach the and, then they can no longer demand the price
Memory supply isn't what held back XServe. We wouldn't need XServe if Apple treated the Mac like a regular computer and supported usable, first-class headless workflows and eGPUs.
The writing has been on the wall since 2019. Apple doesn't like the old way of computing, their goal is to expand the ecosystem by prioritizing install-base and then pushing first-party service offerings like they did with the iPhone. And like they did with the iPhone, Apple is great at ignoring power users to focus on features that make them more money.
You may be waiting a few decades for this type of product, memory supply be damned.
When I was at Apple we never wanted or. We wanted all. If that push to use Chinese memory works out it will be great for us and Apple.
Apple sure doesn't act like it. The Mac is still a minority market share of PCs, and their entrants into spaces like AR do nothing to compete with incumbents.
Now that the Mac Pro is depreciated, Apple's plan to pivot to service offerings seems set in stone. That's the "want it all" attitude they've adopted with the App Store.
They could do both though. The margin from one user buying a $25000 is sky high compared to sixty kids all with the cheapest computer possible.
It's really not. Apple's phone margins have been as high as 30-40% per-unit, it's likely that they make at least ~$80-150 per Macbook Neo sold.
At the $150 mark (which is probably accurate factoring in lifetime service spend), that's a $10,000 minimum return on the 64x Macbook Neos. Apple can charge that type of premium on consumer hardware, but they're in no position to command $10,000 margins on professional hardware. They're not Nvidia, Apple has always been LARPing as an HPC vendor.
Maybe margin was the wrong way to phrase it, but I know which customer I’d choose to have.
But surely you understand how your preferred customer is the less profitable one?
Apple won't subsidize these low-margin enthusiast products with the profits made from services and higher-margin hardware. Tim Cook would much rather ship the 64 Macs, and get ~15-20 school-age kids hooked on Apple One or the App Store for the rest of their life. There's understandably not much patience for catering to people that want to opt-out of the Apple Intelligence service ecosystem, effectively leeching off of more successful products. The volume and opportunity cost kills the concept in the cradle.
I think so, too, and I think it'll end up being a race between Apple & NVIDIA (or NVIDIA partners) to see who realizes this first. It would probably be easier for Apple to do it because it wouldn't require a form factor adjustment [over the Mac Studio they already have]. That said, NVIDIA already offers chipsets for both the lower end (DGX Spark with Vera + GB10, at roughly the $4500 price point) and higher end (DGX Station with Vera + GB300, for $85-100k). The DGX Station is equivalent to ~5-6 RTX6000 GPUs attached to a mid-range CPU server, but far more than most individual developers would want or need. I've heard through the grapevine that NVIDIA's received consistent feedback that they need something like a "GB20" that slots above the Spark/GB10 and can simultaneously run larger models for inference while hosting a dev environment on the same box. You can daisy-chain Sparks just like you can daisy-chain Mac Minis, but you're still constrained on model performance based on what a single device can accommodate.
Form factor is the easy part - both Nvidia and Apple are experienced SOC designers.
The hard part is the GPU architecture. Apple Silicon was designed with a laser focus on raster efficiency (similar to AMD's GPUs) which makes a lot of sense for highly mobile hardware, but is a crippling mistake for high-performance compute. Apple's largest Ultra chips are hamstrung with SOC-tier GPU performance, their highest-end desktops are outperformed by Nvidia's laptop offerings. Apple has to find a way to scale upwards without imposing too much architectural strain on their cheaper hardware like the iPhone and Macbook. Nvidia has already solved this issue; full CUDA compute stacks are usable on extremely cheap GPUs like the Nintendo Switch's Tegra SOC, or the Mac Mini-sized Jetson boards.
In terms of "who needs to redesign more to address the market", Apple has a lot of technical debt to unearth before they catch up to Nvidia. And if they do catch up, Nvidia will still support Linux and other differentiating features that Apple refuses to implement. It definitely feels like Nvidia is closer to a winner with the Spark than Apple is with the Mini or Studio.
I don’t think we should use current prices as landmarks for large scale demand. That Studio’s current prices is inflated because of a (presumably) short term supply crunch, not because the average user is willing to pay $24k for a home AI inference device.
It assumes that RAM remains supply constrained and that none of the existing RAM contracts are cut short.
But Meta and xAI putting A TON of AI compute onto the market. OpenAI and Anthropic are raising the costs of inference (by reducing how much inference users get via subscriptions). And we haven’t seen Oracle / CoreWeave struggle to pay their debts yet, but they will be selling assets once they get close to that point.
What makes you think the supply crunch is short-term?
Unless the raw materials have an inherent limit on mining/production due to the amount present on the planet, why should or would companies not ramp up to eventually meet demand?
Edit: Okay, this doesn’t mean that that’s actually possible in the short-term, so I think you’re right. But that means as the silver lining, in the medium term horizon there’ll be enough supply again? :’)
> why should or would companies not ramp up to eventually meet demand?
Memory is a cyclical market that has historically rewarded conservatism [1].
Counterpoint: there is enough demand from enough capital-rich customers that they may be willing to shoulder the capital risk.
[1] https://www.ldeepai.com/tech-hub/dram-industry-consolidation... Sorry for the slop link, it has a good chart from a solid source
Thanks for the link (and underlying thoughts), I really hadn’t considered that.
So essentially, due to technological progress and other factors inducing price collapses (or at least cycles), you can’t start stockpiling insane amounts of finished-product semiconductor, which means you can’t scale production at current technology levels to infinity either?
I heard that China was spinning up DDR5 (but not HBM?) production in the next couple of years, with the hope of outcompeting Korea and Taiwan in the mid to long term.
It does seem like an opportunity on a silver platter for Chinese newcomers. Huge demand at the moment.
In two to four years, the Chinese will have at least half the memory market worldwide, and once in, they will continue forward and not look back.
I also believe there would be one or two tech companies that will get into memory by taking it in-house to make sure that they won’t have this problem again in the future.
Don't forget price fixing [1] which we are seeing clear indicators of happening right now too.
[1] https://en.wikipedia.org/wiki/DRAM_price_fixing_scandal
For existing producers expanding capacity would be a risky move. But it's the perfect time for any newcomers to enter the market. Low yields and worse product don't matter as much right now, and by the time the market cools down you have everything dialed in and can compete on even ground
> it's the perfect time for any newcomers to enter the market
This is a good hypothesis. Curious if anyone has data on the failure rates of new entrants in semiconductors based on how frothy it was on founding.
On one hand, more demand makes selling easier. On the other hand, a shortage makes your input costs (consumable and capital) pricier.
EDIT: It seems like the 2 to 3 year lead time and a crowding effect from new entrants historically made booting up a fab into a boom a bad bet [1]. (The article argues, convincingly, that this time may be different.)
[1] https://www.uncoveralpha.com/p/every-memory-cycle-ends-the-s...
If demand doesn't fall down or current manufacturers supply go up, somebody (presumably in China) will spin up fabs. Apple wanted to use blacklisted Chinese RAM already.
I am hopeful but I am not confident China has the capability to do it.
They're already doing it
https://www.techspot.com/news/112502-memory-prices-tipped-fa...
As if it requires some unique genes or brains?
If a place can do it, another place, with a huge track record on manufacturing and lately expanding all kinds of tech, can.
It might help to not have much in the way of environmental, safety or labor regulations.
Whether or not you feel like those are good overall (I do), they do actually also slow things down.
>It might help to not have much in the way of environmental, safety or labor regulations.
Yes, like how it helped western industry early on. Or, well into the 70s for the most part.
where is this cheap Chinese RAM? I'd like to buy some
Ebay and Amazon are flooded with it. Especially if you are looking for anything prior to DDR5. DDR2 and DDR3 are especially flooded with weird brands you've never heard of before.
Unfortunately its not so cheap anymore as everyone ramped prices up of course.
Last year I could still get 32GB of DDR4 for under $60 from chinese brands.
DDR3 is from 10 years ago. DDR2 is from 15.
And?
I just upgraded my 2008 Thinkpad R61i to 8GB of DDR2 a few months ago while I was also upgrading to a core2duo.
DDR2 and DDR3 are still in active use by SBC manufacturers.
> I just upgraded my 2008 Thinkpad R61i to 8GB of DDR2 a few months ago while I was also upgrading to a core2duo
Uhhh… ok… good for you I guess…
It's the euv machines that are the bottleneck. Pretty hard to ramp those up any faster in the next few years.
DDR5 is still mostly made with DUV (remember Intel 14+++++++++?), and even though manufacturers have slowly been moving a few layers to EUV the advantage is at the margin. Lack of EUV at scale will not prevent China from ramping useful RAM into this market.
spinning up fabs takes ages, micron has a US fab started bulding in 2023, its still not operational (projected to start mid-2027)
That proves that spinning up US fabs takes ages.
Chinese fabs might not be so tied with red tape and regulation upon regulation (which is a funny reversal, in terms of "communism vs capitalism" bureucracy/inefficiency cold war thinking)
The Chinese don't have access to new EUV machines.
All of their fabrication ability is based on old processes.
Not so sure about that: https://www.reuters.com/world/china/how-china-built-its-manh...
DDR5 is DUV not EUV
1.) China is not communist, even remotely so. China is fascist in every sense of the word.
2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.
3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."
>1.) China is not communist, even remotely so. China is fascist in every sense of the word.
Except in the actual historical sense. They appear to enjoy all sorts of freedoms, increased prosperity, even have elections at different levels but under a single party system. Which is not necessarily that different than a effectively two party system.
>2.) Authoritarianism can move faster than anything. They can just say "wipe out that village, build the coal plant there, data center here, fab here.
Now that China is more effective, "it's easy because they're authoritarian". Before the argument was "authoritarianism can never be as effective as free-market democracy".
>3.) If it's red tape and regulation holding the US back, then that's clearly not "capitalism."
It's real world capitalism, not some fantasy some guy imagined removing all warts.
I think the most ironic fact of the 21st century is that there are less than 20,000 naturalized citizens in China. Western leftists don't really have a good explanation for that one and it definitely leans into the fascist characterization.
If the increased demand is not short term, production capacity will eventually increase. In the meantime, the logistics disruptions and industrial material shortages and energy inflation will disappear as soon as the wars disrupting them stop, which should bring prices down.
you assume demand will remain flat
what if demand keeps rising faster than production capacity is deployed?
If demand and prices keep rising without production capacity being built fast enough, there will likely eventually be a rush leading to overinvestment and price crashes, but there are too many other factors involved; state investment for security, international politics and trade relations, the possibility of an AI bubble burst, etc.
There are wars coming. The prices are not going down.
We are in a bubble which will be burst the moment the world starts retaliating against the US' 20+ year history of supporting genocide and committing war crimes unabated.
Buy the AI toys while you still can.
DRAM doesn't require cutting edge Fabs, and Chinese manufacturers have finally caught up to the DDR5 standard.
> Multiple motherboard and PC component makers move forward with Chinese-made memory validation
https://www.pcgamer.com/hardware/memory/multiple-motherboard...
The only thing standing in the way of a major Chinese DDR5 ramp up is money.
The underlying demand for massive RAM builds is coming from AI hyperscalers.
There are lots of signals that the sector has been overinvested and that corporate customers are pulling back on spending as the cost of the APIs is revealed.
Once the hyperscalers start struggling to bay their debts (it will happen, just a question of time), there will be a supply glut.
So the only question is: do we share the same definition of “short term”.
512GB M3 Ultra is out of stock, not coming back, and there’s nothing like it on the consumer market. That’s the reason they go for so much.
But there will be and his point is they cut off supply to make room for the new M5 ultra which I hope has 768GB or more of memory.
Let’s see, I hope!
By the way, they cut the supply because they run out of RAM. They still sell the 96 GB model. Everything else is gone, no stock, they cannot manufacture them. Even minis are 48GB max!
Yea but how many of them are selling at that price? Dozens?
Funnily the current high end Mac Studio are not suited for current LLMs. M3 Ultra is "quite an old" chip for AI, despite its bandwidth. The issue for running local models (especially LLMs), you need few things to align really well: 1) compute power (affecting PP) 2) VRAM capacity (affecting model size you can load) 3) Bandwidth (somewhat affecting decoding speed).
The issue with the M3 chip is the compute performance, as it doesn't fit well the transformer architecture. This changed with the M5 (apple baked their own matmul into the chip), which would significantly speed up PP (and video/image generation btw), making the M5 Ultra significantly faster than the M3 Ultra and in practice much more usable. You can try to load Kimi or GLM on M3 Ultra, but it's not usable. Now the M5 Ultra is not out yet, but undoubtedly it will be a superior offering, and shilling 15k on 512GB version is actually reasonable (if it's every priced remotely around that tag).
Do you honestly think that’s families buying them?
My money is on small companies or affluent programmers experimenting with some new hobby / business model.
Not the same issue, but makes me nostalgic for these simpler times:
https://www.wired.com/2011/04/amazon-flies-24-million/
I’m sure that hasn’t gone unnoticed.
How much did Apple sell them for?
Yeh I don't know why the emphasise the 'neural engine' - it's their stonking RAM bandwidth that just blows everything else out of the water.
100%
But that would be more for enterprise usage. I cannot imagine a personal computing usage which can justify a 10k machine.
> cannot imagine a personal computing usage which can justify a 10k machine
For me, the privacy pitch wins. I have a friend visiting, however, who spends like $2,400 with Anthropic every year. That's a solid ROI even if the thing becomes obsolete after a couple years. (I'm still on my 2020 MacBook Pro. I love it and will be sad when I have to replace it.)
> That's a solid ROI even if the thing becomes obsolete after a couple years.
How can that be a solid return on investment? There's no model you can run locally to have frontier model level performance. Also who spends 2.4k yearly for personal AI usage, like what's the usecase? If your friend is spending that money for his business then it's not personal computing.
> There's no model you can run locally to have frontier model level performance
I'm betting he doesn't need a frontier model. Sonnet, today, is likely good for 80% of his tasks, which largely involve repretitive, tedious work.
> Also who spends 2.4k yearly for personal AI usage, like what's the usecase? If your friend is spending that money for his business then it's not personal computing
Combination of business and personal.
That's the $200/mo subscription -- not uncommon :)
I do the $100/mo for myself, then about ~$200/mo for startup.
Do you use it for work? I can’t imagine paying this myself for personal use. I just don’t see the benefits vs. the free models available to me.
People with money and things to do. People spend $100+ a month on coffee without even blinking.
I have a friend who is a teacher and spends that much yearly for ChatGPT and Claude. It saves him many hours a year in time assembling documents and creating posters, flyers, and personalized learning materials for his students.
In addition to privacy I’d like to be able to burn as many tokens as the hardware will let me 24/7 without getting a surprise bill at the end of the month. I don’t care if it is slower that the cloud, I’m not in a hurry.
Apple is a premium brand with high brand loyalty. Do you not think even 1 billionaire would want something like that? Even to just say that they bought it. Apple could sell things at a price point much more than $10k.
There's only so many billionaires and at Apple's scale you would not offer such a product for public sale even if you do custom builds for the rich and famous.
Apple makes product lines with assembly lines, its not a hand fab or custom build type of place.
At the end of the day, it’s just an Apple computer, not a Ferrari or an Aston Martin. I hardly think an Apple computer can be considered as a luxury item, unless they release it as a limited edition
Apple does like to market their hardware for education and small business customers, though.
I think "buy this $10,000 box and to easily grant every Macbook Neo on your team safe, private, free AI" could be a real winner.
I just went to Apple.com and specced out the top of the line 14" Macbook Pro, and the price is $9,849.00. Well over 10K with California sales tax.
I could absolutely justify having that machine for the actual work I do, and I'm not even doing any of the really hard AI things. If I were, I might prefer a Thelio Mega workstation from System76, which is $90,383.00 fully loaded.
Alas I can not afford a 10K personal computer right now, which is not the same thing.
> I cannot imagine a personal computing usage which can justify a 10k machine.
If Apple released a machine that would let me run, say, DeepSeek V4 Pro or GLM 5.2 locally at 100 tok/s for $10k, I might hurt myself running to get my credit card.
But then, I'm also posting this sitting in my truck while my family attends an event, with a Vision Pro on my face so I can monitor 7 Claude Code sessions without constantly switching screens on my laptop.
$10,000 divided by five divided by 12 is only $166.66 a month without interest over 5 years a Mac M2 Ultra or Mac M3 Ultra are easily usable computers over five years, so are most of the medium to high-end laptops that Apple sells for one person or locally in a small company.
It’s not necessarily out of reach or unusable over the course of time, the thing I keep hearing over and over is how usable many people find Mac’s very useable over the course of time, obviously software support plays a big part of that.
On top of that, AI models are getting more useful despite their smaller size. The future is a personal computer future, not a mainframe computer one. Yes, larger computers are useful, but most people will not be using that larger computers, which will be relegated to universities and larger companies.
Very few brands have such a distortion field.
See https://en.wikipedia.org/wiki/The_Cult_of_Mac
Apples on stage use cases for their hardware and software makes me wonder if they actually use computers over there, or what a "job" at apple entails.
I am unsure that apple themselves understand why their hardware (top end & bottom end) has been so successful, without this understanding leaning into these use cases isn't really going to be possible.
> I am unsure that apple themselves understand why their hardware ... has been so successful [like I do]
You have a bold career as a technical journalist ahead of you!
Obviously they are playing 12d chess. They stopped selling high memory machines, they stopped selling pro machines. They are the king of local Ai compute, definitely not stumbling backwards into a product category they didn't know existed.
With their apple finger right there on the pulse, they are going hard on the VR/AR glasses (following the lead of the visionary CEO of facebook), cars and folding phones. By the end of the year (tm) we 100% will have all the features that were showcased and demonstrated 2 releases ago.
Having worked at Apple for over a couple of decades, I can attest there are some very (very) capable and intelligent people there.
I trust they know more about their business model than some rando on the internet, sorry.
Those of us that are less capable and less intelligent must just be holding it wrong.
you don’t need to be less intelligent or less capable, but you will necessarily be less informed.
How you hold it is, of course, up to you.
How many of these 'consumer' customers would they get for $10,000 and how would they reach their Apple-typical gigantic margin with that?
Apple is quite hostile to professional users, so no.
Sadly, that is an outdated PoV. It has probably not been valid, since last century.
It's just that Apple isn't really focused on software development professionals, and it's still fashionable to throw shade on them, so we hear a lot of kvetching about it, in communities like this.
Could you please point me out which current Mac model has enough storage for full feature video editing job? Thanks.
I dunno. It's not my bailiwick. I do know lots of pro editors use Macs, but I think they use DaVinci Resolve (not Final Cut Pro). I'm interested in finding out what you use for it.
I've been using Macs for all kinds of stuff, since 1986, so I can definitely state they get work done.
I'm being sarcastic here.
But I still strongly believe that Apple hates pro users because they don't make as much money and because they get in the way of serving laymen. The Aperture fiasco, the Final Cut Saga, the Xcode war of attrition and the never ending chain of failures with MacPro - all suggest that I'm right.
(Developer of a major plugin for DaVinci here)
Good on ya. I'm not interested in fighting about this stuff. I've had people hating on me for using Apple since the 1980s. It gets a bit old. Sort of like high school popularity contests.
I'm an exclusive Apple user for a couple decades so I'm not fighting here :) just acknowledging the fact that Apple has some deficiencies
Fair ‘nuff.
As an Apple developer, since then, I have been incandescent with rage at Apple, many times.
Guess I’m a walking demonstration of Stockholm Syndrome (at least, that’s what I’ve been told).
All companies have deficiencies in the tech world. However, I would rather be in Apple’s position today, in comparison to Intel or Microsoft, particularly over the last 20 years.
And I definitely feel the pain with Aperture, not making it to Apple Silicon.
Video editors use high-speed network storage.
How much internal storage do you use for your full feature video editing, vs attached/network attached storage?
I’m interested to know the ways that Apple doesn’t cater to Software devs the way other OS does. I hate developing on a windows/non-unix machine
I feel that it does, but I’m also a dev. I used to run a multi platform shop for years, and have a pretty good idea of what kind of support various companies give.
That’s just excuses for example how does a relatively small company like Black Magic Design With one of the best programs in its class manage to support three operating systems MacOS, Windows OS and Linux OS simultaneously?
There are many larger software companies that can support all three operating systems, they just make up excuses/reasons as to why they can’t perform over the years.
Ages ago, back when the Macs would come out, my co-workers and I would take a bit of time to configure the most expensive possible configuration --- time was, it was pretty easy to hit six figures, but over time, that has gradually come down.
Who would be the market for such a thing?
People who are good at convincing their boss.
Bring back xServe!
I asked an Apple (via a sales rep who visited our company to showcase in internal iPad healthcare app) to please do this for iCloud when iCloud Drive was in-development. We would have easily paid $50,000 for a rack-able Mac Pro you could point "managed" devices at.
Apple simply cannot comprehend the ask.
They had Xserve, it didn't sell well enough to survive.
How do you design XServe being tied to IBM/Intel the way you want to make it, and why would you, when you’re having problems with Intel, why? would you go any further with Intel hardware wise? Makes no sense. The next four years will be interesting with an engineer CEO in charge.
> Apple simply cannot comprehend the ask.
Apple knows the market demand for this type of device.
You may have paid $50,000 for it, but you’re only one customer. At Apple scale they need to focus their finite resources on the products that serve the largest market demand.
$50,000 rack mount servers are not a large demand.
You'd be surprised. This was the same conversation where Numbers was pitched as an Excel replacement.
Apple just doesn't do "enterprise" well - they need to poach some Microsoft employees to figure it out.
Apple has plenty of enterprise-specific features.
From a historic standpoint though Apple came back from near-death because they differentiated by focusing on the consumer first.
While other companies were recycling the same beige boxes meant to be tucked under desks for home use, Apple came out with products in candy colors.
While Microsoft was rolling out new business process tool SKUs, Apple came out with GarageBand and bundled it in for free.
Apple is not the company prioritizing going after a Fortune 500 company to replace their fleet with Macs. So their focus isn't going to be to design products and features to try to get that deal closed.
Our company would buy at least 10 of those instantly.
If I had the capital I’d make an household inference appliance.
No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.
Even the appliance faceplate would be a display showing stats like an old HiFi stereo.
Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory
You're describing the mac mini/studio with some facelift.
Yeah but like running linux hopefully
so you have invent unified memory for linux first because that’s the limitation today
Fairly sure most iGPUs these days are zero-copy and can dynamically allocate memory so what does "unified memory" mean to you exactly? A wider bus would be nice but it's not exactly a groundbreaking new invention.
I was actually pretty far off:
> Unified memory in Linux creates a single address space accessible to both the CPU and GPU, eliminating the need to manually copy data between system RAM and video memory. It is enabled via NVIDIA's CUDA, AMD's ROCm/HIP, or generic kernel-level Heterogeneous Memory Management (HMM).
So it does exist and is available for platforms that matter.
It is interesting how apple claimed that "unified memory" is something special, and ppl believed them.
Intel and AMD had been doing this for years already, and had linux support for it from day 1.
Cool. Apple was the only one who managed to ship a consumer device with UMA and RDMA support. 2TB VRAM max over RDMA.
I think the REALLY cool thing about apple's shared memory implementation is the ultra-wide memory bus.
Otherwise, AMD is quite close to what Apple has, and Strix Halo is honestly incredible.
Not sure what RDMA brings to the table.
RDMA increases the inference performance by a significant percentage across devices connected via Thunderbolt 5.4x512 is like a 2TB machine.
Raspberry Pi and other SBCs, Android phones and practically all of the embedded devices with a display and microprocessor.
All have unified memory. Linux runs just fine on all of those.
Dude, https://news.ycombinator.com/item?id=48858095
Ah ok. I replied to ~45 minutes stale page.
Or just buy a MI300A server like https://www.servethehome.com/gigabyte-g383-r80-aap1-amd-inst...
"Just". And then GPUs, and RAM? And cooling? Will you really appreciate it when sitting right next to it?
Absolutely, but not under the control of Apple.
build a Xeon / epyc 4u server. 12 channel ram.
Is that the nvidia spark?
Yes, and a lot of others.
A bit too expensive for a home appliance though, isn't it?
Isn't that what what George Hotz is doing over at tiny? https://tinycorp.myshopify.com/
Yes, but for inference. 45k is so far out of the budget of a professional unless you earn ridiculous money and have no dependents.
A professional AI engineer? Earning hundreds of thoundands of dollars a year?
Who lives on one of the coasts where that job likely requires them to be, where rent is $3-5k and mortgages within spitting distance of that, sure.
Yes, just a big cool Cerebras wafer for the closet please.
A single wafer comes with 44GB RAM, the reason why Cerebras is so interesting is because the architecture scales up to 1.6PB RAM.
Central heating / thinking.
Sounds like reinventing the home server.
the pheriphels support, or the appliance faceplate is tens of dollars, that not where you make the saving
95% of the price is going to be in GPU+CPU+RAM
I think the closest to that in existence is the LLM ASIC designed by Taalas:
https://taalas.com/products/
Unfortunately their chatbot, while amazingly fast, doesn't know anything about the company running it.
Anyway I wouldn't mind an ASIC running a diffusion language model locally. Even if eventually it would become dated. Beats outsourcing all that to a company that's running on VC money which in the future might either perish or worse - dominate the market and charge whatever they wish.
Here you go: https://www.truffle.net/
"Login to order"
That's a new one.
I lasted about 25 seconds on that site. Way too much friction for me to endure just trying to figure out what it is
Yeah, I don't know who thought that website was a good idea.
I feel like this is some sort of satire? There's no actual information or substance to anything on any page of that site.
It could heat your home in the winter and your pool in the summer.
Is warming a pool in the summer real where you live?
Yes. Solar thermal heaters on the roof are common in Florida and other parts of the south. Some people also use heat recovery devices attached to the AC condenser. Further north I've only seen natural gas heating (e.g. in very rich NYC exurbs). The amount of shade over the pool has a big effect.
I'm keeping an eye on Tenstorrent for this. Pricing seems like its going to end up being in between a super memory dense unified memory platform, and a purpose built GPU.
Definitely on the edge of what would make sense at home, but its interesting.
What I’m not sure to understand is that if you want to just run Claude code or openclaw type software with llm apis or subscriptions (and not run local models) to benefit from a local file system and always-on capability for ‘second brain’ type of workflows, I guess you don’t need a Mac mini but can run it on a raspberry pi or an old laptop ? Does anyone have experience with that ?
Yup, for openclaw and APIs you dont need a big PC. I run something lightweight on the RPI4 8gb. Many people run local LLMs which is where a mac is useful. Frankly I dont think you can beat the value of an openrouter subscription and API calls.
> Many people run local LLMs which is where a mac is useful
Unless you go for the very expensive options, most of the Mac Minis really aren't suitable for running local LLMs, they're painfully slow with prefill/processing input, and the models you are able to run don't handle long context very well, which these sort of long-running agents perform very differently with when you can.
I'll agree with your latter point, hard to beat the value of using something like OpenRouter or similar remote inference.
Even with local models, you can run the agent software and the inference workload on different hosts, which is what I'm doing at home. Beefy server responsible for inference, tiny VM on other server is running the actual agent software + RPC + bridges and what not.
Why not go direct to the source instead of paying an extra 5.5%? Seems like it'd be trivial to have AI wire up connections to your preferred inference providers and save yourself some money over time.
If you're referring to the markup charged by OpenRouter, you can use harnesses like OpenClaw/Hermes without it and go direct like you're saying. If you're talking about actually "routing", then you don't get that out of the box. However, the popular use of those harnesses doesn't often use the smart routing approach with a single agent. Instead, the approach is to create multiple agents, each with a role and a model tailored to that role by cost and functionality.
If you or anyone else don’t mind, I have a have a question or 2.
I use Claude Pro ($20/m) as a glorified search engine (no ads/SEO) plus simple hobbyist dev things (shell scripts, managing my Mac, apps etc.
I also use it for tasks like - “search the web for top ten selling EVs, put them in a table” and then iterate - pivot tables, charts, additional research”. It could be cars, it could be broccoli. Code Work has facilities to streamline this type of work, but I usually drop into the CLI.
How much if any functionality would I need to recreate if I switch to OpenRouter and would be match my costs with the API approach. I don’t want any cost overruns. With Codex or Claude, if I run of tokens, no big deal, I can wait.
Thanks!
You're not getting it on OpenRouter for anywhere near that cheap.
Pretty much. Running a local harness calling an llm via APIs doesn't necessarily take a lot of resources. But whatever tasks you want that agent to do via tool calling will be limited by the resources of the machine it runs on if you run those locally, so that's what should inform your choice of specs in this case
...or any cheap VPS? I now do most of "second brain" things via pi harness with Opencode Go subscription, and it costs me like 20 bucks a year, with added benefit of "you can have tmux and open session realtime on whatever device".
it's only because you can send texts via imessage on a macmini. that's it.
Buying a new computer at +$500 just to have iMessage access feels insane to me.
OpenClaw supports all the mainstream (and free) chat apps like Discord, WhatsApp, Signal, Telegram... None of them requiring a MacOS machine.
Is it a lack of knowledge from the users or do they really value iMessage integration that much?
What I don't get is where is the Mac Mini Neo at like $350? Neos tiny motherboard in a box with some more ports would be awesome.
A Pi running macOS more or less. Not dissing it though. Killer machine for those who don’t need a lot of power locally. Also a great kiosk for some things.
With how apple seemed to be caught by surprise when it came to Macbook Neo demand, I'm not sure they have the quantities of SoC's around to handle the demand a Mini Neo could drive. Especially if they could do it for $299.
I will go out on a limb and say that's not going to be an Apple product, period. It doesn't fit anywhere in the value envelope.
The relevant questions here are: will the person using this machine also conceivably be wearing a pair of $549 AirPod Max? Or a $399 base Apple Watch? Does that person expect to pay more or less for their largest-screen computing device than their headphones?
Framing that way points toward a $350 price point being a laptop for young children (younger than Apple Watch age, so lower elementary). That's a whole different software experience beyond just the hardware.
Apart from the ports, that’s roughly the AppleTV hardware. A macOS or Linux port to that would be a cute thin client. Not gonna happen, but cute.
And configuring a bot for Telegram is incredibly easy.
It's anecdotal but the kind of people I know that bought Mac Minis for this purpose are what I'd call "light techies." They definitely know how to use an iPhone or a Mac but would struggle on the CLI of a Linux box.
Anyone who wanted the OpenClaw use case that is comfortable with Linux probably already has several Linux machines (including a few Raspberry Pis) on-hand.
> Is it a lack of knowledge from the users or do they really value iMessage integration that much?
My understanding is that the barrier to entry to using iMessage makes iMessage a LOT more secure from spam. If you want to do mass iMessages you have to register as a business with Apple, go through all sorts of checks and attestations, etc.
At any rate, iMessages are a lot more trustworthy than SMS. So being able to spam people via iMessage is very desirable. I recall a few months ago a guy posting his little spam-iMessage-as-a-Service product here on HN. You could build your little iMessage spam army using a bunch of Mac Minis...
Not even. It's literally "yeah bro, you gotta get a Mac Mini".
I do exactly this. You can run the whole thing on a PI. I have actually installed asahi Linux on my Mac and I connect to it remotely so you can be sure I will never upgrade my Mac again because it’s already overbuilt.
Correct. You do benefit from some headroom for things like launching browsers etc but refurbs or mini PCs (with at least 16gb; ideally 32gb of memory) from the likes of Minisforum or GMKTec work well enough if you're wanting to spend a little bit of money.
Yeah I find the Mac mini trend is kind of baffling.
It seems like it's driven either by 1) people hearing Macs are good for AI, buying one, and using Claude for inference, not realizing that you interact with the anthropic API from an internet connected hair dryer. Or 2) people want their agents to have blue bubbles.
I find it hard to believe that enough normal people are doing on device inference is driving Mac Mini's out of stock. And even if they were the Mac mini is not actually a very good platform for it.
I assume people ask an LLM what to use to run locally, and it suggests a Mac mini
One aspect you're missing is that people running a claw type agent thing need to run it on a Mac to automate software in the Apple ecosystem.
Neo-Siri in iOS 27 removes the need for a lot of this, but before then, if you want to ask a robot about information that is stored in Apple notes, or to send an iMessage, a Mac mini is your only practical option.
What does the new Siri do that circumvents the need for a mac? To access Siri you still need one no?
the new Siri does the apple-ecosystem tasks you would have needed a claw type agent for natively
The demand is not coming from 'normal Apple customers' it's coming from people who want a machine that can run local AI.
It has nothing to do with Macs being especially good at AI. It has everything to do with being one of the last 'cheap' devices being sold with that much unified RAM.
I believe it was because it allowed OpenClaw to access iMessage directly
There are two angles to this. One is that if you want to integrate your agent setup into the Apple ecosystem you need a network connected Mac running 24/7.
The second is that the puck is heading towards local models. The people running their own 'Claws are usually experimenting running their own services either to save money or to explore the future where 95% of requests are handled on device.
The people I know getting Mac Minis to run agents (I include myself) are running local LLMs.
You don’t need a Mac Mini just for that, but they’re fairly inexpensive (or were anyway) and quality is very good. The people who buy them may never use all the performance available, but they’re more interested in convenience than getting the cheapest thing possible.
You can sort of justify it by assuming it will last a long time and they’ll use it for other things, too.
Running models on-device on a Mac is immensely annoying though. Figuring out what will work out of BF16, FP8, BF16+FP8, NVFP4, INT8, GGUF ... the list goes on ... is 'non-obvious' at best. Apple do little to support with tooling. There's MLX, but unless you're happy to transform a model to that format yourself you'll be lagging a long way behind.
Apps like LMStudio, Ollama, Draw Things, etc do a great job of simplifying it but it's still a pain.
I dunno I use LMStudio pretty regularly and the MLX folks and the community usually have MLX versions of new model releases up within a day or two.
For some models like the popular coding and chat models, things move faster. For things like images, voice, sound etc they definitely lag a long way behind.
& they clearly delineate all the models that’ll work on your exact machine (but guardrails can be disabled in settings)
Converting a model to mlx is literally a git clone and mlx_lm.convert. 5 minute operation after clone.
Or just ask any model to do it for you.
How is it a pain exactly? It’s just learning and only takes a day or two to get up to speed. We seem to have forgotten that for the past fifty years doing all kinds of tasks on computers has been tedious and involved and time consuming to even get working. My first computer had 48kb of RAM and to play a game you had to load it off cassette for five minutes. That was annoying. Having LM Studio download a model and load so you can chat or attach an agent it is effortless and easy in comparison.
Not to mention you can just ask Codex or Claude Code to set up a local LLM for you. Use one tool to set up the other.
Counterpoint, I'm just a happy user of this local LLM [1] from antirez (of Redis fame).
[1] https://github.com/antirez/ds4
In general I just don't like not using a cross platform architecture of some kind.
MLX is fine, but the cross platform alternatives (ONNX) are terrible, GGUF often lacks and doesn't have the "easy convert" that some of the commenters below say MLX has.
What problems are you running into? Ive been running multiple qwen coder next agents locally and it works fine.
It’s not for the AI inference, it’s for the tool calls and desktop GUI app workloads and browser. There aren’t any on-device models capable enough of real work that can run on lower end Mac Minis. But for running a few browsers and GUI apps, you’re much better off buying a Mac Mini than paying for a more expensive and worse-performing container in the cloud. Browsers were not designed to run in Linux containers but they run optimally on baremetal desktop OSes. An M4 Mac Mini beats the single core performance of any VM you might rent in the cloud, in terms of raw compute per dollar (Geekbench scores).
At their original price points, a set of four were a great solution for my requirements in tokens/second/$.
> Apple's Mac mini and Mac Studio have become the machines of choice for running AI agents, according to Doug Brooks, Apple's senior product manager of Apple silicon.
This is mostly an US phenomenon, no Mac mini nor Mac Studio around here.
Only Thinkpads and Macbooks laptops talking to hyperscalers.
Around where? They're pretty popular for it in the UK right now given our obscene energy pricing as they end up being one of the best low power options for local llm. If you're not in the local llm space you obviously wouldn't see it. It's like saying Tennis isnt popular around here then admitting you dont frequent a tennis court so wouldn't even know.
Around here in my German office, and down in my home country Portugal, even less, given the average salaries.
So, when you said pretty popular, you meant in the local LLM space, which is a niche of a niche.
People bought diesel cars as miles per gallon was higher than petrol.
People are buying apple unified as electricity costs in many countries are very high, so cheaper to run than Nvidia setup.
As non-apple unified memory options increase, many people will have more choose those
This is a paraphrase of https://www.thedeepview.com/articles/how-apple-s-decade-long...
The submission link should have been the original editorial piece at
https://www.thedeepview.com/articles/how-apple-s-decade-long...
Being a Mac switcher since 2003 I am as much of a fanboy as anybody else but this quote from the article caught my attention, and smells like PR.
> Many AI tools are also Mac-first or Mac-only
I fail to recall AI tools Mac-only general purpose AI or agentic tools. Most of the claws, harnesses, studios and inference engines seem to be multiplatform. You can say you can run then in a Mac with a nicer UI wrapper or whatever, but "Mac-first" or "Mac-only"?
one. apfel.
oMLX[1] is the only one that comes to mind but it's not exactly unique to mac, it just runs MLX models and provides a nice gui. It does have the whole paged SSD KV cache thing, not sure if thats working on other platforms.
[1] https://omlx.ai/
I don’t understand why they are not reintroducing the Apple Xserve?
My guess is the OS. People who want a server often enough want to choose the OS, Apple wants to supply the OS and the hardware together so they're not blamed every time the two turn out to be incompatible, as happened the other way round in the 90s.
Target audience - B2B. There are multiple videos of Steve Jobs saying that he hates B2B, because the people using the devices are not the ones making the purchasing decision. It is pretty much against Apples DNA, and all their B2B they have today is a means for them to sell more B2C.
Because they killed the market, no one would now buy a macOS server, when Linux distributions, and to a lesser extent FreeBSD, own the server room.
They would even sell less than Windows Server licenses.
By the way, they are down the same path with the workstation market, now that they only top level answer is the Mac Studio.
Workstation market wants flexible towers that they can customise to their own liking and special use cases.
The main reason Swift exists for Linux, is that app developers need to have servers somewhere, and if they want to share Swift code with the backend, well it isn't going to be on macOS Server.
the SemiAnalysis guy argues that there is a market for macOS servers - running macOS VMs that agents need for computer use. some software is mac only
but Apple needs to change the licensing model, currently you are allowed to run only 2 macOS VMs for every physical one you buy
At the same time Apple made remotely hosting a Mac much easier.
whos going to buy one? You cant trust them not to kill it within a few years and cease all software updates AND make it impossible to install a different OS to keep it going. Until they stop being dicks about what you can do with the hardware you own it's a non starter.
Apple itself was a major user of Xserve, Apples needs for cloud compute are massive and growing, and Apple could probably rent Xserve as a cloud to justify the cost and sell it to privacy focused consumers and businesses.
Apple doesn’t have an ads business, and it is inevitable that Google/openai/Anthropic are going to seek to monetise consumer ai via ads.
So the ad free Apple on device experience will be welcome.
What does this mean? Apple ads bring in 8 billion revenue a year.
They aren’t as aggressive and focus on the customer experience more than others, 8bn isn’t a lot for a company with their reach.
For example Apple radio is a free product with no ads, Apple TV and Apple Music don’t have an ads supported tier.
You think Apple is a saint, but they have a completely locked down mobile computing/phone platform. Why the trust in them doing what is best for the user?
I didn’t say Apple are a saint
I said as they don’t push ads as much as others, the customer may be better
As ads focused services are designed to keep you there as long as possible rather than delivering what you are actually want quickly
It's still wishful thinking. A better strategy is to not root for one player and vote (with your wallet) for more competition.
Advertising accounts for about 1% to 2% of Apple's total overall revenue, bringing in roughly $8.5 to $12 billion annually.
1-2% is rounding error.
Quippy, but untrue
Welcome to the ad-filled experience, we think you’ll love it. – Tim Apple
https://ads.apple.com/maps
Oh please the neural engine is mostly useless for LLMs. Siri in iOS 27 is laughably pathetic and slow compared to GPT Live DESPITE sending personal context to their (attested) cloud to execute anything but the most basic queries. Still years behind.
I think the relevant comparison is the developer beta, which has access to the Gemini-powered Siri that will roll out publicly later this year. From the reviews I've seen, Apple won't be "years behind" (which surely they were) for long.
But not in the EU, because of the DMA :/
I am on the developer beta for iOS/iPadOS/macOS and all I see is still just the OpenAI option.
That said, Siri seems a little bit better now - my subjective opinion. It is a little bit less frustrating.
I did say iOS 27. I am using the new Siri. It’s better, but extremely unreliable (half of all requests fail) and slow. It should be like a Codex running on my phone with the ability to chain skills (intents) to execute a task, but it’s too crappy for that.
And the voice is still a poor text to speech model, very far behind GPT live.
Apple could dominate this niche if they decided, for a while until prices fall, to eat some margin and bump up RAM in high end models. Couple that with a new M series chip with even faster AI performance.
It’s not a huge niche but it’s an influential one. They’d get the engineers and CXOs of AI ventures and a lot of academics and hobbyists.
For the platform it would keep them cemented as the high end vendor. In the long term it would position them to take advantage of any software or training breakthroughs that deliver frontier model performance at that scale.
Except that Mac ultra M3 they talk about is now only being sold in the 96 GB configuration. It’s no longer being sold in larger Ram configurations by Apple Apple because of the global RAM shortage. And you can not add memory after purchase because it’s integrated/soldered on.
> And you can not add memory after purchase because it’s integrated/soldered on.
For as much as I dont like this aspect of modern computing, I understand why it is done from a technical perspective. Power, heat, and performance are all "better" when ram is on the motherboard vs in a "stick".
> "The speed of AI development right now is just crazy," Brooks said. "I can't imagine where we're going to be a year from now, three months from now, or even a month from now," he added.
I don't think I'm taking this out of context when I say this is unintentionally correct. Apple still doesn't know what to do about AI.
Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.
Everyone else is using it as a content scraper and praying nobody will step in to end the piracy/fraud.
The new Siri isnt that exciting at least on my iPhone 15 Pro Max. I know it's beta but it's sluggish and often says try again. I watched many videos on Youtube saying its amazing but maybe not so much on older phones? Also, I need a Siri I can talk in an unfettered manner from my lock screen while Im driving without having to unlock the screen. Probably a big ask for Siri to know my voice via voice fingerprint allowing unfettered access from my lock screen.
Others running the beta now on newer iPhones and enjoying it more so?
The best models for new Siri, on device, requires the iPhone 17 Pro, so your experience on a 15 is going to be degraded.
I use it on my 15 pro max. It's a major improvement, but I still don't use siri much. It's gone from murderously infuriating to passably usable. I still use gpt for any sort of exploratory conversations. I haven't bothered trying to use siri for that because to me that's not what siri is for.
Although given how effed up the voice for chatgpt is now with the latest updates I might talk with siri more.
Because I use carplay in tandem with my phone where the map is on the carplay screen and turn by turn directions are on my phone, it's always unlocked so I haven't run into whatever lock screen issue you brought up.
> The new Siri isnt that exciting at least on my iPhone 15 Pro Max
On my Pro 16 it has its ups and downs - I still can't get it to "play my running playlist on shuffle" whilst running (this is the only thing I used Siri for before the beta and it would improve my life immeasurably if it worked). But it responds to things like "how long will it take to drive to the AirBnb booking in my inbox", and "when is X playing a concert in Y - add a calendar entry with details" perfectly.
This is a beta and I have hopes, but I can imagine it will run better on a 17 and later
Those things also work for me but Siri is often so slow that it would be faster to do myself.
> Luckily, it doesn't matter because it's a solution in search of a problem. Most consumers aren't using AI apart from google search.
This is... a view.
Maybe I live in a strange sphere of strange ("normie"-ish) people, but the people around me are for sure using AI. Mostly chatgpt to be fair. They use it to compare products that they intend to buy, identify plants in nature, create travel plans, find interesting places to visit nearby, give movie suggestions based on what they have previously enjoyed and so on and so forth. AI is becoming a very integrated part of their reality. To "google" something and digging through the search results manually is very rapidly being replaced by asking chatgpt, for better or worse.
Chatgpt is so damn good for cooking it is unbelievable. It will learn your family's tastes over time, you can tell it what pantry staples you keep is stock, and you can take any recipe you find online and ask for stuff like "find a way to make this preparable in 45 minutes instead of 2 hours, what trade offs will I be making?"
I agree! I think it's because there's so much cooking material in its training data. I wonder what proportion of the internet is food blogs and recipes... Probably a lot!
This is true but I can do the same on free ChatGPT without even logging in. I wouldn't pay $5 a month for that functionality, much less $20 (or >$1000 to be able to run it at home).
SOTA AI for "serious" work is in a different position, used by fewer people but with big pockets and sometimes a pathological dependence on it.
> To "google" something and digging through the search results manually is very rapidly being replaced by asking chatgpt, for better or worse.
I can see this kind of low level usage as being perfect for local LLMs... So I can't see a market there for openai etc forever.
Are they using the free chatgpt or a paid one?
Nobody does what to do, they are just throwing trillions at it betting they’ll figure out. If they don’t they’ll be screwed, if they do Apple will quickly catch up with a better and more refined product, as they’ve always done.
If you take just this part of the quote...
>"I can't imagine where we're going to be a year from now, three months from now, or even a month from now,"
I'd say he's making an accurate appraisal of his abilities
When you don't limit on-device AI to "can be used to run coding model", Mac minis are great. My M4/16gb been working on a long term research project using Qwen 2.5 8b for months now. The performance is good enough for processing a lot of small text prompts.
Now just imagine they'd kept making the Mac Pro and enabled compute offload to GPUs. Or even just passthrough to Linux VMs. Would've been quite the AI machine.
The issue with the Apple is that they didn’t really develop any competitive local AI machine. Their strategy/marketing falls flat when you ask them how exactly they implement AI: they buy it from Google cloud. In the future local AI may become a thing but that’s 4-5 years away. I count the 2q-4q and atrocious performance as “local ai” only for the enthusiasts crowd not for people doing competitive work.
> "people often want a system that's under their control, isolated from their primary machine, and capable of running 24 hours a day, seven days a week," said Brooks. "A Mac mini is an amazing system for that," he added.
These execs are so out of touch they believe Apple hardware to be "a system that's under their control", how does it come to this? Besides, a VM without bi-directional sharing of data gives you pretty much the exact same thing.
Did hundreds/thousands of developers really go out there and bought Mac Minis just because one prominent technology semi-celebrity happens to have used a Mac Mini for the development of their thing? Seems bananas people would spend hundreds on monies on something they barely grasp how it works.
I think tech people spend hundreds on tech because it’s fun.
The remote access story for macOS is absolute sadness, without Jump Desktop there would be zero performant ways to access that “system under my control”.
And all of that because Tim Apple fears any feature that could mean people could have less than one iDevice per person.
I'm curious what you mean. I have been accessing Macs remotely over SSH and VNC for like 20 years and it's always been easy and as performant as the network would allow.
The built in high rez screen sharing between Macs works well too. Through Tailscale I’ve accessed my main Mac from both the opposite coast of the US and from the other side of the Pacific and it works great.
Host resolution automatically matches that of client, image quality is great, framerate is decent, latency is minimal. The host creates virtual screens for the connection so connected screens don’t light up and the machine remains locked to anybody accessing it physically too, which is a nice privacy assurance.
Ok, I didn’t want to take the bait but this one’s just too much.
> “He also described a shift toward running AI locally rather than in the cloud – a move motivated by privacy, security, and the rising cost of inference as agents consume more tokens.”
Classic Apple. No more just beating the “security and privacy” drum, now its “tokens are expensive!”
<neanderthal voice/> Cloud scary. Cloud expensive. Mac good. Buy Mac!
> “He also singled out what he calls ‘transparent AI’ on iPhone and iPad, referring to features scattered throughout the operating system and third-party apps that work quietly without announcing themselves as AI.”
<neanderthal voice/> Apple use AI, Apple just not say it. Apple smart, not lagging behind industry! Buy iPhone!
How about you invest in developing your own models, correctly? And provide a secure and private inference cloud service on your fancy Apple silicon? And integrate that into your platform so Siri gets smarter without you farming queries out to Google Gemini? Bill me for it in iCloud+ I’ll probably pay for those tokens.
Was that so hard?
But why should apple invest in developing their own models? Why would it be correct?
Or phrase it in a very similar ask, why don't they invest in power plants? The model space is truly crowded, what do they gain or recover suppose they are SOTA? Across the Pacific they are pumping out free models that are only 6-12 months behind. What business sense does it make for Apple to develop their own models?
apple has invested in training models
https://machinelearning.apple.com/research/introducing-third...
they just suck
The average apple user doesn't care about models, they care about inference.
I agree, apple shouldn't invest in their own models. But they should have close to the best inference + end user design.
They don’t believe in this model and never had. You sound like the people who screamed that Apple was dying because they were not making a netbook style Mac in 2009. Apple is the only big tech company with a non existent financial exposure to the current capex bubble. Let big dogs bark at the moon. They are the loud ones, at least until the moon implodes.
I’m not seeing how it’s bad that a company is pushing in the direction of user hardware ownership. Of course it’s self-serving, that’s what companies do, but with most of the rest of the industry increasingly leaning in the direction of eliminating powerful general purpose computers in favor of thin dumb clients with useful compute being gated by subscriptions, it’s nice to see some dissent.
AI features not being constantly shoved in my face and just selectively silently integrated where it’s most useful is preferred to what the rest of the industry has been doing, too. I think most of us are pretty sick of AI getting tacked onto things that don’t need it and then given prominent promotion and UI positioning, potentially at the cost of features we actually use.
They could be doing more, sure, but directionally this all seems fine?