I'm convinced that this is just guerilla marketing from Apple. When this started spreading a day or two ago, it was all from no-name spam media sites that are paid to publish articles. They all claimed "a source" is where they got the intel, without specifying the source. It was spreading like wildfire on socials.

The same thing happened with Mac Mini's and OpenClaw. Nobody cared about or was using Mac Mini's for OpenClaw, but there were all of these very suspicious posts from accounts that were clearly Apple marketing bots (you could tell by looking at their post history, where they would drop "Mac Mini" into every conversation they could across all different subreddits and unrelated topics). Then it became fairly common.

So Apple's marketing department is seemingly using the same strategy again. Because still, it's impossible to find a reputable source for this claim.

• bensyverson

  today at 2:49 AM

  People really have a twisted idea about how corporate America works. Apple does not have a psyops division cooking up ever more elaborate ways to seed demand for low-end, low-margin Macs via bot armies. They honestly don’t need to, because they’re making an avalanche of cash from the iPhone.

  People really were talking about and buying Minis due to OpenClaw, because they wanted something that was on all the time and had access to all their stuff in macOS. Occam’s Razor.

  • bloppe

    today at 3:23 AM

    Companies that ignore everything besides their legacy cash cow tend to go downhill. I'm sure there are at least a few people at Apple who's main concern is marketing the Mac mini, and those people are highly definitely trying to cook up new marketing strategies all the time

    • brookst

      today at 4:10 AM

      Sure. But thinking that means every article is a complicated psyop is classic correlation / causation.

      If you’ve hot any occam’s razor left in you, consider:

      1. Apple is a masterful media manipulator, carefully planting stories like this by gradually seeding fringe media first, then more mainstream, with a clever and subtle goal of making themselves look dumb in order to trick people into, uh, ok, I’m lost here, but something. Maybe drum up demand for products they can’t keep in stock anyway?

      2. Media outlets chase clicks, and when fringe media finds a hit, mainstream media follows because you can always write a story about the fact that other people are writing stories. Low cost, proven clicks, nobody yells at you for missing a trend.

      Which is more likely? I’ll cheerfully stipulate Apple is pure evil, serves cooked babies in their cafeterias, whatever it takes to discuss based on reason rather than big-evil-corpos.

  • giancarlostoro

    today at 3:55 AM

    I was buying my own Mac specifically for LLMs around this same time, didn't care about OpenClaw, but people I knew IRL were immediately asking me about buying a Mac for OpenClaw, and whether they should get a Mac Mini or what, and these were not programmers or tech nerds. There was genuine hype. Macs are lower cost alternatives to Nvidia GPUs.

  • youniverse

    today at 4:09 AM

    I know multiple people who bought mac minis just to run claude cowork on them

  • mplewis

    today at 3:23 AM

    > Apple does not have a psyops division cooking up ever more elaborate ways to seed demand

    Most companies have that division. It's called marketing.

  • fsckboy

    today at 2:59 AM

    [flagged]

• srslack

  today at 1:39 AM

  I am using Mac studios for local inference, but not for openclaw. It was simply the best bang for the buck at the time, and the software side of things has gotten even better since then. I haven’t run the pricing comparison but I have a feeling with the direction they’re heading that it’s better to hold off for m6 as it may assuage various bottlenecks if you are latency sensitive.

  • bredren

    today at 3:16 AM

    Reportedly, the M6 will not get pro, max or ultra editions. Instead Apple will go straight to M7 for these.

    • pylotlight

      today at 3:31 AM

      Some reporting seems to reject that btw: https://wccftech.com/apple-m6-pro-m6-max-soic-mh-wmcm-packag...

• ipsum2

  today at 2:00 AM

  Apple's marketing isn't that smart. They're very traditional. You give them way too much credit.

  • corndoge

    today at 2:08 AM

    Any justification for this claim?

    • ipsum2

      today at 3:58 AM

      The easiest one to think about is that the larger the company (or division), the more likely there are leaks. Think about how much as leaked in Apple's hardware or software division. If they were doing incredibly underground marketing, someone would've bragged on LinkedIn or to a reporter.

    • mixmastamyk

      today at 3:16 AM

      It's not surprising marketing for a large corporation leans toward the bland. This is not the scrappy Apple from the turn of the century.

    • raydev

      today at 2:35 AM

      All historical evidence.

• nsagent

  today at 2:44 AM

  It's not. This article is weirdly light on details. The original source article from The Information apparently mentions that companies like OpenAI are buying them up to run RL. So it's yet another case of AI companies buying all the compute.

  * I say apparently because The Information wants you to sign up for access to the article, but I found this mentioned in multiple summaries of the article like this one [1].

  [1]: https://tech-insider.org/openai-mac-buying-apple-supply-shor...

• chzblck

  today at 2:46 AM

  This may be your experience

  For mine I was on the fence - waited an hour or so and the delivery date went from early oct to 10-12 weeks so i decided to wait on the 512gb version

• transcriptase

  today at 3:29 AM

  The openclaw crowd were just former cryptobros onto their next grift: performative posts about AI that would generate enough views/engagement/ad revenue to pay for the hardware… assuming they were even telling the truth about having it at all.

  None shipped anything of value. Same with anyone claiming to be running dozens or hundreds of agents 24/4.

  • ygytt

    today at 3:39 AM

    [dead]

• mgh2

  today at 2:36 AM

  Additional source: https://www.youtube.com/watch?v=OpqGf0m5FaQ

• EGreg

  today at 3:32 AM

  Alex Finn was using and touting Mac Minis hehe

  He started that whole trend!

• internet2000

  today at 3:38 AM

  This is straight up a conspiracy theory.

  • insensible

    today at 3:45 AM

    In the sense that publishing marketing-driven press releases is a conspiracy between the publishers and the marketing team, I guess?

• fortran77

  today at 2:55 AM

  Agreed. It’s also nonsense. You need an NVidia gpu to do any serious work.

Apple was also allegedly "caught off guard" by the Macbook Neo demand.

I don't really see how they couldn't see the Local AI demand or demand for a cheaper Macbooks. This just reads like marketing imo.

• InterviewFrog

  yesterday at 11:59 PM

  I have a strong suspicion its because of openclaw mania that happened like 3 to 4 months ago.

  Basically you can have your own 24/7 AI Employee. At-least thats the appeal and folks were buying mac minis massively.

  Just my gut feel. Everybody moved on from that now. But it was a big deal back then.

  • CrimsonCape

    today at 12:00 AM

    If you use OpenClaw now can you tell us how it's going?

    • InterviewFrog

      today at 12:18 AM

      I used it initially. It was great and useful but the api costs were pretty high..

      Now, I use claude code directly on my machine with the remote control flag.. so its like open claw exactly. I can be outside and give commands to it via mobile app

      • jonchurch_

        today at 12:54 AM

        caffeinate -is -t 7200 claude --remote-control

        then Control + Command + Q ftw

        (keep system from sleeping if plugged in, keep system from going idle even if not plugged in, for 2 hours, start claude code w/ remote control enabled, lock my mac w/ the keyboard shutcut)

    • hx8

      today at 12:23 AM

      I've been using it since about February, still using it. It's running on my old M1 MBP. I have a lot of automations on OpenClaw, things like "Every week summarize this data and send me a message about it" or "check when a new version of an NPM package comes out and submit a PR to update it."

      It's quite expensive from a value per token perspective. It's quite cheap from a "send one message from my phone and it does something probably productive" perspective. No major security breaches yet.

      • Gigachad

        today at 1:56 AM

        All of the use cases I see seem like stuff that’s better performed by normal software. Dependabot will watch for new versions and submit a PR for free.

        AI for developing software that runs predictably seems more useful than AI performing actions like a program would.

• vikramkr

  yesterday at 11:43 PM

  I don't think you need the scare quotes. product lifecycles can take years and the actual neo demand was actually pretty insane. They were using it to soak up demand for binned a18 chips and it would have been irresponsible to forecast that they'd have the demand they did when there's another perfectly reasonable universe where 8 gigs was too much of a compromise and it flopped.

  this article's also about enterprise demand specifically. That's a bit surprising to me as well frankly. I'd have thought the primary market for mac studios would be hobbyists/enthusiasts with a bunch of disposable income who are willing to pay 18k for a 512 gb machine to run glm 3.5 flash or 9k to run deepseek v4 flash locally. It's competing with a $200/mo subscription or renting server gpu time for open source models during a memory shortage - and idk if it's going to be powerful enough to train or fine tune so it's really just inference. seems reasonable to be surprised

  • RossBencina

    today at 3:40 AM

    > It's competing with a $200/mo subscription or renting server gpu time

    Not if you're an enterprise that wants or requires on-prem inference.

  • dwaite

    today at 1:46 AM

    > another perfectly reasonable universe where 8 gigs was too much of a compromise and it flopped.

    The target audience didn't evaluate that as a limitation - the majority of the market for Apple devices does trust that they will not produce and sell a computer incapable of support their use.

    Professionals know there are tasks that a baseline computer cannot handle, and even common tasks that a more powerful computer does better, but those people weren't really the target demo for the laptop.

    • odo1242

      today at 2:38 AM

      > The target audience didn't evaluate that as a limitation - the majority of the market for Apple devices does trust that they will not produce and sell a computer incapable of support their use.

      Well, kinda the point. We know this to be fully true in retrospect (and many people correctly predicted it) but decent arguments existed against it at time of release.

      • today at 3:16 AM

  • lumost

    today at 1:29 AM

    A lot of big enterprises struggle with ai adoption. Either they can't get models/tools approved fast enough, can't provision them in their internal network, or get slammed with exorbitant inference costs.

    A big enterprise can drop one (or 4) of these on someone's desk and let them go nuts.

  • Gigachad

    today at 1:54 AM

    I think people just hate subscriptions so much they are willing to make an obviously worse financial choice to buy upfront.

    I’d be willing to pay more to own vs subscribe, but the gap is currently far too large where buying a Mac Studio for AI is a straight up terrible investment.

• raydev

  today at 2:36 AM

  What is useful marketing in saying "we could have made more money but accidentally didn't"?

• chacham15

  today at 12:18 AM

  the machines arent optimized for it is why. the main driving factor is large unified memory which makes large(r) models possible, but there isnt the gpu horsepower to back it up. essentially, it fits the corner of the market that wants large models and is ok with running them slowly which doesnt sound like it would be a large market.

• api

  today at 1:06 AM

  There are probably people in Apple who think desktop is dead or pro-only and didn’t think many people would want a lower end cheaper Mac. Why wouldn’t they just get an iPad?

  • ghaff

    today at 1:16 AM

    Even with the newer keyboard, the iPad just doesn't work for me as a MacBook replacement. And really isn't lighter at that point. Yes, an iPad is better for watching movies on a plane but that's not really a reason to bring one relative to a MacBook Air when I could also just bring a Kindle Paperwhite which weighs nothing. I've tried and an iPad basically doesn't work for me.

There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded.

I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's zero locally.

• kridsdale1

  yesterday at 7:54 PM

  Likewise. I have a huge demand personally to run AI noise-filtering models on many TB per month of raw video files. It takes about 3 days per file.

  Apples ProRes codec is only licensed to run in high quality mode on a Mac, and so my Nvidia PC can’t do what I need. Thus, I own the beefiest Mac Studio you can currently buy. I would pay more for more TFlops.

  I have done local LLM on there but it wasn’t interesting. Far worse performance and intelligence per dollar than the cloud boys.

  There is no cloud offering for my video needs though.

  • tassadarforaiur

    today at 12:59 AM

    Have you looked into those tinycorp nvidia drivers for thunderbolt egpus on Mac?

  • varispeed

    yesterday at 10:09 PM

    Out of curiosity, how would you transfer many TB to cloud and back if such service was available?

    • firecall

      yesterday at 10:32 PM

      My guess is that it wouldn’t be TB sized files.

      It’s probably files that, over the course of a month, add up to multiple TBs.

      Which would suggest that a 1Gbit fibre connection would be adequate. For serious commercial usage, multi Gbit fibre is available in many places around the world.

      Although being video files, they could easily be in the TB range. In which case, it would be interesting to know!

      • jaggederest

        yesterday at 10:36 PM

        10mbps constant = ~3TiB a month, which is close to what I assume they're doing to acquire the video (that's about what a few SD x264 streams would run). So you don't even need a fast connection, per se.

  • veegee

    yesterday at 8:18 PM

    [dead]

• btown

  yesterday at 11:22 PM

  A C-level executive I know is getting a top-of-the-line new Mac simply to function as a personal build server and host for agentic coding instances - they are able to orchestrate so many parallel projects that they're hitting RAM limits from sessions and the builds and local test runs they're kicking off (largly unsupervised). Before AI, they'd only had a MacBook Air; this completely changes their workflows. They talk about how many other executives they've met are equally giddy at having gone from coding few to no projects themselves, to coding more projects in parallel than any of their respective pre-AI technical colleagues.

  I'd suspect that agentic coding has birthed so many new effective engineers, that the entire dynamics of demand for high-end machines have been upended.

• postalcoder

  yesterday at 8:43 PM

  I’ve been happy training and running inference for small language models on my M4 Mac.

  Inference with MLX is surprisingly zippy. I’m running a classification task on the entire HN comment dataset and it’s projected to take about two and a half days, which is not bad considering we’re talking about tens of millions of comments.

  Yes, I could do it much more quickly by throwing Modal GPUs at it but this is low-priority work. I might as well throw my M4 a bone.

• ronfriedhaber

  yesterday at 9:09 PM

  > Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes.

  Modal significantly improves this. Highly recommend.

  • embedding-shape

    yesterday at 9:20 PM

    Is Modal at all similar to Vast.ai or just related because "It's for AI"? I looked at Modal's page for training, and it talks about using some SDK and other junk, can you not just get a beefy instance from Modal with tons of VRAM to do what you want with?

    • cmrdporcupine

      yesterday at 9:40 PM

      Modal is more like AWS Lambda in a way. Computing functions as a service instead of an whole container or whole instance / GPU. Much more granular.

• jmalicki

  yesterday at 8:36 PM

  Are you training something so big you need that much unified RAM though?

  If you can fit it on a GPU, and especially for training, it is so much quicker than a Mac.

• jtap

  yesterday at 8:51 PM

  Same, but with vision models. Unfortunately, I might be at my limit locally. I have three models that I'm using to find and identify objects in pictures. The largest dataset and model now takes about 8 hours per epoch on my Mac M4 with 16G memory.

  • NegativeLatency

    yesterday at 9:10 PM

    Yeah this was what got me to start doing short rentals of bigger gpus in the clouds, upload your parquet files and it takes a couple of hours for a thing that would have my mac at 100% for a couple of days

• ashish01

  yesterday at 8:24 PM

  What are you training on using self play? Like alpha go? Curious what your setup is like .

  • alasdair_

    today at 3:18 AM

    Yes, basically like alphago. I’m teaching it to play magic: the gathering.

    I had to start with some heuristic-based bots that played the decks very simply just to get to the point where the was some signal to learn from. I did behavioral cloning on the bots as a foundation, then self-play.

• mercutio2

  yesterday at 10:07 PM

  I’m doing the same!

  Do you find that CoreML manages to fill up your drive with so many tiny files that a reboot takes hours to clean them up? I keep meaning to get my friends still inside the spaceship to file a radar about that.

  What game are you building?

Caught off guard has been used a lot this year.

https://www.macrumors.com/2026/08/30/apple-unexpected-mac-mi...

https://www.macrumors.com/2026/05/01/apple-was-caught-off-gu...

https://www.macrumors.com/2026/01/29/apple-on-airpods-pro-3-...

Recent work in this space has got me looking at using local models for daily use. I’m waiting for people to start dumping some of the previous generation minis on Facebook marketplace or eBay so I can pick one up. Of course there’s also other options. I’m still hammering out my requirements and what I want to do besides putz around.

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have.

I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me.

Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardware to get there.

• mcotton

  yesterday at 6:28 PM

  A simple example.

  I have an older M2 Mac mini that does the OCR and visual description of all my screenshots. Screenshots are stored on my NAS.

  I like to screenshot things as a quick way to remember. They are things that I would not be comfortable sending a cloud provider (customer data, prototype screenshots, bank dispute details).

  It runs Qwen3.5:9b and glm5.2-ocr with Ollama and uses about 10GB of RAM. It automatically releases the models from RAM after 5 minutes of inactivity so it is pretty seamless to leave running in the background.

  All the details are stored in a simple webapp with a SQLite db that I can search through.

  • Kirby64

    yesterday at 7:27 PM

    > I have an older M2 Mac mini that does the OCR and visual description of all my screenshots. Screenshots are stored on my NAS.

    Doesn't Apple do this already within it's OS all locally? It certainly does it for OCR and categorization.

    EDIT: Also, no reason to use a generic LLM for this. This functionality exists in something like Immich (both OCR and 'context categorization'), and doesn't tie you into the Apple ecosystem either.

    • c0nsumer

      yesterday at 7:40 PM

      I personally use Apple Photos for this. It stores the original, plus makes them nicely searchable, so I have a Hazel action that takes screenshots from the desktop (and from my NAS where mobile devices back them up) and imports them.

      Works out really well.

      • al_borland

        yesterday at 8:42 PM

        The standard system search will also search for text in images now. I think at first it was Photos only, but now its system wide.

    • yesterday at 7:35 PM

  • ern_ave

    yesterday at 7:46 PM

    Would you mind writing that up in more detail and posting it somewhere? It sounds pretty interesting.

    • mcotton

      yesterday at 10:58 PM

      I will write about it at some point but can't get to it today.

      You can grab the code here: https://github.com/mcotton/listing

      I run it in Docker on my laptop or Synology NAS. It uses the OpenAI/Ollama URL schema for processing.

  • jwx48

    yesterday at 6:52 PM

    What is your M2's total memory? I find this application really interesting.

    • mcotton

      yesterday at 8:38 PM

      It is the base M2 with 24GB RAM

  • tiahura

    yesterday at 7:01 PM

    A new base model mac mini is $900. That is 45 month of Gemini. Gemini 4.7 Flash will give better OCR results that Qwen or GLM w/ 10GB.

    • Jeremy1026

      yesterday at 7:03 PM

      That doesn't help with the not wanting to send confidential information to a cloud though. No amount of cost savings can negate that.

      • yesterday at 8:05 PM

      • wolvoleo

        yesterday at 11:50 PM

        100% this. No way I'm giving Google my stuff.

        Neither openai or xai. Anthropic maybe but not likely. Mistral is the most likely one because they're under the EU laws but I think they focus more on commercial these days.

    • jtbaker

      yesterday at 7:40 PM

      having a 64GB mac mini m4 pro the last few years with some increasingly capable usefulness has kept me interested in this stuff in a way that using a paid platform wouldn't have. Similar to running K8s in a homelab, something about interacting with the hardware makes it more engaging/interesting, for me at least.

      In general, I'm a big believer in doing more with fewer resources, within reason, and think having local setups really helps me be mindful with what's happening under the hood with these systems and managing context efficiently to get high quality results.

      • socalgal2

        today at 1:37 AM

        I guess it depends on what you're trying to do. I've run a few LLMs on my 64gig Mac but anything image or video related is ridiculously slow compared to even old NVidia on my PC.

        • jtbaker

          today at 2:58 AM

          Have you been using an MTP setup? I've been having pretty good luck with the Qwen models with the built in MTP heads via https://mtplx.com at around Q4. My main driver rig is an M5 Max MBP work got for me a few weeks ago. Hitting 1000-1200 TPS prefill on Qwen 3.8-Flash-Next there.

    • m4rtink

      yesterday at 7:40 PM

      Do you believe Gemini will costs the same in 45 months or even exist, given Google track record ?

      • Aurornis

        yesterday at 7:42 PM

        The options available across the board are getting cheaper and better all the time.

        There is no reason to believe that equivalent level model output will be more expensive in 12 months, let alone almost 4 years from now.

        Of all the good reasons to use local AI (privacy, etc), worrying about not having access to cheap models in 4 years is not one of them.

        • overfeed

          yesterday at 8:25 PM

          > There is no reason to believe that equivalent level model output will be more expensive in 12 months

          It's almost never a drop-in replacement, and having to check and adjust integrations and workflows with new models gets old fast. My task was perfectly solved by the old model, I don't need a newer, "better" one - especially at higher prices ("more cost-effective" my foot). Local models lets one choose a model and freeze the downstream integrations forever, without being forced on the 6/8-month upgrade treadmill by aggressively short, scarcity-driven hosted model-deprecation schedules.

        • mixmastamyk

          yesterday at 9:06 PM

          The big providers are losing on average tens of billions a year on these services, so yes prices must go up. Even Moore’s won’t help in the medium-term due to shortages and difficulty/reluctance to vastly increase capacity.

          • Aurornis

            yesterday at 11:23 PM

            You can buy tokens on OpenRouter right now from companies who serve tokens as a business and who are not selling at a loss.

            The frontier labs have very high prices for inference. The prices are actually going down, not up.

            • mixmastamyk

              today at 2:53 AM

              I thought the discussion was about "frontier labs" in the cloud versus open at home.

              Others running open models in the cloud is a nice third alternative, but not a solution for those that need frontier models, which I believe continue to need training as well as inference. These are currently heavily subsidized and hardware constrained several years out.

          • yesterday at 10:19 PM

        • wilkystyle

          yesterday at 9:28 PM

          Surely you cannot possibly believe there is no reason.

          Don't get me wrong: I hope you are right, and I am generally optimistic about the future of AI.

          But do you really think, in a world filled with examples of big software companies repeatedly taking away or hamstringing capabilities we've taken for granted, that you can just count on a big tech company hosting cheap inference on incredibly powerful models forever? Surely we have learned by now that these companies do not exist to provide a public service to us, and the government cannot always be counted on to have the best interests of the citizens in mind.

          I mean, how many times have we seen this in just the past decade or two?

          - Consistent attempts to pass legislation weakening or banning the use of encryption

          - Exorbitant Reddit API pricing (still salty about the death of the amazing Apollo app)

          - Google fighting against sideloading on android

          - US gov't issuing export control directive to suspend access to Fable/Mythos

          - US lawmakers considering ways to regulate adoption of open weight models

          - Chinese officials considering restricting overseas access to their most advanced models

          I can absolutely see a much more restricted, closed down, and expensive future due to a combination of government regulations (regardless of which nation is doing it) and big companies rug-pulling as the check comes due on all the billions of dollars spent to get here.

        • hydropax

          yesterday at 9:55 PM

          [dead]

      • azan_

        yesterday at 9:36 PM

        Models equivalent to current version of gemini will likely be much, much cheaper, maybe even something like 100x cheaper.

    • booty

      yesterday at 10:05 PM

      Two thoughts.

      A $20/month Gemini subscription is truly all you need, then yeah, sure.... obviously a homelab setup is a ridiculous alternative on a pure cost basis. For most people doing "real" work with LLMs 40+ hours per week, a more apt comparison would be one or multiple $200/month subscriptions. At which point the break-even point of a homelab is much sooner.

      However, most people running homelabs are doing it for other reasons. Independence, learning, and/or privacy issues.

      • gessha

        today at 2:38 AM

        I love this quote from a homelab reddit:

        - Is that even worth the electricity price compared to api? - We don't ask that here

    • wilkystyle

      yesterday at 7:37 PM

      This is such a tired argument and it seems to be parroted every single time someone talks about local models on hacker news.

      Yes, of course the most economical path is to hand over all your data and become fully dependent on a cloud provider who is already operating as scale, hoping that they won't change/remove models, hamstring capabilities, or raise prices.

      If this were a thread about hosting your own email or blog or cloud photos, you'd have plenty of people out here telling you how easy it is to do it yourself instead of relying on Gmail for email or WordPress/Medium/Substack for blogging, or iCloud for cloud photos.

      And yet, without fail, every single thread about self hosting local models seems to have some copy/paste form of this cost-savings argument.

      Where is the appreciation for this cool thing GP built? Where is the appreciation for the desire to figure out how to host your own version of the incredible capabilities that were not available merely a few years ago? And why, on this site of all places, would someone advocate trading all of the knowledge and independence gained from learning how to host something like this ourselves in favor of throwing it all over the wall to Google?

      Come on.

      • mv4

        yesterday at 8:27 PM

        It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!".

        • jmalicki

          yesterday at 8:39 PM

          > It's quite shocking to me how many experienced, tech-savvy people, who used to care about cookies and ad tracking - are now willingly sending their business strategies, highly confidential contracts, and intimate personal issues to a cloud provider because "it is only $0.0x per million tokens!".

          Because there are more privacy guarantees there, depending on the provider. "But what if they violate their contract!" is some pretty tin-foil hat stuff.

          How is this any different than a business running their website out of the cloud, assuming you are using a provider with appropriate contractual terms?

          You can care about tracking and ads but still be comfortable storing your backups in the cloud, and many have been for quite awhile, even sometimes without encryption - that is totally different than e.g. Meta actively trying to track you and understand your relationship graph and your purchases etc.

          • 0cf8612b2e1e

            yesterday at 8:54 PM

              But what if they violate their contract!" is some pretty tin-foil hat stuff.
            

            The foundation of these businesses is stealing IP in bulk.

            • jmalicki

              yesterday at 9:02 PM

              The foundation of the businesses training AI models.

              So don't use them for inference.

          • wat10000

            yesterday at 9:12 PM

            I'm not sure about the tin-foil-hattedness of worrying about them violating their contract. But that's by-the-by. It is definitely not tin-foil-hat to worry about the data being taken in a breach.

            • jmalicki

              yesterday at 9:21 PM

              It's no different than generally using AWS.

        • notatoad

          yesterday at 10:19 PM

          i think you overestimate the number of tech-savvy people who ever cared about cookies and ad tracking.

      • Danox

        yesterday at 11:29 PM

        I don’t understand the willingness to give up privacy so easily, particularly if you are developing something that you plan to monetize somewhere down the road.

        I’m pretty sure that all of those disclaimers that all the AI model makers have for you to sign off on to say that they’re not responsible for anything that might go wrong if your work gets copied accidentally and used someplace else wink wink?

        You know the lawsuits for that particular aspect are incoming in the future…

        • wolvoleo

          yesterday at 11:53 PM

          Yeah I use it for hobby only. I would use cloud models to develop new stuff, I don't really care if it gets lost because if I were to publish it it would be FOSS anyway. I hate entrepreneurism and monetisation so I'm happiest being a salaried employee with many hobbies :)

          But no way whatsoever I'm uploading my personal files, photos, emails, chats into cloud AI. No way.

      • socalgal2

        today at 1:41 AM

        I think the price is beyond that though. AFAICT, there's not competitively fast image or video generation on Mac. To buy it will cost me $4000-$12000. So I rent.

      • jay_kyburz

        yesterday at 7:59 PM

        And everybody knows advertising is just around the corner.

        It will be horrible to be dependent on an AI who is also be trying to sell you various goods and services.

        We're going to need AI whose loyalty is to us and only us.

    • bahmboo

      yesterday at 10:39 PM

      Yes but I also get a full fledged computer in the deal. I can sell it later. I can use it for all sorts of things like games and browsing and video editing. Paying for Gemini for other tasks is also in the mix but at the end of 4 years I get...nothing.

    • nrmitchi

      yesterday at 7:03 PM

      ...

      > They are things that I would not be comfortable sending a cloud provider

      It's also an old machine that the commenter already has; it's intellectually dishonest to compare it to the price of a brand new, 4-iteration-newer machine.

    • homarp

      yesterday at 7:17 PM

      except

      a) model I pick will not 'suddenly' go away

      b) I am sure my data stays where I want it

      c) my inference mac can run other things if I need to

      I pay for that.

• Aurornis

  yesterday at 1:41 PM

  > it seems really far off from the kind of experience even a basic $20/month subscription gets me.

  The $20/month subs are much stronger than the local models you can run, even with how far local models have advanced lately.

  The appeal of local models is that the data never leaves your network so you can feel safer putting sensitive content into it. It also feels “free” to use when you’ve already paid for the hardware.

  But it doesn’t perform better and if you do the math you’re probably not saving money either. It’s helpful for things that you can’t or don’t want to outsource to a 3rd party.

  • pizza234

    yesterday at 6:42 PM

    There are a few use cases that are (somewhat) surprisingly unsuited for cloud providers:

    - translations: cloud providers can bowdlerize (censor) bad words/content; also, if you want to do a translation for personal use of copyrighted materials, cloud providers may block it

    - image generation: generating drawings with a style that even just resembles a copyrighted one (ie. Disney) may be blocked by cloud providers - for example, generating old cartoons style with GPT may not be possible.

    • raffraffraff

      yesterday at 8:14 PM

      What about a light but bulky AI job, like batch processing 50GB of files? I'm currently doing it on my used Macbook M1 Max 64gb, and it's chugging through it for the cost of electricity (free with my solar).

      • Gigachad

        today at 3:53 AM

        64gb unified memory is incredibly powerful for a local machine. Most of us have 16GB which is almost useless compared to the API models.

    • nomel

      yesterday at 7:40 PM

      Cloud does not mean censored. You can rent gpu time and run whatever model you want, with your data kept as private as any other cloud instance you personally run. Cloud is location, with (for some work) wayyyy cheaper access.

      • ryandrake

        yesterday at 7:52 PM

        Cloud is still not "your computer" so it's probably wise to take that into account and act in accordance with your own threat model.

  • seanmcdirmid

    yesterday at 1:50 PM

    Uncensored models are also popular reasons, although it’s more of a niche.

    • xnx

      yesterday at 2:06 PM

      Aren't those also commonly available from cloud providers?

      • digitalsushi

        yesterday at 2:15 PM

        i'd say 'with' instead of 'from', the cloud provider rents you the hardware, what you run on it, is whatever you show up with

  • segmondy

    today at 12:19 AM

    glm5.3 is matching fable in lots of places and beating it after more than 1 pass in many. now of course when i say this, folks would claim that it's not local, but it can be. if you happen to own a mac studio 512gb, you could run it.

  • varispeed

    yesterday at 6:05 PM

    I don't think it is really about "sensitive", but basically about any content you put in. Why would you give corporations your reasoning (data on how you interact with AI, how you "talk" etc.).

    All of this is private, but not necessarily sensitive. You never know what is happening with this data. They might say they don't log it or don't sell it, then few years later you'll find it all online or read a book that has a story eerily similar to what you chatted about with GPT a year ago.

    • fwip

      yesterday at 8:03 PM

      Although in this use case, it's likely because GPT guided you to write the same story as somebody else. Talking with an LLM about an idea is a great way to make it more predictable and homogenized. If you're fixing a bike or writing software, this is usually a good thing.

      • varispeed

        yesterday at 9:52 PM

        Gemini not long ago, when you said something "useful" said thank you, I will use it to help other users with similar problem. When asked "why would you do that, I thought our chat is private?" it would respond "Apologies. My mistake, of course this chat is private and your information will not be used." Funny.

  • insane_dreamer

    yesterday at 7:35 PM

    > much stronger than the local models you can run

    but depending on what you're doing, you may not need the "bleeding edge" performance

  • tiahura

    yesterday at 7:02 PM

    Banks, Biglaw, and the Pentagon all do it in the cloud. What could an individual be working on that is so secretive?

    • Someone

      yesterday at 7:11 PM

      > Banks, Biglaw, and the Pentagon all do it in the cloud.

      In _a_ cloud: their own virtual private cloud. They also have enough power to negotiate contracts with strong privacy provisions.

      • jmalicki

        yesterday at 7:23 PM

        > They also have enough power to negotiate contracts with strong privacy provisions.

        What privacy provisions would you want to add to AWS? Most of the reasonable strong privacy provisions you'd want are already there and/or available if you want to sign up for it, even including US Govt Top Secret data if you meet some approval.

        • Larrikin

          yesterday at 7:48 PM

          If they decide to ignore the contract and violate the privacy provisions, the US government can punish them, can the company from future contracts, throw executive in jail.

          If they violate their privacy agreement with me there's effectively no punishment I can get that they will actually feel. There would have to be a class action law suit and they always just settle those for some small amount and admit to no wrong doing

          • wolvoleo

            today at 12:15 AM

            > If they decide to ignore the contract and violate the privacy provisions, the US government can punish them, can the company from future contracts, throw executive in jail.

            To be honest, they'll just send a few million and some gold plated trinkets to the right people and it'll go away. You know that's how things work now in the US. Even if you're an actual convicted drug dealer you can have your mother say a few nice things and whoop there's your get out of jail free card. This stuff is peanuts compared to that.

            If not they'll just stall in court and settle for peanuts after everyone is exhausted battling it out. That's what they've always done before the current government made it even easier. Fines are not a penalty, just a business expense.

            Nobody important is going to do any time for this. Not in this America.

      • SR2Z

        yesterday at 7:27 PM

        This kind of stuff is available off the shelf at any major cloud provider.

        • NetMageSCW

          yesterday at 9:14 PM

          Guaranteed privacy is available off the shelf?

          • SR2Z

            yesterday at 10:53 PM

            Yes, insofar as any "guaranteed" is cloud providers just saying "we pinky swear to not decrypt your resources!"

            I know what I'm talking about here - trust me when I say that the difference between a "government" or "large corpo" account and "some dude" is literally a few flags to prevent resources being created in the wrong regions. It's nothing that a random customer can't do themselves.

    • ozim

      yesterday at 8:33 PM

      I like to buy specific brand of soap. I don't want them to know that, it is my right and so is running local LLM "wasting" money on local inference to keep track of my stack of soap.

    • mbreese

      yesterday at 7:07 PM

      Those companies also have data sharing/use agreements that they can get from Cloud AI providers due to their size and spend. The secrecy and data protection is largely what they are paying for. Those types of agreements just aren’t available to individual customers. It’s only when you’re spending $$$ that it becomes worth it for the provider.

  • BurningFrog

    yesterday at 1:45 PM

    It also takes some load off the AI data centers.

    IDK if that might be a concern for Apple or their AI partners.

    • Scene_Cast2

      yesterday at 3:57 PM

      It worsens the supply crunch, no? A unit you use sparingly vs that memory going into a GPU that serves many more people.

      • maherbeg

        yesterday at 6:57 PM

        Those will use different wafers, so unless that memory is allocated for unified memory vs gpu HBM it won't make a difference.

      • SMEbooop

        yesterday at 8:06 PM

        What supply crunch? Tons of RAM available for purchase. It's just expensive. That there is a "supply crunch" is made up to benefit from Trump administration not giving a shit how big corps operate

        There's cloud hosts out there with unused compute. Wasted cycles are all over businesses running unused cloud apps and subscribed to services they don't use.

        Still need a local computer to access the cloud; so a barely used gadget still exists. And this creates duplication of effort; we built RAM for servers AND the edge devices.

        Seems redundant when tech nerds and corporations are really the only people that care.

        And all that data in the web is meaningless yet we create a supply crunch storing it in servers.

        This an out of touch nickel and dime perspective given the big picture to say nothing of the mess of strip mining and manufacturing pipelines that go into every screw, wire, and such

    • articulatepang

      yesterday at 3:03 PM

      Surely this is not something that motivates the vast majority of people using local LLMs.

• jasode

  yesterday at 2:29 PM

  >I realize I’m somewhat limited (16GB RTX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me.

  I just ordered a new Mac Studio M5 Max 128GB $5899 ($6400 with tax) to be able to run the bigger "consumer size" models in the 70B parameter range (~96 GB). That said, I have no illusions that this expensive setup with a Qwen Flash coding LLM will be comparable to a $20/month subscription. Even upgrading to an even more expensive Mac Ultra 256GB for $10000 to hold a bigger model still won't be comparable. Apple hasn't shipped my Mac yet and I'm still considering cancelling it and downgrading to a smaller 64GB RAM config ($4299) to save $1600.

  Why did I initially spend the extra $1600 if I knew ahead of time that it wasn't as good as cloud AI? Because I thought I could use some local LLM for the easy tasks or when I hit cloud rate limits. No issues with privacy so that wasn't part of the motivation at all. I just wanted some local AI capability to augment a subscription. I've not totally convinced myself of the cost/benefit of this.

  Based on today's consumer hardware landscape, you're paying very high prices for crippled capability compared to the cloud AI subscriptions. We're also in a transition period where the next iteration of hardware improvements have some compelling features for local AI. Apple's upcoming M7 (2027 or 2028) is anticipated to have better GPU and neural engine to help with prefill TTFT. AMD Strix Halo is about to release 192GB system which is a big upgrade to their current 128GB ai pc. Maybe apply my $1600 savings towards those newer products. Those future products will still be very expensive but maybe the cost/benefit will be better.

  • pizza234

    yesterday at 6:58 PM

    > Why did I initially spend the extra $1600 if I knew ahead of time that it wasn't as good as cloud AI? Because I thought I could use some local LLM for the easy tasks or when I hit cloud rate limits.

    The maths don't check. With Deepseek Flash one goes a very long way with 1600$ - even 10$/month, for easy jobs, are more than 13 years, and at a higher quality.

    • wolvoleo

      yesterday at 11:57 PM

      Oh no doubt. But one does have the guarantee that no bits left their home and that's a big one for some.

      The low hanging fruit stuff for me is more something I use it for because I have the local LLM setup running anyway. It wasn't the reason I bought it, but now that it's there I might just as well use it as much as I can.

    • yesterday at 11:06 PM

  • try-working

    today at 12:56 AM

    I think the M5U Ultra 96gb is the sweetspot in that price range. It has more compute and bandwidth so you get to run models better sized to its hardware. I believe the Max would be too slow; personally I'm getting this SKU because I think it'd suck to get the 128gb Max and then discover it's too slow, and end up just using cloud providers anyway.

    • kristianp

      today at 1:33 AM

      I agree about getting the Ultra if you're interested in AI (LLM) inference speed. I'm a little perplexed as to why there isn't a RAM option in between 96GB and 256GB, though. For instance, I believe Deepseek v4 flash runs a lot faster on (https://github.com/antirez/ds4) with 128GB than 96. I assume the higher SKUs have low enough sales that Apple doesn't want to be manufacturing too many different RAM configs as that would eat into their margins. (I say this as someone interested in AI hardware in general, even the 96GB is out of my price range).

      • try-working

        today at 2:08 AM

        It's the upsell ladder. This is how they get you to buy a more expensive tier or two. At first I was only going to look at 64gb or 128gb Max, but ended up here.

        • Grombobulous

          today at 2:56 AM

          I also think it’s possible that it may be best to continue waiting on the Mac side of the house despite the increasing prices.

          I think that some of the hardware design folks have been blindsided by AI demand and we haven’t really gotten that next generation AI hardware yet, to the point where buying M5 isn’t going to make sense in a couple of years.

          Rumors seem to be that the M7 is the generation that Apple is looking to push AI performance much further.

          I’m not sure that Apple anticipated this specific route that computer hardware has gone and I don’t think M5 and previous iterations were really specifically architected for local AI performance, more like they happened to be pretty good at it.

  • bertmuthalaly

    yesterday at 6:00 PM

    Local LLMs are improving for fixed hardware, though - a 30b parameter model now is markedly better on the same hardware than one from a year ago.

  • reilly3000

    yesterday at 10:45 PM

    Keep the memory. You’ll be glad you did when you realize that you’re better off with a solid coding model plus a good voice model and also a lightweight all-rounder all running at once isn’t of loading dynamically (slowly). It also helps if you want to be able to run a browser, IDE, and container environment.

  • ac29

    yesterday at 8:16 PM

    > AMD Strix Halo is about to release 192GB system which is a big upgrade to their current 128GB ai pc

    Big upgrade to memory capacity but memory speed is only going up by a few percent, so its still going to be slow with more than a few B active params (I have one)

  • adastra22

    yesterday at 6:19 PM

    Serious question: why not run DGX Spark or Framework Desktop, at 30%-50% lower cost?

    • jnwatson

      yesterday at 6:45 PM

      M5 Ultra has 4-5x the memory bandwidth of both. 1.2 TB/s memory bandwidth opens up good performance on relatively large models.

      • vardalab

        yesterday at 10:10 PM

        you can get 4xGB10 for <20K so that gets you about the same tg and pp will be probably better. Power consumption though will be something like 200W idle so that's a bummer. And you get VLLM and SGLANG unlike them mac where one has to use oMLX (nice but not the same concurrency or cache performance) I have 128GB M5 Max laptop and sill prefer to run things on other boxes in basement because its no fun to have the primary device being overloaded.

    • HNDevsSuck

      yesterday at 11:24 PM

      [dead]

  • j45

    yesterday at 8:09 PM

    It not only about it being an expensive setup (or not), and also other considerations:

    - There's no guarantee of the $20/month service, and it likely has some limits compared to dedicated hardware token wise.

    - Model are becoming more and more efficient, in many cases an M1 Max Mac Studio is still capable with 32 GB. 128 GB ram may not be the necessary baseline.

    - Folks may think they want to only have a general model running locally (it's the comparable after all from the cloud providers), but we have to remember if the tasks we're trying to do ultimately are more specific than general and if there's space for the smaller models to do that.

• drusepth

  yesterday at 6:06 PM

  A huge benefit of local setups at our studio is that a lot of our software can't run headless, so when we're having agents work in Blender or in Unity etc with MCP that'd otherwise eat up our normal computer use. (Try to have two people try to work in the same Unity editor at once... then try ten!)

  We also built some QA agents that are always playing our games from the same builds a player would and flagging things to fix/improve; that alone needs the game focused and front-and-center so it can properly screen-capture for deciding what inputs to take next (and for screenshots/replays), which also means we can't really do any hands-on work at all on the machine when it's running.

  Having a separate (and tiny) machine for all of this has been great. We don't bother with local models because, you're right, the $20/month sub is way better than anything that can run on small consumer hardware atm.

  • tylerflick

    yesterday at 6:16 PM

    > A huge benefit of local setups at our studio is that a lot of our software can't run headless, so when we're having agents work in Blender or in Unity etc with MCP that'd otherwise eat up our normal computer use

    I'm curious about your setup. I've been tinkering with the idea of setting up Blender (cli use) in a container to allow agents to verify the scripts they are generating compile at a minimum. One thing I've found extremely helpful was generating a RAG of the current version of Blender.

    For anyone wondering, I'm running Gemma4 26b A4B on a mini PC with 32 GB of DD4 and a Vega 7 iGPU (llama.cpp w/ Vulkan).

• D13Fd

  yesterday at 2:35 PM

  I’ve been running DeepSeek 4 Flash, Qwen 27B and Qwen 9B on local hardware. They work well for coding and document review tasks. I think Qwen 9B local on a 5090 might be legitimately helpful for small task agents in omp, since it is ridiculously fast. But my motivation is that I have data that I unfortunately can’t share with 3rd parties.

  I have been eyeing a 512 GB Mac 5 Ultra to run full DS4 pro locally, which I expect would be pretty amazing as far as quality/recall. The only downside is that the speed is a lot slower than something like 27B on the 5090.

  • dagaci

    yesterday at 7:34 PM

    I have a RTX PRO 6000 96GB when the pricing was way better than now i also have a RTX 5090 too.

    What I noticed is that (1) the great local models are optimized run inference (diffusion & LLMs) well on 32GB VRAM <= GPU's because that that's what the target has ...

    (2) The quality of local models (esp. in diffusion) is increasing faster than the need for more VRAM - additional reason for the value of these FAST GPUs to increase!

    (3) RTX PRO 6000 96GB is really great for fine tunes (ai-toolkit) :) but doesn't outperform my RTX 5090 with inference by anything significant on the good local models.

    I have never run an AI job on a Mac, i also have doubts about performance and compatibilities - since the reviews almost never compare directly.

• CuriouslyC

  yesterday at 9:36 PM

  Image and Video gen is superior locally, because you can tweak more, use LoRAs, use whatever model fine tunes, and generate uncensored content, plus as you're often cherry picking from multiple gens it ends up being cheaper for comparable quality as well.

  Local coding is a step down but good enough for a lot of things if you have privacy concerns.

• matheusmoreira

  yesterday at 11:19 PM

  Local inference can't compete with cloud on speed, intelligence and economics. It's all about freedom, privacy, control, sovereignty.

  It's about not having to accept any of the stupid "terms" of the corporations. It's about doing things the big labs don't allow you to do, like cybersecurity stuff, or even just chatting with the AI about some wrongthink.

• booty

  yesterday at 9:57 PM

  What models you running? What effort level?

  Wild oversimplification, and benchmarks vary widely, but I've read a lot of benchmarks suggesting that Qwen3.8-27B (xhigh effort) competes with near-frontier models at a lot of coding tasks. To the best of my understanding it's not going to run very feasibly in 16GB of VRAM at usable quants however.

  r/LocalLLM and r/LocalLlama are noisy, but valuable sources of anecdata if you have the time (or the tokens, hah) to comb through them. You are going to see a lot of modest setups there, and also guys with $20K+ of hardware.

  The two things (besides my bank account) that keep me from investing heavily in local are (1) we are not guaranteed to get a steady release of open models in the future (2) a lot of the "fun" stuff LLM stuff that interests me involves orchestrating lots of parallel agents, which of course multiples the hardware you need to achieve it.

  For example, I've been having good results having both Sol and Opus review the same PR, and then I have them cross-review each others' PRs. A next step I'd like to consider is maybe having a swarm of Luna agents review the same PR and have them fight it out... maybe with Sol doing final arbitration? I suspect 5-10 Lunas might outperform a single Opus. Or maybe not. But at any rate, that would be impractical in a homelab without a pretty big hardware (or time) budget.

• TechSquidTV

  yesterday at 6:43 PM

  In my limited experience, not quite yet but we are damn close. Qwen 3.8 27b is it. If I could run this as a decent speed, I would no longer need cloud models at all. I'm actually currently trying it out in the cloud to pay for the inference speed but the model is fully runnable at home.

  I realistically costs $5-10k to replicate a ChatGPT like agent. And it doesn't scale.

  That's still really close. And models and quantization etc keep improving.

  I'm absolutely positive that I'll be switching to mostly local AI in the next 5 years.

  • Grombobulous

    today at 3:02 AM

    I could believe that especially with some of the chips playing catch-up.

    E.g., the M7 chip is rumored to be the one where Apple has poured really serious effort into local AI performance where previous generations seem to have mostly been coincidentally good at it.

    Maybe this is an incorrect opinion but I don’t personally think that the M1-M3 or maybe M4 or even M5 chips were designed with LLM inference in mind at all. These were designed with things like video rendering, image/video ML, and rasterization performance in mind.

  • vardalab

    yesterday at 10:35 PM

    Good 60-70 tg and 2K pp Qwen 3.8 27B FP8 can be had for about 5-6K (2xR9700 + PC) Gives about 3-4 concurrent sessions with full 262K

    Fast 150+ tg and 2-8K pp Qwen 3.8 27B nvfp4 is about 8K (5090 +PC) Gives really only one concurrent session that flies because kv caching is not perfect for ninfer https://github.com/Neroued/ninfer

    both are very serviceable, I prefer FP8 on 2xR9700

    But, yes it doesn't scale that well but in 5 years the same hardware should still be very capable of running some great MoE models, for example Qwen 3.6 35BA3B on 5090 can fly at 600 tg

  • cyanydeez

    yesterday at 6:58 PM

    Qwen 3.8 at 27b, 4bit MTP, Full context, in 72GB blackwell is 2-3x agents.

    If you have a real product and can actually sell it, youre taking a largish risk relying on the cloud.

    From model changes, alignment, to enshittification and the natural cognitive offloading, you could be one day removed and ROI tanked.

    Think of AI like a mafia boss who helpfully supports you untill they need a favor. Thats all cloud AI is in America.

• randomblock1

  yesterday at 9:22 PM

  It's not that far off anymore. On my 7900 XTX 24GB, I can run Qwen3.8 27B with 131K context at Q4_K_M (55 tok/s with MTP). Excluding hardware cost, it's about $0.02 tok/M in and $0.40 tok/M out (cached in $0.0001). On OpenRouter, that would cost more than 10x what it actually costs me.

  Of course, 131k context at 4-bit quant is a trade off, but even then, it's VERY capable. It doesn't feel that far behind something like GPT 5.6 Luna.

• paxys

  yesterday at 1:42 PM

  Local setups aren't going to make sense purely from a cost perspective, and definitely not when you are buying Apple hardware. AI subscriptions are too highly subsidized right now.

  • monatron

    yesterday at 1:50 PM

    I think your last point is exactly why I'm so interested in local models. The current landscape doesn't feel sustainable. The last few months we've seen the big providers (OpenAI, Anthropic) start to play with usage limits, resets, banked resets, pulling models, etc. I think local models are close to the point where, with a sufficiently well-architected harness, you can get results that are on par with the experience you'd have with cloud inference. It is nice to know that I have hardware under my desk that I control with open weight models that I can interact with on my terms.

    • polnoner

      yesterday at 9:48 PM

      It is certainly not sustainable but that is exactly why I want to use $400-$800 of resources a month for $20 while the deal last.

      It is really a once in a lifetime deal.

      Once the deal is over the local models will be better than what I am using now anyway and the hardware will be all the better than what I can get now for the price.

    • htrp

      yesterday at 2:14 PM

      yes, you have to unfortunately adopt a 2025 mindset when working with smaller local models because that's where they are compared to the frontier.

    • julianlam

      yesterday at 2:09 PM

      > local models are close to the point where, with a sufficiently well-architected harness, you can get results that are on par with the experience you'd have with cloud inference

      In my opinion, 98% of the work most devs would send to an AI can be capably achieved with a local model and a frontier-level model is overkill.

      The goalpost moving feeds right into Anthropic and OpenAI's interests.

  • apparent

    yesterday at 6:08 PM

    > AI subscriptions are too highly subsidized right now

    I've been running into annoying limits with Claude recently. It gives me like 5 questions over the course of 15 mins and then tells me to wait 5 hours. When companies can change things up to make the base subscription nearly useless (the last question always gets messed up, too), then you realize the value of owning your own infrastructure.

    • adastra22

      yesterday at 6:23 PM

      On the $200/mo plan I have never hit a five hour limit, and I struggle to use my full credits each week.

      $200/month is vastly cheaper than owning and operating comparable hardware.

      • apparent

        yesterday at 6:53 PM

        You're right that $200/mo is much cheaper than comparable infrastructure. OTOH, you don't get to have a computer that can also be used for other applications, or which works when the internet is down. Also, you remain tethered to whatever pricing the AI companies want to charge. If AI pricing goes like Uber/Lyft did when the VC cash ran out, then we'll be paying much more in a few years' time. We could look back and think "I wish I'd bought my own setup back in 2026" if it's going to be inevitable.

        And all this before you get into privacy/security/compliance stuff.

        • wolvoleo

          today at 12:04 AM

          Ohhh trust me we're going to pay a hell of a lot more. It'll be nothing like Uber's VC running out. Someone has to be make back all those trillions.

          What we're paying now is just akin to the drug dealer's free starting shot.

      • 0x457

        yesterday at 7:07 PM

        > On the $200/mo plan I have never hit a five hour limit, and I struggle to use my full credits each week.

        I started tasking fable with huge projects over the weekend and now I hit at very least fable limit by monday.

        • anslopic

          yesterday at 11:12 PM

          [flagged]

      • limewedge

        yesterday at 11:16 PM

        [dead]

        • wolvoleo

          today at 12:06 AM

          Well it's not just text to text. It's text to code, text to images, research. For me 200 is too much but my 20€ sub yeah I do find that worth it.

          If I actually made money with it (I don't and I don't want to) the 200 one probably would start making sense. If I were using it all day I'm sure it would save me more than that 200 by simply being able to do more work.

          • anslopic

            today at 2:18 AM

            [dead]

      • anslopic

        yesterday at 11:11 PM

        [flagged]

  • yesterday at 2:16 PM

• dghlsakjg

  yesterday at 9:15 PM

  I'm using a Mac to do bulk diarized transcription (STT). Most services run in the $.05-1.00+ per hour of transcribed audio.

  My Mac can do ~200x realtime (1 hour takes 20s or so). I can do several thousand hours per day. Its pretty incredible

  Not sure how much that qualifies as AI vs LLM usage, but it seems to work pretty good

  • bojangleslover

    today at 12:38 AM

    What are you using for this? We tried this a few years back, could simply not find a good diarization engine.

    • dghlsakjg

      today at 2:56 AM

      Parakeet for transcription and senior for diarization.

      Check out the fluid-ml library which packages this up for ANE very nicely.

• inventor7777

  yesterday at 2:14 PM

  One use case I find cost effective is using it as a voice assistant for Home Assistant. API pricing on models is very weird compared to the normal chats, so I use Qwen/GPT-OSS on my Mac Studio via llama.cpp server.

• yesterday at 7:44 PM

• wizee

  yesterday at 10:40 PM

  For software development tasks, Qwen 3.8 27B is genuinely excellent, but you need 32+ GB of VRAM to run it well with decent context, and enough memory bandwidth and compute to run it at a decent pace. With an M5 Max Mac Studio, you can do that decently well.

• spacephysics

  yesterday at 6:29 PM

  Also, the $20/month subscriptions are HEAVILY subsidized, so it's not an apples-to-apples comparison really

  • jmalicki

    yesterday at 7:26 PM

    It is a completely reasonable comparison for me as a consumer, since they're the costs and benefits that I'll actually get.

    • wolvoleo

      today at 12:36 AM

      For now yes. But don't count on it staying that way. Even month to month the actual allowances change.

  • PaulStatezny

    yesterday at 7:09 PM

    For the amount of tokens you get, based on your comment, ALL subscriptions are heavily subsidized, and the most expensive ones are the most subsidized.

    For OpenAI and Anthropic, the $100 subscriptions cost 5x the $20 subscriptions and give you 5x the tokens. And the $200 subscriptions are 10x the cost for 20x the tokens. (Tokens cost 50% as much.)

• SamInTheShell

  yesterday at 7:25 PM

  From what I’ve been seeing, the Mac studios do look like they have potential. I was looking to drop $10k-$15k on one until recently. After comparing a Radeon 7900 XTX vs Ryzen Halos 128GB vs M1 MacBook Pro 64Gb, I landed on just getting an external closure setup with Nvidia RTX 5090.

  The model I’m specifically targeting to use at high speeds is Qwen 3.8 27b @q4ks. This model actually proved to be good at coding (it sits somewhere between Sonnet 5 and Opus 5 capability). M1 got 10 tok/s, Ryzen Halo 20tok/s, and Radeon 7900 XTX 50tok/s (can only do 128k context window in Radeon card).

  The prefill gets extremely slow around 50k tokens in context window (whatever prompt processing stage entails could be wrong about phases here). It takes about 2 hours to fill the context.

  Even with a drafter model intended for speed instead of mtp, I can’t get past 70tok/s, still is extremely slow to process prompts as context grows, and drops down to 40-50tok/s anyway making this config still moot for improvement on my Radeon card.

  The only thing I can point to slowing me down is bandwidth of the card itself.

  I am waiting to actually get my 5090 right now and I am betting that the 1700 Gbps of capacity will fix my prompt processing speeds. I don’t need full PCIe lane bandwidth to serve my house I just need to load the full model into vRAM and let the GPU do its thing.

  Additional benefit to the external enclosure route is being able to migrate the inference between devices more easily. I can develop out the infrastructure then migrate the card to be hooked up to a shared node in the house with all the tools necessary for my family to take advantage of the privacy enhancement that comes with local inference.

  • unsnap_biceps

    yesterday at 7:44 PM

    How are you actually using the local model? I've played with Qwen 3.8 27b on ollama and the coding harnesses (Claude Code and OpenCode) seem to fail way more often then using the cloud models. And by fail, I mean the edits don't apply cleanly, it goes to add python code, but doesn't indent it properly, or the edit doesn't apply and so it tries again and again and eventually wipes out a different function then it "intended". It just gets really frustrating compared to the relative stability of Claude Cloud.

    • SamInTheShell

      yesterday at 7:49 PM

      My use-case is only coding, every model sucks at writing good literature and there is no way around that (have had people try to debate me on this, but it's a taste thing, I have extensive English writing skills from my school years).

      Prior to two weeks ago, I was just using Pi and Ollama.

      I have tried my hand at putting together a few harnesses and I finally landed on what I like. Been working on this small app to handle running llama-server for me from any device that has the llama-cpp stack setup: https://github.com/SamInTheShell/loom

      Qwen 3.8 is the first model I've been using that hasn't been having issues doing edit calls. Here are my llama server settings and GUFF that I use: https://gist.github.com/SamInTheShell/0bf838e8dc5093583b688e...

    • vardalab

      yesterday at 10:46 PM

      try using pi harness, hae not encountered these sort of problem myself also yuou can ask codex to look at the transcript and figure out the solutions to tool call failures that way

      • SamInTheShell

        today at 12:58 AM

        I should have mentioned in my prior comment, I didn't have any issues in Pi either.

  • vardalab

    yesterday at 10:45 PM

    try ninfer once you get your 5090 https://github.com/Neroued/ninfer

    • SamInTheShell

      today at 1:07 AM

      Thanks for sharing, I'm definitely trying it out after I get through my project milestones for the 5090. The README claims 700tok/s for Qwen 3.8 27b, that would be amazing, I'm only expecting an increase from my ~50tok/s on my Radeon to 200tok/s on the 5090.

• rc1

  yesterday at 1:46 PM

  The article implies the demand is for running locally. I’m not convinced, at least with a mac minis. Most people I know and myself buy the mini as it is always on, easy to setup, and isolated from my main computer which is a laptop. The mini is driving the use of the $20/month subscriptions.

  • julianlam

    yesterday at 2:10 PM

    As a thin client to access cloud models this is an astounding waste of money.

  • cyclopeanutopia

    yesterday at 6:07 PM

    Why not use rpi then?

    • happyopossum

      yesterday at 6:18 PM

      Easy integration with iCloud, mac native screen sharing/remote access, iMessage support, and a huge corpus of widely-adopted agents to pick from?

    • adastra22

      yesterday at 6:41 PM

      In my case, I dev on macOS. The env the agent runs in is the same as my dev laptop, configured and in sync. Has access to all the same tools and environment as I would on my laptop.

      • limewedge

        yesterday at 11:17 PM

        [dead]

• gchamonlive

  yesterday at 1:50 PM

  I think 24gb is the bare minimum for a local qwen3.8 based setup. I've used qwen3.6 and it's not as straightforward as "can it replace <insert the most cost-effective cloud solution today>"

  Local llms don't suffer from cloud availability issues. Anyone that used Google models know that sometimes they just don't have capacity whatsoever, at least that was the state of things some months back when I used them. Just bear in mind if needed, cloud providers will prioritise API and corporate customers over subscriptions if availability degrades more.

  Also they don't have the same guardrails as the other models, so for hacking, reverse engineering and black coding (piracy etc...) these local models might be the only options.

• julianlam

  yesterday at 2:05 PM

  16GB VRAM could load a small quantised qwen 27B model but it would be a ways away from a frontier cloud model.

  Though keep in mind not being beholden to shenanigans from said cloud companies (and interference from government entities!) is definitely worth something intangible.

• adamtaylor_13

  yesterday at 1:50 PM

  The principle of KISS keeps coming to mind when I consider local computing. I'm looking forward to the day we can just run Opus-level models at 100 tok/sec on consumer hardware.

  But currently it's really hard to beat anything offered by the cloud companies. And the cost and complexity of setting it all up, just to barely (if at all) touch on Opus-level intelligence makes it seem like we're not quite there for the common man (enthusiasts are a different story.)

  I am very excited for open source local models, and we're nearly there, but it's still too complex and expensive to be my daily driver (yet).

• scosman

  yesterday at 8:35 PM

  Right now sweet spot is voice transcription. Meeting recording apps are genuinely better locally than in cloud. Can run on an M1 easily. Latency matters. I built https://github.com/scosman/Biscotti and see zero reason to use cloud ever again.

  LLMs are harder: not much useful below 12B, and the 700B+ ones are really much better. Models like Qwen 3.8 27b show promise: in a few years pretty good local AI should be in reach for anyone willing to buy a $1000 computer (but who knows what your $20 sub buys you then).

• ololobus

  yesterday at 7:48 PM

  I was looking at $10k Mac Studio with M5 Ultra and 256 GB for local experiments, but then struggled to find what really good modern model I can fit into it. Yes, it can run a good dense 27B at Q8 with plenty of context, but what beyond that? IIUC, some Deepseek flash variants at Q4 are also feasible, but I am not sure if the quality will be good. They also don’t run that fast, like about 30 t/s

  So if I stay within 35B, especially MOE, my M5 Pro 64GB MBP can also run them well, and it can do plenty of other stuff too including gaming. While 256 GB with such RAM bandwidth and powerful GPU sounds like fun on paper, it doesn’t seem to be the next level compared to 64 GB

  Really curious what people run on 256 GB Macs

  • vablings

    yesterday at 7:50 PM

    I feel like for localAI t/s is less of an issue. Just make a PRD and run a ralph loop. For big slogging projects like reverse engineering, or converting a codebase to a new language it actually doesn't matter if it takes a day or seven days.

    • SMEbooop

      yesterday at 8:12 PM

      Yeah this is my experience. My 24GB 3090 + 64GB RAM takes a couple hours to crank out some code with largest Gemma 4 and Qwen3.8 models it can run

      But in the meantime I get dishes done, vacuum, flip laundry... etc etc

      Frontier models also seem in such a rush to emit anything they produce a mess that needs steering all day anyway

      While I have not tested it, it feels like my local setup going slower is better at producing code that works the first time as its not trying to look fast for marketing sake

  • vadansky

    yesterday at 7:50 PM

    Sounds like it's worth waiting for M7 anyways, no point investing too much right now

    https://news.ycombinator.com/item?id=48676795

    • Danox

      yesterday at 11:55 PM

      At least wait until someone gets one in hand and post a review, and if it does work decently well there probably is going to be a long backlog.

• gcoakes

  yesterday at 6:32 PM

  I have a RX 9070 also. I run llama-swap with a fill-in-middle 7B model, local 9B model, and it proxies up to OpenRouter for the bigger stuff. I think that's where the sweet spot is right now.

  I've spent $2 in the last 2 weeks on OpenRouter. I've been trying to only use the medium sized models that I would otherwise be able to run on a nice local setup. That nice local setup would cost ~$4k. I don't know what the operating cost would be, but I would be concerned that my home electricity would cost more than at a datacenter. It just doesn't make sense right now except for privacy reasons.

  I'm probably going to hoarde open weights models in the ~31B range until memory costs fall in a few years. Then, I'll buy some hardware to run at home just so I feel more sovereign over my stack regardless the cost/token speed.

  • BatFastard

    yesterday at 6:43 PM

    I made the same choice, aside from privacy concerns, you can not locally host a cost competitive model.

    But I am looking forward to lower hardware costs!

• N1ckFG

  yesterday at 4:06 PM

  In my experience so far, separately from privacy concerns there's a specific use case where cheap local shines--when unlimited shots on /goal with a dumber model is better than limited time with a smarter model. This looks less like the assisted-coding scenario that's commonly brought up as a good local scenario, because if you're searching for a fast and accurate solution to a single blocking problem, the bare minimum for a model that can do that is a 24GB dGPU or a 64GB Mac. Instead, this looks more like a Hermes agent on an Raspberry Pi driving OpenCode on an old gaming computer with just enough RAM and VRAM to handle an MoE, churning out something overnight that would quickly exhaust the subscription plans, like a knowledge graph for a large document corpus.

• codazoda

  yesterday at 6:59 PM

  In my experience they work well for some jobs.

  I recently built a minimal Dark Software Factory out of an N150 Mini PC. It uses three models; Sonnit, Sol, and Gemma.

  But, I have a LOT of instructions about how I prefer the software it builds. Gemma doesn’t handle all my instructions very well. But it’s close!

  I’m running gemma-4-12b because I have limited RAM and larger models were too slow.

  I do two types of jobs: planning and prototyping. It has done fine at some of my planning rounds.

  I still consider it experimental and don’t use it a lot but I think we’re getting there.

• lumost

  yesterday at 8:18 PM

  There are many enterprise environments where running modern models is... difficult. Rather than fighting security for months, a user could petition for a mac studio and have rough cost parity with a z.ai subscription.

  if they are a heavy user, perhaps they string 4x together.

• pletnes

  yesterday at 7:15 PM

  I’d be curious to use them on larger data sets. Log files, for instance. High volume, might be low value per line but not much cost per token if you already have a gpu to interpret them using a LLM.

• mlboss

  yesterday at 7:17 PM

  Data privacy and "unsafe" models are pretty valid reasons to use local models. If I want to generate violent images/text you cannot do that using cloud models.

• spacedcowboy

  yesterday at 1:50 PM

  I was getting semi-useful results from a 128GB M4 Max. That was a few months ago, and the models have improved (quite a bit) since then, but now I'm happy to send my $20/month to get Claude code.

  It's still frustrating as hell to come down in the morning, having given it a list of tasks to do overnight, with tests to pass before they're "done" and find that it worked for about 20 minutes after I went to bed, and decided that it would stop at "3am" (it wasn't) and "not do significant work this at this late hour". Like WTF ? You're an LLM. You don't sleep.

  Bloody training data full of humans demanding sleep. I tells ya...

  • scrumper

    yesterday at 2:08 PM

    > "not do significant work this at this late hour"

    Is this Claude code? Or your local? I assume Claude? I'm more than a little staggered by this, like, it makes no sense! It doesn't even serve Anthropic's interests (surely better for them if it burns your token quota so you have to buy more the next morning.) The LLM just... decided? I'd be so mad.

    WTF indeed. Can one even file bugs?

    • xienze

      yesterday at 2:17 PM

      > It doesn't even serve Anthropic's interests (surely better for them if it burns your token quota so you have to buy more the next morning.)

      Parent mentioned their $20/month subscription. It's definitely in Anthropic's interests for you to not use it.

  • xienze

    yesterday at 2:01 PM

    > I went to bed, and decided that it would stop at "3am" (it wasn't) and "not do significant work this at this late hour". Like WTF ? You're an LLM. You don't sleep.

    I think that's Anthropic trying to get you to not extract as much value out of that subsidized subscription as possible.

• fisle

  yesterday at 1:46 PM

  Could you elaborate on your experience with local models on your card? I've been thinking of upgrading to 9070 XT, and was thinking the 16GB would be okay-ish to at least run something usable locally, no?

  • fancyfredbot

    yesterday at 1:59 PM

    Usable certainly. But my impression is that useful models still need a bit more than 16GB. Something like Qwen 3.8 27B is useful but squeezing it into 16GB requires fairly aggressive quantisation which will make it unreliable (e.g it'll get stuck in loops) and won't leave enough space for a long context (which qwen 3.8 really likes)

  • Grombobulous

    yesterday at 3:29 PM

    I’m the parent of this thread, the person with the with the RX 9070.

    My understanding would be that if you’re interested in this sort of card for AI that you should go with the AI PRO R9700, which is basically the professional version of the RX 9070XT but with 32GB of memory.

    It’s significantly more money but not crazy like a 5090.

    I just happen to have the 9070XT primarily for gaming purposes.

    I’m not quite sure how to describe my experience using it other than “rudimentary,” and a lot of that is on me for not really understanding the best way to set it up.

  • villish

    yesterday at 2:30 PM

    If you have been using cloud hosted models, you will be severely disappointed with what you’d be able to run on 16GB VRAM. You will spend most of your time fighting with the model to fix its mistakes.

• tristor

  yesterday at 2:10 PM

  I've been experimenting with local models on an M5 Max MBP w/ 128GB of RAM since March of this year. Generally I've had very good results. Where things were lacking initially was with tool calling and the need to rely on tool calling for functionality like web search, which is otherwise well integrated in the cloud models. There is also a lot more work required on the harness side, however at this point (August 2026) there is not only much better tool calling in local models, but community supported projects have built good harnesses. Pi.dev and OpenCode + a SOTA local model is /very/ /very/ capable these days. Using LM Studio's built-in chat with a decent system prompt and proper tuning with local models is /very/ capable these days. Cloud models are still better, but it's the "harness" (not in the desktop app, but in the backend) that makes it so for the most part.

• xienze

  yesterday at 2:00 PM

  IMO local models require a substantial amount of prompt+harness engineering to get in the neighborhood of what you'd get from a cloud model. Which isn't a bad thing, you'll learn a whole lot about how these things work.

  What you'll learn pretty quickly from said engineering is that there's a lot more to a good LLM than just the weights themselves. You need a good search provider (also self-hostable, but sounds easier than it really is). You need (well, it's debatable) a memory system. You need a good system for up-to-date library references like a Context7 (also self-hostable but the options are surprisingly not that good). You need a good set of specialized subagents that can perform various tasks well -- for the sake of "doing things well" but also managing context efficiently.

  When you've got all that, local models can be _extremely_ useful. But there's one other important thing and that's decent hardware, unfortunately. A lot of people try out local models using small consumer GPUs or Macs and are rightfully unimpressed with the performance. And if the performance doesn't get them, usually they have expectations that they'll perform at Claude levels out of the box. Getting in that neighborhood, like I said, definitely requires some work.

  • Grombobulous

    yesterday at 3:37 PM

    What you’re describing is exactly what I’ve experienced in my time testing out this stuff locally, and I had a hard time figuring out what exactly to blame.

    I keep hoping that one day some comment is going to paste a link to some kind of idiot-proof guide or piece of software that’s “90% as good as Claude but running local.”

    And by 90% I don’t mean that the model is 90% as good or runs 90% as fast, more like all the other stuff you mentioned is set up out of the box.

• epolanski

  yesterday at 1:48 PM

  I have multiple 48GB friends that successfully run smaller quantized models for general assistance + light editing (coding, spreadsheets, etc), that don't require very heavy models.

  So yes, they are genuinely very useful, but they are not yet a full replacement unless you have more powerful hardware and or don't need more intelligent ai.

• froggertoaster

  yesterday at 7:33 PM

  You're limited by the manufacturer (CUDA is king, thus NVIDIA is the king right now) and your lack of VRAM will make using a useful model difficult.

  I'm not surprised at all.

  Context: I have a farm of DGX Sparks and several RTX 6000's, and can run very close to foundational models with ~2 sparks

• iLoveOncall

  yesterday at 6:06 PM

  I might be wrong but subscription models don't give you API access. I'm only interested in API access when it comes to personal use, so local models running for free makes sense for me.

  That said I have an RTX 5090, not a Mac Mini, so it's not exactly the same level of performance... The latest open models run at 200 tpm at around 30B params.

  • DaSHacka

    yesterday at 6:42 PM

    Only Anthropic does that AFAIK, at least I can use my $20/month Codex and Kimi subscriptions in pi.

• ivewonyoung

  yesterday at 3:23 PM

  > 16GB RTX 9070

  What's an RTX 9070? Do you mean the RX 9070 or RTX 5070?

  • Grombobulous

    yesterday at 3:28 PM

    Oops yeah I meant RX 9070, fixed it!

• Ajitesh-Rawat

  yesterday at 10:02 PM

  [flagged]

• anslopic

  yesterday at 6:08 PM

  [flagged]

  • adastra22

    yesterday at 6:16 PM

    That’s… not how any of this works. Five paragraphs and literally every one is wrong.

    • HNDevsSuck

      yesterday at 11:22 PM

      [flagged]

      • cindyllm

        yesterday at 11:34 PM

        [dead]

    • anslopic

      yesterday at 11:08 PM

      [flagged]

  • hhh

    yesterday at 6:14 PM

    this is insanely misleading, you can't run anything close to current chatgpt or gemini on local hardware

    • sharms

      yesterday at 7:01 PM

      I am running GLM 5.3 across 2x DGX Sparks and was doing comparisons and it absolutely can beat Gemini. Yesterday it corrected a poor Fable 5 response even

    • colingauvin

      yesterday at 6:24 PM

      GLM 5.3 Flash? Qwen 3.8 Flash Next? I believe those both are as good as the best Gemini, competitive with Terra.

      • notnullorvoid

        yesterday at 6:40 PM

        Yes they are quite good, but are not able to run on a 16GB RX 9070.

        Quantized Qwen 3.8 Flash Next could maybe run eventually on that card with a highly optimized inference engine that dynamically caches the hottest layer experts. Even then you run into some hard limits.

    • NoYouAre

      yesterday at 11:39 PM

      You are insanely non-technical then. Yes you can. Skill issue

      You probably pay $200/mo for text-to-text!

  • throwaway314155

    yesterday at 6:10 PM

    > You have good enough hardware to run good models comparable with Gemini and ChatGPT.

    That is at best misleading and at worst outright misinformation.

    • amazingamazing

      yesterday at 6:24 PM

      If you have 2TB of VRAM you can’t run one of the big models which are comparable?

      • adastra22

        yesterday at 6:57 PM

        The post was replying to someone with 16GB. (And also: no, even the best open weight models are not as good as what you can use on your ChatGPT subscription. They’ve gotten a lot better, but not that much better.)

        • thinkafter

          yesterday at 11:14 PM

          [flagged]

        • anslopic

          yesterday at 11:09 PM

          [flagged]

          • limewedge

            yesterday at 11:18 PM

            [flagged]

            • jdiff

              today at 12:11 AM

              How many of these sockpuppets are you going to create?

              • FoundARetard

                today at 2:09 AM

                [dead]

  • chis

    yesterday at 6:14 PM

    what are you smoking

    • anslopic

      today at 2:26 AM

      [dead]

  • anslopic

    yesterday at 9:06 PM

    [flagged]

It's fun to see that even an extremely large company can find unexpected product market fit [0]. Per this article, "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy." That sounds insane in retrospect, but I think there's just inherent uncertainty in what people actually need and will use things for.

[0]https://pmarchive.com/guide_to_startups_part4.html: "In a great market—a market with lots of real potential customers—the market pulls product out of the startup... The product doesn’t need to be great; it just has to basically work."

• yardie

  yesterday at 2:33 PM

  You should listen to the podcast Acquired, specifically Nvidia and then Jensen Huang. They basically lucked into AI. Some researcher was using Nvidia gaming cards, and reached out to them about questions on CUDA. That email eventually turned them into a trillion dollar question.

  • BeetleB

    yesterday at 8:32 PM

    What year are you talking about? When I was in grad school, around 2007, Nvidia was aggressively marketing GPUs for high performance computing. They would go to campuses, talk to professors, etc.

    Yes, the whole Deep Learning thing was luck, but as with most lucky things, they ensured they were positioned to capitalize on it.

    • cyberclimb

      yesterday at 11:21 PM

      Probably cerca 2014 as that's when AlexNet was released, demonstrating that neural networks could beat traditional ML models at image recognition tasks. I recall the researchers used Cuda to optimize their training setup.

      AlexNet kicked off a new wave of research around neural networks by demonstrating they could be scaled well and trained on GPUs.

      • BeetleB

        yesterday at 11:49 PM

        Yeah - definitely by 2014 they were well entrenched within academia with their CUDA offerings. By that point it wasn't "luck".

  • caycep

    yesterday at 6:34 PM

    to their credit, there was a lot of work behind "luck". Jensen showed up in person in 2017 in NEURIPS and he and likely a lot of his top brass basically sat down and read the entire conference proceedings/abstracts; there was likely a lot of work behind the scenes to behind the ML research pivot.

    • jldugger

      yesterday at 9:12 PM

      And 2017 was _late_ in their pivot. They'd been active for much, much longer. Last winter break I sat down to watch every GTC keynote, going back to 2009[1]. Even then, he's talking about expanding to non-graphics workloads. Google's GPU paper[2] just slotted naturally into their existing narrative and were happy to support it. "fortune favors the prepared" as they say.

      [1]: https://www.youtube.com/watch?v=fYuH2Kl_b98 [2]: https://scholar.google.com/citations?view_op=view_citation&h...

    • adastra22

      yesterday at 6:44 PM

      Yeah, The NVIDIA Way goes into a lot of detail on how and why the pivot from graphics to AI happened. This is a prime example of “you make your own luck.” Jensen engineered an organization that was primed to recognize and pounce on the next big thing, and it ended up being AI. But they saw it coming WAY in advance (like 2011/2012, not 2017) because they were explicitly on the lookout.

      • adolph

        yesterday at 7:09 PM

        > an organization that was primed to recognize and pounce on the next big thing

        e.g.: previous crypto hype-cycle

        https://www.pcgamer.com/nvidia-cmp-graphics-card-availabilit...

        • kridsdale1

          yesterday at 8:31 PM

          Crypto was a stupid fad, but Nvidia certainly made a lot of money, so being a vendor to a fad is not stupid.

          • adolph

            yesterday at 10:10 PM

            Which makes it a Exempli gratia of "an organization that was primed to recognize and pounce on the next big thing." In addition, they also recognized early on some of the weaknesses of crypto-mining as an industry and limited their exposure while making a pivot to the next thing.

    • georgeburdell

      yesterday at 7:44 PM

      AlexNet was 2012 and they explicitly called out the use of NVidia GPUs

  • edelbitter

    today at 1:07 AM

    I might have believe this story, if not at the same time Intel had made an expensive bet on producing not-quite-gaming cards, later looked at the same trillion dollar question.. and then almost decided that this did not bring enough luck to keep spending.

  • noosphr

    yesterday at 8:14 PM

    In 2006. The next 20 years of cuda support weren't luck, as anyone trying to use AMD will know.

• SaltyBackendGuy

  yesterday at 1:40 PM

  Maybe a bit of hindsight bias / the outside view here, but I feel like they're completely asleep if they didn't anticipate strong demand for this specific use case.

  • articulatepang

    yesterday at 3:08 PM

    I think a reasonable story could have been told that goes like this: local models aren’t as good as frontier models with a $20/month subscription, and the hardware costs a lot. So only a few enthusiasts will buy Apple machines for this purpose.

    This story turned out to be false but I think smart, reasonable people a couple years ago could have believed it with conviction. It doesn’t really seem like “completely asleep” to me.

  • scosman

    yesterday at 8:38 PM

    They were investing in ANE and Metal before everyone in consumer. Hardly asleep. They just underestimated the market size, as pretty much everyone did.

    • porkpieshoe

      yesterday at 9:15 PM

      [dead]

  • adjejmxbdjdn

    yesterday at 1:42 PM

    I don’t understand how that’s possible. They should have had a better idea of what was happening in the memory markets than pretty much any other entity.

    • Nevermark

      yesterday at 2:15 PM

      Their universal RAM strategy is so obviously helpful for AI. (1) GPU/NPU <--> CPU RAM copies eliminated. (2) All (most) RAM available for GPU/Neural, when local models are typically kneecapped by limited GPU RAM sizes vs. the much larger RAM options for M/Max/Pro/Ultras.

      They have been taking NPU's seriously on their phones, tablets and laptops since the M1.

      Then they enabled fully-connected RDMA for 4 x 512GB MacStudio's = 2TB RAM. Perfect for a large Mixture-of-Experts model.

      It would be very strange if they didn't notice their product line had landed in a new sweet spot.

      • pizzafeelsright

        yesterday at 2:35 PM

        I am curious about the corporate disconnect from the frontline to the generals.

        While the company I am in is embracing AI the disconnect and delay between what is available and possible versus what is approved and permitted is a three month window. The State employees I speak to are just now getting around to writing their usage policies for internal AI usage.

        • wolvoleo

          today at 12:48 AM

          Same here. For every little use of AI we have to fill out a 3 page proposal to get a PoC for 3 weeks in copilot studio (the worst AI builder around but due to MS lobby the others are banned). Then we have to find a business sponsor to decide it's useful, find 2 financial controllers to underwrite its token cost (even if very little), go through rounds of lifecycle and governance approvals. Then we can move to preprod but nooo we're not there yet. Now comes the security review and DPIA. That's another hugely complicated process that takes months. Eventually when both are done we can go the the deployment team and go through their process which I haven't even seen yet because most of our projects got cut off beforehand.

          All the while the top of the company is of course praising AI and saying we should do everything with it right away.

          It's a joke really. We had to go through all this rigmarole just for an agent that does some preliminaries on a service support ticket (making sure all data was filled in correctly and contacting the requester of not) before sending it to a human for final review. It couldn't action anything, not a single thing. Just assist with the prerequisites.

          And yet they call our company 'innovative'. We're certainly innovative at inventing bureaucracy.

      • bigyabai

        yesterday at 4:48 PM

        FWIW, the reported reason for OpenAI buying Macs has nothing to do with the memory by the sounds of it. Every single outlet I can find reporting on this seems to repeat that the intended use case is for agentic workloads and generating training data for reinforcement learning. They don't appear to be doing inference nor any sort of training AFAICT.

        • Nevermark

          yesterday at 5:56 PM

          For OpenAI, their data center archipelago is their own "local" and "personalized" AI.

  • FireBeyond

    yesterday at 7:43 PM

    Tim Cook has been touted as the greatest supply chain logistics person on the planet and revolutionizing Apple's product delivery, securing exclusive contracts years in advance, etc., etc.

    But "oops, we missed that people are interested in AI work on our machines" seems like a really fucking big myopia. But then again, Tim's off to retire on a bed made of cash this week, so...

• xattt

  yesterday at 1:40 PM

  Was this the case in the past?

  My vibes were that Apple wound down the “actual work” side of their operations (including machines like Xserve), because Ives couldn’t handle the unsexiness and unpredictability of business requirements in hardware.

  He was self-indulgent and only wanted to work on things that “vibed” with him, rather than what the customers needed. It’s easy to be creative when you get to do what you want to do, it’s hard when you have hard constraints.

  • tonyedgecombe

    yesterday at 2:01 PM

    I think Jobs was quite sceptical about courting enterprises. Personally this is one of the reasons I choose Apple over Microsoft.

  • giantrobot

    today at 1:01 AM

    Apple has not in the past three decades really courted the capital E Enterprise market. They'll definitely sell to Enterprise customers and have Enterprise sales teams for big customers. But they're not and never have been Dell or HP.

    Enterprise sales sucks. There's infinite amounts of politicking and glad handing and buyers will get all sorts of sweet brib..."sales dinners" then go with the cheapest option. Margins on hardware sucks and the only money is in support contracts. Apple instead invests in consumer sales/support primarily and all the other channels are side businesses.

    Stuff like the Xserve existed mostly for Apple internal purposes and ended up being sold externally to goose the scale enough to make them not a huge loss. At one point a large percentage of the offices on Bubb road were packed with Xserves running portions of the iTunes Music Store and the Apple online store. More offices were packed with Xserves doing media ingest and encoding for iTMS. Just about every building had racks of them as build and file servers.

• DannyBee

  yesterday at 1:41 PM

  It's also fun to see how many people here believed this was all some clear deliberate strategy in the first place rather than an accident.

  • GeekyBear

    yesterday at 8:27 PM

    They didn't "accidentally" add tensor units to the GPU cores in the M5 generation.

    However, I don't think they expected the level of Enterprise interest they saw.

• 1over137

  yesterday at 1:48 PM

  No ‘staff focused on developer relations’ is entirely unsurprising based on what I see from the outside.

  • mercutio2

    yesterday at 10:17 PM

    That raw statement is completely and totally false.

    “Not as fully staffed as some people might hope” or “Developer Relations isn’t as responsive as I’d like” are both at least not obviously false.

• ghostly_s

  yesterday at 9:22 PM

  > "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy"

  This is clearly a mis-statement, they have a whole annual conference for developers. Maybe they mean specifically AI devs.

Slightly tangential question: Does having a spruced up Mac Studio help learning the ropes of model tuning and/or development as good as having a CUDA device?

Mac Mini's were really nice HTPC candidates, too, before the AI boom. Like all things genuinely useful and affordable, they were snatched from the hands of normal consumers by a bunch of schmucks chasing the latest gold rush.

• ThreeFinger

  yesterday at 7:10 PM

  Isn’t a Mac mini annoying to use as an HTPC? You have to deal with a remote, software, and a full OS, compared with an Apple TV, which has a good remote and is optimized for TV use.

  • reaperducer

    yesterday at 7:42 PM

    Isn’t a Mac mini annoying to use as an HTPC? You have to deal with a remote, software, and a full OS, compared with an Apple TV, which has a good remote and is optimized for TV use.

    I've been using one for about a decade as a media server.

    It just sits in the cabinet happily running the macOS TV program with the video files on an external hard drive. Playback on the TV is handled by the AppleTV's built-in Computer app. Works beautifully.

    I have more movies and TV shows on that box than I could watch in my lifetime — a combination of ripped DVDs (Netflix, public library, and purchased) and OTA recordings.

    When the cable goes out in my neighborhood (frequently), or a big storm screws up satellite reception (seasonally), I just don't care because I'm all localhost. As long as the lights stay on, everything is fine.

    No ads. No privacy violation. No fees. No bandwidth congestion. No buffering. No subscription rate increases. All I pay for is electricity.

• procgen

  yesterday at 6:40 PM

  Our Blessed Homeland / Their Barbarous Wastes

• scrumper

  yesterday at 2:11 PM

  I need a new little Mac for my music studio, currently an M2 MacBook Pro. I thought I'd be fun to experiment with some local models as well. Well, let's price up an M5 Pro. $3,019 with 64GB RAM and a 1TB HD. Three thousand American dollars for a Mac Mini. Beefy spec for sure but not comically so.

  Frankly even the entry price is a bit high - I remember buying one for my son a few years ago (M1 mini) and it was a few hundred; now we're up to $900 for the base model.

  • flyingshelf

    yesterday at 7:24 PM

    Kinda sad that in 2026 1TB and 62GB of RAM is considered "beefy". We had 1TB iPhones for 5 years.

    The only reason for this huge speedbump is that chip makers have been dragging their feet for the last 10 years with "just enough" memory.

    • tom_

      today at 12:12 AM

      How long have we had 15-core iPhones with 64 GB RAM and active cooling though?

  • alasdair_

    yesterday at 7:45 PM

    I bought a 5090 a year an a half ago for $2000. The same card, now a year and a half older, is $4000. Then there is the RAM - I bought 96GB, wishing it was 128, and now the price on my old RAM has doubled.

    Stuff is crazy expensive.

    • embedding-shape

      yesterday at 9:27 PM

      > Then there is the RAM - I bought 96GB, wishing it was 128, and now the price on my old RAM has doubled.

      I also bought 96GB some while ago but after the initial increases, thinking I'll wait it out. Now 128GB is far more expensive than it was when I first looked. Luck has it I want DDR5 RDIMM as well, which seems the hardest hit when it comes to RAM prices, fun stuff.

  • intrasight

    yesterday at 7:49 PM

    The first Mac, which had 128k of ram and an 8mhz CPU and cost more than $6500 in today's dollars. Humans are spoiled.

  • cma

    yesterday at 10:20 PM

    Isn't OP complaining that the price went up because people want them for AI use? I guess it also went up because of cloud AI use increasing component costs, but those local models you want to run probably were trained in the cloud..

  • happyopossum

    yesterday at 6:25 PM

    > I remember buying one for my son a few years ago (M1 mini) and it was a few hundred; now we're up to $900 for the base model.

    The base price of a mini has only gone up $200 from $699 in 2020 to $899 today, and for $699 you only got 8GB of RAM instead of 16. Yeah the price has gone up but not nearly as much as you seem to be remembering...

    [0] https://theapplewiki.com/wiki/Mac_mini_(M1,_2020)

• neverrroot

  yesterday at 8:46 PM

  This is just so incredibly disrespectful to so many people.

  • hananova

    yesterday at 9:41 PM

    Yes, sociopathic tech bros making everything insanely expensive for regular people is indeed incredibly disrespectful!

    • qeternity

      yesterday at 10:23 PM

      You're not entitled to this stuff any more than they are.

      What planet do you live on?

Totally saw this coming. My team just grabbed three Mac Studios for local LLMs; the unified memory is a game-changer.

I really hope with Ternus taking the helm Apple starts to remember that it has products outside of iPhone.

• catoc

  yesterday at 2:51 PM

  I wish they would stop with the new-iPhone-every-year nonsense and refocus on quality, fix some bugs… but yeah, not gonna happen

  • stetrain

    yesterday at 6:09 PM

    At this point it's basically like car models years. They rev them annually and make a few improvements but they're actually going longer between major redesigns. If they didn't bump the numbers every year people walking into stores would be worried about paying lots of money for an old phone that will soon be replaced by a much newer model.

  • Sir_Twist

    yesterday at 6:16 PM

    I feel like iOS 27 is a step in this direction, in terms of sanding off the rough edges of iOS 26.

    • catoc

      yesterday at 7:20 PM

      Wish they would sand off liquid glass

      • kridsdale1

        yesterday at 8:33 PM

        That would make it frosted glass.

        Incidentally ios27 has a user control slider for glass frostiness.

Not just the high end stuff. The Neo is sold out until late September on the budget end, it seems like it is a smash for HS and college kids.

I hope Apple can take all this cash and do some stability releases like they used to do, bugs around things like Family Sharing, the painful "update" to Settings App, etc could all use a lot of love.

• alistairSH

  yesterday at 7:35 PM

  Huh, glad I grabbed my Neo two weeks ago. It's the "top" spec version, but still a good bit less than a MBA - seemed like a pretty reasonable replacement for the M1 iPadPro it replaced (wanted to go back to a normal laptop vs tablet).

What’s going on isn’t Apple behind in AI model building I thought I read that somewhere on the MacRumors site in the last two years, that Apple is behind its tech peers and Apple might as well just close the doors.

I always thought Apple was in a good position because unlike their peers they didn’t burn billions of dollars trying to build an AI model that has no financial moat around it.

I still think they’re in a good position in comparison to their tech peers and we will know even more when some of the new computers get into the hands of some of the tech reviewers.

I believe the new computer’s will be pretty good hardware wise what I’m interested in, is the Apple software support for connecting several Mac computers together, and some of the other (new?) software that Apple may have written in house to support those who want to run AI software locally that is just as important as the new hardware.

• rglover

  yesterday at 11:12 PM

  They're making the smartest possible move: let others burn insane amounts of capital and time finding the quirks and once they see a viable lane, execute.

  It's old Steve Jobs logic. Works backwards from the customer experience to the technology (they're the only big player I see doing this).

  • pertymcpert

    today at 1:56 AM

    I feel like they failed upwards into the right strategy. I doubt it was deliberate and not because of the failure of their foundation models to decent.

Apple trying to sit on two chairs. Make bank selling hypeware while keeping distance to not sully their brand too much.

Most of those people will be dissapointed when they experience Q4 variants of those models getting stuck in loops.

I would wait till the ram crisis is over to fetch a future 64gb ram gpu to run Q8 models. Cloud inference until than.

• jms703

  yesterday at 11:45 PM

  I'm excited to hear the ram crisis will be over. But will it?

• etoxin

  yesterday at 10:44 PM

  Switching from Q4 to Q8 was a game changer when I upgraded

No they weren’t.

• stetrain

  yesterday at 6:13 PM

  The part where people starting buying Mac minis just to run OpenClaw was a pretty sudden movement in the market. These computers aren't powerful enough to run big local LLMs but are still being purchased for AI workflows.

  A year ago you could get an M4 Mac mini for $399 on sale and now the same one used goes for over $700. The general AI RAM/SSD spike is part of that but there was also a huge demand spike for small, powerful, desktop machines that could be easily configured with these workflow tools.

• nullbio

  today at 1:03 AM

  Sad that Apple is stooping to the level of spamming the web with bots and fake news to advertise their products. Are there laws against this sort of thing?

Time to bring back the Xserves, I guess.

If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8.

It's true, most people don't run models, but being the default platform for running open weights seems like it has plenty of advantages right now. Just like sales benefited from developers defaulting to MacOS for most open source languages like Ruby, Go, Rust, and TypeScript.

• mirekrusin

  yesterday at 2:05 PM

  32GB is not enough, it's unified/shared memory, you need to have space for usual system and user apps/services.

  64GB+ or dedicated 48GB (2x24 on GPUs) is IMHO absolute minimum.

  • redox99

    yesterday at 7:50 PM

    32GB of fast unified memory is enough for Qwen 3.8 27B.

    - 16GB for the weights at Q4

    - 9GB for the full 256K context at Q8

    - 7GB spare for overhead and system.

    The problem is that these Macs have 32GB of slow unified memory.

    Edit: I'm thinking of a headless Mac mini, if you meant running it on the same machine you're using of course you'll need more memory, but LLMs are best served from a headless server so that's what I'd recommend.

    • hawk_

      yesterday at 9:21 PM

      Is this for setup for agentic coding? Why not also run the IDE compiler etc... on the same machine to use those CPU cores as well?

      • 0x457

        yesterday at 11:00 PM

        Keep in mind that if you want MTP it adds a few gigs. If you use sub-agents it turns already slow generation into even slower generation. Won't be doing any compling (so rust, c and probably go are not avaiable) becase those add memory pressure during compiling.

        32gb of unified memory is enough enough for system to be used for anything other than LLM generation.

      • redox99

        yesterday at 10:04 PM

        You can, you just need a beefier PC, and it's more annoying in terms of noise and heat vs throwing something on your server closet. Plus you don't need to worry about other software stealing resources and whatnot.

    • qeternity

      yesterday at 10:27 PM

      > Edit: I'm thinking of a headless Mac mini, if you meant running it on the same machine you're using of course you'll need more memory, but LLMs are best served from a headless server so that's what I'd recommend.

      What? LLMs are best served from a massive PD disaggregated cluster of B300s connected via NVLink.

      If you're running LLMs on a Mac Mini, it's because you want to run local, not because it's the best setup.

• shagie

  yesterday at 6:59 PM

  > If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8.

  https://www.canirun.ai (five months ago: https://news.ycombinator.com/item?id=47363754 377 comments)

• tristor

  yesterday at 2:03 PM

  > If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8.

  32GB is not enough RAM. I don't even own a device with less than 36GB at this point, and that device I only have because my employer is being cheap. 64GB is a reasonable starting point for running local LLMs + normal tasks. 128GB let's you really run most smaller models like Qwen 27B and 35BA3B with good context. Even Qwen3.8-Flash-Next runs in 128GB with a 4-bit quant.

  32GB would be limited to running models like Gemma4 12B and smaller dense Qwen versions like 9B unless you were using very small quants which damages quality of response.

  • Xeoncross

    yesterday at 5:15 PM

    You are mistaken. I'm running Qwen 3.7 28B 4bit (MLX) with a 200k context window and everything total is 32GB RSS.

    Is this the best? No. That's why I said the sweet spot. Getting from 16GB macs to 32GB is perhaps possible. Jumping to 64GB or 128GB as the default is simply unreasonable right now.

    • e28eta

      yesterday at 6:32 PM

      I have a similar machine, and briefly poked at running a local LLM, but got discouraged after a couple days. The quality, responsiveness, and impact on the rest of the system didn’t seem worth it to me.

      What sorts of things are you doing with the local LLM? Anything interactive? Should I take another look?

      • Xeoncross

        yesterday at 9:08 PM

        Yes, 15-30 t/sec is pretty slow for local models so I recommend running local LLM tasks overnight where (vs paid plans) there isn't a risk of chewing through your token budget from a rogue loop or sub-agent. Even if it takes hours, you're sleeping anyway so no concern. herdr + pi works great for this but there are lots of harnesses.

    • Foobar8568

      yesterday at 6:54 PM

      Memory used : 38GB, and I haven't even started a LLM nor podman, I always fight with memory when using LLM on my mac with 48gb.

      And I don't remember to have been able to have pushed to 200k context Qwen 3.6. 3.8 is running on my RTX 5090.

      • redox99

        yesterday at 7:59 PM

        Qwen 27B runs very comfortably on a 5090. You need to use Q4 quants and Q8 KV cache. Here's the math

        https://news.ycombinator.com/item?id=49514141

    • tristor

      yesterday at 6:49 PM

      I assume you mean Qwen 3.8-27B? Yes, you can run this in 32GB of RAM, but it's very context limited. With KV cache compression and other techniques, it's better now than in the past, but I'd still want more RAM, personally.

      EDIT to add that you need to reserve 8GB for the system if you don't want to cause problems on macOS, which means 32GB RAM = 24GB max for model + context. It takes 18-19GB to load a 4-bit quant of Qwen3.8-27B, so I'd be really surprised if you can actually get a 200k context window. You need to fit within a 24GB WSS (which is generally a more constrained RSS) to get stable performance on 32GB RAM.

      • jckahn

        yesterday at 9:26 PM

        I run Qwen 3.8 27B just fine on my Mac mini M4 24GB. I use Unsloth's Q3 XXS with 128k context. It successfully completes long horizon tasks with OpenCode.

More customers is generally a good problem to have in most businesses. Just that the situation is very paradoxical given the supply shortages.

• pmontra

  yesterday at 7:40 PM

  If those customers are in the market you want to develop. If they are not, money is money but if it comes from the wrong people it might slow you down.

I get the impression they want AI marketing points but don’t actually want people to use local AI on their products.

I have no idea why. They could be so successful if they leaned into local AI.

• m463

  yesterday at 11:40 PM

  I don't think they're ready for local ai. They have memory + memory bandwidth, that's it.

  I also don't think they do "technology". For example, containers have been around for a long time, and apple didn't show up. (I know they have some support now). Imagine an apple-native docker/podman doing something like FROM macos:10.12

  I was actually surprised when they did their own chips. I figure it was about control.

• jonplackett

  yesterday at 11:33 PM

  I think they’re too scared to ‘own’ it - it would be someone else’s model and potential security issue.

  But apple have to own everything they do so they’re in a bind

> Apple's unusually timed announcement of new Mac mini and Mac Studio models this week was driven by unexpectedly strong enterprise appetite for AI hardware, according to The Information.

Obviously; no one else can justify the expense.

Maybe it's not anything specific to Apple? There's high demand and short supply elsewhere due to AI, so it doesn't seem all that odd that many companies would try to buy gear from Apple too.

So for people who don't understand, there are two markets for Apple hardware in this space:

1. Running an agent like OpenClaude. The $599 Mac Mini was an insanely good deal for this. I happened to buy a M5 Pro Mac Mini for $999 last year for other reasons. The equivalent is now almost $2000; and

2. Hardware for running inference on local models. This to me is the far more interesting market because Apple has a real opportunity to disrupt NVidia's stranglehold on the market.

With current architecture, the largest model you can reasonbly run is the amount of memory on the GPU and is a function of the quantization (eg int4, int8, fp8, fp16, etc) available and the number of parameters. NVidia aggressively segments the market. The most VRAM on a "consumer" card is 32GB on the 5090, which allows you to run ~31B parameter models.

In comparison, the RTX 6000 Pro has only slightly more CUDA units than a 5090 but has 80GB of VRAM. A few months ago they were $10-11k. Now they're ~$16k.

Macs use a shared memory architecture. Apple has previously sold Mac Studios with up to 512GB of RAM. Almost all of that memory can be used to hold much larger models without taking a penalty for interconnections between different GPUs or machines. Plus Apple interconnects between computers are actually relatively good by chaining TB5. It's still slow but it's about the best non-enterprise option available.

But the previous Mac Studios just didn't have the raw FLOPS and memory bandwidth. The M5 Ultras are up to 1.2TB/s of memory bandwidth. M3 Ultra had ~900GB/s. RTX 5090s and RTX 6000 Pros are 1.8TB/s. The current best HBM3 NVidia DC GPUs are at 3.2TB/s IIRC. But the M5 Ultra has a claimed ~4.5x the FLOPS of the M3 Ultra.

We don't have our hands on these yet but it probably means they are going to be much closer to a 5090. I expect ~50% of a 5090's inference speed. That may sound bad but it's actually really good because a 256/512GB Mac Studio can probably locally run the best Flash models. With NVidia hardware you'll need to spend many tens of thousands for that.

We'll see what the inference speed is but I expect it to be usable. DeepSeek v4 Flash, for example, will be entirely runnable. We're not at DeepSeek v4 Pro local yet.

• jubilanti

  yesterday at 8:37 PM

  > 1. Running an agent like OpenClaude. The $599 Mac Mini was an insanely good deal for this.

  I still have zero clue how "Buy a $599 Mac Mini to have a sandboxed LLM API caller" became the default. If you're not doing local inference and don't need to inject into iMessage or iCloud, all you need to run openclaw-style harnesses that call external APIs is a Raspberry Pi 4B, an N100, an HTPC, or that 10 year old laptop sitting in your desk.

  • nullbio

    today at 1:05 AM

    Apple has a huge marketing budget, and evidently, they are not beyond using unethical tactics to sell their products. That's how it started.

  • MaxikCZ

    today at 12:29 AM

    People bought the mac mini and then spent more money to have someone install openclaw for them, it was wild.

• subarctic

  yesterday at 7:53 PM

  You have the m4 pro right? I thought the m5 pro mac mini was only just announced

Sounds like people want those bespoke servers that Apple has been rumored to have developed.

Classic monopoly move: Control the user base, then control hardware. Any decent always-on local LLM setup with Apple devices will have to compete with these behemoths now. Great.

• GeekyBear

  yesterday at 8:13 PM

  > Classic monopoly move: Control the user base, then control hardware

  Classic monopoly move by who?

  Apple created MLX as an open source framework to allow users to run any open model locally.

  • bel8

    yesterday at 8:19 PM

    just to clarify, models already ran locally without MLX years before it existed, on non-Apple environments.

    MLX was just Apple's bridge to what already ran in other hardware.

    • GeekyBear

      yesterday at 8:52 PM

      MLX is an open source framework that allows you to run open models in a manner optimized for Apple's hardware.

      You can take advantage of larger amounts of memory, higher memory bandwidth, and clustering multiple systems.

      How is using an open source framework to run open models a monopoly move?

    • Danox

      today at 12:24 AM

      Apple Pay is a bridge too because at the time the other pay services didn’t care about doing anything for the Mac, and over the years that’s the way it’s been that’s the way it goes when you’re the last vertical computer company left from the 1980s, the Apple Watch and Apple Maps is also a bridge for the same reason…

I wish they sold something that could go in a colo - redundant power supplies, lights out management, etc. you know they have them internally...

• dewey

  yesterday at 1:25 PM

  > you know they have them internally...

  What makes you think that? There's a lot of data centers that sell you access to colocated Mac Mini's, they have added FileVault unlock via SSH in the boot process which also makes things easier. There's not that many reasons to run a Mac in the cloud unless you have some very specific Mac related workload.

  • giancarlostoro

    yesterday at 1:37 PM

    Because there's been photos of Apple building server racks with Apple Silicon, but also a recent leak.

    https://www.macrumors.com/2026/08/26/leaked-images-of-apple-...

    https://www.reuters.com/business/apple-begins-shipping-ai-se...

  • scrlk

    yesterday at 1:34 PM

    Apple built internal M5 servers for private cloud compute:

    https://wccftech.com/apples-private-cloud-compute-server-m5-...

    https://security.apple.com/blog/private-cloud-compute/

    • dewey

      yesterday at 1:50 PM

      Thanks, missed that article.

  • N_A_T_E

    yesterday at 1:39 PM

    At this point it’s a well known secret that Apple has real rack mount servers for their internal processes. They actually have officially released video of their servers in the WSJ report on their chip supply chain.

    https://forums.macrumors.com/threads/photos-of-apples-own-ne...

    • reaperducer

      yesterday at 7:53 PM

      it’s a well known secret that Apple has real rack mount servers for their internal processes

      Only if by "secret" you mean "announced in multiple press releases and a public event with federal, state, and local officials at its new sever factory in Houston."

      https://www.apple.com/newsroom/2026/08/apple-opens-advanced-...

  • ndiddy

    yesterday at 1:35 PM

    > What makes you think that?

    There’s articles about them, Apple uses them internally for AI services. https://forums.macrumors.com/threads/photos-of-apples-own-ne...

  • unrented7977

    yesterday at 2:09 PM

    Apple has to have significant build infrastructure to support internal iOS development, surely? They can't just be using whatever is at the developers' desk, or a big pile of Mac minis in a closet. That's far too pedestrian for Apple internal works.

    Plus they did sell rackmount servers for some time.

  • shepmaster

    yesterday at 1:36 PM

    > What makes you think that?

    Here's a leaked / rumor image of Apple servers themselves.

    https://www.macrumors.com/2026/08/26/leaked-images-of-apple-...

  • abtinf

    yesterday at 1:38 PM

    > [cites a convoluted work around]

    > [still claims there is no reason]

• detourdog

  yesterday at 1:38 PM

  They did. Now think they feel a stack on Minis or Studios fills the reduce needs better. The multiple machines one gets software redundancy in addition to everything else.

This sounds like advertisement, disguised as an "article".

I hope Apple does not gain some exclusive enterprise tier for hardware. Part of what I love about them is that everything is available to consumers. A lowly home user can buy the exact same 256 (or 512) gigabytes of memory in a Mac from Apple, as long as they have a couple dozen thousand dollars to spare. I'd be really sad to lose that.

is this a manufactured demand meme

• nullbio

  today at 1:06 AM

  That's exactly what it is.

just a real bummer that they raised the prices so much

• Danox

  today at 1:15 AM

  Which is why Apple is going to need to design around again and do something about that in house if you can design and engineer a processor or a modem you can certainly do something about memory and SSD’s, Apple buying PA Semi, Intrinsity and Anobit led to Apple Silicon, buying Infineon led to a new Apple modem the capabilities is there and the money is there in house it may take two-four years but long-term I don’t think there is any other choice.

  One thing to watch for when Apple introduces the new phones coming up shortly is whether or not Apple has replaced Qualcomm in their flagship smart phones because that is coming up soon Qualcomm has given warning to their investors.

• snarkyturtle

  yesterday at 1:18 PM

  It definitely puts it out of the range of every day users but a non-insignificant proportion of people who use it for ai have become multi-millionaires because of ai. So there's definitely no shortage of people who have no problem paying those high prices.

  • bigyabai

    yesterday at 4:55 PM

    > but a non-insignificant proportion of people who use it for ai have become multi-millionaires because of ai

    I can't even name one person who fits this mold, let alone a non-insignificant proportion of people. Who are you thinking of?

  • noman-land

    yesterday at 1:21 PM

    Citation needed.

• AlexandrB

  yesterday at 1:17 PM

  I suspect it can't be helped at this point. RAM is the new gold. Valve even had to increase the price for a 4 year old piece of hardware[1].

  [1] https://tech-insider.org/ca/steam-deck-price-increase-2026/

• lenerdenator

  yesterday at 1:21 PM

  [flagged]

  • DarmokTanagra

    yesterday at 1:45 PM

    They use the datacenters for that.

    • lenerdenator

      yesterday at 7:28 PM

      No, that's for generating the hot air used to help inflate their egos. Heating is a very separate line item on the budget.

      • DarmokTanagra

        yesterday at 9:42 PM

        Ah you're right, I have a hard time distinguishing the difference between hot air and SV dev farts.

• VCFundedGenYer

  yesterday at 2:36 PM

  I don't know if you're young, or new to this industry, or what, but take a look around - the prices went up across the board. It's not just Apple.

Translation: Tim Cook was caught off guard.

Imagine if Apple truly went the route of local AI and every Mac came with a fully local, open weight Siri. They could put Anthropic and OpenAI out of business.

• dhosek

  yesterday at 10:52 PM

  Presumably this is on their roadmap. The Jobs-era thing would be to come in with something that people on HN would say, “yeah, I could do this myself with open source tools” but in practice falls into the category of “I could build this $30 thing for $15 for parts and then another $15 for the parts I had to replace because I screwed up the build the first time and then $30 to just buy the damn thing” that would really blow non-HN people away even if it is more expensive than most other local computing systems, but I don’t know what the Ternus-era Apple will be like.

  • Danox

    today at 12:44 AM

    Apple is almost certainly building the software end of it to go along with the computers that they currently sell to the public they have the time to do it right because their biggest competition in that area would be Microsoft, but Microsoft is currently listing at sea with Copilot, Xbox, and the Surface line of computers.

    Memory probably is the biggest holdup/obstacle at this moment.

I imagine Apple could also grow their business in the EU by marketing to companies who want powerful AI features but can't leverage 3rd party AI services due to GDPR. If only they could come up with iOS, App Store, and developer policies that respect the sovereignty of the EU.

Apple has a huge opportunity here to lead the market for machines to run local models if they step into it. Their stuff is already better than what nVidia is offering with stuff like the DGX Spark.

It's a niche market but it's a market that overlaps heavily with professionals in the AI space and lead developers, so it's a market that gets them customers in those roles.

If I were running Apple I'd call the RAM price bubble for what it is and temporarily eat some margin to offer machines with more RAM than competitors, especially these models that are great for edge AI, and capture market share.

• mwcampbell

  today at 3:33 AM

  > Their stuff is already better than what nVidia is offering with stuff like the DGX Spark.

  How so? In tokens per second when running major open-weights models, or something else?

• Danox

  today at 12:50 AM

  Apple isn’t the company that eats margins but they are company that would design around the problem and I think that’s what they will do after all, they have the design and engineering and plenty of money because they didn’t burn it on AI models or data centers.

• bigyabai

  yesterday at 4:50 PM

  Apple doesn't design GPUs on-par with Nvidia's efficiency yet. They need an architectural overhaul to be a serious competitor, which is what I'm expecting is queued up for M7.

  Nvidia has CUDA, AMD has CDNA, and Apple has... compute shaders, I guess?

  • Danox

    today at 12:58 AM

    Probably two generations away. I’m more interested in how much uplift/speed and more importantly what is the power usage is required for the new computers Apple is shipping particularly for the Studio versions.

  • nicce

    yesterday at 6:35 PM

    > Apple doesn't design GPUs on-par with Nvidia's efficiency yet

    How much it matters in inference? Most GPUs have enough computing for that and the bottleneck is the RAM speed and size. And M5 Ultra is becoming to challenge this.

    • bigyabai

      yesterday at 7:56 PM

      For prefill, it's basically all that matters. Long-horizon agent tasks, session compaction, file reads and context manipulation will all hit the compute bottleneck in regular usage, incurring several minutes of latency on most Apple Silicon chips, regardless of RAM.

      It's kinda why memory bandwidth is an enormous red herring, even for datacenter applications. Nvidia's huge advantage is a compute-optimized GPU architecture and their Infiniband networking, their memory controllers aren't really the star of the show.

  • newsclues

    today at 12:32 AM

    Apple has metal and mlx

Were they really caught off guard or is it a long-term play by Apple who knows that consumers may just want machines capable of local models. Build-or-buy (subscribe) options. I for one would get a Mac Studio over a DGX Spark because you get a general Mac machine as a bonus. The big news is Apple being caught off guard by Nvidia buying Hugging Face. That should have been Apple's.

Off guard? :'D

• throw9399383838

  yesterday at 1:38 PM

  Apple did not raise prices fast enough

There are so many used M4 Mac Minis for sale on marketplace for $600+ now that the people I guess got bored of the local models and decided to see if they could make their money back as they probably bought them when they were $399 on sale earlier this year. I lowball them every time.

• chung8123

  yesterday at 6:16 PM

  Have you been successful with the lowball offers? That would help indicate where the supply/demand for them are.

• mixdup

  yesterday at 6:15 PM

  When the M6 Minis were announced I pre-ordered and Apple gave me $480 trade-in on my entry level M4 Mini that I paid $499 for about 14 months ago

• okdood64

  yesterday at 8:51 PM

  > I lowball them every time.

  And?

This may sound a little wacky, but one potential use case I'm considering is robotics. Say you want to use a fine-tuned mid-size VLM model right on the robot. You pretty much have to use the Jetson line, which has a great ecosystem and everything (depth cameras, lidar, SLAM, small segmentation models, etc) runs on it, but it's gets very expensive very fast if you want to run LLMs on it. The Jetson Thor lines are 3-5k depending on memory etc.

One very efficient option today is to have the cheapest Jetson (Orin Nano) run the classical robotics stack, then have a base mac mini run nothing but the VLM. The Mac mini is considerably cheaper and faster at these workloads than the mid-range Jetsons.

I think this wonky situation is because Apple us under immense consumer pressure to absorb the ridiculous memory prices, while the Jetson is aimed at "business" and much more likely to fluctuate with the market. Last year I bought a Jetson Orin Nano 8GB for $375CAD, today that official nVidia Amazon page is out of stock and other sellers have it listed for $900-$1100CAD. Absolutely bonkers pricing.

This is the distinguishing angle Apple can take with AI.

Local inference solves so many of the privacy and inconsistency problems with these frontier subscriptions.

• nobodyandproud

  yesterday at 10:59 PM

  Interesting downvote.

Now if only they hadn't discontinued the Mac Pro. Could be quite the AI machine with multiple compute GPUs at higher bandwidth than an external Thunderbolt enclosure.

It would be nice if I could upgrade my 16GB M2 MacBook Pro some time in this lifetime. This AI bullshit is getting annoying.

[dead]

[dead]

I am missing something in the article. From what I am reading, AI companies are so desperate for memory they are buying Apple Systems and other hardware and striping them for RAM and maybe other components.

Is that what others read ?

• Leftium

  yesterday at 2:06 PM

  Apple hardware uses a unified architecture. That means the CPU and RAM are integrated together: it is very difficult/impossible to strip the RAM from these systems.

  This unified architecture makes Apple hardware very good for AI work, where latency between RAM and CPU is very important

  Even the SSDs (which are normally more strippable) use a proprietary hardware form factor.

  I think the article mentioned the real draw:

  > Apple noticeably promoted the ability to link multiple Mac Studios together into a single, more capable system for running large frontier AI models, a feature aimed at business and developer customers rather than everyday consumers.

  (Couldn't read the source cited, might have more info: The Information)