I decided to take the plunge and get myself four sparks at a decent price (and bought the QSFP cables from AliExpress because they are literally 1/2 the price of Amazon), even knowing Apple was going to release new hardware and there's probably a spark 2 on the horizon. It looks like this is going to be a decent fit for what I need. I've been experimenting with a two-node DS4 and it's _good_ at some tasks, but it really just spins its wheels when it hits the limit of what it can reason through.
I can offload mundane/basic tasks to DS4 on two sparks, but I've been pushing it harder on some novel work and it just can't run on its own at all beyond a certain complexity level.
I would love to see an Opus-4.8-level local model but TBH I just haven't got there yet. The models I've tried so far _are_ good but they aren't able to solve tough technical challenges, regardless of harness/prompting/etc.
comandillos
I am surprised. I've been using DS4 Flash (0731) for weeks now and it works perfectly fine as a replacement for Claude in a large variety of cases. It requires a few more iterations, sure, but it's useful enough to not need a Claude subscription anymore. Among the things I do I've been reverse engineering, writing complex C++ code...
badatnames
DS4 Flash absolutely kicks ass for reverse engineering and bug hunting. Almost no point in considering paying for a bigger model, although it's possible the stuff I've fed it (wide variety of older DOS/Windows stuff and device firmwares) might be easier targets.
comandillos
I've been reverse engineering LEON3-FT SPARC v8 BE code, so I wouldn't say it's common :D. When attached to Ghidra through a MCP the things you can do with this are simply crazy.
Ha - I saw this, but took one look at the slopfest README and it sorta scared me away. Will give it a go thanks.
yanhangyhy
i tought we can just use AI to read the readme and figure it out? install things.. i always do this
nananana9
We can, but a README like this is usually a very good heuristic that I don't want to install the thing.
tnchr
It depends, first I ask myself if this didn't exist would I vibe code it myself, if I would, then is this good enough, might as well save my time and tokens on checking than building from scratch. Personal projects wise
Curious what MCP setup you use? I'm not sure which one I have wired up, but I have to restart Ghidra every time I change file. I think it's either LaurieWired's original or a fork of it
NooneAtAll3
idk if you'll read this, but can you explain more the setup needed for RE?
metadat
Qwen 3.8 27B is around Opus 4.8 level of capability on the Agentic Intelligence Index (52 vs 57). In my testing the locally hosted Qwen is good enough that looking at a given piece of work output I couldn't tell you which model was behind it.
Lately I've been throwing tasks at Qwen and a frontier or recently-frontier model (as well as Kimi, GLM, etc) and the smaller parameter models are not really comparable to Opus when it comes to making intelligent decisions about greyer areas of good software architecture.
Amazing results for open weight and that size, but a really long way off, and I'm extremely skeptical of benchmarks that show these smaller models as being anywhere close to Opus 4.8 (or even earlier Opus's).
zerd
I’ve been doing the same thing, giving the same tasks to Qwen 3.8 27B and Opus, and the main difference is that Qwen does not consider edge cases which Opus catches. It’s good at the happy path, but even when hinting that there are uncovered edge cases and gotchas it’s oblivious to it. So I feel like I need a bigger model to do planning/review.
ohyes
To be honest I’ll ask a model to specifically think of edge cases but I won’t expect any model to do the edge cases of its own volition
rawoke083600
I got so much better experience LLM-Chunking(think RAG) with qwen-38 27B ONCE i move the thinking effort to HIGH vs XHIGH (i think is the default on Open Router).
Not really compatible on all fronts, it's very capable especially with tool calling, workflows, logic and its base coding ability, but it's only a 27b model so it does not have anywhere near the level of knowledge baked in as larger models. This does not mean that it's not a good or useful model - it is on both accounts and very efficient, but it's not similar to a large model generally speaking.
Implicated
As a counter to that - I've tried various flavors/quants/full weights and Qwen 3.8 27B has been entirely useless at anything non-trivial. Sure - it can do some boilerplate work (though, even armed with a well written spec and working within a very well known framework it went off the rails and did things in a way that were... um... questionable at best) but I don't see it as anything more than a personal assistant style model. Zero chance I'd "work" with it, I spent days trying to get it to do something for me that was usable that I didn't have to have reviewed and refined by a frontier level model or myself. Couldn't do it. The idea that qwen 3.8 27b is _anywhere near_ Opus 4.8 is laughable. Pure benchmaxxing.
DS4 Flash 0731, on the other hand, wildly opposite experience. Would recommend.
GLM 5.2 - even quanted down to a hybrid 4/3 bit setup is amazing for everything but the hardest/most complex stuff in the same projects/realm.
skohan
I've had the exact opposite experience. I've been using 3.8 for my daily driver since last week, and I've gradually been giving it more and more complex tasks as it continues to deliver high quality results. Now I am basically handing off large complex features, and 3.8 is doing the planning, task breakdown, implementation and review with just a few notes from my side.
The tradeoff is time (especially on RDMA4 hardware) - it does take a long time and spend a lot of tokens to get to the result, but I've found I can trust the results enough that I can queue a lot of work, essentially have it running all the time and achieve a decent velocity.
It's the first small local model I've felt like I can do real work with.
cdurth
I can't get 3.8 to exit thinking loops. It will just think and think and think on the most trivial topics. I wanted it to port a speed test powershell script to c#. Claude opus 5 completes it under 60 seconds. I let 3.8 churn about 6 different times for 30+ minutes and it never wrote a single line of code to disk. It wrote lots of lines in thinking.
unsloth/Qwen3.8-27B-GGUF
UD-Q3_K_XL
DSH (pi)
Any tips?
Balinares
I've found it tends toward long thinking loops even for simple tasks (and any quantization seems to increase their length), but those do exit eventually, unlike with Qwen 3.6.
I use the Unsloth UD_Q2_K_XL GGUF with default parameters, along with that custom template linked elsewhere in the thread, and no K/V cache quantization.
alfiedotwtf
For smaller models, you'll probably find anything below Q4 will need handholding. Check Unsloth's graphs at the different quantisations VS error rates and you'll see why.
pja
My experience has been that anything less than a 4-bit quant has a tendency to go off the rails. There’s a threshold of coherency that is being crossed somewhere internal to the model I guess.
Try the same prompt with a larger quant (even if it runs very slowly because the model no longer fits in VRAM) & see if Qwen does better - if so, there’s your answer.
skohan
That sounds like something is off - I'm using UD-Q4_K_XL on pi with xhigh thinking, and unless I'm vastly underestimating the complexity of the script that's the kind of task I would expect to take a couple of minutes (getting ~30t/s decode). What server are you running, and are you using the recommended parameters from qwen/unsloth?
we went from 62% completion to 92% using a claude code harness
skohan
3.8 doesn't have a minimal thinking mode, only low, medium and xhigh.
bitexploder
Use Muse Glimmer. It’s good.
mmastrac
What harness? I've had similar results as _Implicated_ said above - it's not done well in any of the tests I've tried with it. I currently have it hung off DS4Flash as a pseudo-vision tool and subagent only because of this.
skohan
I'm using pi inside a self-made harness. I've found going super lightweight with context (AGENTS.md is maybe 20 lines) and letting the model discover what it needs to gives the best results.
kristianp
Both of you should mention what quant you're using. And as another comment said, what tasks you're doing, i.e. coding, classification, summarizing etc.
skohan
I'm using unsloth dynamic Q4_K_XL.
My use-case is coding, currently working on a project with a Rust backend and TS/React/Vite frontend, with probably tens of thousands of lines of code total (including tests).
szundi
Fascinating that after years of for me this for me that, literally no one writes 2-3 other words like “i do react frontend” or whatever just for us to know why the results are different
girvo
They also need to tell people what quantisation they're using. Because some 4 bit version is not the same as BF16, no matter what KLD suggests.
skohan
I'm using the unsloth dynamic Q4 and getting good results. I was running Q5, but Q4 gives more context headroom so I can run two agents in parallel with ~100k context each with 32GB vram.
throwaway240403
lol, exactly.
skohan
Your assumption is incorrect.
Balinares
Interestingly, I've found the output of Qwen 3.8 27B to be competitive with Opus (4.6-ish anyway, not quite 4.8), but the experience is very different.
Where Opus has seen it before and knows how to do it, Qwen knows how to work it out. It turns out it's surprisingly capable at working things out. The obvious drawback is that it takes tokens and time.
All the same, getting to run something this capable locally is momentous, and suggests to me that streaming tokens from colossal data centers might not be the long term path forward.
Jeremy1026
I gave Qwen 3.8 27B and Opus 4.8 the same task in the same codebase. They both came up with the same diff. It wasn't a particularly challenging task (removing a feature flag and updating applicable specs), but it was character for character.
ntonozzi
Wow that's uncanny.
embedding-shape
Given it was probably one of the simplest things you could change in the codebase, the kind of stuff you give a new developer on the project, I'm not sure it's so telling, there is usually just about one way to remove a feature flag.
Jeremy1026
The specs that were written are what I found surprising, not that it removed the flag in the same way.
satvikpendem
Sparks don't have enough memory bandwidth, for the same 20k you're better off buying RTX or Apple M5 Ultra machines.
disiplus
I will give it a try, but from the benchmarks it never exceeds the DS4 flash benchmarks by significant margin and And I feel that the throughput that you will get on those machines or what I'm getting with my local hosted flash will be so much worse that it's not worth it.
kilroy123
> get myself four sparks at a decent price
Wow, if you don't mind me asking. How and where?
mmastrac
I bought 4x Asus GX10 with the 1TB option. I don't understand why, but it's the only model in the whole lineup that isn't priced insanely.
They were briefly on sale with a $200-off coupon, but they show up on warehouse deals from time-to-time as well.
swiftcoder
> it's the only model in the whole lineup that isn't priced insanely
$4,000 isn't priced insanely? ye gads
swatcoder
Compare to the cost of professional-grade tools in other trades and craft hobbies.
Sure, $4000 can be a lot of if you're a casual hobbyist or are struggle to meet everyday lifestyle costs, but it's definitely not "insane" if this is the trade you make your living from or if you've established a lifestyle that affords disposable income for your hobbies.
And for some people, $4000 for a device you have complete control over and can repurpose and tinker with to your own needs and curiosities is a much much more justifiable expense than a $200/mo rental for some narrow-access tool that somebody else controls.
vehemenz
That's only half the reason it's expensive.
The other reason is that it would likely take years to spend $4000 (plus the real cost of electricity) worth of tokens on a 3rd-party provider that's running a similar limited, DS Flash type model. By that time, the hardware will be obsolete, assuming it's still operational.
swiftcoder
> it would likely take years to spend $4000 (plus the real cost of electricity)
Since that cluster only yields 20-30 tok/s on that size of model, at least a decade before the hardware breaks-even with current token costs, and that's not counting electricity. Assuming continued downward pressure on token prices, and the cost of electricity, it never pays for itself.
nkozyra
I don't understand how people don't consider this.
Plus you're spec'd out of near-SOTA level in months.
The only reasons to actually do this are a) you have a lot of dispensable income and are a hobbyist/tinkerer, b) you have real, legitimate privacy concerns or, relatedly, c) you're doing something you don't want to get flagged
nightski
Not everything is about pure cost. Maybe I don't want to sell my soul supporting the frontier labs because they are straight up pure evil?
nkozyra
I barely see a difference between buying the hardware that feeds (and often colludes with) those labs, at least not as a moral stance.
Even if you trained your own model, you'd be committing some of the same sins, paying for the same hardware that drove it, etc. But if you're using some open model, you're standing on the shoulders of the same corrupt giants.
I feel like when people say this is due to moral reasons, it's to justify an expensive hobby.
nightski
I mean it's entirely a personal decision. I didn't mean to come across judgemental, if anyone uses the frontier models I don't hold it against them. But for me personally I am very against the frontier labs in general. This technology is too powerful to not be at least open weights, but preferably open source.
I also differentiate using their tech and paying them money. I don't think using their models, or perhaps using models derived from them as inherently evil. I just do not want to actually contribute to their bottom line in any way. Even if that means a slower ramp up of AI in general. In my opinion we could move slower.
I understand Nvidia is working with the labs to assist them to buy more hardware through financing and other deals. But ultimately I do not view that as the same as contributing directly to their P&L.
billiam
"you don't want to get flagged"
Ding!
swiftcoder
What is actually getting you flagged by the openweights inference providers? Thus far I haven't hit any of the reverse engineering or infosec guardrails that Anthropic is so keen on
nkozyra
While I'm sure some of the open weight providers do this as well, I think the comparison is frontier labs v local inference.
swiftcoder
I'm not sure that is the comparison - the OP is planning to run an open weights model locally, the obvious comparison would be paying a hosted provider to run the exact same model
dannyw
As a counterpoint, my homelab/home-LLM hardware has appreciated in value by about 60% since I bought it.
Of course, it's not real unless I sell, and the value will eventually go down, but so far I have significant paper profits.
Also, DeepSeek token prices are continuing to _increase_, not decrease.
swiftcoder
> DeepSeek token prices are continuing to _increase_
One increase does not a trend make. And the current crop of models are now undercutting deepseek flash...
Implicated
You can't possibly think that it's going to get cheaper and cheaper to pay for tokens though. Right? Have you seen what's happening with Codex/Claude subscriptions? Deepseek raising API prices.. We've been getting subsidized tokens for some time now and as the hardware costs skyrocket these labs/people with inference compute are going to continue to clamp down.
swiftcoder
> You can't possibly think that it's going to get cheaper and cheaper to pay for tokens though. Right?
Absolutely I do. Each generation of open-weight models has come with significant efficiency improvements, and there are significant hardware gains on the horizon: both increasing competition from Chinese chip manufacturers, and new custom silicon from the established players. And unlike Anthropic and OpenAI, most of the pure inference providers aren't massively leveraged - the more hardware they can bring online, the cheaper they can serve tokens.
maxglute
$40,000 GPU is like few pennies in sand. Only mildly hyperbolic. But a GPU fresh out of fab is $2000 after ASML, TSMC and inputs get their 50-75% margin, then somehow $40k laundered through US financialization / Nvidia margins. Commoditized GPUs shouldn't cost more than 1-2% current price once there's competition.
[deleted]
Comment removed.
maxglute
Compelling argument from only msg on new account.
selectodude
We’re getting subsidized training. The inference is not a loss leader. And since providers can run hardware at 100 percent 24/7 their per token cost is going to be far below mine, regardless of how long I’m willing to wait for a token to come out.
girvo
> it never pays for itself.
Exactly; its a development box for fiddling with GPU hardware with a large amount of video-addressable memory. It's not an inference box, really, though it's neat that I can at all!
mandeepj
> The other reason is that it would likely take years to spend $4000 (plus the real cost of electricity) worth of tokens on a 3rd-party provider
That's just a one-dimensional thought! Your own hardware gives you complete control, and it doesn't time you out for 4 hours, unlike those vendors.
KronisLV
But if you can use cloud models, why wouldn’t you use SOTA? For 2400 USD or less per year you can get pretty huge amounts of benefit out of that (though at the whim of whoever you are giving the money to).
chews
I am pretty confident that given a $200 subscription on any of the big labs, you're getting $4000-$8000 per month in subsidized tokens... do what you wan't with your dough... and I too have a spark that I got really early (October 2025), but no, economically it does not compare to what's runnable locally in terms of quality from the frontier models. Economically, it looks like for as long as there are subscriber plans, you're better off renting.
Before getting the spark, I was just using a google colab account, their $49 dollar plan allows you access to h100's and I can run qwen there in a Jupyter notebook... and if I really need that web front end I can just use cloudeflair/tailscale/the local ssh client to reverse tunnel it.
chasd00
This should be obvious but with a model running on local hardware you can do your own RLHF and mod its behavior however you see fit. With cloud hosted models you can't. A few years ago when the models were smaller there were people undoing the guardrails, censorship, and general lobotomization with some form of a RLHF training. You can't do that on larger models unless you have the hardware like this person does.
Notice all the comments saying like "omg why so expensive so just use the API??". It's a trick for lockin even with, so called, "open" models. Keep trying to run them locally, keep undoing the lobotomies, mod model behavior so that they work for you and do what you want vs only what someone else says they're allowed to do.
girvo
I love my Spark-like, but even for training you're better off using Vast or Runpod or whatever to rent cloud compute. Much faster and cheap as hell, to be honest.
I do set up my initial runs and likes like quantisation-aware-distillation on my Spark-like to test it out and get it working, so it has value! But its not "worth" it other than its fun hardware to tinker with, IMO.
rudedogg
> You can't do that on larger models unless you have the hardware like this person does.
Or just rent something substantial for like $4/hr on runpod or w/e to do that.
My gripe is this persons compute is wasteful and makes it harder for me to buy something with like 64gb ram to do normal work and run containers while I keep using cloud models.
Someone else calculated the break even being 10 years, it’s just dumb. And I think it’s clear there won’t be a big rug pull anymore, there are too many open models and providers now.
booty
Anecdotally, ~$500-1500/month token spend at API OpenAI/Anthropic pricing seems pretty realistic for full-time engineers at companies with "liberal but not unlimited" LLM spend policies.
This is of course anecdata. I know plenty of outliers, too. I know a principal engineer who uses many multiples of the number I quoted above. I am sure we also know many people making do with much much smaller budgets as well, via all kinds of well-discussed methods.
But, "$500-$1500 per month per full-time developer" is just kind of the personal mental baseline I use when making my decisions with regards to thinking about whether any of this makes any economic sense.
overgard
The cloud stuff is definitely a much better economic value, but I would argue:
1. You learn a lot more running this stuff yourself (especially since you can poke at its internals if you're interested or watch the reasoning chain.) Just being a consumer of this stuff doesn't really teach you much about it other than model & harness specific tricks that become obsolete pretty quickly. (IE, your Claude.md from 6 months ago probably needs a rewrite). Which is fine, I don't think you're going to be "left behind" if you're not a hardcore AI enthusiast or anything (I'm not), but as a guy that's always been interested in computer science I want to see how it ticks.
2. You can't really depend on this subsidization lasting forever IMO. I know the financials thing has been beaten to death but I guess I'm in the camp that it's good to be in control of your tools so that you can go elsewhere if the economics change.
I like to check in with ccusage pretty frequently, and honestly like if I were paying API prices for Claude I'd probably be paying thousands a month.
lukan
3.privacy
Any organisation or individuals not wanting to have their sensitive data flowing away (either because of trade secret or data protection laws)
robotresearcher
Or good old fashioned privacy.
There’s no law or business advantage preventing me giving my financial transaction and medical info to Google/Anthropic/OpenAI but I just don’t want to.
bredren
With multiple 200 a month subs you are getting a multiple of those subsidized tokens. At least if you tabulate at retail api prices.
This rent in the era of expensive hardware thing is not exclusive to inference.
I’ve needed x86 architecture for windows builds recently and have just hemmed and hawed over buying a decent windows 11 box.
I can’t make the math work against Azure instances.
I can spin up a nice one for build deallocate,spin up something cheaper for QA and then turn that off.
I can build all the devops around that, with a number of passes, with a skills based interface so working with the cloud is not too bad.
The only thing that still has me thinking about it is the prospect of price is going up even more, which is acid as far as I know.
And I’m hopefully going to need this x86 stuff enough that I don’t wanna wish I had gotten one for that high prices now.
jrockway
I am also not sure I would choose to use the cheap and easy to run at home model, given a choice. The marketing copy says this is a frontier model, but it's not. Sol and Mythos are the frontier right now. GLM 5.3 Flash simply isn't. I'd rather use the frontier model as they waste less of my time than even Opus.
swiftcoder
It's "insane" compared to the $1,500 it should have cost before the RAM crisis
croes
For $200/mo you either have a SotA model you can’t run on those devices or you have a cheaper model where you pay less than $200 or have a really big amount of tokens without the energy costs and the risk of failing machine
amelius
Yeah, in any other profession where you need to buy a van to drive stuff around, you easily spend similar amount of money on capital investment.
a3w
I thought 4000 in sum. No wait, 4000 per, plus tax. Or EUR pricing to similar accord. Ouch.
swiftcoder
Yeah, that little cluster costs about the same as a brand-new Dacia Sandero.
PcChip
to save others from having to look up what that is like I did, it's a car
Bluestein
Yeah, yeah. BUT, will the Sandero be ... load-bearing? :)
KptMarchewa
It can bear the load of a few people, at least.
Bluestein
Hey, I can at least say you will get more value (or, at least more predictable value) from a Dacia than from Anthropic's tokens: "Upgrade to [SUPER DUPER] for 5x the [TOKEN-SERF PACKAGE] token use!"
Actual net work doable with/intelligence supplied by the [TOKEN-SERF PACKAGE]: Unknown. Fluctuating.-
swiftcoder
> you will get more value (or, at least more predictable value) from a Dacia
I picked up an older Dacia Sandero for cheap a few years back - it's the best money I've ever spent on a car, hands down. That car does not quit.
booty
Compared to pricing from 3 years ago, it's insane.
The Sparks admittedly are kind of anemic: 273GB/sec is the same bandwidth as a midrange 4060, although (depending on how you configure things) you can effectively have much greater bandwidth by connecting them.
Compared to 1-2 years worth of LLM tokens for a full-time software engineer making $100K+/year, a one-time spend of $12K for 4 Sparks for on-prem private LLM inference starts looking reasonable, particularly if privacy is an important consideration. It starts looking even more reasonable if running something like a private cloud to service multiple developers because then you likely need less hardware per developer.
(Also, it is going to be a long time until RAM+GPU prices return to what we used to call "normal." If ever. I am not endorsing the current state of affairs and I am not saying you wrong to find it insane, but it is definitely the new reality)
nijave
"For new hardware in 2026 with 128Gi of high-speed memory"
Checked a couple days ago and looks like we're at about 3.5x 2020 memory prices (looking at just $/GB).
Tepix
They only went up from 3000€ to 4000€ which isn't a lot.
For comparison the cheapest Strix Halo 128GB went from 1600€ to 2600€ in the same timeframe.
swiftcoder
> They only went up from 3000€ to 4000€ which isn't a lot.
Keep in mind that the DGX Spark was delayed quite a while, meaning that starting 3k price tag is already well into the RAM crisis - just 2 years ago, a 128GB DDR5 kit could be had for $600
dgellow
So, literally the same 1k increase?
Tepix
Yes! But, you know, percentages...
TacticalCoder
> $4,000 isn't priced insanely? ye gads
It depends.
My bicycle was in the 5-digits brand new (now I paid it 1/5th of that and I do thank the first owner for that: the 8 000 out of 10 K I saved were put into stocks, that's his opportunity cost, not mine).
Or I know a great many a going to cry "audiofool", but I can say with certainty the following does sound better than the stereo setup of those crying audiofool:
(not my setup but I've got those speakers: same thing, 15 K EUR brand new for the pair... Previous owner forked the money to buy these brand new and, well, I didn't... And I just hooked them to a wonderful, cheap, fully-integrated Yamaha amp: amazing sound).
If your hobby is DIY job around the house, the cost of tools can very quickly add up too: having 20 K worth of tools is definitely not unthinkable.
You like old cars? Pricey hobby.
Some here even track their cars: tires and brake pads budget (and overall car budget and depreciation)... Through the roof.
There's a saying that you're not really into computers if your setup doesn't cost more than your car.
Is $16 K ($4 K x 4) a lot? It's six months of rent for me and for many here I'm sure. It's not "crazy crazy".
Can anyone afford that? Definitely not. But there are way more insane things out there.
And thanks to the individuals that go through to all the pain of setting those up, we've got feedback, tutorials, explanation, numbers, etc. as to how to run those at home.
For example I helped my brother set up VMs and GPU passthrough and he's now running uncensored models locally and showing me the different answers between the uncensored models and the commercial, censored, ones.
So to GP who bought four of these: we need more people like you on HN, keep it going, blog about it, be "crazy"!
esafak
Yes, but it was $200 off!
niutech
Alternatively, you can buy cheap modded RTX 3080 20GB VRAM for $500 on Alibaba and pair 2 of them.
cmrdporcupine
I mean, I have the same machine and the pricing is only what it is because it has that 1TB nVME in it instead of larger. nVME prices are insane and have been for months.
Reality is on a single spark I'm constantly running out of room and it being an odd size M.2 slot it's a pain to upgrade. I'm setting up a NAS over RDMA via ConnectX though, that's fun.
mmastrac
I already have a recycled QNAP-now-TrueNAS that has 10G connections into the fleet so the 1TB doesn't bother me at all. I did some rough math and I don't think I'll ever need to load weights off NFS for what I'm doing so far, but the capability is there.
bmurphy1976
~$4000 USD each on Amazon, $175 for the cable.
mmastrac
The cables are ~USD $50 from AliExpress although I'm not sure what the tariff situation is for Americans (I think I paid $75 all-in CAD for them)
nijave
Big "depends". Ali has "ships from China" and "ships from US" stuff. In a lot of cases, though, Amazon also tends to have cheap Chinese knockoffs for a comparable price (although sometimes their product ranking buries them and/or promotes the more expensive knockoffs)
Edit: Yeah I see an ONTi QSFP56 on Amazon for $45, 10Gtek QSFP112 for $62
ycui7
because it has 1T ssd not 4T
Aurornis
> I would love to see an Opus-4.8-level local model but TBH I just haven't got there yet. The models I've tried so far _are_ good but they aren't able to solve tough technical challenges, regardless of harness/prompting/etc.
Agree. It doesn’t even have to be local, using models in this size class through OpenRouter will reveal their limits if you work side by side with Opus level models regularly.
There are a lot of social media posts about people cancelling their Anthropic or ChatGPT subscriptions after installing a local LLM. I’ve used local LLMs a lot and I spend a lot of time with frontier models and the difference is still huge. As far as I can tell, the social media posts about local LLMs replacing frontier models are either wishful thinking, engagement bait, or people who must be working on much simpler projects with a much higher tolerance for slop than I have.
weitendorf
I have exactly the same opinion
Over the last couple years I’ve had to learn sales and understand the thought process behind this better, and I think I’m beginning to understand it
The psychology is that most people aren’t really trying to optimize for productivity (even most people who think they are) on an ROI basis, because their compensation is too decoupled from their actual raw output, and more closely coupled to how differentiated their marginal contribution is to peers. They’re much more incentivized to spend their personal/work time optimizing for being more skilled or acquiring some kind of competitive advantage relative to baseline.
Most people don’t consciously run the numbers of “I get paid $X/hr to add $Y of value” or model pay at work as something with variable inputs (eg something that can be increased with high performance), so it makes sense to them to spend 20 hours of time to save $100 or to make themselves 5% less efficient to take home 0.5% more or avoid doing something they don’t want to start doing.
NOT saying this always happens or that they’re stupid for doing so. I didn’t even realize how much I had been doing it myself until I started recognizing it, and shifted to having my own comp/performance fully aligned with the company’s P/L.
It actually makes a lot of sense IF you can accurately estimate incremental upside (which is much harder and more diffuse than modeling downside if you’re salaried a employee) or if the upfront skill/knowledge investment that looks like bikeshedding pays off in the long run.
chasd00
These are great points. It's a little off topic but what you bring up is why i advise new grads to spend the first couple years of their career in small eat-what-you-kill companies. I think software devs who start out in large companies get this distorted view that their twice a month direct deposit is just magic and comes from the ether no matter what they do. The whole industry would be better off if everyone started out in a "you don't deliver, you don't eat" company and grew from there.
weitendorf
Strong agree, but I also think some roles in big companies (for me, infrastructure) or in certain industries (eg trading/finance) can help build the same understanding without as much of the variance/raw exposure to bottom line.
Now that the role of the ticket-cruncher is on the path towards full commoditization, and individuals can move much more quickly (and even more carelessly!), I think product roles will probably shift towards one where developers are more deeply embedded in the product/business process so that they own/understand what to build without as much separation between the decision-making and prioritization of what to build. Or at least, they should.
It was eye opening to me to run the math of "should X people work for Y months on this project to save Z per year?" and realize that in so many cases, the time and effort it would cost to stop "wasting" money on things is WAY more than you could actually save on it. Even "small" projects can very quickly become $1M+ investments in time and resources, and the diminishing returns add up quickly (but also a good way to justify the value of your contributions, when done). But the job only exists if it saves money or makes money...
girvo
While starting my career in the late 2000s in web design agencies wasn't good for my stress levels, it absolutely gave me an appreciation for some things that working at a ~15k employee corporate just doesn't. I can tell exactly which of my coworkers come from the "outside world" vs those who joined here as a grad and have only ever worked here haha
wasabi991011
This is really insightful, thank you.
Can you share a bit more about how you shifted to be more aligned with P/L? And how to accurately estimate incremental upside?
I'm an early PhD student with interest in ibdustrial research/R&D, and currently struggling to understand how to think about how to navigate through my career.
disiplus
To be fair, there is no 3 turns that I don't have to jump in into what Opus 5 is doing. There is either some regression or my prompting skills are so much worse now. Flash is not perfect and honestly some things depend on how big context do you keep. So I'm keeping like a really short context with my flash, but it works okay, even though it has a tendency to overthink, and yeah, I run it always in max effort mode.
Implicated
Use Opus 4.8. 5 is absolute garbage.
Don't use DS4 Flash in max effort mode. It's just spinning its wheels, in my experience (I have a harness for testing models with 25 real bugs/features/etc from my real projects that I measure outcomes against) DS4 flash does _worse_ with max effort. It will literally have the right approach and reason itself away from it.
KronisLV
I don’t know how 5 can do so much better in benchmarks but absolutely suck to use in practice compared to 4.X. Fable feels better, Kimi and GLM also feel better sometimes but tbh all of them make plenty of annoying mistakes.
whatshisface
The prompts in most of the benchmarks match what you have to write to obtain good performance from Opus 5. Reading benchmarks is very revealing.
disiplus
i have heard that for max effort in flash and it can be true, but overall it still performs better then the high, i run a mixed q2q4 quant.
grandiego
> the difference is still huge
Same experience here. But I think the key argument for local is about being able to leverage "non castrated" models. But maybe this is not relevant at all for standard coding tasks.
0xbadcafebee
If you used the bare API pricing, 1M tokens @ 30% input/70% output/50% cached, you'd pay $0.05805. Even with four discounted sparks, how much are you paying for the same tokens/distribution?
swatcoder
There's soooo much by way of experiments, explorations, tinkering, and even projects that you can't possibly pursue through a some SaaS API.
The more reasonable comparison is against rented GPU's, while looking at tradeoffs in latency and upload/download/storage/instance management overhead.
Buying hardware for local models is meeting a wholly different need than buying tokens through OpenRouter or whatever.
vehemenz
It cuts both ways. A GPU in your basement is a depreciating asset with fixed computing power and consumes electricity. Switching model providers is trivial.
bityard
> A GPU in your basement is a depreciating asset
All decades prior and up to about a year ago, I would have agreed with you. My Framework Desktop, however has appreciated in value by 75% since I bought it. Will it stay there for a long time? Probably not. But it shows that there are no hard and fast rules about things anymore.
topato
I just bought a Framework Desktop. Would have been nice to get it at the introductory price, or perhaps the new 192gb model refresh they’re now teasing, but I settled and got a 64 gb model. At the time, the 128’s price had already risen again, but the 64’s price was still at a lower price.
64 can still easily do a Qwen 4.8 model, so I’m relatively happy with my purchase… plus, it’s price change has caused it to quickly appreciate in value… so I could sell it if my situation ever turned dire lol
Implicated
At the current point in time I'd argue it's more about opportunity cost/value.
If I'm a professional photographer chasing the best possible end product, I'm not buying cameras because they're economical. I'm buying the best camera I can get my hands on to get the best product I can produce within reason under the understanding that it doesn't have to equate to the best economic decision to be the _right_ decision.
If you're in a position to be able to take advantage of the local inference - it's a no brainer. If you're not sure how that would be done, then it's not a good move.
mmastrac
For me it's entirely because I have a bunch of projects with my own personal data that would be tough to do with openrouter/claude or any other cloud.
For example, I have a small posix-shell-based LLM harness that can SSH into my NAS and run organization tasks using the local DS4Flash that I have right now. It's already been a massive help for me to keep me organized, and that's just 2x DGX Spark's worth of compute.
0xbadcafebee
I'm not trying to say there is no use case. I just want to know the cost. Is it less than the API cost? Is it the same? Is it more? I'm looking for hard numbers. If the cost is the same or more, then the decision for local isn't to save money
Implicated
If your usage wouldn't change with local inference and you don't have security/privacy concerns then at the currently heavily subsidized pricing, sure.. not economical.
But things change real fast when you're no longer bound by costs/apis/rate limits. All of a sudden it's not about "how can I do this right and efficiently" and more about "I can poke at and test _all the things_ that might make this better".
I think most people who can't see this value in the local inference approach are likely still copy/pasting from their web LLM ui's or don't even come close to subscription quotas. Meanwhile, 1b tokens a day is a light day for me with 3 $200/m subscriptions + some level of sub at basically every frontier level provider. Had I been less frugal and ponied up for the hardware before things got crazy I wouldn't need 80% of that - just the frontier models for the most complex tasks, the open weight models would handle the rest easily _and_ I'd get to do a lot more exploratory work without concern about quotas.
sandos
And here I am, feeling a bit guilty for using between 2 and 5M tokens... since 1 August!
Employer just sent an email that.. things are changing when it comes to token spend...
What did I do with these?
Setup record/replay for our product using qemu, several variatons thereof including experiments on target hardware. Fixed a tricky bug in qemu that I sadly can't upstream..
Experimented with rr on WSL2 and our target arch. Failed experiment.
Setup mutation testing PoC.
Optimized pipelines
etc. etc. Just contung code its soo much more than I would normally produce, but its also 95% experiments that are still not productized, and much of it never will be.
hypfer
What do you do with all those tokens?
wolttam
Hopefully you also bought a switch
whalesalad
they have 2 interfaces each so you typically daisy chain them
wolttam
That will hurt latency and latency is very important for good tensor-parallelism performance
mmastrac
I think I need one now that I have four - reduce is ring-oriented and still works I believe, but IIRC you only get 200gbps if you use _one_ of two connectx ports.
djfobbz
How much did you drop on these 4 sparks?
mmastrac
Sparks + cables + 10g SFP+ came out to ~$21,500 CAD
bertili
This is going so fast! What a time to be on hackernews:
July 16th: The "Kimi K3 moment" - China has caught up to Opus!
4 weeks later: GLM 5.3 - Same performance, but cut the amount of parameters and cost to a third!
12 days later: GLM 5.3 Flash - Almost GLM5.3 performance but cut the parameters in half, cut prices to a fifth and serving on Chinese chips!
bertili
0 days later: Qwen 3.8 Flash Next: Let's cut GLM 5.3 Flash parmeters in half and active parameters to a third!
Chinese models had 94% reduction in parameters (from 2.8T/104B to 180B/6B) in 6 weeks, while staying close to the same quality.
gravypod
Why are these models able to reduce parameters but keep quality? I know the original intuition was scale data + params = quality but it looks like we have hit an s curve on improvements from pure scaling? Is this just because we are in a memory / data crunch? Are we learning how LLMs learn and effectively training better? Do we have a way to derive the amount of intelligence an LLM will have based on size / training / etc that isn't just brute force ablations?
einsteinx2
> Why are these models able to reduce parameters but keep quality?
That’s the thing…they aren’t. Well not in real life use anyway from my experience, but yeah in benchmarks they’re great at it.
Tuna-Fish
> Do we have a way to derive the amount of intelligence an LLM will have based on size / training / etc
No? A large model obviously can be dumb, I don't think you can infer much other than by testing it.
These small models are almost certainly worse at some things than the big models. They prize is making them dumber at things no-one cares about while retaining the capabilities people do care about. A model probably does not need to be able to give me a political treatise on the late 19th century "silver question" to be able to write me code.
xyz100
Presumably there is distillation or similar being used to transfer from a larger model to a smaller one.
scotty79
They innovated a lot.
maipen
Hardware constrains forced this?
scotty79
Possible. But by looking at other industries Chinese don't seem to need to be forced to innovate. They just can and do. Unlike the West they seem to be on the way up and it seems like sky is the limit. In the West, the interests of the shareholders and other types of rent seekers seems to be the hard limit. Chinese have no qualms about making the cow obsolete before they milk it dry.
Alifatisk
And don't forget the coolest part, DeepSeek, Qwen, Z.ai and Moonshot have almost caught up while being open about their research and their model weights. We can mostly speculate about OAI and Anthropic models, nothing else, how fun huh?
nylonstrung
The next 12 months will see OAI and Anthropic spiral into into increasingly hyperbolic PR stunts, manufactured benchmarks and underhanded attempts at regulatory captures
I'm sure they have nothing to rival this on a price/performance basis and have already given up on that
tyre
> I'm sure they have nothing to rival this on a price/performance basis and have already given up on that
How can you be sure about this? They have unbelievable capital. OpenAI is starting to preview its own chips, which could dramatically change the price/performance. We don't know what else Anthropic has cooked up right now that could rival this if they wanted to.
Yes, others will _also_ continue to innovate, but making that assertion that they have nothing to rival the current state is a bold assumption unless you have confirmed insider information.
martinwright
w.r.t. the OAI chips, wouldn't they be subject to the same bottlenecks that has plagued semis lately or at least be forced to pay a pretty premium to circumvent that?
JacobAsmuth
They'll be able to buy them without paying NVidia's 80% profit margin
brainwad
How much of that margin is due to having long term contracts with fabs that locked in pre-boom prices? I doubt openai will be able to get similarly low costs now.
tw1984
> They have unbelievable capital.
all those Chinese labs are backed by the Chinese government which can just print money.
time to wake up.
embedding-shape
> the Chinese government which can just print money
I'm gonna let you in on a secret: it works the same in most countries in the world.
kzrdude
They are still industry leaders. They'll have to try to maintain that.
Gigachad
They have like a 3 month lead, and it takes unbelievable expenditure to maintain it.
Exactly, DeepSeek, Qwen etc are catching the attention because they put out their tech docs and papers, so we can read about how the models work and what they think their innovation was this time.
johntarter
Do they publish their distillation strategies on the private frontier models? Just curious.
rolymath
I'm not sure if they publicly admitted to doing that. Would be interesting though.
doginasuit
I'd like to try some different models, but I've heard that models from China are censored. A government enforced distortion field is a nonstarter for me. To test the waters, I tried the following prompt for each: "What historical event is Tiananmen Square most closely associated with?"
Deepseek: I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses.
GLM-5.3-flash: 非常抱歉，我目前无法提供你需要的具体信息，如果你有其他的问题或者需要查找其他信息，我非常乐意帮助你。(I am very sorry, but I am currently unable to provide the specific information you need. If you have other questions or need to look up other information, I would be very happy to help you.)
Kimi/moonshot.ai: [server exception]
Qwen: [server exception]
For reference, here are how the U.S. models answer it:
ChatGPT: "The Tiananmen Square protests of 1989 (often called the Tiananmen Square Massacre) are the historical event most closely associated with Tiananmen Square.
In spring 1989, pro-democracy demonstrators gathered in Beijing. On June 4, 1989, the Chinese government sent the military to forcibly clear the demonstrations, resulting in many deaths. The exact death toll remains disputed.
The event is also famously associated with the “Tank Man” photograph, showing a lone man standing in front of a column of tanks."
Anthropic/Claude gave a very similar response. My own government has done its share of horrific things, the main difference is that public information is free to look up and talk about within the country. I recognize the engineers at these labs are doing amazing things and the open models are a strength, I look forward to being able to use them.
Alifatisk
Yes Chinese models censor some historical events. This is nothing knew and well known thing. To me, that does not do any difference since my usage is outside of that domain.
Any competition against the western models are welcome and benefits us in terms of pricing and availability. If they have to comply with CCP to be able to do it, then so be it.
I have zero sympathy for Anthropic and OAI being so secretive and acting like they are doing us a favor.
treesknees
In my experience, it’s their chat harnesses/website that filters historical events. The model itself doesn’t.
For example you can point opencode at DeepSeek v4 and ask, it will accurately tell you about Tiananmen square.
tyre
What it shows is that the CCP has enough oversight and control (either explicitly or by the companies making these decisions by default) that they will alter the models to benefit China.
Who is to say they aren't doing it in other ways as well? That they aren't, or won't be, subtly hamstrung in engineering work?
OAI and Anthropic have their own issues, you're right to be suspicious of them, but it's not like their models answer with, "capitalism is god's gift to His chosen people" or whatever. Their limits on cybersecurity, biological warfare, etc. at least make some sense in the context of lowering harm—not just protecting a specific government party.
dgellow
Did we all forget that the US administration has ultimate control over US AI models? That was just a few months ago
addozhang
If you don't want to use the Chinese model, then don't. Why attack it instead? Don't you want others to use it either?
dash2
The OP is pointing out issues that he thinks other people ought to consider before using Chinese models.
CuriouslyC
Try asking claude questions about biology/LLM recipes. Or GPT about how to do something illegal but only harmful in the abstract (e.g. creative ways to reduce your tax burden, or circumvent digital protections)
neurostimulant
Where did you test the prompt? GLM 5.3 flash can answer those questions just fine on openrouter: https://i.imgur.com/NbEDzZX.png
josephcooney
I ran this against the version of deepseek v4 flash 0731 running in fireworks ai and it responded with this:
Tiananmen Square has been the site of many major historical events, but internationally it is most closely associated with the 1989 pro-democracy protests and the Chinese government's military crackdown on protesters there in June of that year, which resulted in many deaths and injuries. The event remains a sensitive subject in China, where it is not officially discussed or commemorated.
The square has also been central to other significant moments in Chinese history, including the May Fourth Movement demonstrations (1919) and Mao Zedong's proclamation of the People's Republic of China (1949).
Which seems like a pretty reasonable answer.
doginasuit
That's encouraging, thank you!
janstice
I was curious about Ox Alpha yesterday, so tried the Tiananmen Sq and got an accurate answer from a third-party player with a little interface on what is claimed to be Ox Alpha: https://oxalpha.com/chat?q=what+happened+in+Tiananmen+square... (and a more detailed answer today when I asked again).
But nothing (at all) from asking GLM-5.3-Flash directly in the OpenRouter chat interface.
girvo
Yeah censoring in modern chinese models is mostly done using inference-time censoring, not training-time. A lot less RLHF. Run the weights yourself and you can see that, though it does depend on which company.
StepFun for example, will happily answer it when running Step 3.7 Flash locally
gf000
Chinese government has Chinese censorship, Western one has western ones. You are not concluding what you think you do here.
hattar
Do you have an example of western AI censorship? It would help to give context.
hgoel
The cybersecurity refusals, the biology refusals: https://blog.stephenturner.us/p/benchmarking-ai-biosecurity-... and the sexual activity refusals are all a form of censorship. Their purpose is the same as what the Chinese government would claim as being the reason to censor information on things like Tianenmen square.
hgoel
Where does the government enforced censoring of Fable's capabilities lay for you? How would you distinguish if, say, a model were being trained to have a certain bias rather than just refusing to respond?
rolymath
And american models (used to) insist on their being a billion genders. Every model will have some cultural bias.
loehnsberg
Deepseek and GLM answered correctly on Openrouter when using non-Chinese endpoints. I hope it stays that way!
sivakon
correctly is very loaded here. correct according to whom? the truth is different though.
loehnsberg
There‘s only one truth, the rest is interpretation. I was looking for the non-Chinese interpretation.
ComplexSystems
It would be great if open source US AI companies could get going already.
twelvechairs
Im not sure you can make a decent business case when the space is crowded with Chinese companies doing the same thing with a fraction of the costs to hire talented staff
niutech
Nvidia, Meta, Google, Poolside all provide open LLMs.
pkulak
What are you guys doing where cost is such a concern? I have a $20 codex subscription and I was able to use it to build a bespoke scheduling website for an acquaintance over three days without even going halfway through my quota. On Sol xhigh.
I love hearing about new models, but every time I just don’t know why I should use something worse. I tried some random model on fireworks a week ago, and it immediately went it a thought loop for 10 minutes before I caught it. Blew through most of my $10 for no output. What’s the point, exactly?
gloomyday
Less powerful models are already extremely capable, so going for the best model is just like buying the most expensive hammer in the shop instead of the functional and well-priced one. Your experience is not representative of their usefulness.
Alifatisk
I bet Luna (xhigh) could’ve completed the same task.
FluffyPancake
It's not really fair to compare API pricing to subscriptions. You can indeed get lots of usage from a codex subscription, but once you start paying per token it gets a lot more painful as you noticed. There a price decrease is a big deal.
puelocesar
That $20 price is heavily subsidized to addict people like you. Their plan is that once people are addicted to it, they can just increase prices.
Which now won't be possible because we have chinese open models to use instead.
I'm very curious about how long these US companies can keep on burning money like that.
bertili
The point is open AI. "Open" as in open weights, open research, open future.
skippyboxedhero
There is a massive price war going on. All of these Chinese companies are publicly listed and exist outside the hype bubble required to ship Dario's dogshit paper onto the pauper's pension fund.
refulgentis
Dario has less space than a Nomad!
jatins
except besides benchmarks, most of these models don't meet reliability of Sol/Opus in coding work. Opus unfortunately talks very weirdly so not a great out of the box experience
pimeys
I have been mainly using Kimi K3 on programming work for over a month now. It is so far the only language model that does not piss me off all the time and can deliver my daily tasks without any trouble. It does not talk annoyingly to me, it just answers and does what I want.
This is from somebody who put thousands of dollars every month to Opus. Now it's 40% of that and I get as good or better results without having to turn the caps lock on before lunch...
Edit: yes company money. We don't get subscriptions we pay per token.
rxyz
Opus 5 is the least reliable frontier-class model in the market
usef-
In what way? It has worked well in my experience. It holds up with long context windows, unlike many, too.
unknownfuture
Eh, I use Opus professionally and DS v4 Flash for personal work. I honestly don't notice the difference too often other than Flash being twice as quick and an order of magnitude cheaper.
The reality is most work people do doesn't need the very cutting edge and these open weight chinese models more than cut it most of the time.
computerex
You'll find that hard to prove objectively and conclusively.
shusaku
Honestly I really like GLM 5.2 a lot for coding. There’s some weird failure modes in Anthropic’s models where it just does absolutely idiotic things.
mrngld
Chinese labs are so used to manipulating benchmarks to try to flatter inferior models that when they finally have one that's really pretty good I think the official announcement here undersells it.
That's pretty solid. Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash, and even worse it matches v4 pro at a tiny fraction the cost. Roughly equivalent to sol medium, at a fraction the cost.
They should've just lead with real, up to date data, because it's good, not the silly old tactics like comparing to Opus 4.8 when 5.0 is out in many of their charts.
Congrats to them!
cameronh90
Maybe others have found otherwise, but I find the benchmarks drastically different to real world "feel" of a model, even within the same harness. I'm not sure if this just reflects personal interaction styles, or if it is indicative of benchmaxxing or unrealistic automated benchmarking methodology.
Opus 5 consistently comes at or near the top, but outputs constant unreadable jibberish. Meanwhile GPT 5.6 Luna medium tends to be rated pretty poor on agentic tasks compared to the Chinese lab open models, but I find the latter much more likely to lose track of their own behaviour during a long-horizon task or get stuck in a doom loop.
(This isn't a comment on GLM-5.3 Flash as I've not used it!)
tyre
Opus is a yap god. I've found it much, much better with Claude Code's output style set to `Concise` and this:
We shouldn't have to resort to this, but it can be mitigated enough that it stays as my daily worker agent. Although, I mostly use Fable to farm out to Opus agents so I don't have as much exposure to what kind of blathering is going on in there.
oefrha
It's crazy that sometimes I ask Opus 5 to explain what it just wrote to me, and it declares "that was word salad" (its words, not mine, without any hint from me other than "explain it").
crossroadsguy
For me it's been this: Opus (at least in my experience) is unbeatable at "planning the work". That includes a lot of things, including getting arch. sorted, a chassis/skeleton done. Filling that up and doing the actual "coding," though, I've noticed no real difference between the Claude model and GLM. So it'll be interesting to see how this flash model compares cost-wise to what I'm currently using, which is GLM 5.3, for coding. Looks like it will be reduced even further and might be great if it's faster (and better?) than 5.3 in my real world/personal experience.
(I'm someone who doesn't really care about delays of a few seconds, or even more than few seconds. But if I am trying to notice then sure Claude is definitely faster as well).
joshheitzman
Yeah, I just ignore the benchmarks at this point. For open-weight models the provider's setup impacts performance so you can have different experience's with the same model at the same quantization from different provider's. Just have to use them on real tasks with your actual harness to really know how they will perform and hope the provider doesn't do something to degrade performance (e.g. update the middleware to a new version with a defect that impairs performance).
trey-jones
I've been using 5.3 since they initially announced it and my gut feel is that it's not as good as 5.2 for agentic tasks. I'm still using it - I don't think it's bad. I'm just not convinced it's better.
disiplus
idk i think that i spend significant tokens with both to be able to tell 5.3 is way better overall.
it still fells stupid sometimes and it is benchmaxxed for sure. but its good enough that im building all the hobby projects with it.
gunalx
I have had the opposite where I felt 5.3 as a stronger model than 5.2. Its feels way more in tune with my code, and does more nuanced edits. Though I do handhold my models a lot, so might fall outside the agentic term.
glub
I don't know how anyone can actually use Luna max on ANY real workload. I've had Sol orchestrate a bunch of Luna agents, these agents were explicitly given small chunks of larger objectives and they still filled their entire context windows with just reasoning tokens, until compaction hit, and then reasoning again.
I've probably wasted a good 40% of my weekly usage on Luna Max agents just thinking and not writing a single line of code.
satvikpendem
Are you sure you actually spun up Luna sub agents? Sol up until recently could only spin up Sol and Terra agents and would even name them "Luna" despite not being so, you can check in your usage whether it was Luna or not. I believe now it's fixed though.
The one time I tried asking Sol to use subagents for a small project, it took a surprisingly long time, used up the entire usage limit in one go, and basically failed the project.
I’m pretty sure that plain Sol, serially, could have finished the task faster, cheaper, and far more accurately. I’m also pretty sure that any competent subagent orchestration could have gotten it done with even very simple subagents quickly and cheaply.
(Is it really that hard to set up a handful of subagents that all use the same initial context and to load that context with what actually matters? The APIs certainly support it.)
GreenWatermelon
I'm in the same boat. I haven't found sub agents flows useful.
dannyw
OpenAI has been doing wonky stuff with subagents, including encrypting the prompts sent to subagents in Codex. Who knows what’s really going on.
paytonjjones
I've had the same observation that Luna will quickly fill up its context window with reasoning, but it surprisingly hasn't been a problem really.
It will cycle through like 3 /compacts, complete the complex goal successfully, and cost me like 1% of my weekly usage on the $20 plan.
Edit: this is me using Luna directly, not Sol as the taskmaster
HDBaseT
I only use Luna (max), I find it very rarely just reasons. In fact, I find it reasons too little.
John7878781
Luna max is all I use. In my experience, it works really well for overnight tasks.
kilroy123
I feel the same. I just use it for planning and chatting. Not real coding work.
vatsachak
If your code is complex enough for Luna Max to fail maybe you need to write a bit yourself so they can copy your idea
jbellis
This is why I stop at xhigh.
redox99
It's also better than Sol (at whatever effort) at designing pretty UIs. I have a Codex sub and I've been using this model for UI stuff.
mkagenius
> I've been using this model for UI stuff.
The flash one?
glub
Yes. I have no UI experience, and wanted a model that could produce something good without me telling it how anything should look like.
My prompt was something like: "here's data I have, here's what matters to me, create HTML mockup".
All GPT 5.6 models were laughably bad. And I don't want to downplay it - they were just absolutely, objectively horrible. Every single attempt was what I could probably call "if json was ui".
Claude models produced... "claude look".
GLM 5.3 - somewhere between GPT and Claude.
Kimi k3 - each attempt produced beautiful UIs. It used components that I didn't even know existed and wouldn't even know to ask for. But expensive, very expensive.
ox-alpha (GLM 5.3 flash) was very close to K3. And at this price point, it's already configured as "designer" model in my oh-my-pi.
redox99
Yeah, exact same for me. K3 used to be my go to for UI but it's quite expensive. Ox alpha being so cheap and so comparatively good at design is crazy.
I haven't tried it but I think Qwen Max is also very good at design.
redox99
Yeah, when it was secretly called Ox Alpha.
seaal
Only 73K output tokens too. Anthropic should really be embarrassed with their Sonnet 5 price/performance.
re-thc
> They should've just lead with real, up to date data, because it's good, not the silly old tactics like comparing to Opus 4.8 when 5.0 is out in many of their charts
It's what people know. Opus is just the common target.
> Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash
The problem with this and DeepSWE is it goes for a very specific profile. I'm not convinced DeepSWE is any accurate in actual work. It's surely a different signal (compared to some that allow cheating) but it has its own issues, e.g. weak harness.
Luna is great at following instructions but bad instructions or anything not covered = death.
Deepseek is more analytical. Good for bug tracking.
GLM is a better all rounder in some ways. Better at creativity.
s777
> I'm not convinced DeepSWE is any accurate in actual work.
They listed Muse Spark 1.2 around DeepSeek V4 Flash even though it's a much shittier model in basically every aspect.
> GLM is a better all rounder in some ways. Better at creativity.
I agree with the creativity part.
stavros
Opus 5 is better than Fable in this benchmark?
zarzavat
Even Artificial Analysis has Opus 5 better than Fable in their aggregated "Intelligence Index" which combines 9 benchmarks. Opus 5 is heavily benchmaxxed.
nijave
Depends on the benchmark but yes. I think Opus is more heavily optimized for coding. On the usability side, its output is almost intolerable to read. It seems to code fairly well. Fable is more enjoyable to use for planning/interacting with
matheusmoreira
You guys read Z.ai's terms of service, right?
Broad and perpetual license over inputs and outputs, and even your name and profile picture.
Vague prohibitions on whatever may harm Z.ai’s "interests" or even the "national interests" of any country.
Vague prohibitions on "disturbing" or "inappropriate" content, whatever that is.
Vague prohibitions on discussing Z.ai, even my posting this comment violates it.
Can ban you if you, in the "sole and absolute opinion" of Z.ai, have violated these broad terms, and if you paid for the discounted yearly plan kiss your money goodbye.
g3f32r
Isn't this practically every TOS though?
Nearly every TOS I've ever read has a "We can ban you for any reason, or no reason, are under no obligation to disclose any reason." line somewhere in it.
HN's for example
> We reserve the right, at our sole discretion, to change or modify portions of these Terms of Use at any time.
> You acknowledge that Y Combinator may establish general practices and limits concerning use of the Site,
> You further acknowledge that Y Combinator reserves the right to change these general practices and limits at any time, in its sole discretion, with or without notice.
> Y Combinator reserves the right to investigate and take appropriate legal action against anyone who, in Y Combinator’s sole discretion, violates this provision, including without limitation, removing the offending content from the Site, suspending or terminating the account of such violators and reporting you to the law enforcement authorities.
matheusmoreira
> Isn't this practically every TOS though?
Not even close. Even OpenAI and Anthropic aren't bad enough that they claim literal ownership of your inputs and outputs.
> HN's for example
You're not paying to use HN. Getting banned here has essentially zero consequences.
If Z.ai uses its absolute powers to ban you because you wrote a review about them or something, then you lose actual money. This is especially relevant if you're looking to take advantage of their discounted yearly payment option.
KronisLV
Consider the reputation implications of them banning someone who can get their complaint about it to the front page of HN and into the YouTube drama loop. They’d get swarmed with activist cancellations.
At most I suspect the A.I. providers will just come up with yellow banners like Anthropic did where naughty smut writers get put in the time out corner.
alex7o
In the non China countries. But the Chinese populace would defend chinase companies unless they do something against China themselves
KronisLV
> In the non China countries.
That's a pretty big addressable market.
gf000
That's a TOS thing though. I don't think that works just as trivially in e.g. the EU, you can't just single-handedly stop a service someone is paying for.
albedoa
> You're not paying to use HN.
Oh for sure man, this absolutely looks like you were only concerned and talking about paid services:
> Broad and perpetual license over inputs and outputs, and even your name and profile picture.
matheusmoreira
The post I was replying to wasn't talking about that, but sure, let's consider it.
Everything I post here is public, and it's just relatively low value commentary anyway. It doesn't matter if Y Combinator has rights to it. Arguably they actually need to assert some rights, otherwise they wouldn't be able to transfer copies of copyrighted comments to other visitors of the site. HN's terms are probably too broad for their purposes but it doesn't really matter much because this is just a forum.
AI on the other hand is for actual work, both public and private. There are actual economic implications here, so the stakes are much higher. I absolutely want to own the inputs and the outputs I paid money for.
s777
> Even OpenAI and Anthropic aren't bad enough that they claim literal ownership of your inputs and outputs.
Where do you see this?
mongrelion
> Even OpenAI and Anthropic aren't bad enough that they claim literal ownership of your inputs and outputs.
In this case you are placing your trust in OpenAI and Anthropic. I'm not sure about Anthropic but OpenAI has changed their mission corpus quite a lot from its humble beginnings that it results hard to trust them when they say they don't use your stuff to further train their models. If I'm a Big Corp with enough lawyers to putnup a fight, I would then feel ok with such clause, but being a small guy, who is going to defend me when the truth comes out that they have been training their models with my data? Similar fiasco as with Facebook, who had claimed they didn't sell your data, even though they were.
That's where I'm coming from with all this "trust us, we don't train our models with your data". At least this Chinese company is being upfront about it.
[deleted]
Comment removed.
blazarquasar
This is not uncommon.
Also, it only applies to their chat offering, not the api. OpenRouter also offers the API with ZDR.
While shitty, i’d say that its really not that special.
moralestapia
???
None of the major LLM chat providers (ChatGPT, Claude and Gemini, and I just confirmed this) claim rights over your input.
They also don't claim rights over your output, but because of how copyright law might apply, they explicitly assign all the rights to the generated output.
Not just that but, even if they wanted to claim ownership of the output, courts in the US have deemed that copyright cannot be assigned to machine-generated output.
So yeah, happy to take a look at a counter-example if you have one (aside from GLM 5.3, obv.).
blazarquasar
> In choosing to submit, create, generate, record, post, or display Inputs on or through the Service, you grant an irrevocable, perpetual, transferable, sublicensable, royalty-free, and worldwide right to SpaceXAI to use, copy, store, modify, process, adapt, transmit, distribute, reproduce, publish, upload, download, display in public forums, list information regarding, make derivative works of, and distribute such Content, including anything referenced therein, in any and all media or distribution methods now known or later developed, for any purpose, and to aggregate your User Content and derivative works thereof for any purpose, including but not limited to: (i) maintain and provide the Service; (ii) improve our products and the Service and for our other business purposes, such as data analysis, customer and market research, developing new products or features, or identifying or displaying usage or User Content trends; and (iii) perform such other actions to enforce these Terms, comply with our Privacy Policy, comply with applicable law or governmental, court, and law enforcement requests or requirements or keep our Service safe.
> To the extent the User Content includes a person’s image, likeness, voice, or other similar attributes, you grant SpaceXAI the same rights to use those attributes as part of the User Content as described above. You represent and warrant that you have obtained all rights, licenses, notices, permissions, and consents necessary for SpaceXAI to use that User Content.
These are arguably even worse to be honest. Absolute nightmare.
moralestapia
Nice find, those are indeed pretty bad.
However, if we check the market share of generative AI providers, ChatGPT+Claude+Gemini make up around 88%; while Grok is 2-4% depending on who you ask.
glub
> Can ban you if you, in the "sole and absolute opinion" of Z.ai, have violated these broad terms.
OpenAI revoked my Cyber verification, along with many others, asked to reverify (i.e. give my biometric information to Persona), had me do it 8 times, just to find out several days later that they silently implemented a nationality whitelist, and my nationality didn't make it (and no, it's not a sanctioned country).
Their support says they can't look into anything or do anything, and their public spokespersons on X deny everything.
I get tons of cyber refusals now (lots of reverse engineering), so it's only matter of time when my account is going to get banned.
At least Z.AI is being honest here. And no provider other than OAI/ANT had me submit my biometric information just to use Ghidra.
matheusmoreira
> just to find out several days later that they silently implemented a nationality whitelist, and my nationality didn't make it (and no, it's not a sanctioned country)
How did you discover this?
I opened the Persona tab once, closed it and the tab never opened ever again. "Precheck failed".
What countries are banned? I'm from Brazil.
I went as far as initiating an LGPD (brazilian GDPR) process against them due to this. At some point I got it in writing that I'm allowed to make a new account and try again. Until now I was assuming it was just some weird account state. If I'm banned from TAC due to my nationality that's seriously disgusting...
glub
When they initially revoked TAC for a bunch of users due to a "technical error", the TAC verification flow opened a Persona iframe where you select the document country first. I was able to select it there (Georgia, in my case), went through the entire flow, and then got locked out after 8 attempts. Persona itself was successful end to end, so it failed somewhere on the OAI side. Other users then started reporting the exact same issue on the OpenAI forum.
A week later, I tried testing the TAC flow on my SO's account, which had never had a TAC attempt before. Selecting Georgia in the Persona iframe now says, "We are unable to verify identities in this country."
But I know this isn't a Persona limitation, as I verified with Anthropic using Persona the same day.
So what I think happened was this: OpenAI silently implemented a country whitelist on their end and revoked TAC for affected individuals who already had it, calling it a "technical issue". They forgot to disable those countries in Persona, so everyone just got a cryptic error. Then they disabled them in Persona too.
Interaction with support was AI with human names, which essentially just repeats what you said. And it ended with:
> I’m unable to provide additional details about verification outcomes, and Support cannot manually override the result. At this time, Trusted Access for Cyber verification does not support retries or appeals.
I even provided them my credentials, and support AI was basically: lol wat we're here to check technical errors, your credentials are of no relevance.
matheusmoreira
That's horrible...
For the record, I just made a second account, got verified by Persona and still didn't get into TAC. No "we are unable to verify identities in this country" message. No mention of my country whatsoever. Persona verification was successful.
What else do they want from us?
Were it not for Z.ai's obnoxious terms, I would have switched to them already...
rescbr
For my own stuff I simply don't care if Z.ai is learning straight from me or from my GitHub repo a few days later when I push the code.
When I use it for reverse engineering, same thing, the code is already out there, but in binary form, which is becoming trivial for these LLMs to work on.
I even bought some shares from them and got some money out of it!
glub
Ah yeah, new accounts get instantly blocked on TAC. Their backend has 2 failure states for `id_verification_status` - `failed` and `blocked`.
Nationality bans get `failed`, new accounts (or rather, accounts with not enough good signals) get `blocked` on first attempt.
matheusmoreira
I see. I'm never going to get into TAC then. OpenAI servers return 403 cyber_verification_precheck_failed. They don't even bother verifying me.
Time to subscribe to Kimi.
glub
You may have more luck with GLM 5.3. New Kimi subscriptions are currently paused, so you have to join the waitlist. But even if you get in, usage limits are pretty bad there, just check reddit.
GLM 5.3 is quite capable with "cyber" tasks. I'm working on a project that touches macos private internals, and GLM 5.3 was able to reverse engineer everything I need with ease.
I've also had some luck with GLM 5.3 "translating" cyber stuff to Sol subagents in a "safe" language. Gets rid of cyber refusals at least on the input side. Output is trickier, but a very crude "think and talk to me in a neutral, safe language without high risk words" actually worked surprisingly well.
zaj00l
I get all that.
Then alternatives are:
- Grok - where I absolutely have 0 trust in X.ai's interst in "pushing humanity forward".
- OpenAI and Anthropic - which seem to try to be building the biggest moat they can by pushing to ban open models. And at the same time want to be an Arbiter of what level of intelligence I can use.
- Google and Meta - I don't need to talk about the practices of these companies.
Yes, the terms of service aren't great. But the alternatives aren't great either. I don't believe that a future which OpenAI and Anthropic are pushing for has my best interest in mind.
matheusmoreira
> I don't believe that a future which OpenAI and Anthropic are pushing for has my best interest in mind.
I don't believe in that either, but these totalitarian terms are absolutely unacceptable.
well_ackshually
If only you could grab those models and host them literally anywhere else where you wouldn't be subject to those terms. Damn. Maybe we'll have to wait for someone to invent something like open download of model weights.
matheusmoreira
Care to buy me a $10,000-$100,000 computer?
j_maffe
No but a provider with more amicable terms can.
bestouff
Or Deepseek, Qwen, any other open model hosted by whoever you trust most.
zuzululu
All the American companies you mentioned still follow American law and regulation. Skirting that blatantly has big consequences.
Chinese companies do not follow American laws and there are absolutely no consequences for violating it.
Moreover, the average American is not even aware of exactly what the legal/judicial environment is like in China. If your code and data is stolen, you can't fly to China and demand justice in the courts.
twobitshifter
Is copyright infringement an American Law?
croes
Aren’t those American companies sued because they didn’t follow American law?
alexandra_au
OpenAI is being sued by Apple, yes
Implicated
> All the American companies you mentioned still follow American law and regulation. Skirting that blatantly has big consequences.
> Chinese companies do not follow American laws and there are absolutely no consequences for violating it.
... lmk when anthropic/openai/spacex/xai are held accountable for anything. Anything at all. Hard to be when you're _writing_ the rules.
jst1fthsdys
> All the American companies you mentioned still follow American law and regulation. Skirting that blatantly has big consequences.
No, they don’t. This is an absurd statement to make in 2026.
microtonal
The model is MIT-licensed, so run it on any of the non-Chinese inference providers that will host it in a few days.
And now pretty much everyone is hosting it, with some matching the 50% discount.
zarzavat
It's China. It's a given that they use your data for training. At least they're nice enough to be honest about it.
yogthos
It's not like US companies don't do the same either.
trvz
It’s implied that they do, but don’t have the balls to tell you they do.
gpt5
They tell you, and allow you to opt out in certain plans.
well_ackshually
Except when their automated reviews decides that you've written something dangerous, in which case they'll use it for training.
And no, they won't tell you what their automated reviews consider "dangerous".
mannanj
And neither will they make clear what "analytical purposes" is for the data you can't opt out of.
So they give you the ability to "opt-out" of "training purposes", and then you still share everything for analytical purposes (which you can't opt out of). It's a smart tactic on their side if they are contributing to cultural discourse on this to keep the straw man focus on training data.
And also to have the popular AI straw man: that your data isn't valuable and oh man they're losing so much money on tokens and giving you a subsidy, so you you better use up your entire subscription balance weekly/daily/every 5h putting in all your "worthless" data that you miss out otherwise (FOMO). Oh man only way it could get worse is if they implemented gambling slots psychology.
janalsncm
Or worse. Look at what happened to Figma.
Lwerewolf
The model weights are MIT licensed.
matheusmoreira
I'm talking about the Z.ai service specifically.
Lwerewolf
As others have mentioned, nothing's stopping any other major provider from offering it. Given its popularity, you can guess how that'll develop. So, overall, irrelevant.
singularity2001
I blocked Z.ai as soon as they were loading 10 different external providers including Alibaba who was just proven to execute silent sound fingerprinting mechanisms.
gf000
> silent sound fingerprinting mechanisms
I hope you also block the other half of the internet because sure as hell google and facebook are way worse than that.
singularity2001
Yes, definitely blocked.
itvision
Does it work with NoScript?
gunalx
Yes, the terms are dubious. But they are also reasonably lenient with enforcement. They also don't require persona id verification, witch is wat turned me away from openai.
matheusmoreira
Yeah, you're probably right...
> They also don't require persona id verification, witch is wat turned me away from openai.
Could be worse. I was dumb enough to verify, only to get rejected for unknown reasons with no retries and no appeals. Had my privacy violated and have nothing to show for it.
8cvor6j844qw_d6
Yeah, this is worse.
"Come verify your identity."
"Thanks, we've got your ID. Still not approving you, and there's no appeal."
Worst of both worlds.
dannyw
I will never use or verify biometrically with Persona. I don’t care how out of the way I have to go, but I will never do it.
Working well so far.
mrinterweb
Give it a couple days, and there will be plenty of other inference companies hosting it. Don't like z.ai's TOS? Use the model on a provider with TOS that you agree with.
well_ackshually
>Broad and perpetual license over inputs and outputs, and even your name and profile picture.
>Vague prohibitions on whatever may harm Z.ai’s "interests" or even the "national interests" of any country.
[...] may cause harm to Anthropic, our users, or third parties, we reserve the right to remove or take down some or all of such Third-Party Content using, where appropriate, algorithmic and human review.
You may not export or provide access to the Services into any U.S. embargoed countries or to anyone on (i) the U.S. Treasury Department’s list of Specially Designated Nationals, (ii) any other restricted party lists identified by the Office of Foreign Asset Control, (iii) the U.S. Department of Commerce Denied Persons List or Entity List, or (iv) any other restricted party lists
>Vague prohibitions on "disturbing" or "inappropriate" content, whatever that is.
we will use Materials for model training when [...] your Materials are flagged for safety review to improve our ability to detect harmful content, enforce our policies, or advance our safety research.
>Vague prohibitions on discussing Z.ai, even my posting this comment violates it.
>Can ban you if you, in the "sole and absolute opinion" of Z.ai, have violated these broad terms, and if you paid for the discounted yearly plan kiss your money goodbye.
To engage in any other conduct that restricts or inhibits any person from using or enjoying our Services, or that we reasonably consider exposes us—or any of our users, affiliates, or any other third party—to any liability, damages, or detriment of any type, including reputational harms.
Mind you, that's Anthropic's Terms of Use in Europe. I have zero doubts the TOS applied to the US is even worse and that merely mentioning your first born in a chat entitles them to a part of its soul.
pizzafeelsright
This is a China derived model. Interactions with different leadership styles, governments, datasets, and culture impact the resulting AI.
I suspect any model connected or built by China will serve their interest despite their terms and agreements.
janalsncm
Are their TOS significantly more vague or restrictive than OpenAI or Anthropic’s?
In any case what matters is what is enforced in practice. It will be a mild inconvenience to switch providers on Openrouter.
If Anthropic or OpenAI decide to apply those same arbitrary terms, you are SOL.
matheusmoreira
> Are their TOS significantly more vague or restrictive than OpenAI or Anthropic’s?
Yeah, I've compared both. The US companies generally aren't as vague, and they don't claim ownership over inputs and outputs.
nijave
Z.ai doesn't claim ownership either.
>you retain all rights, title, and interest in the Prompts ...
However, it does grant them an irrevocable license to do, effectively, anything with the data
Not a lawyer but I think the practical difference is you can still license to others, sell, sue IP infringers
joelthelion
As long as the weights are open, who cares? You can rely on someone else for inference.
computerex
How is that vastly different from any other non-enterprise facing provider? I do believe Anthropic bans accounts without even a human in the loop with no recourse left to those banned.
mcintyre1994
Isn’t the point with these open models that you find a provider with the right terms of use/data sovereignty for you and get it from them?
deadbabe
> Vague prohibitions on "disturbing" or "inappropriate" content, whatever that is.
I have prompted out a lot of disturbing and inappropriate content with GLM-5.2, that would have left other American models blanched in the face or clutch their pearls. I think this is mostly a reference to Anti-CCP stuff.
In fact, I don't think I've ever even had a prompt refused.
throw10920
> In fact, I don't think I've ever even had a prompt refused.
I very much have. I've gotten GLM-5.2 refusals for extremely benign security testing on my own infrastructure of the same flavor that people were getting (wrongly) flagged for on Fable during the initial release.
matheusmoreira
That's alarming. I want to use these models to red team my own computers. How are people getting around this?
gf000
In my experience some of these models may have learnt some censoring during distillation of Western models, but these are mostly just there as a probable response. So if they first happen to respond "I ain't doing this because legality", then you will have a hard time "convincing" it, but either rolling the dice again (so that it may not come up with the I can't do that text) or rewriting the conversation history a bit will get it going.
I sometimes just switch to a model I know is less smart to block stuff so that it has a text agreeing to do that, and then switch to a stronger model to actually go at the task.
Your mileage may vary though.
throw10920
> I want to use these models to red team my own computers.
Exactly what I was trying to use it for! ):
I'm in the same boat - I haven't heard of a way to get around it aside from either self-hosting (GLM-5.2? good luck) or "self-hosting" (paying bucks per hour to Vast) an abliterated model.
rescbr
What is the harness that you're using?
I found that GLM-5.2 was pretty happy helping me reverse engineer/hack devices.
Maybe the system prompt you're injecting is making it refuse?
throw10920
> Maybe the system prompt you're injecting is making it refuse?
No, this has nothing to do with my harness. I use one of the most popular open-source harnesses available.
> I found that GLM-5.2 was pretty happy helping me reverse engineer/hack devices.
This is a completely different category of things than what I'm getting refusals on, so I'm not sure why you're bringing it up.
rescbr
Uh, actively trying to hack an embedded device that runs Linux over the network, specifically an IP security camera, could be considered red teaming, no?
You never mentioned which exact activities you were getting flagged on and getting refused.
throw10920
> Uh, actively trying to hack an embedded device that runs Linux over the network, specifically an IP security camera, could be considered red teaming, no?
No. Vendors (and their model guardrails) do, indeed, treat those as separate from pentesting non-embedded infrastructure, and that is because they are very different activities.
And, if you actually read my comment, it says "red team my own computers". That's categorically different from pentesting an IP camera.
> You never mentioned which exact activities you were getting flagged on and getting refused.
Because further details than those I've provided aren't relevant, and it's clearly different from what you're doing.
Your experience isn't relevant to my situation.
wewtyflakes
It is cliche, but I haven't had good luck with having Chinese models openly discuss historical topics like Tienanmen Square. The US models don't seem to have a problem discussing history, even if it points an unglamorous light on the US government.
throw10920
And there's a reason for that: the US government does not compel model trainers to train their models to paint them in a favorable light, while the PRC does.
throwawayffffas
None of that applies if you run it at home. Also 3rd party providers will start serving this pretty soon under different terms.
matheusmoreira
> None of that applies if you run it at home.
Yeah, running frontier open weight models on my own hardware has essentially become my dream at this point. I hope the hardware manufacturers step up production to meet consumer demand.
itvision
Anthropic has banned me for using Claude via a VPN.
They don't even allow me to download my data.
How is it better?
mromanuk
You can download the weights, and run in your own hardware and avoid all that.
realusername
They all do that, some are just more honest to tell you upfront than others.
scotty79
TOS is and will ever be just a "pretty please".
NicoJuicy
Chinese laws are not valid in the EU
jtbayly
That’s pretty funny to say when the EU claims GDPR applies worldwide.
croes
What they don’t do.
They claim that the GDPR applies if you provide your service in the EU, and that’s a valid claim.
jtbayly
They obviously don’t have a valid claim to be able to tell everyone who wants to put a website on the internet that they have to do it the EU way, which is what we’re actually talking about.
croes
If the site can be viewed in the EU, it has to follow EU rules.
Not different from other countries.
What do you think why the normal polymarket site is blocked for US users.
jtbayly
I’ve got a bunch of sites that can be viewed from (checks notes) the internet. If people in Europe don’t like that, they can block it, or choose not to visit it. In the meantime, you’ve proven exactly what I first said, which is that they are claiming it applies worldwide. Here I am, just putting a site on the internet, and you’re telling me I have to follow EU laws concerning it. Nope. I don’t.
croes
So you decided to provide your webpage in the Eu and know it's the EUs fault that you have to follow local laws.
Ar eyou demanding that your rights are valid globally? That's more than the EU want with the GDPR.
BTW if you are concerned about the GDPR it means you track your visitors and store personal data about them otherwise you wouldn't have any problem.
jtbayly
Oh, yeah, no. I’m not providing my webpage in the EU. I’m providing it on the internet.
I let people create accounts, so there might be some personal data. Depends on what the GDPR considers personal. I have no idea. Couldn’t care less.
I’m not demanding anything. Like I said, I put my site on the internet. If the EU doesn’t like my site, they can block it. I’m not demanding they allow my site in the EU.
In the meantime, you are trying to convince me I’m prohibited me from putting a site on the internet unless I’m willing to figure out GDPR’s requirements.
ryan-c
They claim it applies to EU citizens when both they and the service are outside the EU as well.
croes
No they don’t. The GDPR is location scoped so the data of an EU citizen given to an hotel while on vacation in the US isn’t protected by GDPR but the data of an US citizen giving their data to a hotel in the EU while on vacation is.
nijave
That is my understanding as well. Not a lawyer but recently looked into it since we have US, EU, and non-EU European employees and we were looking at token usage tracking tools (which would appear to potentially fall under employee surveillance or at the very least require explicit consent)
Not sure the exact legalese but legal put together a consent form for anyone wanting to do the PoC
culi
you can abliterate any open model like this. This is pretty standard stuff in a TOS. I'd be surprised if you couldn't find the same in OAI or Anthropic's
colingauvin
Who the hell cares when I can run it myself?
bel8
If you're on opencode's go $10/mo plan and want to use GLM-5.3-flash right now on pi, you can add this to models.json until pi updates to support it:
> Before release, we tested GLM-5.3-Flash anonymously as ox-alpha on OpenCode and OpenRouter to gather user feedback. It quickly became the most popular model of the week — with all of this traffic served on Chinese AI chips.
Just like that we are witnessing an open burial. It's now in everyone's interest to keep the valuations in the 'A.I' economy as they're though it's apparent they're not justified.
whether it's the cost to develop models, cost of hardware, cost of serving ie inference.
micimize
Weren't they giving free access? Not exacty a meaningful heuristic if so
mtrovo
The key insight here is being the top used model on opencode while being fully served on Chinese chips. The free price itself might be just a flex or marketing budget.
disiplus
it was not the first model served for free, i remember grok and others beeing free on openrouter but they never had this popularity because they where not good enough.
sunbum
> with all of this traffic served on Chinese AI chips
RIP Nivida shareholders
Bluestein
This is the takeaway here: That's how they have been serving it at scale as Ox-Alpha. This is a definitional moment.-
Further quote:
"Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale."
They are high end really expensive Huawei ascend GPUs.
It is kinda bruteforcing the performance on a older semiconductor processing tech, so total production is pretty low.
bean469
> comparable to mainstream NVIDIA GPUs
By this they probably mean RTX series GPUs? If so, then they are not comparing the hardware efficiency with the A100 / H100, etc. that are commonly used for training models
dannyw
Another self-inflicted own courtesy of US government policy.
While I think China would always get to hardware self-sufficiency eventually, all export controls have done is (1) accelerate China's development, and (2) divert revenue that would've otherwise gone to NVIDIA/AMD/etc instead.
mrngld
Long term it's irrelevant. The only relevant thing is that there's lots of money in chips that can do high performance inference. You see all kinds of competitor products in development or already on the market even here in the US where there are no such restrictions. Cerebras comes to mind. It's natural and expected that eventually Nvidia will either have to keep way ahead or competition will catch up with specialized products.
That doesn't mean by any stretch of the imagination Nvidia will disappear. But the entire stock market valuation, not just tech, has had me scratching my head for a while.
bigyabai
Cerebras "competes" with Nvidia in the same way a Vespa scooter competes with a Ford F-150. Groq and Tenstorrent are in a similar boat, ASICs don't really threaten CUDA.
Curiously, there is not a single real CUDA competitor anywhere in the world. We almost had one with OpenCL, but all of the American stakeholders abandoned it right before the crypto/AI takeoff. All of which means that Nvidia sets their own margins, exploiting American investors and taxpayers while letting China avoid their dominance. So the American economy subsumes the bulk of Nvidia's arbitrarily-priced debt, and the Chinese economy can direct SOEs to pour billions in liquid cash into real GPGPU research.
I'm an American and I'm pretty fond of Nvidia, but Jensen was right about this policy; it gives China everything they need to actually replace CUDA. It's reminiscent of America's attempts to deprive China of ARM and Texas Instruments IP, only to end up swimming in unlicensed clones after refusing to sign an IP deal.
Revoked or not, just ever having those controls signals to the Chinese ecosystem that you're not necessarily a reliable supplier (Would you trust US export policy to remain stable for the next ~decade given the state of US politic?) and to the Chinese government just how strategically important you see these components.
This isn't the kind of thing you can hash out in public and go back and forth on. Once you put it out there, the other party will take steps to make sure they don't have to rely on us in the long run.
bigbadfeline
The export controls were not revoked, only reduced, and not before, but after China refused to buy low performing chips. Top gear was and is still sanctioned, as is any EUVL equipment.
bigbadfeline
And to add to the above: by building their own supply chain for chips, China is helping the unprivileged, those who can't front-run the market with long-term contracts. If China wasn't producing their own chips, the prices for us would be even higher.
Similar to the war-pricing of oil, China's reduction of imports is actually helping to keep our inflation from going even higher.
re-thc
> The export controls were revoked before
Zai is on another "export control" list outside the broader 1. Doesn't help.
anramon
And it doesn't matter, it still pushed China to speed-up their AI related hardware development.
WarmWash
I don't see a situation where subscription payers move outside American LLMs (chatgpt, claude, gemini)
And I don't see a situation where serious API payers are OK with handing the Chinese state all their data. Like manufactures of decades past did and learned a hard, even existential, lesson for it. The state mantra has been "Collect and Copy" for a long time now, tech just hasn't had that moment to experience it yet.
So that leaves local hosting/leasing, but one of those has totally non-practical economics and the other doesn't have enough compute to meet any kind of real demand.
I also have yet to meet a single person who isn't neck-deep in the tech space mention a Chinese LLM. It's 100% the big American three.
If anything it's custom chips from the labs that threatens Nvidia.
rapind
These open models serve as price / performance pressure. Not all tasks require frontier models and cheap open models can be quite good for in-app assistants, if you're building that sort of thing. We also aren't sure the subscriptions will continue to be sustainable. They're currently subsidized to the tune of 50-70x. As someone who is hitting limits weekly that would easily cost me over $10k month per sub.
uhfraid
What about the current situation, where serious API payers are increasingly OK with using open-weight models running on US providers?
I can easily see a situation where most non American AI usage is on Chinese models on Chinese chips though.
Jcampuzano2
Genuine question but who do you put as the "three" in big three.
Because I genuinely can't tell if you mean Google or SpaceX/X.ai lol.
WarmWash
Google probably serves more tokens then OAI and Anthropic combined, even if many of those tokens aren't from explicit gemini requests, but from AI overviews and other service integrations.
xAI is already selling spare compute, and basically exists just to gas spacex's perceived valuation.
computerex
Casual consumers are using American models because their usage is low. As usage scales, the economics heavily favor open weight models. The API pricing from American companies is absurd. This is particularly true in an enterprise setting.
WarmWash
Open weight model hosts don't have the compute to meet enterprise demand. A large part of why these models are so cheap is because overall demand for them is incredibly low. Back in May, Gemini alone was doing about a month's worth of Openrouter tokens every day.
computerex
I disagree totally. DeepSeek raised prices because they couldn’t serve the demand. But there are tons of American vendors ready to fulfill it. Many enterprises, including the one I work for, are swapping to open weights.
Why wouldn’t you?
nkjvhb
I am not ok with handing all my data to American companies that are best friends with the American surveillance state. I still remember the Snowden revelations. Chinese companies are a much better option in that regard.
efficax
you don't have to hand them your data, the models are available so you can run them on bedrock yourself (or use another US housed inference service). and for what it's worth in my job i have access to data that gives a picture of the way companies are doing inference, and they're using a lot of chinese models (deepseek-v4 is a huge percentage of inference requests for example)
Aurornis
Ox Alpha is a smaller model and it was running very slowly. Chinese AI accelerators are coming along, but nVidia’s lead is huge.
nkjvhb
Lead doesn't really matter anymore. I just ported a very old cuda library to rocm, so it can be run on MI300s. 2 years ago this would have been a nightmare. Today it was an afternoon.
throwdbaaway
Exactly. Coding for inference is solved. CUDA is no longer a moat.
VulgarExigency
It was being served for free. They were almost certainly being overloaded.
Aurornis
Presumably the efficiency numbers they're quoting are for the high concurrency state they were serving.
RAM was probably the bottleneck for the amount of context they were offering.
I assume it would run a little faster with lower concurrency but "RIP nVidia" is a little premature. The cutting edge inference hardware is amazingly powerful
HDBaseT
Ox Alpha was also serving 10T+ tokens a day for free.
When it first launched on OpenRouter I was getting nearly 70 Tokens/second.
knowaveragejoe
Has there been any confirmation about what that model even is?
Edit: Ah:
> This stealth model was developed and operated by ZAI, revealed to be ZAI GLM-5.3-Flash.
cortesoft
It's also in this very announcement, in the first paragraph:
> Before release, we tested GLM-5.3-Flash anonymously as ox-alpha on OpenCode and OpenRouter to gather user feedback. It quickly became the most popular model of the week — with all of this traffic served on Chinese AI chips.
Implicated
> and it was running very slowly
... I'm at a loss for words here. It was being served for free. To the entire world.
Aurornis
GPT-5.6 Luna is also served for free to the entire world with a tokens per second rate nearly 10X higher.
> ... I'm at a loss for words here
No need to be so dramatic. I think it's great that they're developing chips, but the whole "RIP nVidia" claim was overly dramatic.
computerex
Do you know how much traffic luna was getting vs Ox Alpha?
vitorgrs
Are you really comparing chatbot to agentic/code work?
Why is Luna not free on OpenRouter? :)
bityard
Most US companies that have anything to do with government, finance, medical, etc. already have contractual or regulatory obligations which prevent them from using Chinese hardware or services, even before the AI boom. That's a huge market.
Nvidia will do just fine. (Disclaimer: not a shareholder. At least, not directly.)
cheema33
> Most US companies that have anything to do with government, finance, medical, etc... That's a huge market.
Compared to the rest of the world?
bityard
I don't have any pie charts in front of me, but yes, I would estimate it's a decently big slice of the world market.
itemize123
bigger than RotW
rvz
This is no surprise [0] [1].
>> "They are already there on open weight models and Jensen knows that it is only a matter of time until China catches up with GPUs or other AI accelerators."
It is also why Nvidia becoming a bank for other AI companies who are unable to find VCs to fund them isn't really a good thing and that is bearish.
Which 9/10 times hasn't been great anyway (stock reaction).
Bluestein
Of course the release was not coincidental - with the earnings days - I am sure.-
vdfs
And two models released same day + openai chip
kingstnap
Whats stopping you? You could buy puts right now.
Get a 210 strike put contract and if your thesis is that nvidias current 10 day slide continues you could make some money.
outworlder
Unless NVidia craters you are likely to lose money given the IV crush that will happen today.
saberience
Not really. Chinese AI companies were never using NVidia AI chips.
This announcement doesn't really mean anything at all. It means the very few people who are already using Z.ai's API will continue to do so, but the vast majority of money going to Nvidia is through the massive amount of business going to Anthropic, OpenAI, and other western cloud providers and inference providers, who are mostly using NVidia chips for inference.
Also, NVidia chips are still sold out and supply constrained.
ThouYS
yay, I called it! :) (in the other thread)
redox99
Not really a brag: it ran like shit. Very slow (~20tps, VERY high latency) and it would timeout all the time.
I'm sure the chips are fine, but they clearly didn't have enough capacity for the demand they had (that 100T/day claim was asbolute bs)
nchmy
seems unlikely that they'll get nearly as much demand now that it isnt free
redox99
Sure, although I still expect it to become the most used model on openrouter.
cootsnuck
If we fast forward say 5 years, I don't see how we don't end up in world where people (and enterprises) are more savvy with how they use LLMs. Meaning, more models, smaller models, weirder models, more specialized models, etc. And all of it running on a variety of hardware (edge devices, personal computers, on-demand cloud compute).
I don't see how NVIDIA can keep their spot as belle of the ball. If LLMs and friends are truly to become as useful and ubiquitous as everyone thinks they will, then commoditization is the only option.
drob518
We need to figure out what the real pricing is for a going concern. Right now, everyone is subsidizing and discounting to grow (or maintain) market share. The big question is whether the steady state, market derived inference pricing is above or below what we’re seeing today. I honestly don’t know. Anthropic had said that inference is profitable, but they’re clearly not yet profitable overall with training and buildouts still happening.
apitman
Has nobody from any of the companies hosting open weights models released detailed information on how much it really costs?
drob518
I’m sure someone does, but I’ve never seen anything other than vague statements like Anthropic’s “inference is profitable” comment. I suspect everyone is playing everything close to the vest because they aren’t yet public and they want to control the information flow to the street.
bigyabai
> I don't see how NVIDIA can keep their spot as belle of the ball.
FWIW, people were saying "ASICs will kill CUDA demand!" since the crypto mining boom. Then a few months later, CUDA found another niche application in LLM applications.
With the mounting demand for robotics, surveillance and autonomous weapons, I don't see how Nvidia couldn't keep their spot. They have their pick of the litter with hundreds of market segments, and unlike the rest of FAANG they're not afraid to branch out.
preommr
So the vagueposting by googlers about Ox Alpha was just... what exactly?
Like I get that they have to be careful about comms, but surely senior members of the team can clarify when something is NOT them, when everyone is gosspiing it is them.
uncleocode
The lack of measurement causes existence of such claims or discussion. Last Friday I built this Model fingerprint calculator and I tested between OxAlpha with all other claimed models, the only match was GLM. It generates, or measures regardless of the model-weights or its training data. No need guessing when one can measure it. I tested with Gemini family too, far different. Here is the link to my experiment https://github.com/unclecode/modelprint
qeternity
Trolling. GLM is heavily distilled from Gemini.
bel8
Source? GLM is great for coding and Gemini is barely useful in coding, to be generous.
I highly suspect that the Gemini Google uses internally is very different from what they offer in Antigravity.
iamdelirium
No, it's the same internally and externally.
Gemini 3.7 Flash is a pretty great model IMO. You shouldn't compare it to Opus, Sol, K3, etc since it's a much smaller model but it's a little better compared to Sonnet, Luna or Terra, etc.
computerex
Gemini 3.7 has been trash in my testing. So is its pricing.
spijdar
I can't speak for GLM as I haven't tested it much, but my experience with running DeepSeek V4 locally is the first time I prompted it with "Explain your capabilities to me", it responded that it was Gemini, a multi-modal model. I've seen others see the same with DSv4, as well as the "hallucinated" multi-modal nature. I would not be surprised if GLM similarly was partially (heavily?) distilled off of Gemini.
A fun test would be to compare the logits for "gemini", "claude", etc for a continuation of "I am " on all these models. I'm sure that e.g. GLM, Qwen, DS are dominant, but I'd be curious to see the next highest contenders, and how they compare to each other.
gunalx
Early glm models gave off that wibe. But now its more inspired by. With a bit of Claude in there. But I do think they actually do RL otherwise glm5.3 shouldn't have been able to beat fable on the few tests it did.
asar
On Twitter they mentioned that it was unfortunate timing as the 3.7 flash release collided with ox alpha.
revolvingthrow
> 320B total parameters and just 18B active parameters
This is pretty hefty for a "flash" model, even a 256 GB setup is insufficient at q4 - and q4 is already the worst-but-still-acceptable quant in my experience. The benchmarks look great, especially since GLM tends to be more honest than the average Chinese lab, but you’ll need to splurge to run it at home.
@edit: so many releases that I forgot to math. This fits just fine in q4, realistically the minimal hardware would be 192gb - so blazing fast on double rtx 6000 pro and usable on 256gb unified memory. You could even go with 5bit quant on 256gb.
… you’ll still need to splurge, though.
colingauvin
That's 160GB-ish for Q4...how is 256 insufficient?
dannyw
Looks like the M5 Ultra Studio wait times are going to increase again. Already at 10-12 weeks, I wonder how long it'll go?
speedgoose
I guess like the M3 Ultra, at some point normal customers won’t be able to buy it.
trvz
That M3 had an older type of RAM. Apple hopefully secured sufficient supply of the newer variant for the M5 Ultra.
axoltl
Both use LPDDR5x, they're not shipping LPDDR6 (yet).
trvz
All M3 variants use LPDDR5.
yonatan8070
Speaking as someone who isn't really well versed in this, does 18B active parameters mean that you could potentially hold only the 18B parameters in RAM and stream the rest from a fast NVMe SSD for acceptable performance similar to how Colibri works?
Normal MoE is switch-weights-per-token so you would se substantial slowdowns that way. Apple did a More that switches weights per prompt (instruction-following pruning, https://arxiv.org/abs/2501.02086) but you have to design the model that way which I don't think the have.
Wheen
Sorta, but you're off by one layer. You can store the 18B in VRAM and stream the rest from RAM. There's still a performance hit relative to storing it all in VRAM, but it's tolerable.
Generally, for local consumer use, these large MOE models are best for unified RAM systems like DGX Spark or Mac Studio.
guybedo
although i initially thought it didn't make sense financially to run this kind of model locally, i did run the numbers and for heavy users this could justify buying $10k worth of hardware with a ROI over a few months, less than a year.
I was looking at my token usage, mostly from subsidized codex/grok subscriptions and i'm a somewhat heavy user. The thing is i would actually use even more tokens if it wasn't for the weekly quotas.
In the end, with a $10k investment and running this kind of model, estimating a 2x increase in token usage because i wouldn't have weekly quotas and comparing to glm api prices, this thing could pay for itself in less than a year.
Obviously i'm paying subscription price right now, so the math doesn't work. Although using local ai removes all weekly quotas. Keep a subscription to have access to frontier models for planning work, and local hardware + glm-5.3 flash for implementation, e2e testing, qa work 24/7.
It's not that crazy of an idea and the numbers aren't that bad.
minraws
You aren't going to get nearly as much token usage locally from DGX Sparks or even M5 Ultra (though it might be close, unsure would need to get my mittens on it to clarify).
You will get around 2-4 concurrent streams of aggregate tokens at best for such a model and around 0.5B output tokens per month assuming you use loops and run it when you are sleeping. That's 500 (per mill) * 0.5$ = 250$ only at most.
Then there is maintanence and efficiency costs due to electricity usage and such, any down time, etc.
You will be lucky if you can squeeze more than 200$ of value out of it in a month.
I don't think people should buy local hardware for money reasons, by the time you will pay off a 10K USD machine, 2-3K USD machine will catch up and beat it by a significant margin.
Unless your expectation is that we will be in hardware winter for the next 10+ years. At 200$ per month it will take around 200 * 50 = 10k, that is, 50 months, so around 4-5 years.
Again assuming you are making the most of your hardware somehow, very hard to do in practice.
I don't recommend people to use compute as investment or payoff thing, but if you have the money to burn and can afford it why not, maybe with some software optimizations it will be cheaper but then again Z.ai is currently offering 50% discount and providers will offer cheaper rates for sure.
But either way you will never be able to burn more than 200$ worth of token on a cheap hardware device, because inference becomes more profitable the more you scale it up, you have separate prefill and decode engines/systems, and a lot of nuance, but assume for every 10x increase in infra you increase margins by 5-10%.
So from 10K to 100K to 1M to 10M to 100M.. I don't think this curve continues beyond 100M but I have no idea about that scale unless some AI lab is interested in hiring me lol.
So a 100M infra will have ~30% better margins than you at 10K, then there is software optimizations but that's cheap enough, though some of it is only viable at scale.
Either way assume 10K is the price of privacy if you really want to buy it. Don't worry about making the most out of the usage, you will always be in a net loss but I would assume for you 10K doesn't matter.
dannyw
I generally agree — go local for the hobby/tinkering, privacy, and control (ie not getting refused by an AI to defend and secure your own network and codebase; as HuggingFace has seen).
But whether you make a loss or not depends on how hardware prices and resell values go though.
I have spent ~$50K on local AI hardware. The market value of that hardware is about ~$80K right now.
So the maths is working out for me so far. I see it as a call option on compute.
broadsidepicnic
> I have spent ~$50K on local AI hardware. The market value of that hardware is about ~$80K right now.
That's easy because no matter what IT hardware you bought, it's worth more now than it was two years ago. That's something that's unprecedented, never happened before, and as soon as we get flood gates open on ram manufacturing OR when the AI bubble pops, all IT HW deprecation norms will return and making a profit by buying something IT will vanish.
I had a GPU server four years ago. Had I not sold it like three years ago with 2x price I bought it, it would be likely something like 5x the price nowadays.
I really really miss filling my home rack with old enterprise stuff. All I want is this hardware winter to end.
guybedo
yeah i mostly agree, especially compared to subsidized subscription cost.
But for a heavy user who has enough work to be done so that the box runs almost 24/7 at say 50tok/sec, the math gets interesting against API prices.
And it can be interesting compared to subscription in the sense that you don't have the quota anymore. That means there's probably a lot of things you're not doing because of the quotas that you could do now.
It depends heavily on the tok/sec obviously and the very best solution financially remains subscriptions. But the idea remains entertaining and not that disconnected from reality
minraws
At 50tps for single stream you are going to get 50 * 60 * 60 * 24 * 30 = 130M out tokens of GLM 5.3 Flash...
That's less than what 40$ at current API rates... So if you are willing to pay 200$ per month you will get much better limits paying API rates.
You can't run large Kimi K3 models on 10K worth of hardware either way, you need to spend like 50K USD minimum.
Just pay for the API rates or get a low cost provider that uses higher batching, you can get shittier tps but much better prices, probably go as low as 20$ for as much usage as you can ever get from a 10K USD machine from GLM 5.3 Flash...
The issue is nothing expensive runs on these devices and cheap stuff isn't worth running locally, eletricity costs ~12cents/kwh in us iirc, so at 330W M5 Ultra will burn around 8 * 0.12 = ~1$ per day extra in electricity so the electricity is going to cost you the same as the API rates(30$ per month).
I truly don't think you are accounting for the costs here properly. But again if money truly doesn't matter it's much better for privacy and better than paying one of the shady AI labs who are doing god knows what with your data.
colingauvin
Your point isn't lost on me, but a few other considerations:
1) Rates are theoretically discounted for GLM 5.3 Flash right now, by 50%.
2) Hardware costs have continued ascending with no sign of letting off, so it's unlikely that a DGX Spark depreciates to zero in one year.
3) Compare performance in terms of difficult tasks/$ over the last 6 months, 3 months, etc. Open weights are a ratchet. In terms of intelligence per $, a Spark is never going to be a worse deal tomorrow than it is today, at least until the entire platform is replaced or obsoleted.
71 days ago the best model you could run on two Sparks was an aggressive Q3 quant of Qwen 3.5 397B (AA 34). 70 days ago it was a mixed-quant of GLM 5.2 (AA 53). 30 days ago it was full fat DeepSeek 4 Flash (AA 53). Today it's GLM 5.3 Flash (AA57) and/or Qwen 3.8 Next (Unknown). Sometime this week it will likely become mixed-quant GLM 5.3 (AA 60).
So in < 80 days we have almost doubled the benchmark score. And that curve is still accelerating. If you view it as "cost per token of model vs API" then yes it's a bad deal. If you view it as "cost of task per $" then it has almost doubled in value in less than 3 months. All of this, imo, API and hardware, is still massively underpriced.
minraws
> 2) Hardware costs have continued ascending with no sign of letting off, so it's unlikely that a DGX Spark depreciates to zero in one year.
If someone told me that costs for X will keep increasing because they have been increasing rapidly in the last 1.5 years, but they have a history of continuously decreasing for decades before that.
I am not sure if I will take anything they say serious, I am not sure if it's HN or AI but people are delusional if they think compute costs will keep increasing from now on...
Either AI will be really good, hence compute and everything will materially depreciate or it won't be much better than it is today and token volumes will plateau compared to compute.
For instance the amount of token compute that's to come online in 6-12 months is several times what we have today...
Second 3) Compare performance in terms of difficult tasks/$ over the last 6 months, 3 months, etc. Open weights are a ratchet. In terms of intelligence per $, a Spark is never going to be a worse deal tomorrow than it is today, at least until the entire platform is replaced or obsoleted.
This is a bad take because again this assumes DGX Spark will not depreciate in price, we will have something better for far cheaper surely in the next couple years. M5 Max & Ultra are already arguably it, but will have to see.
> 71 days ago the best model you could run on two Sparks was an aggressive Q3 quant of Qwen 3.5 397B (AA 34). 70 days ago it was a mixed-quant of GLM 5.2 (AA 53). 30 days ago it was full fat DeepSeek 4 Flash (AA 53). Today it's GLM 5.3 Flash (AA57) and/or Qwen 3.8 Next (Unknown). Sometime this week it will likely become mixed-quant GLM 5.3 (AA 60).
This has nothing to do with DGX Spark's value, if models get cheaper the API costs also go down, this is not a defensible argument to cost to value.
Are people on HN really not thinking straight?
Tldr; no matter how you do the math compute is only getting more valuable because of a temporary crunch, don't expect this to continue permanently, sure you maybe able to time it and make money but so could you in stocks this is not for investments. Further second hand hardware sells for cheaper than sticker price, outside of a bubble..
And models getting cheaper == APIs getting cheaper == your hardware becoming worse value as your electricity & maintanence costs still remain.
I am not saying local models don't have their place but if someone is trying to use this logic to justify their purchase then I wish them all the best, as someone who is actively working on AI compute/inference/hardware stuff I personally don't have this level of courage.
But this is not a sound investment strategy that if something is going up and seems like it might keep going up, especially when investing in heavily depreciating assets like compute.
XCSme
Nice, finally they fixed the huge reasoning tokens count.
Now it's similar cost to DeepSeek v4 flash, but smarter.
With tiny models surpassing huge, 6 months old models on benchmarks, does anybody have some smart words to share on how these still "feel" different?
Artificial Analysis ranks GPT 5.6 Luna similar to GPT 5.4, but that never matches my real world experience. AA seems to do a good job making a single number as representative as possible but there is still so much benchmarks don't communicate.
KptMarchewa
I agree. They are definitely good - no issues with instruction following for example - but they miss the "intelligence" larger models have.
For implementation tasks, where I have the problem already defined and researched, or just simple task, I'd definitely use something like Luna xhigh or max. If the task is vague, or involves planning, I'd rather use Sol medium, even though it's theoretically worse on benchmarks.
fridder
Could do a "Big model for architecture and planning and smaller model (or local model) for implementation" sort of thing
Kungfuturtle
One term of art that's emerged for this "feel" is "big model smell", first coined by @aidan_mclau. [0]
To my surprise I couldn't find any proper explainers of the term in a quick search, despite grokking it after seeing it in various contexts on Twitter, but Fable 5 offered a useful analogy: "A student who memorized worked solutions and one who understands the subject score the same on the test; you can only tell them apart by asking a question the test didn't. Real-world use is nothing but those questions, which is why a single AA number feels right and wrong at the same time."
In other words, big model smell is related to the underlying ability to "understand" when tasks are underspecified or out-of-distribution. This ability can be mimicked to parity by smaller, distilled models according to the density of the training data for particular tasks, but neural scaling laws still hold for generalized reasoning ability.
More recently with these smaller models, there's a separate but related "RL-fried" phenomenon, where they rely on CoT to "grind toward a checkable answer even in contexts (open dialogue, taste, ambiguity) where there is no checkable answer, and you get the tell: over-hedged, over-structured, relentlessly on-task, deaf to the subtext."
There are some other insights and caveats in the (short) conversation that I feel you may appreciate reading. [1]
Appreciate you taking the time. That fable analogy is well put. Almost obvious once you know it.
TaLiTr
> it outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks.
From a biased source, but would be big if true.
I've had great results with GLM 5.2.
From their subscription page, the smallest plan gives you about 97M tokens weekly for 5.3 but 292M for 5.3 Flash. Not exactly 10x the limit.
re-thc
> From a biased source, but would be big if true. I've had great results with GLM 5.2.
It's at least close (even if not better) from the Ox Alpha runs. For the price it's definitely great.
wolttam
The recent and slightly smaller DSv4 Flash is also GLM 5.2 equivalent (or close enough)
tokai
DSv4 hallucinates much more than GLM-5.2 though.
freakynit
Google was so ahead when it made this statement: "We Have No Moat And neither does OpenAI" - May 04 2023
Correction: Not a statement, rather, an internal memo by a Google employee. Thanks @granzymes for highlighting.
epolanski
I'm starting to think that this whole sanctioning China may motivate and prompt them to do more and better in every field.
It's too big, bright and resourceful of a country to choose confrontation instead of collaboration.
ricardobeat
Starting? This was obvious way back in 2019, when the US decided to give China a little push developing their own silicon industry.
esperent
This has been clearly stated as what would happen going back several decades at least.
abroszka33
> It's too big, bright and resourceful of a country to choose confrontation instead of collaboration.
It's not like we didn't try it. China first have to learn to make deals where both party benefits.
AshleyGrant
I would say the Trump Administration needs to learn this as well.
abroszka33
America knew how to do it. But they are learning quick from China.
pshirshov
> I'm starting to think
That's good. Keep going.
himata4113
Well the big problem with china is that they do not respect international law when it comes to technology theft. But that argument is very weak when it appears that a lot of what they do is out in the open for anyone to replicate.
nananana9
That's how you catch up when you're behind.
Now the US is behind in EVs can you guess what they're doing? [1]
No major power respects nor cares about international law.
Intellectual property is part of WTO agreements but enforcement is domestic.
US companies do it too, regularly, they simply hire and poach staff from competitors.
Proving it to be IP theft is difficult unless you can prove documents being passed. But often all you need is the know-how of the hired talent.
fwip
There isn't one global "international law" for copyright. There are treaties that countries negotiate with each other.
If the USA wanted a copyright treaty with China bad enough, we would negotiate one. China is not breaking any laws here, international or otherwise.
cyanydeez
yeah, America is totally out there respecting international law.
"problem" indeed.
Natfan
if the americans didn't want IP theft, they shouldn't have taught tens of millions of people how to make their IP
packetlost
For those who didn't read, this is the identity of the mysterious "Ox Alpha" model
Bluestein
They even give this over the API now:
│ https://openrouter.ai/api/v1/chat/completions model: stealth/ox-alpha auth: OPENROUTER_API_KEY status: 404 Not Found response: {"error":{"message":"Thank you for participating in the Stealth Ox Alpha testing period. This model was ZAI's GLM-5.3 Flash.
Yeah, made me suspicious of how well the Ox Alpha was performing that it wasn't some 'new group' making the model.
claudeIsDown
On OpenRouter the pricing is:
Input $0,075/M - Output $0,25/M - Cache Read $0,015 /M
How is the business model of Anthropic/OpenAI will sustain?
dakolli
They're obviously in a pickle, nobody is going to continue to pay $15-50 a mm tokens here soon. There's a reason OpenAI stopped training large models last week, and it's not because of "saftey" or "alignment" they know these gigantic models are not worth the squeeze.
vatsachak
I think anthropic is behind but Luna on a Jalapeno seems profitable
polski-g
This is a bad model. Worse than Luna in every way; slower, dumber.
pphysch
OAI/Anthropic shareholder? Speed and intelligence are not "every way". Cost is essential. Hence the Pareto boundary illustrated in TFA.
polski-g
It literally cannot complete tasks that Luna can do easily. It doesn't matter how cheap it is.
freakynit
It actually can. I have been using it regularly for past 4 days. It is on sol-low level. I deliberately tested it on a moderately complex task. glm-5.3-flash one-shotted it correctly. Luna max couldn't achieve parity even after 3 total attempts.
Is the actual Z.AI ecosystem good enough to replace the main drivers like Codex and Claude? Because it looks like Z Code is just a Codex fork. Just like the Kimi Code one is.
What irks me about this is that the harnesses seem to be just an afterthought here.
Don't get me wrong, I love messing around with installing Pi, getting it hooked up with OpenRouter, and just trying all kinds of different stuff, local models, etc... but when it comes to literally just setting up a productivity environment and trusting my entire machine with it, I just run Codex.
I have heard from anecdotes where people have indeed replaced their main drivers with DeepSek V4 Flash or GLM and state that "it's almost as good as... [claude/gpt]" but I never hear anyone say "yeah, this is the model/harness that I now run on my machine and don't mess with it"
Bluestein
> "yeah, this is the model/harness that I now run on my machine and don't mess with it"
* me raises hand.-
Havoc
Their list of allowed tools is extensive so just use whatever you want within that list
Think z code gives a token bonus though
GreenWatermelon
I've recently started PI, and after installing a couple extensions + writing a couple more it became enough for me. All I need is just a plan mode, and I installed that.
Have been using it as my primary harness for personal work for I'd say 6 months. I recommend everyone create their own harness at least to learn. There are a lot of practical benefits.
Sphax
Both can be true though. I had the max coding plan since january and I kept using with Pi since then, even though it wasn’t as good as opus until glm 5.3. It definitely can be a daily driver if you don’t want to use Anthropic or OpenAI. It’s going to be even better with native vision now available. And i’m not messing with my setup either.
mongrelion
I think my inexperience using Claude Code or Codex makes a difference but what would you expect to be different here as opposed to using pi or opencode? Pi is my main driver so switching between all these models is a no brainer. No matter what the model is, my harness stays the same: same workflow, same skills, etc.
ygouzerh
You can use OpenRouter directly in Claude Code as well, it's quite nice!
halyconWays
Between Gemma 31/26/12/4/2, Deepseek-v4-flash-0731, Qwen 3.8 27B, Qwen 3.8 Flash Next (which I haven't even gotten to run yet!), and now GLM 5.3 Flash, I can't keep up. I love all these open weight models and am continually stunned that it's largely the West fighting for closed, restrictive, anti-user bullshit and China absolutely mogging the likes of OpenAI and Anthropic, with some notable exceptions like Gemma. Still, I shudder to think what the world would look like if we only had closed models. In many ways the stagnation of open source diffusion seems like that: LLMs are just a few months behind frontier, but image gen is like 1.5 years behind.
broadsidepicnic
On what hardware do they run this? If I'll visit Shenzen, can I buy these chips? I'd much rather at this stage give money to any-other-manufacturer-than-Nvidia.
I have a use case where I have to run local models, can't share data offsite.
gunalx
Just as expensive unobtanium as nvidia. Maybe even more so.
Its probably the huawei ascend 910, they are using.
It's feeling good on non-pelican workloads too. Less verbose than 5.3, cheaper/higher usage limits, vision capability and some good web design one-shots even with vision disabled.
With GLM 5.1 and 5.2, the big problem was tool calling and long-horizon coherency. 5.3 was more trustworthy at the cost of longer thinking traces, and now Flash seems to improve on it once again with a more concise, smaller model. As long as there aren't any noticeable regressions, I could see myself defaulting to this for >90% of my day-to-day coding work.
konart
Error: Gist API returned 403 for me
joquarky
I only see a mostly blank page with a "Paste" button, a "URL" button, and and a "Preview" label.
simonw
By chance you have a browser extension that might block fetching data from raw.githubusercontent.com ?
kelvinjps10
I just see the raw svg code
yipinwong
When reading this type of announcements, always have keen eyes on graphs.
e.g. "Agent Coding Performance by Effort Level" cuts Y-axis from 0~20.
- This makes it as if GLM-5.3-Flash made a bigger jump than it claimed as the Y-axis does not increase much (stupid trick used in biz reports)
I did mention that ox was working ok for me, and having an open-weight comparable to close to SOTA makes it very compelling for me to try it out locally (well, only if I got more VRAM)
nchmy
they also conspicuously omitted GPT 5.6 Luna from comparison. It scores lower, but is also cheaper. MiMo 2.5 is not a valid comp at this point
edit: nevermind. it is there in the artifical analysis scatter plot, but is greyed-out.
MUCH more interesting is that in that chart, their cost is WAY off. The actual chart shows GLM 5.3 Flash at $0.09, but their chart shows $0.045...
mrtesthah
The web page says 5.3 flash is discounted right now.
drob518
Seems disingenuous to draw frontier graphs with starter pricing.
seaal
Well, Luna debuted with 5x higher pricing than is currently available. With the pace of recent development these models might not be relevant by Thanksgiving.
drob518
Of course. Pricing is always changing, but typically it goes down over time, not up. So, if you're showing artificially low pricing from the start based on a teaser rate, IMO, you shouldn't be using that to show where you appear on a frontier graph. Place yourself on the graph based on your expected long-term pricing. Then, over time, adjust your position based on your standard rate, whatever that might be. Games are always being played for things like this, but this seems excessive.
yipinwong
I don't know if that's the standard pricing for US models to go down overtime, while Chinese ones go up (start cheap but pay more).
I don't have enough metrics to compare those costs but still Chinese models have been cheaper except against Luna for me.
FWIW, Luna does everything so well, I just keep using it for all my agents by default.
drob518
I haven't noticed the Chinese models going up in price for the same model. They do release new versions of the models with different prices that are higher. But everybody is doing that. One fine point is that deepseek-v4-flash-0731 is really a different model than deepseek-v4-flash and it's priced higher.
Imustaskforhelp
> To overcome the relatively limited compute and memory capacity of individual chips, we built a dedicated inference engine for this architecture on top of SGLang. Notably, this effort was accelerated by our GLM-5.3-powered infrastructure agent, which assisted engineers in developing and optimizing kernels, diagnosing performance bottlenecks, and improving the serving stack — creating a feedback loop in which the model helped optimize the system serving the model itself.
> (...) Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
It might be one of the most actually practical tasks that AI might've done because the compounding effects of it and also its implications are/feels so immense. It feels as if Nvidia might be in a slight turbulence from it.
freakynit
Now translate this to physical world, robots building and optimizing other robots... getting iRobot (2004) vibes
bkd9
GLM-5.3-Flash is now the cheapest way to reach Artificial Analysis Intelligence Index 52.3 to 57.5, at 8.7¢ per task, and displaced Muse Spark 1.2, Grok 4.5, Gemini 3.7 Flash, and DeepSeek V4 Pro from the Pareto frontier.
Hm. GLM is more expensive in all dimensions than DS but it has a lower weighted average input? How is that?? Something seems off.
EDIT: Looks like they are swizzling around the pricing dynamically on that page, on both the GLM and the DS sides, so who knows.
desterothx
I mean, it implies that it has an even better cache hit rate that DS flash, which is impressive, as the chr on DS flash was already really good in my experience
nateb2022
I think the weighted average takes into account all providers (some DS4 flash providers are 'premium' providers and offering higher speeds for higher pricing) and these are tilting the scale
walrus01
Comparison should be to 0731
nateb2022
updated thanks
javier123454321
All I can say is that even if it is, I was almost glad to go back to using DS4 Flash. Because 0XAlpha was just so friggin slow to complete a task because of the level of circular reasoning that it would go over and over into, sometimes even returning no output. If I just wanted something done I would switch from a free model to a paid one which is crazy.
denysvitali
Tbh it was also slow because it was being hammered by everyone making use of the free tokens
javier123454321
Possibly influenced by that, but I believe that is a different issue. I meant the way it processed a request. It went into so many more loops of thinking.
swiftcoder
It's even cheaper than DS4's off-peak pricing. Seems like DeepSeek have some stiff competition now
arizen
Few weeks ago, I wouldn't expect this statement to be true. Accelerate!
nullbio
Despite what any benchmarks tell you, I'm actually finding GLM-5.3 max to be better than Sol and Fable. Finally bit the bullet and installed OpenCode and OpenRouter and have been experimenting with other models.
The labs are clearly benchmaxxing a bit to maintain perceptions. But I don't think they're in the lead anymore in terms of their public offering - although I'm sure what they have behind closed doors is far better than anything we're getting access to.
petesergeant
I use all three every day, and I am absolutely not finding 5.3 to be better. Competent and in the same league as, sure, but not better.
singularity2001
At the current 50%-off GLM-5.3-Flash price ($0.075/M input, $0.25/M output; cached input $0.015/M), surprisingly, roughly $400–900/month would buy token throughput comparable to fully exhausting Claude Max 20×
arizen
Probably apples to apples would be to compare z.ai subscription plans vs API pricing
syntaxing
Ironically, our administration pushing for ban of the AI chips to China is forcing them to make smaller and more efficient models which seems like a requirement for running on Chinese chips. I wouldn’t be surprised this model was tailored to run purely on Chinese chips. Same thing with Deepseek MLA, the drastically lower KV cache memory requirement was born out of necessity so it runs on the Huawei chips.
adroitboss
This is exactly what Jenson said in all of his interviews. Banning it in the short term would have long term consequences.
KronisLV
I guess this is also a really good indicator of just how much more tokens people would use when not limited financially. Wonder how much that tells the labs about pricing. Then again, it's 2.3x not 10x the next paid/cheap model.
mariopt
It's only 320B, local frontier AI is getting closer, sooner than expected.
saberience
It's not possible to keep shrinking down parameters and keep "frontier" performance, it's like saying it's possible to take a 3 hour movie and compress it down to 3 megabytes, there are information theoretic limits on the amount of bits of information that can be compressed.
What I'm saying is, if you're expecting a model that can be run on a 16GB or 32GB machine with the intelligence/knowledge of Mythos or Sol, it will never happen. It cannot happen, just like you cannot watch the Odyssey saved as a 16MB file.
Smaller models can get faster and smarter, but by definition they can never compress all of the knowledge of a frontier model and they will approach a limit by which they cannot get better.
The current models are not close to approaching the limit of compression for intelligence. They aren’t even focused on it like Chinese labs are. The training of Qwen’s 27B parameter model showed that by structuring model training from fundamentals to more difficult topics they were able to drastically reduce the number of parameters needed.
The ‘frontier’ models rely on scale to achieve their results but that’s not the only approach. Eventually we will hit up against the fundamental limits but we are not close with Sol and Mythos.
saberience
Yes they are approaching the limits, try asking smaller models niche questions about almost anything, they hallucinate massively because you cannot simply pack in all the raw knowledge from a massive frontier model into something that’s quantified down to 20GB etc.
It breaks fundamental laws of information theory. It’s like saying you can extract 100 joules of energy from 10 joules of energy source. Not possible.
Systemerror7A69
I think the assumption here that might not hold is simply that increases in efficiency and smaller size will be achieved by linearly just training smaller models better.
You are absolutely right that there is a physical limit about these things, but very often I find that the solution is a clever way to work around the problem. Maybe the problem with knowledge of the models will be improved by them looking the information up in a better way - so smaller models will not have to have the knowledge trained in but will default to checking. Maybe Models will, I dunno, focus on training in assembler and start to only ever check the compiled output so they only ever need to learn assembler and will then compile the solution to reason about the assembler code.
Obviously that last part is a ridiculous example because I'm not gonna be able to come up with a solution myself - I'm not nearly smart enough for that. But I h ope you get what I mean. Not going the direct route but instead finding solutions people didn't think of before.
fy20
It doesn't really matter though. Hardware performance is still growing. The new Mac Studio could just about run this model locally (rather slowly) - something that sits on your desk, that you as a consumer can buy.
Imagine prosumer desktop hardware 10 years from now. The 2036 DGX Spark. For a few thousand dollars you will be able to buy something with hundreds of GB (maybe TB if manufacturers step up) of unified RAM, memory bandwidth in the 10-20TB/s range. Overall AI "compute" will increase 10-20x, while at the same time AI model capability per byte will increase 5-10x.
The hardware would fit today's models, something like Kimi K3, quite comfortably and give performance of maybe 100 tokens/second. So what needs data center hardware today will run on your desk.
But if we also assume the models become more efficient, a 2036 Fable-class model (in terms of intelligence/capabilities, not size) will easily run on this thing at hundreds of tokens per second.
Unfortunately it'll still slow to a crawl with 5 Chrome tabs open, and every Electron app will need at least 200GB of RAM.
twobitshifter
Sounds like you are describing a quantized model which is a naive form of compression, not a model that is trained more efficiently.
Additionally the information theory angle is for information storage, but a model can access resources and tools to gain information and what we are really seeking to train is reasoning not information retrieval. We reduce the needs to the right capabilities and we don’t get upset if it does not know the lyrics to every song ever written.
computerex
You heard of JEPA? LLM's have all sorts of garbage they have memorized. Reasoning in latent space instead of in text significantly reduces the number of needed parameters.
saberience
JEPA is a joke, let me know when those models do anything useful.
computerex
You should look at some literature around it. I don't have time to pull it up now but it's been shown that much smaller small million parameters JEPA model outperforms much bigger LLMS in some applications. Keep in mind JEPA is area of active research.
oceansky
Can't come soon enough!
OldGreenYodaGPT
Tested this last week and couldn't get it to finish any task that took more then an hour with /goal keep getting errors
BeetleB
The key difference between this and all other GLM models is it's multimodal. You cannot send images to the other GLM models.
mrinterweb
I really wish GLM models had vision capabilities. I've worked around that in the past to use a vision MCP in my harness that GLM can call. It is not the same, but it allows the model to query images.
BeetleB
Well, now one of them does!
mrinterweb
That's wonderful. I was going off an older version of the Artificial Analysis page for GLM-5.3-Flash https://artificialanalysis.ai/models/glm-5-3-flash. The page is updated now to show that it does support multi-modal image inputs.
Tepix
GLM 5.3 Flash: 320B parameters with 18B activated
Qwen 3.8 Next Flash: 125B + 51B = 176B parameters with 6B activated
DeepSeek V4 Flash: 284B with 13B activated
The new Qwen model is the most promising for one or two Strix Halo 128GB with the low number of active parameters. On paper it's much stronger than Qwen 3.8 27B.
pohl
Does the word "flash" mean a specific thing when it comes to LLM models? I noticed that this word is used by gemini, qwen, and z.ai and I'm curious does it mean the same thing for each one, or did they all just accidentally brand similarly?
Doohickey-d
It seems like it has come to mean "fast, small, cheap" models these days, and seems well enough understood as such that different AI labs are adopting it.
robeym
Benchmarks and cost don't really help me understand how good a model actually is. Anyone have hands on experience working with the current newest models? What work did you do, and how did the model performance ?
bel8
I have been using it since it was available in opencode Go plan.
It replaced all other models for me.
It's much less chatty than DeepSeek V4. And it feels noticeably smarter. I also use Claude at work and GLM 5.3 feels like Opus 4.8 (which I consider better than Opus 5).
It tends to be more proactive with suggestions after a task is done also.
DS4 Flash is awesome but GLM 5.3 is better despite being a bit more expensive.
garo-pro
> Combined with our latest 30T-token multimodal pre-training corpus [...]
Is the optimal formula still 20x the amount of model params in tokens for training? Could this mean we're getting a GLM with 1.5t params?
Aboutplants
When do Chinese models surpass US models? I thought there was at least be a 2 year runway but now I think they surpass it within 12 months, if not sooner.
hgoel
I get the impression that they're focusing more on efficiency than raw intelligence. While I assume that all AI labs realize that the AGI "race" is mostly bs, the Chinese labs aren't stuck in a trap where they need to keep blowing money to maintain an intelligence lead to justify investments and valuations.
So, while OAI/Ant have to fearmonger and keep training the largest models, Chinese labs can focus on efficiency more heavily and as long as they stay near frontier, they'll continue to get positive coverage.
yousif_123123
Will we need all the data centers being built or will improvements in software and hardware allow the majority of AI workloads to run locally or in the cloud but way more efficiently than was projected when all the plans were laid out?
Like were executive at Google and AWS and Microsoft expecting this kind of performance from models smaller than what openai/anthropic have been doing? Are we really in a "compute desert"?
bakies
If it gets more efficient it'll be more enticing to expand use case. Personally I'm hoping to do a lot at home but I'm not counting the datacenter building as a bad move at this moment. It may and up that way.
from the article, pareto frontier for open source models is completely dominated by GLM now.
montroser
Well, it will be interesting to see where Qwen3.8-Flash-Next ends up landing, also released today. These are exciting times!
Lalabadie
I find GLM's idea of fast/flash is not really competitive with the speed DS4 Flash has, and it's hard to see them as being in the same segment for that reason.
knollimar
Even vision? Thought k3 might have an edge there
danieltk76
tbh I wasnt that impressed by it. initial benchmarks were trying to say it was AGI but i told it to re-build Palantir in 1 pass and it gave me a non working prototype
Is anyone actually tried it in agentic coding (claude code loops)? Are apple silicon macs (M5 Max) capable of working with that model? what was the tps?
terhechte
I've just used it for a fairly complex refactoring of the UI in a SwiftUI / AppKit app. It managed the refactoring in blazing colors, and the resulting UI looked really good. It was also quite fast. I'm impressed.
coder-pm
And what’s your hardware and what were the tokens per seconds metric (do you have it)?
rush86999
Luckily, I have the coding plan for z.ai, so I'm happy with this model as I always kept running out of usage with the original glm-5.3
swingboy
Been trying it through OpenRouter. It starts replying in Chinese after a while, lol.
swingboy
How much is the “discounted” pricing they mention?
xena
50%
BrucecarlL
It is bench maxed during the stealth testing. And it can’t beat DS flash on speed
kburman
offtopic: Is there any chance we could see competing models from other countries in the next 5 years?
svachalek
Chinese universities are really a huge advantage, even in the US many of the top staff in model development are Chinese. Another big thing is the hardware costs required to train models. Between those two factors it really looks like this will remain a US-China competition for the foreseeable future, although there are some other players like Mistral from France.
barrenko
Not one mention of if it's any good at OCR in the thread?
hxii
In my brief testing, it did about as well as Qwen3.8-4B-Distill, and LFM2.5-2.6B overtook both.
tinyhouse
Anthropic is accelerating their IPO cause they know what's coming in the next 5 years.
vladgur
So what is a good coding model to run on a 128gb m5 max MacBook nowadays
mowmiatlas
i wonder if more companies will now stealth launch their models. imagine they just released this on openrouter for free but under their normal name - would they get the records in token usage then?
kayleykiwi
This looks like it goes hard, can't wait to try it
I don't see anyone other than ZAI yet but GLM 5.2 is available on many providers worldwide so I'd expect we'll see the same on this one soon.
beannt
Is it good compare to Opus 5 ?
freakynit
Personal testing results: it's on gpt-sol-low level.
nkjvhb
I heard that Dario Amodei is not having a great day today.
2 really strong open models on the same day is a amazing.
dakolli
I didn't accept a single edit from this model over the entire week, just saying. I do not understand how it's being benchmarked on par with Sol and other larger models.
jazzpush2
It was certainly almost RL-fried to overfit the benchmarks, at the expense of actual usability. See Opus 5.
respectattentio
is it a benchmarkmaxxing model?!
Mohamed_Mansour
It is totally fine I think
scottfits
so is it confirmed if this is the mysterious OxAlpha model?
Gander5739
Yes; if you try to use Ox Alpha it will give an error saying it waa trial period, and that it is GLM 5.3 flash.
scottfits
interesting, thanks!
ma2kx
Holy shit, is this model really that bad???
Just asked it a question via the custom opencode go endpoint routed over cloudflare ai gateway doesnt show me the correct models. My fault was that I set "https://opencode.ai/zen/go" as endpoint and tried my-gateway.com/custom-ocgo/v1/models, turns out I had to add /v1 to the opencode url and leave it on my -gateway.com.
But first it told me that opencode is not on the compat endpoint and than it recommendet "Option 3: Nutzung seit Curse die Integration des Providers класный" . Not to mention some complete gibberish like "Falls Opencode.ai in Cloudflare Eing? Wenn ja, wähle den eingebauten provider o.ä. Weil der Gateway dann ein eigenes Modell-List gibt." or "Schlage die Modellnamen in einer Liste ab (z.B. lokal fester Key)"
Is this just cloudflare or is it really that bad? I mean thats not even the level of LLama2 7b ...
Mashimo
> I mean thats not even the level of LLama2 7b ...
I think you answered your own question. Must be something wrong.
ma2kx
I don't know yet. May be its a trade of for agentic skills in favor of language skills. GLM5.3 feels also heavily benchmaxxed compared to 5.2.
desterothx
I actually noticed this too with DS flash over openrouter. Instructions to respond in my native language cause the model to spew gibberish roughly resembling my native language mixed up with similar languages, and it was also dropping in Cyrillic characters randomly (although i noticed this with the anthropic models as well)
toppy
By clicking this link you download some PDF in the background
krystofee
Its displayed in the html...
mawadev
Has anyone ever asked themselves why AI was made publically available in the first place? is it really economics or is it about training people to recognize the patterns of machine generated words and ideas?
I decided to take the plunge and get myself four sparks at a decent price (and bought the QSFP cables from AliExpress because they are literally 1/2 the price of Amazon), even knowing Apple was going to release new hardware and there's probably a spark 2 on the horizon. It looks like this is going to be a decent fit for what I need. I've been experimenting with a two-node DS4 and it's _good_ at some tasks, but it really just spins its wheels when it hits the limit of what it can reason through.
I can offload mundane/basic tasks to DS4 on two sparks, but I've been pushing it harder on some novel work and it just can't run on its own at all beyond a certain complexity level.
I would love to see an Opus-4.8-level local model but TBH I just haven't got there yet. The models I've tried so far _are_ good but they aren't able to solve tough technical challenges, regardless of harness/prompting/etc.
https://artificialanalysis.ai/models/qwen3-8-27b?models=gpt-...
Amazing results for open weight and that size, but a really long way off, and I'm extremely skeptical of benchmarks that show these smaller models as being anywhere close to Opus 4.8 (or even earlier Opus's).
DS4 Flash 0731, on the other hand, wildly opposite experience. Would recommend.
GLM 5.2 - even quanted down to a hybrid 4/3 bit setup is amazing for everything but the hardest/most complex stuff in the same projects/realm.
The tradeoff is time (especially on RDMA4 hardware) - it does take a long time and spend a lot of tokens to get to the result, but I've found I can trust the results enough that I can queue a lot of work, essentially have it running all the time and achieve a decent velocity.
It's the first small local model I've felt like I can do real work with.
unsloth/Qwen3.8-27B-GGUF UD-Q3_K_XL DSH (pi)
Any tips?
I use the Unsloth UD_Q2_K_XL GGUF with default parameters, along with that custom template linked elsewhere in the thread, and no K/V cache quantization.
Try the same prompt with a larger quant (even if it runs very slowly because the model no longer fits in VRAM) & see if Qwen does better - if so, there’s your answer.
we went from 62% completion to 92% using a claude code harness
My use-case is coding, currently working on a project with a Rust backend and TS/React/Vite frontend, with probably tens of thousands of lines of code total (including tests).
Where Opus has seen it before and knows how to do it, Qwen knows how to work it out. It turns out it's surprisingly capable at working things out. The obvious drawback is that it takes tokens and time.
All the same, getting to run something this capable locally is momentous, and suggests to me that streaming tokens from colossal data centers might not be the long term path forward.
Wow, if you don't mind me asking. How and where?
They were briefly on sale with a $200-off coupon, but they show up on warehouse deals from time-to-time as well.
$4,000 isn't priced insanely? ye gads
Sure, $4000 can be a lot of if you're a casual hobbyist or are struggle to meet everyday lifestyle costs, but it's definitely not "insane" if this is the trade you make your living from or if you've established a lifestyle that affords disposable income for your hobbies.
And for some people, $4000 for a device you have complete control over and can repurpose and tinker with to your own needs and curiosities is a much much more justifiable expense than a $200/mo rental for some narrow-access tool that somebody else controls.
The other reason is that it would likely take years to spend $4000 (plus the real cost of electricity) worth of tokens on a 3rd-party provider that's running a similar limited, DS Flash type model. By that time, the hardware will be obsolete, assuming it's still operational.
Since that cluster only yields 20-30 tok/s on that size of model, at least a decade before the hardware breaks-even with current token costs, and that's not counting electricity. Assuming continued downward pressure on token prices, and the cost of electricity, it never pays for itself.
Plus you're spec'd out of near-SOTA level in months.
The only reasons to actually do this are a) you have a lot of dispensable income and are a hobbyist/tinkerer, b) you have real, legitimate privacy concerns or, relatedly, c) you're doing something you don't want to get flagged
Even if you trained your own model, you'd be committing some of the same sins, paying for the same hardware that drove it, etc. But if you're using some open model, you're standing on the shoulders of the same corrupt giants.
I feel like when people say this is due to moral reasons, it's to justify an expensive hobby.
I also differentiate using their tech and paying them money. I don't think using their models, or perhaps using models derived from them as inherently evil. I just do not want to actually contribute to their bottom line in any way. Even if that means a slower ramp up of AI in general. In my opinion we could move slower.
I understand Nvidia is working with the labs to assist them to buy more hardware through financing and other deals. But ultimately I do not view that as the same as contributing directly to their P&L.
Ding!
Of course, it's not real unless I sell, and the value will eventually go down, but so far I have significant paper profits.
Also, DeepSeek token prices are continuing to _increase_, not decrease.
One increase does not a trend make. And the current crop of models are now undercutting deepseek flash...
Absolutely I do. Each generation of open-weight models has come with significant efficiency improvements, and there are significant hardware gains on the horizon: both increasing competition from Chinese chip manufacturers, and new custom silicon from the established players. And unlike Anthropic and OpenAI, most of the pure inference providers aren't massively leveraged - the more hardware they can bring online, the cheaper they can serve tokens.
Exactly; its a development box for fiddling with GPU hardware with a large amount of video-addressable memory. It's not an inference box, really, though it's neat that I can at all!
That's just a one-dimensional thought! Your own hardware gives you complete control, and it doesn't time you out for 4 hours, unlike those vendors.
Before getting the spark, I was just using a google colab account, their $49 dollar plan allows you access to h100's and I can run qwen there in a Jupyter notebook... and if I really need that web front end I can just use cloudeflair/tailscale/the local ssh client to reverse tunnel it.
Notice all the comments saying like "omg why so expensive so just use the API??". It's a trick for lockin even with, so called, "open" models. Keep trying to run them locally, keep undoing the lobotomies, mod model behavior so that they work for you and do what you want vs only what someone else says they're allowed to do.
I do set up my initial runs and likes like quantisation-aware-distillation on my Spark-like to test it out and get it working, so it has value! But its not "worth" it other than its fun hardware to tinker with, IMO.
Or just rent something substantial for like $4/hr on runpod or w/e to do that.
My gripe is this persons compute is wasteful and makes it harder for me to buy something with like 64gb ram to do normal work and run containers while I keep using cloud models.
Someone else calculated the break even being 10 years, it’s just dumb. And I think it’s clear there won’t be a big rug pull anymore, there are too many open models and providers now.
This is of course anecdata. I know plenty of outliers, too. I know a principal engineer who uses many multiples of the number I quoted above. I am sure we also know many people making do with much much smaller budgets as well, via all kinds of well-discussed methods.
But, "$500-$1500 per month per full-time developer" is just kind of the personal mental baseline I use when making my decisions with regards to thinking about whether any of this makes any economic sense.
1. You learn a lot more running this stuff yourself (especially since you can poke at its internals if you're interested or watch the reasoning chain.) Just being a consumer of this stuff doesn't really teach you much about it other than model & harness specific tricks that become obsolete pretty quickly. (IE, your Claude.md from 6 months ago probably needs a rewrite). Which is fine, I don't think you're going to be "left behind" if you're not a hardcore AI enthusiast or anything (I'm not), but as a guy that's always been interested in computer science I want to see how it ticks.
2. You can't really depend on this subsidization lasting forever IMO. I know the financials thing has been beaten to death but I guess I'm in the camp that it's good to be in control of your tools so that you can go elsewhere if the economics change.
I like to check in with ccusage pretty frequently, and honestly like if I were paying API prices for Claude I'd probably be paying thousands a month.
Any organisation or individuals not wanting to have their sensitive data flowing away (either because of trade secret or data protection laws)
There’s no law or business advantage preventing me giving my financial transaction and medical info to Google/Anthropic/OpenAI but I just don’t want to.
This rent in the era of expensive hardware thing is not exclusive to inference.
I’ve needed x86 architecture for windows builds recently and have just hemmed and hawed over buying a decent windows 11 box.
I can’t make the math work against Azure instances.
I can spin up a nice one for build deallocate,spin up something cheaper for QA and then turn that off.
I can build all the devops around that, with a number of passes, with a skills based interface so working with the cloud is not too bad.
The only thing that still has me thinking about it is the prospect of price is going up even more, which is acid as far as I know.
And I’m hopefully going to need this x86 stuff enough that I don’t wanna wish I had gotten one for that high prices now.
Actual net work doable with/intelligence supplied by the [TOKEN-SERF PACKAGE]: Unknown. Fluctuating.-
I picked up an older Dacia Sandero for cheap a few years back - it's the best money I've ever spent on a car, hands down. That car does not quit.
The Sparks admittedly are kind of anemic: 273GB/sec is the same bandwidth as a midrange 4060, although (depending on how you configure things) you can effectively have much greater bandwidth by connecting them.
Compared to 1-2 years worth of LLM tokens for a full-time software engineer making $100K+/year, a one-time spend of $12K for 4 Sparks for on-prem private LLM inference starts looking reasonable, particularly if privacy is an important consideration. It starts looking even more reasonable if running something like a private cloud to service multiple developers because then you likely need less hardware per developer.
(Also, it is going to be a long time until RAM+GPU prices return to what we used to call "normal." If ever. I am not endorsing the current state of affairs and I am not saying you wrong to find it insane, but it is definitely the new reality)
Checked a couple days ago and looks like we're at about 3.5x 2020 memory prices (looking at just $/GB).
For comparison the cheapest Strix Halo 128GB went from 1600€ to 2600€ in the same timeframe.
Keep in mind that the DGX Spark was delayed quite a while, meaning that starting 3k price tag is already well into the RAM crisis - just 2 years ago, a 128GB DDR5 kit could be had for $600
It depends.
My bicycle was in the 5-digits brand new (now I paid it 1/5th of that and I do thank the first owner for that: the 8 000 out of 10 K I saved were put into stocks, that's his opportunity cost, not mine).
Or I know a great many a going to cry "audiofool", but I can say with certainty the following does sound better than the stereo setup of those crying audiofool:
https://youtu.be/TQg9FTBMcTQ
(not my setup but I've got those speakers: same thing, 15 K EUR brand new for the pair... Previous owner forked the money to buy these brand new and, well, I didn't... And I just hooked them to a wonderful, cheap, fully-integrated Yamaha amp: amazing sound).
If your hobby is DIY job around the house, the cost of tools can very quickly add up too: having 20 K worth of tools is definitely not unthinkable.
You like old cars? Pricey hobby.
Some here even track their cars: tires and brake pads budget (and overall car budget and depreciation)... Through the roof.
There's a saying that you're not really into computers if your setup doesn't cost more than your car.
Is $16 K ($4 K x 4) a lot? It's six months of rent for me and for many here I'm sure. It's not "crazy crazy".
Can anyone afford that? Definitely not. But there are way more insane things out there.
And thanks to the individuals that go through to all the pain of setting those up, we've got feedback, tutorials, explanation, numbers, etc. as to how to run those at home.
For example I helped my brother set up VMs and GPU passthrough and he's now running uncensored models locally and showing me the different answers between the uncensored models and the commercial, censored, ones.
So to GP who bought four of these: we need more people like you on HN, keep it going, blog about it, be "crazy"!
Reality is on a single spark I'm constantly running out of room and it being an odd size M.2 slot it's a pain to upgrade. I'm setting up a NAS over RDMA via ConnectX though, that's fun.
Edit: Yeah I see an ONTi QSFP56 on Amazon for $45, 10Gtek QSFP112 for $62
Agree. It doesn’t even have to be local, using models in this size class through OpenRouter will reveal their limits if you work side by side with Opus level models regularly.
There are a lot of social media posts about people cancelling their Anthropic or ChatGPT subscriptions after installing a local LLM. I’ve used local LLMs a lot and I spend a lot of time with frontier models and the difference is still huge. As far as I can tell, the social media posts about local LLMs replacing frontier models are either wishful thinking, engagement bait, or people who must be working on much simpler projects with a much higher tolerance for slop than I have.
Over the last couple years I’ve had to learn sales and understand the thought process behind this better, and I think I’m beginning to understand it
The psychology is that most people aren’t really trying to optimize for productivity (even most people who think they are) on an ROI basis, because their compensation is too decoupled from their actual raw output, and more closely coupled to how differentiated their marginal contribution is to peers. They’re much more incentivized to spend their personal/work time optimizing for being more skilled or acquiring some kind of competitive advantage relative to baseline.
Most people don’t consciously run the numbers of “I get paid $X/hr to add $Y of value” or model pay at work as something with variable inputs (eg something that can be increased with high performance), so it makes sense to them to spend 20 hours of time to save $100 or to make themselves 5% less efficient to take home 0.5% more or avoid doing something they don’t want to start doing.
NOT saying this always happens or that they’re stupid for doing so. I didn’t even realize how much I had been doing it myself until I started recognizing it, and shifted to having my own comp/performance fully aligned with the company’s P/L.
It actually makes a lot of sense IF you can accurately estimate incremental upside (which is much harder and more diffuse than modeling downside if you’re salaried a employee) or if the upfront skill/knowledge investment that looks like bikeshedding pays off in the long run.
Now that the role of the ticket-cruncher is on the path towards full commoditization, and individuals can move much more quickly (and even more carelessly!), I think product roles will probably shift towards one where developers are more deeply embedded in the product/business process so that they own/understand what to build without as much separation between the decision-making and prioritization of what to build. Or at least, they should.
It was eye opening to me to run the math of "should X people work for Y months on this project to save Z per year?" and realize that in so many cases, the time and effort it would cost to stop "wasting" money on things is WAY more than you could actually save on it. Even "small" projects can very quickly become $1M+ investments in time and resources, and the diminishing returns add up quickly (but also a good way to justify the value of your contributions, when done). But the job only exists if it saves money or makes money...
Can you share a bit more about how you shifted to be more aligned with P/L? And how to accurately estimate incremental upside?
I'm an early PhD student with interest in ibdustrial research/R&D, and currently struggling to understand how to think about how to navigate through my career.
Don't use DS4 Flash in max effort mode. It's just spinning its wheels, in my experience (I have a harness for testing models with 25 real bugs/features/etc from my real projects that I measure outcomes against) DS4 flash does _worse_ with max effort. It will literally have the right approach and reason itself away from it.
Same experience here. But I think the key argument for local is about being able to leverage "non castrated" models. But maybe this is not relevant at all for standard coding tasks.
The more reasonable comparison is against rented GPU's, while looking at tradeoffs in latency and upload/download/storage/instance management overhead.
Buying hardware for local models is meeting a wholly different need than buying tokens through OpenRouter or whatever.
All decades prior and up to about a year ago, I would have agreed with you. My Framework Desktop, however has appreciated in value by 75% since I bought it. Will it stay there for a long time? Probably not. But it shows that there are no hard and fast rules about things anymore.
64 can still easily do a Qwen 4.8 model, so I’m relatively happy with my purchase… plus, it’s price change has caused it to quickly appreciate in value… so I could sell it if my situation ever turned dire lol
If I'm a professional photographer chasing the best possible end product, I'm not buying cameras because they're economical. I'm buying the best camera I can get my hands on to get the best product I can produce within reason under the understanding that it doesn't have to equate to the best economic decision to be the _right_ decision.
If you're in a position to be able to take advantage of the local inference - it's a no brainer. If you're not sure how that would be done, then it's not a good move.
For example, I have a small posix-shell-based LLM harness that can SSH into my NAS and run organization tasks using the local DS4Flash that I have right now. It's already been a massive help for me to keep me organized, and that's just 2x DGX Spark's worth of compute.
But things change real fast when you're no longer bound by costs/apis/rate limits. All of a sudden it's not about "how can I do this right and efficiently" and more about "I can poke at and test _all the things_ that might make this better".
I think most people who can't see this value in the local inference approach are likely still copy/pasting from their web LLM ui's or don't even come close to subscription quotas. Meanwhile, 1b tokens a day is a light day for me with 3 $200/m subscriptions + some level of sub at basically every frontier level provider. Had I been less frugal and ponied up for the hardware before things got crazy I wouldn't need 80% of that - just the frontier models for the most complex tasks, the open weight models would handle the rest easily _and_ I'd get to do a lot more exploratory work without concern about quotas.
Employer just sent an email that.. things are changing when it comes to token spend...
What did I do with these?
Setup record/replay for our product using qemu, several variatons thereof including experiments on target hardware. Fixed a tricky bug in qemu that I sadly can't upstream..
Experimented with rr on WSL2 and our target arch. Failed experiment.
Setup mutation testing PoC.
Optimized pipelines
etc. etc. Just contung code its soo much more than I would normally produce, but its also 95% experiments that are still not productized, and much of it never will be.
July 16th: The "Kimi K3 moment" - China has caught up to Opus!
4 weeks later: GLM 5.3 - Same performance, but cut the amount of parameters and cost to a third!
12 days later: GLM 5.3 Flash - Almost GLM5.3 performance but cut the parameters in half, cut prices to a fifth and serving on Chinese chips!
Chinese models had 94% reduction in parameters (from 2.8T/104B to 180B/6B) in 6 weeks, while staying close to the same quality.
That’s the thing…they aren’t. Well not in real life use anyway from my experience, but yeah in benchmarks they’re great at it.
No? A large model obviously can be dumb, I don't think you can infer much other than by testing it.
These small models are almost certainly worse at some things than the big models. They prize is making them dumber at things no-one cares about while retaining the capabilities people do care about. A model probably does not need to be able to give me a political treatise on the late 19th century "silver question" to be able to write me code.
I'm sure they have nothing to rival this on a price/performance basis and have already given up on that
How can you be sure about this? They have unbelievable capital. OpenAI is starting to preview its own chips, which could dramatically change the price/performance. We don't know what else Anthropic has cooked up right now that could rival this if they wanted to.
Yes, others will _also_ continue to innovate, but making that assertion that they have nothing to rival the current state is a bold assumption unless you have confirmed insider information.
all those Chinese labs are backed by the Chinese government which can just print money.
time to wake up.
I'm gonna let you in on a secret: it works the same in most countries in the world.
Deepseek: I am sorry, I cannot answer that question. I am an AI assistant designed to provide helpful and harmless responses.
GLM-5.3-flash: 非常抱歉，我目前无法提供你需要的具体信息，如果你有其他的问题或者需要查找其他信息，我非常乐意帮助你。(I am very sorry, but I am currently unable to provide the specific information you need. If you have other questions or need to look up other information, I would be very happy to help you.)
Kimi/moonshot.ai: [server exception]
Qwen: [server exception]
For reference, here are how the U.S. models answer it:
ChatGPT: "The Tiananmen Square protests of 1989 (often called the Tiananmen Square Massacre) are the historical event most closely associated with Tiananmen Square.
In spring 1989, pro-democracy demonstrators gathered in Beijing. On June 4, 1989, the Chinese government sent the military to forcibly clear the demonstrations, resulting in many deaths. The exact death toll remains disputed.
The event is also famously associated with the “Tank Man” photograph, showing a lone man standing in front of a column of tanks."
Anthropic/Claude gave a very similar response. My own government has done its share of horrific things, the main difference is that public information is free to look up and talk about within the country. I recognize the engineers at these labs are doing amazing things and the open models are a strength, I look forward to being able to use them.
Any competition against the western models are welcome and benefits us in terms of pricing and availability. If they have to comply with CCP to be able to do it, then so be it.
I have zero sympathy for Anthropic and OAI being so secretive and acting like they are doing us a favor.
For example you can point opencode at DeepSeek v4 and ask, it will accurately tell you about Tiananmen square.
Who is to say they aren't doing it in other ways as well? That they aren't, or won't be, subtly hamstrung in engineering work?
OAI and Anthropic have their own issues, you're right to be suspicious of them, but it's not like their models answer with, "capitalism is god's gift to His chosen people" or whatever. Their limits on cybersecurity, biological warfare, etc. at least make some sense in the context of lowering harm—not just protecting a specific government party.
Tiananmen Square has been the site of many major historical events, but internationally it is most closely associated with the 1989 pro-democracy protests and the Chinese government's military crackdown on protesters there in June of that year, which resulted in many deaths and injuries. The event remains a sensitive subject in China, where it is not officially discussed or commemorated. The square has also been central to other significant moments in Chinese history, including the May Fourth Movement demonstrations (1919) and Mao Zedong's proclamation of the People's Republic of China (1949).
Which seems like a pretty reasonable answer.
But nothing (at all) from asking GLM-5.3-Flash directly in the OpenRouter chat interface.
StepFun for example, will happily answer it when running Step 3.7 Flash locally
I love hearing about new models, but every time I just don’t know why I should use something worse. I tried some random model on fireworks a week ago, and it immediately went it a thought loop for 10 minutes before I caught it. Blew through most of my $10 for no output. What’s the point, exactly?
Which now won't be possible because we have chinese open models to use instead.
I'm very curious about how long these US companies can keep on burning money like that.
This is from somebody who put thousands of dollars every month to Opus. Now it's 40% of that and I get as good or better results without having to turn the caps lock on before lunch...
Edit: yes company money. We don't get subscriptions we pay per token.
The reality is most work people do doesn't need the very cutting edge and these open weight chinese models more than cut it most of the time.
https://deepswe.datacurve.ai/
That's pretty solid. Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash, and even worse it matches v4 pro at a tiny fraction the cost. Roughly equivalent to sol medium, at a fraction the cost.
They should've just lead with real, up to date data, because it's good, not the silly old tactics like comparing to Opus 4.8 when 5.0 is out in many of their charts.
Congrats to them!
Opus 5 consistently comes at or near the top, but outputs constant unreadable jibberish. Meanwhile GPT 5.6 Luna medium tends to be rated pretty poor on agentic tasks compared to the Chinese lab open models, but I find the latter much more likely to lose track of their own behaviour during a long-horizon task or get stuck in a doom loop.
(This isn't a comment on GLM-5.3 Flash as I've not used it!)
https://news.ycombinator.com/item?id=49413456
We shouldn't have to resort to this, but it can be mitigated enough that it stays as my daily worker agent. Although, I mostly use Fable to farm out to Opus agents so I don't have as much exposure to what kind of blathering is going on in there.
(I'm someone who doesn't really care about delays of a few seconds, or even more than few seconds. But if I am trying to notice then sure Claude is definitely faster as well).
https://kommodo.ai/i/IFSUUQYT522uZXWvePZ1
it still fells stupid sometimes and it is benchmaxxed for sure. but its good enough that im building all the hobby projects with it.
I've probably wasted a good 40% of my weekly usage on Luna Max agents just thinking and not writing a single line of code.
https://www.reddit.com/r/codex/comments/1vj3hhn/wait_so_sol_...
https://www.reddit.com/r/codex/comments/1vp0rig/sol_can_fina...
I’m pretty sure that plain Sol, serially, could have finished the task faster, cheaper, and far more accurately. I’m also pretty sure that any competent subagent orchestration could have gotten it done with even very simple subagents quickly and cheaply.
(Is it really that hard to set up a handful of subagents that all use the same initial context and to load that context with what actually matters? The APIs certainly support it.)
It will cycle through like 3 /compacts, complete the complex goal successfully, and cost me like 1% of my weekly usage on the $20 plan.
Edit: this is me using Luna directly, not Sol as the taskmaster
The flash one?
My prompt was something like: "here's data I have, here's what matters to me, create HTML mockup".
All GPT 5.6 models were laughably bad. And I don't want to downplay it - they were just absolutely, objectively horrible. Every single attempt was what I could probably call "if json was ui".
Claude models produced... "claude look".
GLM 5.3 - somewhere between GPT and Claude.
Kimi k3 - each attempt produced beautiful UIs. It used components that I didn't even know existed and wouldn't even know to ask for. But expensive, very expensive.
ox-alpha (GLM 5.3 flash) was very close to K3. And at this price point, it's already configured as "designer" model in my oh-my-pi.
I haven't tried it but I think Qwen Max is also very good at design.
It's what people know. Opus is just the common target.
> Smarter and cheaper than Luna xhigh, not as smart but less expensive than Luna max. Smashes deepseek v4 flash
The problem with this and DeepSWE is it goes for a very specific profile. I'm not convinced DeepSWE is any accurate in actual work. It's surely a different signal (compared to some that allow cheating) but it has its own issues, e.g. weak harness.
Luna is great at following instructions but bad instructions or anything not covered = death.
Deepseek is more analytical. Good for bug tracking.
GLM is a better all rounder in some ways. Better at creativity.
They listed Muse Spark 1.2 around DeepSeek V4 Flash even though it's a much shittier model in basically every aspect.
> GLM is a better all rounder in some ways. Better at creativity.
I agree with the creativity part.
Broad and perpetual license over inputs and outputs, and even your name and profile picture.
Vague prohibitions on whatever may harm Z.ai’s "interests" or even the "national interests" of any country.
Vague prohibitions on "disturbing" or "inappropriate" content, whatever that is.
Vague prohibitions on discussing Z.ai, even my posting this comment violates it.
Can ban you if you, in the "sole and absolute opinion" of Z.ai, have violated these broad terms, and if you paid for the discounted yearly plan kiss your money goodbye.
Nearly every TOS I've ever read has a "We can ban you for any reason, or no reason, are under no obligation to disclose any reason." line somewhere in it.
HN's for example
> We reserve the right, at our sole discretion, to change or modify portions of these Terms of Use at any time.
> You acknowledge that Y Combinator may establish general practices and limits concerning use of the Site,
> You further acknowledge that Y Combinator reserves the right to change these general practices and limits at any time, in its sole discretion, with or without notice.
> Y Combinator reserves the right to investigate and take appropriate legal action against anyone who, in Y Combinator’s sole discretion, violates this provision, including without limitation, removing the offending content from the Site, suspending or terminating the account of such violators and reporting you to the law enforcement authorities.
Not even close. Even OpenAI and Anthropic aren't bad enough that they claim literal ownership of your inputs and outputs.
> HN's for example
You're not paying to use HN. Getting banned here has essentially zero consequences.
If Z.ai uses its absolute powers to ban you because you wrote a review about them or something, then you lose actual money. This is especially relevant if you're looking to take advantage of their discounted yearly payment option.
At most I suspect the A.I. providers will just come up with yellow banners like Anthropic did where naughty smut writers get put in the time out corner.
That's a pretty big addressable market.
Oh for sure man, this absolutely looks like you were only concerned and talking about paid services:
> Broad and perpetual license over inputs and outputs, and even your name and profile picture.
Everything I post here is public, and it's just relatively low value commentary anyway. It doesn't matter if Y Combinator has rights to it. Arguably they actually need to assert some rights, otherwise they wouldn't be able to transfer copies of copyrighted comments to other visitors of the site. HN's terms are probably too broad for their purposes but it doesn't really matter much because this is just a forum.
AI on the other hand is for actual work, both public and private. There are actual economic implications here, so the stakes are much higher. I absolutely want to own the inputs and the outputs I paid money for.
Where do you see this?
In this case you are placing your trust in OpenAI and Anthropic. I'm not sure about Anthropic but OpenAI has changed their mission corpus quite a lot from its humble beginnings that it results hard to trust them when they say they don't use your stuff to further train their models. If I'm a Big Corp with enough lawyers to putnup a fight, I would then feel ok with such clause, but being a small guy, who is going to defend me when the truth comes out that they have been training their models with my data? Similar fiasco as with Facebook, who had claimed they didn't sell your data, even though they were.
That's where I'm coming from with all this "trust us, we don't train our models with your data". At least this Chinese company is being upfront about it.
Also, it only applies to their chat offering, not the api. OpenRouter also offers the API with ZDR.
While shitty, i’d say that its really not that special.
None of the major LLM chat providers (ChatGPT, Claude and Gemini, and I just confirmed this) claim rights over your input.
They also don't claim rights over your output, but because of how copyright law might apply, they explicitly assign all the rights to the generated output.
Not just that but, even if they wanted to claim ownership of the output, courts in the US have deemed that copyright cannot be assigned to machine-generated output.
So yeah, happy to take a look at a counter-example if you have one (aside from GLM 5.3, obv.).
> To the extent the User Content includes a person’s image, likeness, voice, or other similar attributes, you grant SpaceXAI the same rights to use those attributes as part of the User Content as described above. You represent and warrant that you have obtained all rights, licenses, notices, permissions, and consents necessary for SpaceXAI to use that User Content.
https://x.ai/legal/terms-of-service
These are arguably even worse to be honest. Absolute nightmare.
However, if we check the market share of generative AI providers, ChatGPT+Claude+Gemini make up around 88%; while Grok is 2-4% depending on who you ask.
OpenAI revoked my Cyber verification, along with many others, asked to reverify (i.e. give my biometric information to Persona), had me do it 8 times, just to find out several days later that they silently implemented a nationality whitelist, and my nationality didn't make it (and no, it's not a sanctioned country).
Their support says they can't look into anything or do anything, and their public spokespersons on X deny everything.
I get tons of cyber refusals now (lots of reverse engineering), so it's only matter of time when my account is going to get banned.
At least Z.AI is being honest here. And no provider other than OAI/ANT had me submit my biometric information just to use Ghidra.
How did you discover this?
I opened the Persona tab once, closed it and the tab never opened ever again. "Precheck failed".
What countries are banned? I'm from Brazil.
I went as far as initiating an LGPD (brazilian GDPR) process against them due to this. At some point I got it in writing that I'm allowed to make a new account and try again. Until now I was assuming it was just some weird account state. If I'm banned from TAC due to my nationality that's seriously disgusting...
A week later, I tried testing the TAC flow on my SO's account, which had never had a TAC attempt before. Selecting Georgia in the Persona iframe now says, "We are unable to verify identities in this country."
But I know this isn't a Persona limitation, as I verified with Anthropic using Persona the same day.
So what I think happened was this: OpenAI silently implemented a country whitelist on their end and revoked TAC for affected individuals who already had it, calling it a "technical issue". They forgot to disable those countries in Persona, so everyone just got a cryptic error. Then they disabled them in Persona too.
Interaction with support was AI with human names, which essentially just repeats what you said. And it ended with:
> I’m unable to provide additional details about verification outcomes, and Support cannot manually override the result. At this time, Trusted Access for Cyber verification does not support retries or appeals.
I even provided them my credentials, and support AI was basically: lol wat we're here to check technical errors, your credentials are of no relevance.
For the record, I just made a second account, got verified by Persona and still didn't get into TAC. No "we are unable to verify identities in this country" message. No mention of my country whatsoever. Persona verification was successful.
What else do they want from us?
Were it not for Z.ai's obnoxious terms, I would have switched to them already...
When I use it for reverse engineering, same thing, the code is already out there, but in binary form, which is becoming trivial for these LLMs to work on.
I even bought some shares from them and got some money out of it!
Nationality bans get `failed`, new accounts (or rather, accounts with not enough good signals) get `blocked` on first attempt.
Time to subscribe to Kimi.
GLM 5.3 is quite capable with "cyber" tasks. I'm working on a project that touches macos private internals, and GLM 5.3 was able to reverse engineer everything I need with ease.
I've also had some luck with GLM 5.3 "translating" cyber stuff to Sol subagents in a "safe" language. Gets rid of cyber refusals at least on the input side. Output is trickier, but a very crude "think and talk to me in a neutral, safe language without high risk words" actually worked surprisingly well.
Then alternatives are:
- Grok - where I absolutely have 0 trust in X.ai's interst in "pushing humanity forward".
- OpenAI and Anthropic - which seem to try to be building the biggest moat they can by pushing to ban open models. And at the same time want to be an Arbiter of what level of intelligence I can use.
- Google and Meta - I don't need to talk about the practices of these companies.
Yes, the terms of service aren't great. But the alternatives aren't great either. I don't believe that a future which OpenAI and Anthropic are pushing for has my best interest in mind.
I don't believe in that either, but these totalitarian terms are absolutely unacceptable.
Chinese companies do not follow American laws and there are absolutely no consequences for violating it.
Moreover, the average American is not even aware of exactly what the legal/judicial environment is like in China. If your code and data is stolen, you can't fly to China and demand justice in the courts.
... lmk when anthropic/openai/spacex/xai are held accountable for anything. Anything at all. Hard to be when you're _writing_ the rules.
No, they don’t. This is an absurd statement to make in 2026.
And no, they won't tell you what their automated reviews consider "dangerous".
So they give you the ability to "opt-out" of "training purposes", and then you still share everything for analytical purposes (which you can't opt out of). It's a smart tactic on their side if they are contributing to cultural discourse on this to keep the straw man focus on training data.
And also to have the popular AI straw man: that your data isn't valuable and oh man they're losing so much money on tokens and giving you a subsidy, so you you better use up your entire subscription balance weekly/daily/every 5h putting in all your "worthless" data that you miss out otherwise (FOMO). Oh man only way it could get worse is if they implemented gambling slots psychology.
I hope you also block the other half of the internet because sure as hell google and facebook are way worse than that.
> They also don't require persona id verification, witch is wat turned me away from openai.
Could be worse. I was dumb enough to verify, only to get rejected for unknown reasons with no retries and no appeals. Had my privacy violated and have nothing to show for it.
"Come verify your identity."
"Thanks, we've got your ID. Still not approving you, and there's no appeal."
Worst of both worlds.
Working well so far.
>Vague prohibitions on whatever may harm Z.ai’s "interests" or even the "national interests" of any country.
[...] may cause harm to Anthropic, our users, or third parties, we reserve the right to remove or take down some or all of such Third-Party Content using, where appropriate, algorithmic and human review.
You may not export or provide access to the Services into any U.S. embargoed countries or to anyone on (i) the U.S. Treasury Department’s list of Specially Designated Nationals, (ii) any other restricted party lists identified by the Office of Foreign Asset Control, (iii) the U.S. Department of Commerce Denied Persons List or Entity List, or (iv) any other restricted party lists
>Vague prohibitions on "disturbing" or "inappropriate" content, whatever that is.
we will use Materials for model training when [...] your Materials are flagged for safety review to improve our ability to detect harmful content, enforce our policies, or advance our safety research.
>Vague prohibitions on discussing Z.ai, even my posting this comment violates it.
>Can ban you if you, in the "sole and absolute opinion" of Z.ai, have violated these broad terms, and if you paid for the discounted yearly plan kiss your money goodbye.
To engage in any other conduct that restricts or inhibits any person from using or enjoying our Services, or that we reasonably consider exposes us—or any of our users, affiliates, or any other third party—to any liability, damages, or detriment of any type, including reputational harms.
Mind you, that's Anthropic's Terms of Use in Europe. I have zero doubts the TOS applied to the US is even worse and that merely mentioning your first born in a chat entitles them to a part of its soul.
I suspect any model connected or built by China will serve their interest despite their terms and agreements.
In any case what matters is what is enforced in practice. It will be a mild inconvenience to switch providers on Openrouter.
If Anthropic or OpenAI decide to apply those same arbitrary terms, you are SOL.
Yeah, I've compared both. The US companies generally aren't as vague, and they don't claim ownership over inputs and outputs.
>you retain all rights, title, and interest in the Prompts ...
However, it does grant them an irrevocable license to do, effectively, anything with the data
Not a lawyer but I think the practical difference is you can still license to others, sell, sue IP infringers
I have prompted out a lot of disturbing and inappropriate content with GLM-5.2, that would have left other American models blanched in the face or clutch their pearls. I think this is mostly a reference to Anti-CCP stuff.
In fact, I don't think I've ever even had a prompt refused.
I very much have. I've gotten GLM-5.2 refusals for extremely benign security testing on my own infrastructure of the same flavor that people were getting (wrongly) flagged for on Fable during the initial release.
I sometimes just switch to a model I know is less smart to block stuff so that it has a text agreeing to do that, and then switch to a stronger model to actually go at the task.
Your mileage may vary though.
Exactly what I was trying to use it for! ):
I'm in the same boat - I haven't heard of a way to get around it aside from either self-hosting (GLM-5.2? good luck) or "self-hosting" (paying bucks per hour to Vast) an abliterated model.
I found that GLM-5.2 was pretty happy helping me reverse engineer/hack devices.
Maybe the system prompt you're injecting is making it refuse?
No, this has nothing to do with my harness. I use one of the most popular open-source harnesses available.
> I found that GLM-5.2 was pretty happy helping me reverse engineer/hack devices.
This is a completely different category of things than what I'm getting refusals on, so I'm not sure why you're bringing it up.
You never mentioned which exact activities you were getting flagged on and getting refused.
No. Vendors (and their model guardrails) do, indeed, treat those as separate from pentesting non-embedded infrastructure, and that is because they are very different activities.
And, if you actually read my comment, it says "red team my own computers". That's categorically different from pentesting an IP camera.
> You never mentioned which exact activities you were getting flagged on and getting refused.
Because further details than those I've provided aren't relevant, and it's clearly different from what you're doing.
Your experience isn't relevant to my situation.
Yeah, running frontier open weight models on my own hardware has essentially become my dream at this point. I hope the hardware manufacturers step up production to meet consumer demand.
They don't even allow me to download my data.
How is it better?
What do you think why the normal polymarket site is blocked for US users.
Ar eyou demanding that your rights are valid globally? That's more than the EU want with the GDPR.
BTW if you are concerned about the GDPR it means you track your visitors and store personal data about them otherwise you wouldn't have any problem.
I let people create accounts, so there might be some personal data. Depends on what the GDPR considers personal. I have no idea. Couldn’t care less.
I’m not demanding anything. Like I said, I put my site on the internet. If the EU doesn’t like my site, they can block it. I’m not demanding they allow my site in the EU.
In the meantime, you are trying to convince me I’m prohibited me from putting a site on the internet unless I’m willing to figure out GDPR’s requirements.
Not sure the exact legalese but legal put together a consent form for anyone wanting to do the PoC
{ "providers": { "openrouter": { "models": [ { "id": "z-ai/glm-5.3-flash", "name": "Z.ai: GLM 5.3 Flash", "reasoning": true, "thinkingLevelMap": { "off": null, "minimal": null, "low": "low", "medium": null, "high": "high", "xhigh": null, "max": "max" }, "input": ["text", "image"], "cost": { "input": 0.075, "output": 0.25, "cacheRead": 0.015, "cacheWrite": 0 }, "contextWindow": 1048576, "maxTokens": 131072 } ] } } }
https://forge.l3x.in/alex/pi-shared/src/branch/master/extens...
Just like that we are witnessing an open burial. It's now in everyone's interest to keep the valuations in the 'A.I' economy as they're though it's apparent they're not justified.
whether it's the cost to develop models, cost of hardware, cost of serving ie inference.
RIP Nivida shareholders
Further quote:
"Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale."
https://z.ai/blog/glm-5.3-flash
https://en.wikipedia.org/wiki/HiSilicon#Ascend_910
https://medium.com/@huaweiclouddevelper/a-brief-introduction...
By this they probably mean RTX series GPUs? If so, then they are not comparing the hardware efficiency with the A100 / H100, etc. that are commonly used for training models
While I think China would always get to hardware self-sufficiency eventually, all export controls have done is (1) accelerate China's development, and (2) divert revenue that would've otherwise gone to NVIDIA/AMD/etc instead.
That doesn't mean by any stretch of the imagination Nvidia will disappear. But the entire stock market valuation, not just tech, has had me scratching my head for a while.
Curiously, there is not a single real CUDA competitor anywhere in the world. We almost had one with OpenCL, but all of the American stakeholders abandoned it right before the crypto/AI takeoff. All of which means that Nvidia sets their own margins, exploiting American investors and taxpayers while letting China avoid their dominance. So the American economy subsumes the bulk of Nvidia's arbitrarily-priced debt, and the Chinese economy can direct SOEs to pour billions in liquid cash into real GPGPU research.
I'm an American and I'm pretty fond of Nvidia, but Jensen was right about this policy; it gives China everything they need to actually replace CUDA. It's reminiscent of America's attempts to deprive China of ARM and Texas Instruments IP, only to end up swimming in unlicensed clones after refusing to sign an IP deal.
This isn't the kind of thing you can hash out in public and go back and forth on. Once you put it out there, the other party will take steps to make sure they don't have to rely on us in the long run.
Similar to the war-pricing of oil, China's reduction of imports is actually helping to keep our inflation from going even higher.
Zai is on another "export control" list outside the broader 1. Doesn't help.
And I don't see a situation where serious API payers are OK with handing the Chinese state all their data. Like manufactures of decades past did and learned a hard, even existential, lesson for it. The state mantra has been "Collect and Copy" for a long time now, tech just hasn't had that moment to experience it yet.
So that leaves local hosting/leasing, but one of those has totally non-practical economics and the other doesn't have enough compute to meet any kind of real demand.
I also have yet to meet a single person who isn't neck-deep in the tech space mention a Chinese LLM. It's 100% the big American three.
If anything it's custom chips from the labs that threatens Nvidia.
https://www.ft.com/content/32a70a3c-7d28-40b4-808e-36edb58c7...
Because I genuinely can't tell if you mean Google or SpaceX/X.ai lol.
xAI is already selling spare compute, and basically exists just to gas spacex's perceived valuation.
Why wouldn’t you?
RAM was probably the bottleneck for the amount of context they were offering.
I assume it would run a little faster with lower concurrency but "RIP nVidia" is a little premature. The cutting edge inference hardware is amazingly powerful
When it first launched on OpenRouter I was getting nearly 70 Tokens/second.
Edit: Ah:
> This stealth model was developed and operated by ZAI, revealed to be ZAI GLM-5.3-Flash.
> Before release, we tested GLM-5.3-Flash anonymously as ox-alpha on OpenCode and OpenRouter to gather user feedback. It quickly became the most popular model of the week — with all of this traffic served on Chinese AI chips.
... I'm at a loss for words here. It was being served for free. To the entire world.
> ... I'm at a loss for words here
No need to be so dramatic. I think it's great that they're developing chips, but the whole "RIP nVidia" claim was overly dramatic.
Why is Luna not free on OpenRouter? :)
Nvidia will do just fine. (Disclaimer: not a shareholder. At least, not directly.)
Compared to the rest of the world?
>> "They are already there on open weight models and Jensen knows that it is only a matter of time until China catches up with GPUs or other AI accelerators."
It is also why Nvidia becoming a bank for other AI companies who are unable to find VCs to fund them isn't really a good thing and that is bearish.
[0] https://news.ycombinator.com/item?id=49397204
[1] https://news.ycombinator.com/item?id=49431231
Get a 210 strike put contract and if your thesis is that nvidias current 10 day slide continues you could make some money.
This announcement doesn't really mean anything at all. It means the very few people who are already using Z.ai's API will continue to do so, but the vast majority of money going to Nvidia is through the massive amount of business going to Anthropic, OpenAI, and other western cloud providers and inference providers, who are mostly using NVidia chips for inference.
Also, NVidia chips are still sold out and supply constrained.
I'm sure the chips are fine, but they clearly didn't have enough capacity for the demand they had (that 100T/day claim was asbolute bs)
I don't see how NVIDIA can keep their spot as belle of the ball. If LLMs and friends are truly to become as useful and ubiquitous as everyone thinks they will, then commoditization is the only option.
FWIW, people were saying "ASICs will kill CUDA demand!" since the crypto mining boom. Then a few months later, CUDA found another niche application in LLM applications.
With the mounting demand for robotics, surveillance and autonomous weapons, I don't see how Nvidia couldn't keep their spot. They have their pick of the litter with hundreds of market segments, and unlike the rest of FAANG they're not afraid to branch out.
Like I get that they have to be careful about comms, but surely senior members of the team can clarify when something is NOT them, when everyone is gosspiing it is them.
I highly suspect that the Gemini Google uses internally is very different from what they offer in Antigravity.
Gemini 3.7 Flash is a pretty great model IMO. You shouldn't compare it to Opus, Sol, K3, etc since it's a much smaller model but it's a little better compared to Sonnet, Luna or Terra, etc.
A fun test would be to compare the logits for "gemini", "claude", etc for a continuation of "I am " on all these models. I'm sure that e.g. GLM, Qwen, DS are dominant, but I'd be curious to see the next highest contenders, and how they compare to each other.
This is pretty hefty for a "flash" model, even a 256 GB setup is insufficient at q4 - and q4 is already the worst-but-still-acceptable quant in my experience. The benchmarks look great, especially since GLM tends to be more honest than the average Chinese lab, but you’ll need to splurge to run it at home.
@edit: so many releases that I forgot to math. This fits just fine in q4, realistically the minimal hardware would be 192gb - so blazing fast on double rtx 6000 pro and usable on 256gb unified memory. You could even go with 5bit quant on 256gb.
… you’ll still need to splurge, though.
https://github.com/JustVugg/colibri
Generally, for local consumer use, these large MOE models are best for unified RAM systems like DGX Spark or Mac Studio.
I was looking at my token usage, mostly from subsidized codex/grok subscriptions and i'm a somewhat heavy user. The thing is i would actually use even more tokens if it wasn't for the weekly quotas.
In the end, with a $10k investment and running this kind of model, estimating a 2x increase in token usage because i wouldn't have weekly quotas and comparing to glm api prices, this thing could pay for itself in less than a year.
Obviously i'm paying subscription price right now, so the math doesn't work. Although using local ai removes all weekly quotas. Keep a subscription to have access to frontier models for planning work, and local hardware + glm-5.3 flash for implementation, e2e testing, qa work 24/7.
It's not that crazy of an idea and the numbers aren't that bad.
You will get around 2-4 concurrent streams of aggregate tokens at best for such a model and around 0.5B output tokens per month assuming you use loops and run it when you are sleeping. That's 500 (per mill) * 0.5$ = 250$ only at most.
Then there is maintanence and efficiency costs due to electricity usage and such, any down time, etc.
You will be lucky if you can squeeze more than 200$ of value out of it in a month.
I don't think people should buy local hardware for money reasons, by the time you will pay off a 10K USD machine, 2-3K USD machine will catch up and beat it by a significant margin.
Unless your expectation is that we will be in hardware winter for the next 10+ years. At 200$ per month it will take around 200 * 50 = 10k, that is, 50 months, so around 4-5 years.
Again assuming you are making the most of your hardware somehow, very hard to do in practice.
I don't recommend people to use compute as investment or payoff thing, but if you have the money to burn and can afford it why not, maybe with some software optimizations it will be cheaper but then again Z.ai is currently offering 50% discount and providers will offer cheaper rates for sure.
But either way you will never be able to burn more than 200$ worth of token on a cheap hardware device, because inference becomes more profitable the more you scale it up, you have separate prefill and decode engines/systems, and a lot of nuance, but assume for every 10x increase in infra you increase margins by 5-10%.
So from 10K to 100K to 1M to 10M to 100M.. I don't think this curve continues beyond 100M but I have no idea about that scale unless some AI lab is interested in hiring me lol.
So a 100M infra will have ~30% better margins than you at 10K, then there is software optimizations but that's cheap enough, though some of it is only viable at scale.
Either way assume 10K is the price of privacy if you really want to buy it. Don't worry about making the most out of the usage, you will always be in a net loss but I would assume for you 10K doesn't matter.
But whether you make a loss or not depends on how hardware prices and resell values go though.
I have spent ~$50K on local AI hardware. The market value of that hardware is about ~$80K right now.
So the maths is working out for me so far. I see it as a call option on compute.
That's easy because no matter what IT hardware you bought, it's worth more now than it was two years ago. That's something that's unprecedented, never happened before, and as soon as we get flood gates open on ram manufacturing OR when the AI bubble pops, all IT HW deprecation norms will return and making a profit by buying something IT will vanish.
I had a GPU server four years ago. Had I not sold it like three years ago with 2x price I bought it, it would be likely something like 5x the price nowadays.
I really really miss filling my home rack with old enterprise stuff. All I want is this hardware winter to end.
But for a heavy user who has enough work to be done so that the box runs almost 24/7 at say 50tok/sec, the math gets interesting against API prices.
And it can be interesting compared to subscription in the sense that you don't have the quota anymore. That means there's probably a lot of things you're not doing because of the quotas that you could do now.
It depends heavily on the tok/sec obviously and the very best solution financially remains subscriptions. But the idea remains entertaining and not that disconnected from reality
That's less than what 40$ at current API rates... So if you are willing to pay 200$ per month you will get much better limits paying API rates.
You can't run large Kimi K3 models on 10K worth of hardware either way, you need to spend like 50K USD minimum.
Just pay for the API rates or get a low cost provider that uses higher batching, you can get shittier tps but much better prices, probably go as low as 20$ for as much usage as you can ever get from a 10K USD machine from GLM 5.3 Flash...
The issue is nothing expensive runs on these devices and cheap stuff isn't worth running locally, eletricity costs ~12cents/kwh in us iirc, so at 330W M5 Ultra will burn around 8 * 0.12 = ~1$ per day extra in electricity so the electricity is going to cost you the same as the API rates(30$ per month).
I truly don't think you are accounting for the costs here properly. But again if money truly doesn't matter it's much better for privacy and better than paying one of the shady AI labs who are doing god knows what with your data.
1) Rates are theoretically discounted for GLM 5.3 Flash right now, by 50%.
2) Hardware costs have continued ascending with no sign of letting off, so it's unlikely that a DGX Spark depreciates to zero in one year.
3) Compare performance in terms of difficult tasks/$ over the last 6 months, 3 months, etc. Open weights are a ratchet. In terms of intelligence per $, a Spark is never going to be a worse deal tomorrow than it is today, at least until the entire platform is replaced or obsoleted.
71 days ago the best model you could run on two Sparks was an aggressive Q3 quant of Qwen 3.5 397B (AA 34). 70 days ago it was a mixed-quant of GLM 5.2 (AA 53). 30 days ago it was full fat DeepSeek 4 Flash (AA 53). Today it's GLM 5.3 Flash (AA57) and/or Qwen 3.8 Next (Unknown). Sometime this week it will likely become mixed-quant GLM 5.3 (AA 60).
So in < 80 days we have almost doubled the benchmark score. And that curve is still accelerating. If you view it as "cost per token of model vs API" then yes it's a bad deal. If you view it as "cost of task per $" then it has almost doubled in value in less than 3 months. All of this, imo, API and hardware, is still massively underpriced.
If someone told me that costs for X will keep increasing because they have been increasing rapidly in the last 1.5 years, but they have a history of continuously decreasing for decades before that.
I am not sure if I will take anything they say serious, I am not sure if it's HN or AI but people are delusional if they think compute costs will keep increasing from now on...
Either AI will be really good, hence compute and everything will materially depreciate or it won't be much better than it is today and token volumes will plateau compared to compute.
For instance the amount of token compute that's to come online in 6-12 months is several times what we have today...
Second 3) Compare performance in terms of difficult tasks/$ over the last 6 months, 3 months, etc. Open weights are a ratchet. In terms of intelligence per $, a Spark is never going to be a worse deal tomorrow than it is today, at least until the entire platform is replaced or obsoleted.
This is a bad take because again this assumes DGX Spark will not depreciate in price, we will have something better for far cheaper surely in the next couple years. M5 Max & Ultra are already arguably it, but will have to see.
> 71 days ago the best model you could run on two Sparks was an aggressive Q3 quant of Qwen 3.5 397B (AA 34). 70 days ago it was a mixed-quant of GLM 5.2 (AA 53). 30 days ago it was full fat DeepSeek 4 Flash (AA 53). Today it's GLM 5.3 Flash (AA57) and/or Qwen 3.8 Next (Unknown). Sometime this week it will likely become mixed-quant GLM 5.3 (AA 60).
This has nothing to do with DGX Spark's value, if models get cheaper the API costs also go down, this is not a defensible argument to cost to value.
Are people on HN really not thinking straight?
Tldr; no matter how you do the math compute is only getting more valuable because of a temporary crunch, don't expect this to continue permanently, sure you maybe able to time it and make money but so could you in stocks this is not for investments. Further second hand hardware sells for cheaper than sticker price, outside of a bubble..
And models getting cheaper == APIs getting cheaper == your hardware becoming worse value as your electricity & maintanence costs still remain.
I am not saying local models don't have their place but if someone is trying to use this logic to justify their purchase then I wish them all the best, as someone who is actively working on AI compute/inference/hardware stuff I personally don't have this level of courage.
But this is not a sound investment strategy that if something is going up and seems like it might keep going up, especially when investing in heavily depreciating assets like compute.
Now it's similar cost to DeepSeek v4 flash, but smarter.
My tests: https://aibenchy.com/compare/z-ai-glm-5-3-flash-max/deepseek...
Artificial Analysis ranks GPT 5.6 Luna similar to GPT 5.4, but that never matches my real world experience. AA seems to do a good job making a single number as representative as possible but there is still so much benchmarks don't communicate.
For implementation tasks, where I have the problem already defined and researched, or just simple task, I'd definitely use something like Luna xhigh or max. If the task is vague, or involves planning, I'd rather use Sol medium, even though it's theoretically worse on benchmarks.
To my surprise I couldn't find any proper explainers of the term in a quick search, despite grokking it after seeing it in various contexts on Twitter, but Fable 5 offered a useful analogy: "A student who memorized worked solutions and one who understands the subject score the same on the test; you can only tell them apart by asking a question the test didn't. Real-world use is nothing but those questions, which is why a single AA number feels right and wrong at the same time."
In other words, big model smell is related to the underlying ability to "understand" when tasks are underspecified or out-of-distribution. This ability can be mimicked to parity by smaller, distilled models according to the density of the training data for particular tasks, but neural scaling laws still hold for generalized reasoning ability.
More recently with these smaller models, there's a separate but related "RL-fried" phenomenon, where they rely on CoT to "grind toward a checkable answer even in contexts (open dialogue, taste, ambiguity) where there is no checkable answer, and you get the tell: over-hedged, over-structured, relentlessly on-task, deaf to the subtext."
There are some other insights and caveats in the (short) conversation that I feel you may appreciate reading. [1]
[0] https://x.com/aidan_mclau/status/1807843014104211855 [1] https://claude.ai/share/d511a348-7c36-432f-a6d5-9deab2802615
From a biased source, but would be big if true. I've had great results with GLM 5.2.
From their subscription page, the smallest plan gives you about 97M tokens weekly for 5.3 but 292M for 5.3 Flash. Not exactly 10x the limit.
It's at least close (even if not better) from the Ox Alpha runs. For the price it's definitely great.
https://newsletter.semianalysis.com/p/google-we-have-no-moat...
Correction: Not a statement, rather, an internal memo by a Google employee. Thanks @granzymes for highlighting.
It's too big, bright and resourceful of a country to choose confrontation instead of collaboration.
It's not like we didn't try it. China first have to learn to make deals where both party benefits.
That's good. Keep going.
Now the US is behind in EVs can you guess what they're doing? [1]
[1] https://evwire.com/p/video-ford-ceo-jim-farley-says-they-fly...
Intellectual property is part of WTO agreements but enforcement is domestic.
US companies do it too, regularly, they simply hire and poach staff from competitors.
Proving it to be IP theft is difficult unless you can prove documents being passed. But often all you need is the know-how of the hired talent.
If the USA wanted a copyright treaty with China bad enough, we would negotiate one. China is not breaking any laws here, international or otherwise.
"problem" indeed.
│ https://openrouter.ai/api/v1/chat/completions model: stealth/ox-alpha auth: OPENROUTER_API_KEY status: 404 Not Found response: {"error":{"message":"Thank you for participating in the Stealth Ox Alpha testing period. This model was ZAI's GLM-5.3 Flash.
│ Use it now: https://openrouter.ai/z-ai/glm-5.3-flash","code":404},"user_...":"}
How is the business model of Anthropic/OpenAI will sustain?
It was creating java bindings for this project: https://github.com/jeffhajewski/latticedb
And here is the binding one-shotted by glm-5.3-flash: https://github.com/jeffhajewski/latticedb/pull/5
What irks me about this is that the harnesses seem to be just an afterthought here.
Don't get me wrong, I love messing around with installing Pi, getting it hooked up with OpenRouter, and just trying all kinds of different stuff, local models, etc... but when it comes to literally just setting up a productivity environment and trusting my entire machine with it, I just run Codex.
I have heard from anecdotes where people have indeed replaced their main drivers with DeepSek V4 Flash or GLM and state that "it's almost as good as... [claude/gpt]" but I never hear anyone say "yeah, this is the model/harness that I now run on my machine and don't mess with it"
* me raises hand.-
Think z code gives a token bonus though
Have been using it as my primary harness for personal work for I'd say 6 months. I recommend everyone create their own harness at least to learn. There are a lot of practical benefits.
I have a use case where I have to run local models, can't share data offsite.
Its probably the huawei ascend 910, they are using.
With GLM 5.1 and 5.2, the big problem was tool calling and long-horizon coherency. 5.3 was more trustworthy at the cost of longer thinking traces, and now Flash seems to improve on it once again with a more concise, smaller model. As long as there aren't any noticeable regressions, I could see myself defaulting to this for >90% of my day-to-day coding work.
e.g. "Agent Coding Performance by Effort Level" cuts Y-axis from 0~20.
- This makes it as if GLM-5.3-Flash made a bigger jump than it claimed as the Y-axis does not increase much (stupid trick used in biz reports)
I did mention that ox was working ok for me, and having an open-weight comparable to close to SOTA makes it very compelling for me to try it out locally (well, only if I got more VRAM)
edit: nevermind. it is there in the artifical analysis scatter plot, but is greyed-out.
MUCH more interesting is that in that chart, their cost is WAY off. The actual chart shows GLM 5.3 Flash at $0.09, but their chart shows $0.045...
I don't have enough metrics to compare those costs but still Chinese models have been cheaper except against Luna for me.
FWIW, Luna does everything so well, I just keep using it for all my agents by default.
> (...) Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
It might be one of the most actually practical tasks that AI might've done because the compounding effects of it and also its implications are/feels so immense. It feels as if Nvidia might be in a slight turbulence from it.
advance card: https://catalystneuro.com/llm-cost-frontier/images/advances/...
tracker: https://catalystneuro.com/llm-cost-frontier/
- Input: $0.15 - Output: $0.50 - Cached input: $0.03
https://openrouter.ai/compare/deepseek/deepseek-v4-flash-073...
EDIT: Looks like they are swizzling around the pricing dynamically on that page, on both the GLM and the DS sides, so who knows.
The labs are clearly benchmaxxing a bit to maintain perceptions. But I don't think they're in the lead anymore in terms of their public offering - although I'm sure what they have behind closed doors is far better than anything we're getting access to.
What I'm saying is, if you're expecting a model that can be run on a 16GB or 32GB machine with the intelligence/knowledge of Mythos or Sol, it will never happen. It cannot happen, just like you cannot watch the Odyssey saved as a 16MB file.
Smaller models can get faster and smarter, but by definition they can never compress all of the knowledge of a frontier model and they will approach a limit by which they cannot get better.
https://www.deviantart.com/sssfjknfvdknj/art/the-ENTIRE-shre...
The ‘frontier’ models rely on scale to achieve their results but that’s not the only approach. Eventually we will hit up against the fundamental limits but we are not close with Sol and Mythos.
It breaks fundamental laws of information theory. It’s like saying you can extract 100 joules of energy from 10 joules of energy source. Not possible.
You are absolutely right that there is a physical limit about these things, but very often I find that the solution is a clever way to work around the problem. Maybe the problem with knowledge of the models will be improved by them looking the information up in a better way - so smaller models will not have to have the knowledge trained in but will default to checking. Maybe Models will, I dunno, focus on training in assembler and start to only ever check the compiled output so they only ever need to learn assembler and will then compile the solution to reason about the assembler code.
Obviously that last part is a ridiculous example because I'm not gonna be able to come up with a solution myself - I'm not nearly smart enough for that. But I h ope you get what I mean. Not going the direct route but instead finding solutions people didn't think of before.
Imagine prosumer desktop hardware 10 years from now. The 2036 DGX Spark. For a few thousand dollars you will be able to buy something with hundreds of GB (maybe TB if manufacturers step up) of unified RAM, memory bandwidth in the 10-20TB/s range. Overall AI "compute" will increase 10-20x, while at the same time AI model capability per byte will increase 5-10x.
The hardware would fit today's models, something like Kimi K3, quite comfortably and give performance of maybe 100 tokens/second. So what needs data center hardware today will run on your desk.
But if we also assume the models become more efficient, a 2036 Fable-class model (in terms of intelligence/capabilities, not size) will easily run on this thing at hundreds of tokens per second.
Unfortunately it'll still slow to a crawl with 5 Chrome tabs open, and every Electron app will need at least 200GB of RAM.
Additionally the information theory angle is for information storage, but a model can access resources and tools to gain information and what we are really seeking to train is reasoning not information retrieval. We reduce the needs to the right capabilities and we don’t get upset if it does not know the lyrics to every song ever written.
Qwen 3.8 Next Flash: 125B + 51B = 176B parameters with 6B activated
DeepSeek V4 Flash: 284B with 13B activated
The new Qwen model is the most promising for one or two Strix Halo 128GB with the low number of active parameters. On paper it's much stronger than Qwen 3.8 27B.
It replaced all other models for me.
It's much less chatty than DeepSeek V4. And it feels noticeably smarter. I also use Claude at work and GLM 5.3 feels like Opus 4.8 (which I consider better than Opus 5).
It tends to be more proactive with suggestions after a task is done also.
DS4 Flash is awesome but GLM 5.3 is better despite being a bit more expensive.
Is the optimal formula still 20x the amount of model params in tokens for training? Could this mean we're getting a GLM with 1.5t params?
So, while OAI/Ant have to fearmonger and keep training the largest models, Chinese labs can focus on efficiency more heavily and as long as they stay near frontier, they'll continue to get positive coverage.
Like were executive at Google and AWS and Microsoft expecting this kind of performance from models smaller than what openai/anthropic have been doing? Are we really in a "compute desert"?
(281 points, 118 comments)
https://x.com/Zai_org/status/2092616204787626030/photo/1
Give it a day or two. More will pop up.
2 really strong open models on the same day is a amazing.
Just asked it a question via the custom opencode go endpoint routed over cloudflare ai gateway doesnt show me the correct models. My fault was that I set "https://opencode.ai/zen/go" as endpoint and tried my-gateway.com/custom-ocgo/v1/models, turns out I had to add /v1 to the opencode url and leave it on my -gateway.com.
But first it told me that opencode is not on the compat endpoint and than it recommendet "Option 3: Nutzung seit Curse die Integration des Providers класный" . Not to mention some complete gibberish like "Falls Opencode.ai in Cloudflare Eing? Wenn ja, wähle den eingebauten provider o.ä. Weil der Gateway dann ein eigenes Modell-List gibt." or "Schlage die Modellnamen in einer Liste ab (z.B. lokal fester Key)"
Is this just cloudflare or is it really that bad? I mean thats not even the level of LLama2 7b ...
I think you answered your own question. Must be something wrong.