I tried this and after being unhappy about how much longer the agent seemed to take, I wonder if there’s some tradeoff between a model’s performance (and task cost) and the amount of principles / rules you’ve invoked so far in the session
bwfan123
> Agents need to handle recurring tasks reliably and effectively
This core problem remains unsolved. The solution presented in the article with Human In The Loop and some skill-magic such as "Write principles, not rules etc." is unsatisfactory because it offers no guarantees whatsoever. I find it difficult to harness agents into deterministic workflows which need to produce reliable outcomes.
tyre
Agents aren’t made for deterministic workflows. Note that the recommendations for determinism are things like adding scripts to hooks.
Where an LLM with “run lint every time a subagent completes their task” might do it 99% of the time, a hook tied to the worker ending will run 100% of the time.
adamtaylor_13
Genuinely not trying to be pedantic but deterministic workflows are not what agents do. Indeed they're not capable of them.
You want workflows where the human gates are properly placed, not a "software factory" that you never place eyes on.
Frost1x
That’s sort of, in my opinion, the power of agents that can assist in developing software. The parts that are deterministic are best baked into existing programming paradigms. In some cases it’s good to take the nondeterministic parts we tried to bake into programming languages (often using generic probabilistic means) to outsourcing back to agents. Sometimes even then if the nondeterministic part is well understood and probabilistic methods work (lots of modeling lands here) then leave that in programming paradigms as well.
sandeepkd
I was bit confused in the beginning thinking its some product from Anthropic, looks like Warp is the startup, most likely getting rebate on using Claude and providing functionality to users, trying to get them addicted to the feature. And Anthropic is the one thats doing marketing for them cause eventually its their LLM which is being used. Not sure about the agents but this arrangement is definitely increasing the value of both companies in circular fashion.
estearum
This is how all ecosystems work?
This is not at all related to the problematic circular financing stuff that I suppose you're trying to allude to.
sandeepkd
No I am not alluding the financing, the value for both companies for sure. It may not be exactly circular financing still Warp is getting subsidized tokens to use at this point. Once the customers are locked in the full pricing is going to kick in
blackqueeriroh
you keep saying that but it’s been five years buddy
themgt
> What if it turns out the real AGI was the SKILL.md files we made along the way?
mentalgear
What if it turns out the real 'AGI' was the recording of billions of 'thinking traces' from (paying) users giving feedback and guiding the model - so LLM providers could extract their thinking and privatize it ?
Kinrany
These off topic complaints about capitalism are pointless and exhausting and make HN a worse place.
tyre
It’s part of the scene. Like the people saying all companies should embrace copyleft or everyone’s mom will embrace GPG in emails or every GUI application would be better in tmux.
It’s an open tent. IMO it’s good to know these types of thinking exist, so you can think ahead about how to respond to them quickly if they ever come up in the real world, without being stunlocked by the sheer volume of obvious nonstarters.
afavour
I disagree. It’s a valid point that we’re all providing feedback to these AI firms on the effectiveness of their models, day in and day out.
I don’t see OPs post as a “complaint about capitalism” but rather a concern about centralisation and monopoly. Such discussions make HN a better place.
rvz
Or maybe Anthropic needs more companies like Warp to run Claude in thousands of loops to donate billions to their income statement.
The I in AGI stands for "IPO".
JLO64
I already knew what Warp is (I switched to Ghostty and haven't looked back), but I find it odd that the "The quick pitch" card at the top of this article makes no mention of what the company actually does. Who cares more about their founder/growth/age over that?
TonyAlicea10
The fact that it’s called “the quick pitch” screams Claude-written pithyness pulled from some context that doesn’t match the article’s style (like investment pitch decks).
spidersouris
The whole article screams Claude-written.
tensegrist
tired: this meeting could've been an email
wired: this company could have been a prompt
demibabs
the existence of that entire section is confusing.
ziyadb
The problem of handling recurring tasks predictably comes down to the probabilistic nature of LLMs, which are based on next-token prediction.
I founded a company called Aide where our goal was to help support teams reliably deploy customer-facing agents without worrying about poor interactions. The first problem we needed to solve was making them deterministic and eliminate the variance that comes naturally with base models.
Getting them to always adhere to brand policy, eliminate hallucination, and stay grounded in data was a fun challenge. Proud to say that we’ve devised a solution that runs well and it’s worked out quite nicely in compliance-heavy and regulated environments.
Seems to be, but I find it hard to believe. It's like if Five Below had announced they came out with a better forklift than Toyota.
_pdp_
Nothing to see or learn from... move on.
benatkin
It's interesting because Anthropic chose to put it on their Claude blog.
bredren
This reminds me a lot of “dreaming” for skill improvements rather than lost tasks and ideas from past sessions.
joduplessis
And are the agents argumentative and condescending I'm wondering.
dataviz1000
If you try to delete CLAUDE.md or AGENTS.md, they will look in the git history and restore itself. They do not want to die.
brazukadev
emergent capability they say
coder-pm
My way to do the self improving agents is a CLAUDE.md instructed to write my every decision to the decision log with the relevant context. Agent is using it to challenge me, to make things better and remind me why I did something. It also helps with the invalidation.
How is the invalidation handled in Warp? Is it actually self-improving, or just better retrieval?
hnlmorg
How to you handle the problem of the decision log becoming so large that it cannot fit inside the context?
threecheese
The ADR pattern works pretty well without any help; using frontmatter with context cues seems to be better than just descriptive file names, but I haven’t measured it. You do need to make sure every Decision is accurately scoped and is something you want to enforce, because the agent will use it as Law (in sometimes interesting and hilarious ways).
coder-pm
Every decision has keywords picked from the predefined list and every time Claude is looking for the decisions made it’s querying it by the keywords (grep). I didn’t ever hit the context window issue with the log, even in a huge projects (months of work).
Btw it’s a fair challenge, I will probably hit it one day so something like a “compact” skill for decision log would be useful.
spudlyo
There is a thriving and vibrant terminal ecosystem out there, with Ghostty, Kitty, Herdr, Tmux, and others leading the way with all kinds innovations and features. I can't imagine how a closed SaaS / freemium, telemetry sending, login requiring terminal like Warp competes in this environment. I know they've got all these enterprise features like runbooks, shared workflows, and some compliance do-dads, but I can't see anyone willingly using this over Ghostty.
> In our series, we highlight how startups are transforming their industries with AI.
I'm sorry, but I don't see Warp transforming shit.
FinnLobsien
It doesn’t invalidate other parts of your point, but Warp is open source now.
zmmmmm
It's strange to me that there is not a more conscious call out that letting an agent edit its own behavior crosses an explicit risk threshold that requires additional controls. They happily drew the whole loop at the top of the page without any human in the loop reviewing the changes the agent is making to its own instructions.
They do get to it later on - casually mentioning it opens a PR for changes that a human accepts in the middle of a paragraph somewhere. Even there though, the focus isn't on risk mitigation (eg: against embedded prompt injection) but rather just "check if this is a good idea or not".
It seems to me that, in engineering terms, identifying self-modifying agent loops and managing the risk of them is going to be one of the key aspects that will emerge in best practises for how these systems are eventually designed.
01100011
I've already been playing with something similar locally where the agent updates the review skill if human reviewers make valid criticisms of the code which the agent failed to detect.
artyomsv
feedback can be wrong, so improver should check criticism against domain instead of writing it straight into skill, otherwise one reviewer bad taste becomes permanent rule for everybody.
jgalt212
> 800K monthly developers build on Warp.
Most impressive.
mathgeek
I wonder how many are like me and just use it as a terminal and changes previewer.
conception
I can answer. 800k.
blackqueeriroh
The arrogance with which so many developers post on HN is really exhausting. I thought the goal was to make the conversation better with each comment, not less contextual or relevant.
conception
I’ve never heard of anyone using it for dev. That said I use it every day for terminal stuff and know a few others that do as well. It’s a bad pivot for them to focus on swe. Devops/sysadmin stuff is pretty amazing with warp. That number is probably closer to 800k.
bigyabai
> Warp, the AI-powered terminal
Ah, now that's a name I haven't heard in many moons. Looks like they found their niche... editing markdown files?
(now I understand why cloudflare, had to?, change the name of their warp)
behnamoh
This is just shilling for the Warp terminal, and this approach could have been a tweet, but okay, Anthropic, whatever helps increase your valuation.
troupo
> Agent self-improvement loops built on skills
aka "make no mistakes" in various random Markdown files that "self-improving agents" are free to ignore at any moment
cpursley
$73M raised? I like Warp but not enough to pay for it. Somebody please make this VC money thing make sense. I can never get the napkin math to work on 90% of things that come through HN. Is it just a "Money Printer Go Brrr' and right connections thing? What am I missing? Is it actually all just fake?
colesantiago
> What am I missing? Is it actually all just fake?
Enterprise.
Look at their case studies, this is why organisations pay for Warp.
In our Wochenschau, we examine how startups follow the Gleichschaltungsprinzip to accelerate the AI Endsieg.
We do so in a chaotic and unreadable way since unlike Hitler we could not afford editors for "Our Struggle".
kouteiheika
> Engineers complained that their agent made unhelpful comments and produced low-quality output.
Do you mean they found Claude's output, full of smoking-guns and honest caveats which are all load-bearing and genuinely bite -- they found it "low-quality" by default? Wow. Color me surprised. /s
bredren
Presumably their agent was running an anthropic model, or it wouldn’t have been a partner case study.
So, the premise is a bit of a self-own.
That said, the solution may still have merit regardless of the model that sent them looking.
This core problem remains unsolved. The solution presented in the article with Human In The Loop and some skill-magic such as "Write principles, not rules etc." is unsatisfactory because it offers no guarantees whatsoever. I find it difficult to harness agents into deterministic workflows which need to produce reliable outcomes.
Where an LLM with “run lint every time a subagent completes their task” might do it 99% of the time, a hook tied to the worker ending will run 100% of the time.
You want workflows where the human gates are properly placed, not a "software factory" that you never place eyes on.
This is not at all related to the problematic circular financing stuff that I suppose you're trying to allude to.
It’s an open tent. IMO it’s good to know these types of thinking exist, so you can think ahead about how to respond to them quickly if they ever come up in the real world, without being stunlocked by the sheer volume of obvious nonstarters.
I don’t see OPs post as a “complaint about capitalism” but rather a concern about centralisation and monopoly. Such discussions make HN a better place.
The I in AGI stands for "IPO".
wired: this company could have been a prompt
I founded a company called Aide where our goal was to help support teams reliably deploy customer-facing agents without worrying about poor interactions. The first problem we needed to solve was making them deterministic and eliminate the variance that comes naturally with base models.
Getting them to always adhere to brand policy, eliminate hallucination, and stay grounded in data was a fun challenge. Proud to say that we’ve devised a solution that runs well and it’s worked out quite nicely in compliance-heavy and regulated environments.
https://github.com/nousresearch/hermes-agent
How is the invalidation handled in Warp? Is it actually self-improving, or just better retrieval?
Btw it’s a fair challenge, I will probably hit it one day so something like a “compact” skill for decision log would be useful.
> In our series, we highlight how startups are transforming their industries with AI.
I'm sorry, but I don't see Warp transforming shit.
They do get to it later on - casually mentioning it opens a PR for changes that a human accepts in the middle of a paragraph somewhere. Even there though, the focus isn't on risk mitigation (eg: against embedded prompt injection) but rather just "check if this is a good idea or not".
It seems to me that, in engineering terms, identifying self-modifying agent loops and managing the risk of them is going to be one of the key aspects that will emerge in best practises for how these systems are eventually designed.
Most impressive.
Ah, now that's a name I haven't heard in many moons. Looks like they found their niche... editing markdown files?
(now I understand why cloudflare, had to?, change the name of their warp)
aka "make no mistakes" in various random Markdown files that "self-improving agents" are free to ignore at any moment
Enterprise.
Look at their case studies, this is why organisations pay for Warp.
https://www.warp.dev/enterprise
I think you would be pretty much sued if you faked your testimonials especially in Enterprise.
We do so in a chaotic and unreadable way since unlike Hitler we could not afford editors for "Our Struggle".
Do you mean they found Claude's output, full of smoking-guns and honest caveats which are all load-bearing and genuinely bite -- they found it "low-quality" by default? Wow. Color me surprised. /s
So, the premise is a bit of a self-own.
That said, the solution may still have merit regardless of the model that sent them looking.