today at 7:22 PM
The headline graph is kind of bizarre.
For some reason they're comparing their harness running on Astra xhigh to Codex with Astra max?
---
Also worth noting that OpenAI just added support for async tool calling to their harness, which isn't 1:1 with this approach, but is slowly ramping up in being able to provide something similar.
A big part of why Codex uses so many tokens is that it basically hot loops on polling tasks it starts for... absolutely no good reason: https://www.reddit.com/r/codex/comments/1wdlp7q/weve_discove...
I fixed it on my fork of Codex too, also back in Jan/Feb – I keep this patch rebased, for anyone who wants it: https://github.com/tekacs/codex/commit/9ffcf8db9078eae43d411...
It results in token savings similar in scale to those displayed here by Unreal.
---
My harness has used a slightly fancier version of the approach that Unreal is using since ~Feb, and... it definitely works excellently, but it's also assuredly smoother with Astra and other recent models that are more aware of async tool calling.
today at 7:14 PM
I think this is what I’ve been waiting for! Live interactions with models are fundamentally asynchronous! This will be great for interactivity.
What a good time for harness design. Just while OpenCode is growing up a bit and focusing on their harness. I’m delighted to see people focusing on good general solid harness design principles.
Someone make an OpenCode API (the best general agent end UI API I know) compatible server for it!
Sucks that I haven’t upgraded my client to V2 yet and I don’t really want to build on an outdated API . ..
today at 7:25 PM
Sounds like a trademark issue when Epic ships a wildly popular Unreal Engine
today at 7:28 PM
But I don't think you can trademark a generic word like Unreal...
today at 7:47 PM
They've trademarked it for quite a few closely-related domains: https://tsdr.uspto.gov/#caseNumber=87709072&caseSearchType=U...
>Computer software, namely, game engine software for video game development and operation; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of video games; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of content for virtual worlds and 3D platforms; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of motion pictures, television shows, videos, 3D animations, 3D simulations, 3D visualizations, virtual reality motion pictures, virtual reality television shows; Computer software, namely, software development tools for the creation of computer-generated imagery and graphics for the production of virtual reality video games; Virtual reality game software; Virtual reality software for creating multimedia content; Augmented reality game software; Augmented reality software for use in mobile devices for integrating electronic data with real world environments for the purposes of entertainment
today at 7:43 PM
>"But I don't think you can trademark a generic word like Unreal..."
Sure you can, trademarks are contextual. If they were a landscaping business it wouldn't matter. But within the same industry absolutely.
today at 6:59 PM
I don't really understand how async tool calls translate to token savings
It says that it removes tokens wasted while a model is waiting on synchronous tool calls. What tokens exactly is Pi using when waiting?
today at 7:24 PM
I think a running agent periodically checks if a process it started has finished.
today at 7:37 PM
today at 7:38 PM
omp.sh does this way better by just allowing structural toolcall execution in eval with python/js.
today at 7:26 PM
Why don't you explain what's different?
today at 7:25 PM
Honestly the fact that we're still modeling agent harnesses like chat and strapping them into shell sessions instead of building async actor systems with sandboxed OS functionality access actors is bananas to me. So much easy wins can be had just by building on right abstractions... Hopefully will get enough time to play with this idea soon on my own.
today at 6:18 PM
today at 7:37 PM
Mildly disappointed that this has nothing to do with Unreal Engine, the popular game engine.
today at 7:42 PM
Yeah, I was hoping it would be something to make it easier for agents to interface with Unreal Engine games.
today at 7:45 PM
I was imagining a talking head avatar rendered in Unreal Engine, with lip sync and facial expressions driven by a multimodal LLM that produces the speech.