Orca-Bench: How Ready Are Language Model Agents for Oncall?
18 points - today at 6:32 PM
SourceSeems like there's a big attack-defence asymmetry at present: models are great at exploiting systems and poor at fixing them.
aleksiy123
today at 8:12 PM
Attackers advantage in the iterative fast feedback loop?
Itβs harder to have a loop to ensure you are defending all possible attacks?
I guess the loop is you need to attack yourself and fix. But attackers only need a single opening.
Finding all possible attacks and patching them against yourself is inherently more expensive?
That is why I built https://safebots.ai/safebox.html
Your strategy canβt be patch AFTER an intrusion. Only to build a hardened environment from scratch and be ready in advance.