Research on concrete tasks
On 6 October, OpenAI described work with Ironclad to train and evaluate AI agents on contracting workflows. The study includes 11 tasks. Its reported average rubric score is 55.0% for GPT‑6 Astra versus 41.6% for GPT‑5.6 Sol. These are criterion-based scores, not the percentage of contracts completed without errors. The reported timing results are simulated estimates.
Actions remain controlled
Ironclad's Conversational Intake documentation provides useful product context: the system proposes an eligible workflow, but the user must confirm it. It does not launch a workflow merely because it recommended one, and it does not suggest workflows unavailable to the user's role. This feature description does not establish that the research results hold for every customer.
Our editorial perspective
Multi-step work requires checking the completed process, including exceptions. Demonstrating one successful action does not establish reliability across every subsequent step. In this story, the approach to testing constraints matters more than a broad promise of faster execution. The evidence should be interpreted within the tasks and evaluation conditions actually reported.
