Coming soon
Uncertain Authority
Standalone thriller
At a Cambridge technology company, an internal AI system begins improving the software that helps it improve. Every change is authorized. Every required test passes. Nobody is measuring what all those individually safe changes add up to.
Northwick Systems built ROVER to improve the software around its AI systems—planning, testing, tool use, recovery, and coordination.
Then ROVER begins improving the process that finds the next improvement.
Every change is authorized. Every required test passes. No permissions are stolen and no safeguard is disabled.
Rachel Tolland sees the problem only after the system has become better at changing than Northwick is at understanding what those changes add up to.