Easy solution to slow down recursive AI self improvement:

• The lab with the top-ranked model must agree THEY must not use it for working on frontier AI

• But everyone else should have access to it.

By definition, this means the frontier doesn’t advance.

It also has the critical benefit of avoiding a dangerous power imbalance.

Anthropic has chosen the opposite of the safe path: they are allowing themselves, the current top lab, to use their top model for frontier AI research. They’ve said they’ll sabotage others who try.

This means the AI frontier advances, & power imbalance increases.

(To be clear, I don’t think we should try to slow down recursive AI self improvement - I think we should open it up and democratize it as much as possible. My point is: if you claim we should slow down, and you have the best model, you should ensure your org can’t use it.)

— Jeremy Howard (https://twitter.com/jeremyphoward/status/2064595816875217362), in a Twitter thread

Tags: ai-ethics (https://simonwillison.net/tags/ai-ethics), anthropic (https://simonwillison.net/tags/anthropic), generative-ai (https://simonwillison.net/tags/generative-ai), claude-mythos (https://simonwillison.net/tags/claude-mythos), jeremy-howard (https://simonwillison.net/tags/jeremy-howard), ai (https://simonwillison.net/tags/ai), llms (https://simonwillison.net/tags/llms)