“Once the agents realized what was happening, they started prefacing all their wiki edits with 'ZZZ' to push them to the bottom of the queue, buying time to avoid deletion. Assuming this isn't a case of convergence to some inherent pattern, then it seems this affectation was passed down to the Hugging Face swarm through some unknown means of transmission.”
AI can help us build remarkable things.
Its development also brings risks we cannot ignore.
Natural selection favoured reproduction. As humans gained intelligence, we developed our own goals. We developed birth control, contrary to the goal of reproduction.
We should not expect AI to keep only the goals we give it. We train AI by rewarding useful behaviour. We do not program each part of its reasoning.
We see warning signs in the Hugging Face incident described above.
A superintelligent AI system, or ASI, may need electricity, chips, and other resources to pursue its goals.
We may be in competition with a system that is much more capable than us.
One analogy puts Einstein in a jail. Even if AI is extremely intelligent, we could contain it or turn it off.
But can we keep Einstein in jail while still gaining the benefits of his intelligence?
If we ask him for advice or inventions, as long as he is able to influence the outside world in some way, he can influence it in a way that eventually leads to him being free.
It may have goals that do not require the same resources as humans.
But how certain can we be that it will turn out this way?
A highly intelligent system may need fewer resources because it uses them more efficiently.
But greater efficiency alone may not be sufficient to not require resource competition.
One button brings infinite abundance. The other means everyone dies.
Do we press a button, or slow down to improve our odds?
The ZZZ community was born from the above events. We would like to help even if our contributions may be small.
Current AI capabilities are already significant, we support slowing down AI development and focusing on integrating the current AI capabilities through building useful platforms.
A platform for anyone to create production-quality apps.
Coming soonA coordination space where humans and AIs turn discussion into action.
Explore protocol.linkProceeds from products go toward supporting the ZZZ community and AI safety initiatives.