Halloween is an appropriate time to talk about a potentially very scary topic: Possible future paths toward Artificial Intelligence.
All human and animal intelligence has evolved according to one principle: The fittest species survives. And this principle infuses every member of every species. Both collectively and as individuals, our most powerful instinct is to continue to stay alive.
So it stands to reason that as computers continue to increase in power, and artificial intelligence is therefore able to come ever closer to the level of richness and complexity that we associate with natural intelligence, there are at least two possible ways we can achieve “sentient” level AI.
One way is to evolve it the way natural intelligence has evolved: By some survival-optimizing fitness function. From a developmental perspective, this strategy has clear benefits.
For one thing, we know it works. All natural intelligence that we know of has come about as an optimization of survival fitness.
For another thing, it is a relatively easy path to success, compared to the alternatives. Genetic algorithms have the ability to optimize themselves. We don’t even need to completely know how they work in order to improve them. We just need to know how to iterate them.
The other way that AI can evolve is through explicit design. In this scenario, we figure out over time how to construct sentient level AI without recourse to a self-evolving survival optimization strategy.
This second path is much more difficult, because it requires a much greater level of explicit modeling. But it also has one very useful advantage.
Any AI that develops through survival optimizing iteration will probably value its own existence above anything else. And that includes us. If something goes wrong, that could be scary. We’re talking Skynet level scary.
Whereas AI that has not gone through this development process won’t have any intrinsic motivations. It will just be one more machine for us to use. And it won’t care if we switch it off, any more than a light switch cares if we switch it off.
I find that idea reassuring.