I just watched this interview with Dr. Ben Goertzel from the Singularity Institute for Artificial Intelligence. One of the issues of prime concern for me when I have participated in SIAI forums online was the willingness of some of their thinkers to develop an AGI without built in controls. To me it was like willingly starting an atomic chain reaction without control rods. In this interview, Ben states plainly that an uncontrolled “takeoff” AI is not acceptable that he plans to create an architecture where “ascent” is gradual.
I am grateful that leading AI thinkers are addressing this.
I also think an AGI should be contained. Certainly it should have all the web available to it, but a copy of the web without external access. Google has made a copy of the web on its servers, why not for the first AGI capable of surpassing human intelligence?
We isolate and contain Nuclear, Chemical and Biological agents. Why not intelligent agents?