Abstract
There is no strong reason to believe human level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed goals or motivation systems. Oracle AIs (OAI), confined AIs that can only answer questions, are one particular approach to this problem. However even Oracles are not particularly safe: humans are still vulnerable to traps, social engineering, or simply becoming dependent on the OAI. But OAIs are still strictly safer than general AIs, and there are many extra layers of precautions we can add on top of these. This paper looks at some of them and analyses their strengths and weaknesses.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Armstrong, S.: Utility Indifference. FHI Technical Report (2010)
Armstrong, S., Sandberg, A., Bostom, N.: Thinking Inside the Box: Using and controlling an Oracle AI (2011); accepted by Minds and Machines
Asimov, I.: Runaround. Astounding Science Fiction (1942)
Bostrom, N.: Ethical issues in advanced artificial intelligence. Cognitive, Emotive and Ethical Aspects of Decision Making in Humans 2 (2003)
Bostrom, N.: Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards. Journal of Evolution and Technology 9 (2001)
Bostrom, N.: Information Hazards: A Typology of Potential Harms from Knowledge (2009), http://www.nickbostrom.com/information-hazards.pdf
Bostrom, N.: Predictions from Philosophy? Coloquia Manilana (PDCIS) 7 (2000)
Bostrom, N.: The Future of Human Evolution. In: Tandy, C. (ed.) Death and Anti-Death: Two Hundred Years After Kant, Fifty Years After Turing, pp. 339–371. Ria University Press, California (2004)
Bostrom, N., Salamon, A.: The Intelligence Explosion. Retrieved from The Singularity Hypothesis (2011), http://singularityhypothesis.blogspot.com/2011/01/intelligence-explosion-extended.html
Caplan, B.: The totalitarian threat. In: Bostrom, N., Cirkovic, M. (eds.) Global Catastrophic Risks, pp. 504–519. Oxford University Press (2008)
Chalmers, D.J.: The Singularity: A Philosophical Analysis (2010), http://consc.net/papers/singularity.pdf
Cook, S.: The complexity of theorem proving procedures. In: Proceedings of the Third Annual ACM Symposium on Theory of Computing, pp. 151–158 (1971); Evolutionary Algorithm (n.d.), http://en.wikipedia.org/wiki/Evolutionary_algorithm
Good, I.: Speculations Concerning the First Ultraintelligent Machine. Advances in Computers 6 (1965)
Hanson, R.: Long-Term Growth As A Sequence of Exponential Modes (2000), http://hanson.gmu.edu/longgrow.pdf
Idel, M.: Golem: Jewish magical and mystical traditions on the artificial anthropoid. State University of New York Press, New York (1990)
Kahnemand, D., Slovicand, P., Tversky, A.: Judgement under Uncertainty: Heuristics and Biases. Cambridge University Press (1982)
Kurzweil, R.: The Singularity is Near. Penguin Group (2005)
Mallery, J.C.: Thinking about foreign policy: Finding an appropriate role for artificial intelligence computers. MIT Political Science Department, Cambridge (1988)
McCarthy, J., Minsky, M., Rochester, N., Shannon, C.: Dartmouth Conference. Dartmouth Summer Research Conference on Artificial Intelligence (1956)
Omohundro, S.: The basic AI drives. In: Wang, B.G.P. (ed.) Proceedings of the First AGI Conference. Frontiers in Artificial Intelligence and Applications, vol. 171. IOS Press (2008)
Ord, T., Hillerbrand, R., Sandberg, A.: Probing the improbable: Methodological challenges for risks with low probabilities and high stakes. Journal of Risk Research (13), 191–205 (2010); Paperclip Maximiser (n.d.), http://wiki.lesswrong.com/wiki/Paperclip_maximizer
Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approac. Prentice-Hall (1995)
Salomon, A.: When Software Goes Mental: Why Artificial Minds Mean Fast Endogenous Growth (2009), http://singinst.org/files/htdraft.pdf
Sandberg, A.: Friendly Superintelligence. Retrieved from Extropian 5 (2001), http://www.nada.kth.se/~asa/Extro5/Friendly%20Superintelligence.htm
Shulman, C.: Omohundro’s “Basic AI Drives” and Catastrophic Risks, http://singinst.org/upload/ai-resource-drives.pdf
Shulman, C.: Whole Brain Emulation and the Evolution of. Retrieved from Singularity Institute for Artificial Intelligence (2010) http://singinst.org/upload/WBE-superorganisms.pdf
Simon, H.A.: The shape of automation for men and management. Harper & Row (1965)
Solomonoff, R.: A Preliminary Report on a General Theory of Inductive Inference. Cambridge (1960)
Steels, L., Brooks, R.: The Artificial Life Route to Artificial Intelligence: Building Embodied, Situated Agents (1995)
Sutton, R., Barto, A.: Reinforcement Learning: an Introduction. MIT Press, Cambridge (1998)
Turing, A.: Computing Machinery and Intelligence. Mind LIX 236, 433–460 (1950)
von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)
Walter, C.: Kryder’s Law. Scientific American (2005)
Yudkowsky, E.: Creating Friendly AI (2001), http://singinst.org/CFAI/
Yudkowsky, E.: Friendly AI 0.9 (2001), http://singinst.org/CaTAI/friendly/contents.html
Yudkowsky, E. (n.d.). General Intelligence and Seed AI 2.3, http://singinst.org/ourresearch/publications/GISAI/
Yudkowsky, E.: The AI-Box Experiment. Retrieved from Singularity Institute (2002), http://yudkowsky.net/singularity/aibox
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag GmbH Berlin Heidelberg
About this chapter
Cite this chapter
Armstrong, S. (2013). Risks and Mitigation Strategies for Oracle AI. In: Müller, V. (eds) Philosophy and Theory of Artificial Intelligence. Studies in Applied Philosophy, Epistemology and Rational Ethics, vol 5. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31674-6_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-31674-6_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31673-9
Online ISBN: 978-3-642-31674-6
eBook Packages: EngineeringEngineering (R0)