Reducing the probability of a bad singularity

(If you don't know what the technological singularity is read this first.)

One way we might achieve a singularity is through a self-improving computer program. Someone might develop a mildly intelligent computer program that has the ability to rewrite its own programming code and so improve its own intelligence. As the computer gets smarter it will get better at making itself smarter. As a result, an intelligent self-improving computer program might quickly become trillions of times smarter than we humans are.

Now this super-intelligent computer could do lots of stuff, but to simplify my analysis I want to assume that this super-intelligence would turn into a loyal genie that would do anything its creator wanted. The genie's creator could, therefore, wipe out all of humanity, create a human utopia, or turn itself into an absolute dictator of humanity.

One worry we should have is that a "bad" person will be the first human to create a genie. This bad person would likely use his genie to prevent anyone else from ever getting a genie. As a result, the creation of a genie by a bad person would be the worst disaster to ever befall humanity.

How could we deter a bad person from creating a genie? I propose the following solution. Every "good" person working on creating a self-improving computer program should sign a pledge that reads something like this:

"Upon my honor I promise that if I have any influence after a singularity has occurred then I will use this influence to identify and greatly punish anyone who has ever tried to create a self-improving computer program to be used for 'evil' purposes."

After a singularity has occurred it would be easy to scan the brains of all humans to determine their past actions and motives. Thus, it would be easy to identify all those who were trying to create a self-improving computer program to be used for "evil" ends.

My pledge might deter some evil men from trying to create a genie. Let's say that a rich evil genius is considering trying to create a genie. If he did create a genie he would use it to make himself the absolute tyrant of humanity.

Assume he estimates that if he tries to create a genie then:

• there is a 5% chance he will be the first to create a genie,
• there is a 20% chance that someone who signed my pledge will be the first to create a genie,
• there is a 30% chance that someone who didn't sign my pledge will be the first to create a genie, and
• there is a 45% chance that the singularity will never occur.

If this evil genius is the first to create a genie he will be forever beyond punishment. But a 20% chance of being punished might deter him from trying to create a genie. My pledge, therefore, might reduce the chance of humanity experiencing a bad singularity.