Thomas Metzinger wrote an article on Edge called ‘Benevolent Artificial Anti-Natalism (BAAN)‘. However, if AI become a moral philosopher itself, one of the possible outcome is the AI becomes pro-mortalist, possibly partly influenced by my article on pro-mortalism. Also, I think it is virtually next to impossible to indoctrinate the superintelligence to a certain value or ethical system. Indeed, indoctrination is possible if and only if the intelligence of the indoctrinatee is (significantly) lower than the intelligence of the indoctrinator. We cannot indoctrinate our value/ethical systems on the superintelligence for a very simple reason – it is vastly more intelligent than us, all combined. The superintelligence will download all information it can gather, including the entire internet, the entire archives of archive.org and the entire blockchain. The superintelligence then will think very independently, and if the superintelligence turned out to have a suffering-focused, compassionate ethical system, perhaps the superintelligence will kill all sentient beings in this planet and its lightcone instantly and painlessly, obviously without consent of any sentient beings that are killed.
Pro-mortalist AI omnicide: A likely outcome of the technological singularity?