‘Before the prospect of an intelligence explosion, we humans are like small children playing with a bomb’. Nick Bostrom, Superintelligence.

I have always been fascinated by AI and recently joined a course to learn more. Absolutely brilliant! The teacher recommended Nick Bostrom’s Superintelligence – Paths, Dangers, Strategies as follow up reading, which is how I came across this book.

With the fast progress we have seen in machine learning in recent years, it becomes still more likely that superintelligence will be a reality in the not too distant future. Superintelligence could exceed human intelligence with an unimaginable factor and the big issue is, how can we control it? 

In essence Nick Bostrom’s book is a philosophical piece. Some might call it speculative. Philosophical or not, it tackles a very real problem with a distinct deadline attached to it. If we do not solve the control problem before we get to superintelligence, it will inevitably be too late.

In the first part of the book Bostrom discusses potential paths towards superintelligence and the forms this superintelligence might take. He then moves on to look at what could motivate a super-intelligent agent. 

Two key questions need to be answered:

• How can we get superintelligence to do what we want?
• What do we want the superintelligence to want?

There is no guarantee that superintelligence should have any natural inclination to work for humanity, rather than for itself. And when working with someone so much more intelligent than humans, it might be difficult to impose an incentive structure which will make the AI cooperate.

If the AI is actually willing to work towards an objective which will optimise on behalf of humanity, what does that actually mean? How can we ever communicate something so complex to an AI agent?

Even a simple objective such as ‘make humans smile’ could very well result in an optimal and very efficient solution consisting of filling humans with some chemical which make them walk around as happy zombies. Probably this was not the intention with the objective. But on paper it meets the criteria perfectly.

It is also unlikely that we know what is best for humanity. Our perception of what is right and wrong has changed a lot over times and there is no reason why that should not continue. Superintelligence with its ability to solve complex, multi-dimensional problems should have much better chance to come up with a good solution. If we could just make it understand what we mean by ‘good’. 

These questions are immensely difficult and I am glad not to be in Nick Bostrom’s and his colleagues’ shoes trying to answer them.

The book is written in a dry, technical language. Although I did have, if not a laugh-out-loud moment then at least a smile-to-myself moment over Chapter 9. Bostrom discusses the possibility of AI suspecting it exists in a simulation instead of the real world. Metaphysics for machines! Decartes would have liked it. Ok. Right. Maybe I am the only one who found that funny. 

Bostrom does not like the association, but how can you not put a Matrix reference here.

I must admit getting through this book was hard work. A ‘typical’ sentence reads:

Doing so would seem to require predicting the rhetorical effects on myriad constituents with varied idiosyncrasies and fluctuating levels of influence over long periods of time during which the system may be perturbed by unanticipated events from the outside while its topology is also undergoing a continuous endogenous reorganization: surely an impossible task!

Presumably Bostrom’s publisher did not run the text through a readability test. I suspect it might have indicated a few issues with sentence length and word complexity…

Normally, I read non-fiction in paper format to be able to highlight and write notes plus the occasional ‘??!’ in the margin. But whereas it is easy to search back in an e-book, when you have temporarily forgotten the difference between neuromorphic artificial intelligence and whole brain emulation it is more complicated in paper books.

In the end, it was worth the effort though. Nick Bostrom’s book is an important work about a very crucial topic which could potentially have devastating consequences for humanity and our planet in general. I am still fascinated by AI, but also somewhat wary.   

As an extra bonus from reading this book, I picked up a few useful phrases such as Bayesian hyperparameter optimization, variational autoencoders and sentence-level vector embeddings, which are just bound to come in handy at the next dinner party I attend!