Do not worry, but worry
Interviewer: Paolo Tamagnini, KNIME
Some time ago, after his/her keynote talk at an important conference, a colleague of mine, made the statement that nowadays, every presentation about artificial intelligence (AI) [and related challenges for the future] need to be at least apocalyptic. It is true. By now, most documentaries, shows, TED talks, keynotes, and similar presentations clearly use a very technophobic tone. How much of this is true?
Are we really doomed and should we just surrender to AI?
Is AI going to take over humanity?
Is AI black magic?
Is it true that once you set an AI application into production you cannot control it anymore?
How many data scientists out there are able to control AI applications?
These and other questions we have asked AI experts Rosaria Silipo from KNIME and Diego Arenas from the University of St Andrews.
Rosaria Silipo, PhD, principal data scientist now at KNIME, has spent 25+ years in applied AI, predictive analytics and machine learning at Siemens, Viseca, Nuance Communications, and private consulting. Sharing her practical experience in a broad range of industries and deployments, including IoT, customer intelligence, financial services, social media, and cybersecurity, Rosaria has authored more than 50 technical publications, including her recent books: “Guide to Intelligent Data Science” (ed. Springer) and “Codeless Deep Learning with KNIME” (ed. Packt).
Diego is a Research Engineer working on data science and data engineering projects and finishing his doctorate in Computer Science. Diego has 15+ years of experience working in data related projects; he has an MSc in Data Science and has worked as a consultant and machine learning expert for different industries such as banking, finance, telecom, retail, HR, education, transport, manufacturing. Diego is an open source contributor and a Data for Good enthusiast.
[Paolo] Let’s start from a popular question. Is AI like black magic?
[Rosaria] AI is made of a set of algorithms and data processing techniques. It is all math, statistics, and data handling in the end. So, no. It is not black magic, it is just math. It is not more black magic than the engine of my car, which makes my car move from A to B even without me understanding all the details of its mechanic.
Comparisons like this one create a very technophobic perception of AI, pushing people to reject all of it. With this kind of panic attitude, the only solution is to say goodbye to everything, from your social media to all messenger applications, from your vacuum cleaner to your car, from the microwave to your toothbrush, giving up on all the benefits of modern technology.
[Diego] For the layman indeed looks like black magic but as Rosaria said, it is not. When you see magic tricks, you might think something supernatural is going on. However, once you take a closer look, you will come to discover that there is a logical explanation to what you quickly labelled as magic.
The same applies to AI, people may concede powers to AI that it doesn’t have, and when you look at it closer, in the end can be explained by mathematical operations and working in consort gives the impression of something beyond comprehension.
[Paolo] AI applications are able to learn from the data. So, is it true that once you set an AI application into production you cannot control it anymore?
[Diego] We need to look at an AI application as a helper. A helper that can do the same task we were doing before but more accurately or more efficiently. This changes who performs the task. However, the helper should still be monitored to check that it is not deviating the results.
You can, and should, have control over the deployments and be able to pull the plug if the AI system is not behaving as expected. AI systems should be designed and implemented with this in mind. An AI system should not perform tasks that are ethically questionable or that a human would not perform for ethical reasons.
Once you put an AI application into production you would like to see how it performs. It is often the case that you can update or replace your AI system with newer versions (trained with more recent data), or just take the AI system down because the results are wrong or not as good as expected.
Another thing to consider is that as soon as you start working with the results given by your AI application, you may be changing the nature of the original problem. Therefore, it is necessary to monitor its performance to measure any significant deviations.
[Rosaria] Many AI applications are based on machine learning and deep learning algorithms. In both cases, a model is trained to perform a specific task: giving recommendations, producing a churn score, providing targeted advertising, predicting mechanical failures, etc …
Yes, such algorithms learn from the data and their knowledge is periodically or continually refreshed with the new incoming data, even during their production life cycle. However, from there to say that you cannot control them anymore, the step is just a bit too big. They are controlled: they are constantly monitored to make sure they keep performing well. They can be taken out of service for ever or for temporary maintenance any time.
[Paolo] How many data scientists are out there who can control AI applications?
[Rosaria] The professional qualification of data scientist is a broad specification: from the data engineers, responsible for the whole data architecture, collection, storage, and preparation, to the machine learning/deep learning engineers building and training models; from the business analysts and domain experts, defining the specs for each project and using the end solution, to the IT production experts, actually responsible for the execution and maintenance of the AI applications in the real world on real-world data. All of these people control at least one part, if not all parts, of an AI application.
According to an article on KDNuggets, “In June 2017, the Kaggle community crossed 1 million members, and Kaggle email on Sep 19, 2018 says they surpassed 2 million members in August 2018.“ Considering that mainly machine learning engineers belong to the Kaggle community, and that this is just a fraction of all machine learning engineers in the world, and considering the large range of other qualifications involved in building an AI solution, then the number of professionals who can control the AI end solution is definitely much higher than just a few.
[Diego] There are many people who specialise in particular types of problems in their daily jobs. This could be a certain data science or machine learning technique or algorithm. An AI application implies a few different domain areas so the best strategy here is to modularize development and deployment and work around teams of specialised people. In my opinion, people who can successfully deploy an end-to-end solution in AI are scarce in companies. Often it will be the analyst learning by themselves new techniques and tools.
Having a balanced team for the type of workload that the company has is key. Balanced in terms of roles and experience. Forming collaborative and cohesive teams is key for the survival of analytic teams at companies.
[Paolo] Is there a difference in the perception of AI from people working in the field and from people not familiar with the techniques?
[Diego] Yes, it is different. People in the field will know what techniques to use for a particular problem. They will understand that they are just using programs and algorithms to get an approximated solution to their problem. And that an AI system is often a mix of different machine learning models working together.
Sometimes you can see overexcitement from people that are not in the field but want to solve everything with AI. And that is worrisome. There is a responsibility to first understand the technology and evaluate consequences of using it before coming up with any kind of solution. Accountability is something we should see more in the coming years from experts and non-experts regarding the use of AI.
[Rosaria] Yes, certainly. People working in AI know that algorithms are just algorithms. People not familiar with AI must rely on what divulgative docu-fictions might say. You understand that this technophobic apocalyptic approach does not help to prepare for a rational discussion on how to solve the real problems.
[Paolo] So, are you saying that AI is not going to take over humanity?
[Rosaria] I really do not think it will. It might change our way of life, for the better or for the worse, but I do not think that any algorithm is anywhere close to developing some kind of consciousness or being able to act on its own. I think that in today’s state of the art robots and algorithms will not escape our control. They might replace jobs, but that is another story.
[Diego] Imagine we have an AI system capable of using other AI systems, let’s call it a global AI. Given a task, it is able to select the right AI to solve it. This global AI is able to use and control other AI systems to perform different tasks. Imagine now that the controlling system can improve the other AI systems and itself by retraining and/or readapting to new tasks. At some point, the capabilities of the AI system may surpass average human intelligence. Soon after this, it could be able to train itself without human help or intervention. What happens next? This is the scenario we fear. And there are often two sides in conversations about this topic, one that says that the scenario is impossible to achieve. On the other side, there are people who believe that this is a potential threat, therefore we should invest efforts to mitigate any unplanned consequence from this scenario.
If you choose to pick the concerned side of this conversation, we must say that we are far from reaching this level of independence. It is unlikely that an AI system can take over the world but a more immediate concern would be the controllers of powerful AI systems. We require more transparency on algorithms and systems using our data and influencing our decisions.
[Paolo] You talk about human jobs being replaced by robots. What are the other real challenges for AI in the future?
[Rosaria] There are many challenges for AI in the future!
Many will be related to data security and to privacy protection. All this data available, all my data available, who can access them? Who can do what with them? What does the law allow companies to do with my data? Does the law allow other private citizens to see my data? Is my privacy protected enough?
What if there is a data leakage? What if there is a hacker attack? Is my data protected enough? Is my sensitive data protected enough? Who has my data, can also control me? How can I avoid that kind of power over me? Those are all questions more than technological answers, and require legal and before that political answers.
Another kind of challenge for AI will be about responsibility. If an algorithm makes decisions, who will be responsible for the wrong decisions? From self-driving cars to loan assignment, from medical diagnosis to facial recognition, who will be responsible for mistakes? Who will be responsible for the propagation of fake news? Or for the “free speech” on social media? Where does free speech ends and offensive speech starts? Again, political and legal answers are required here, before the technological answers.
Algorithms are based on data and data is biased, like the world it comes from. Shall we make an effort to correct the algorithm bias towards some ethnic groups or shall we purely rely on the data? If we are to make an effort, are not we interfering with the current world system?
AI will replace some jobs. It is undeniable that AI will have an economic impact onto the future world. We need to prepare people for the transition and most of all we need to prepare the next generations for more AI compatible jobs.
[Diego] I agree with Rosaria.
To me, some of the great challenges are around Fairness, Accountability, Transparency and Ethics. Like any tool, AI can be used to increase or reduce inequalities in the world. The use of models to make recommendations or influence decisions on people’s life puts a tremendous responsibility on the companies using AI systems for automatic decisions and we don’t see many companies working on ethical assessments of their machine learning models or AI systems to be used.
We can expect more AI systems and projects to include transparency as part of the development process. Including components to the system which will explain the decisions made by the AI system to a layman user.
Data agency is another challenge, how much the users own their own data. How much are they able to share and be informed about the insights produced with their own data.
If it is true that AI will take over some repetitive jobs, governments should work to retrain the working force to improve people’s lives. Hopefully the controlled and transparent use of technology and AI systems can have a positive impact in the world.
[Paolo] So, you are saying “do not worry, but worry”.
[Rosaria] [laughing] Yes, that is exactly what I am saying.
There are challenges posed by AI for the future. We need to address those issues now to have as little problems as possible when entering into a more automated and AI powered world. Dealing with those issues will require more legal and political decisions than technological development.
At the same time, we need to remain rational and evaluate the pros and cons of the technology fairly without depicting AI in apocalyptic terms, as more complex and more uncontrollable than it really is.
[Diego] There are many concerning issues that we need to work out, for example, improve legislation on AI systems, regulate corporate accountability, and in general work towards a fairer and transparent system. These are exciting times for the use of AI applications, I think our focus should be on fixing our more immediate problems before dreaming of an AI controlled world.
So, yes. Do not worry, but worry.