AI to bring massive benefits, but also cause great concern
In this interview with Help Net Security, Matthew Rosenquist, CISO at Eclipz.io, explains the benefits as well as the disadvantages of AI, and the tremendous impact it could have on our society.
Artificial intelligence is gaining momentum. What could be the benefits for enterprises?
The applications for Artificial Intelligence are growing every day. AI is great at determining optimal paths and resource utilization as well as identifying objects or interpreting data such as voice recognition. These are powerful tools that can be integrated into many different systems to improve effectiveness, efficiency, or create entirely new capabilities.
Autonomous driving is a great example where AI is employed to understand the surrounding objects and their behaviors while plotting an optimal trajectory around obstacles en route to the destination. Regular logic is not sufficient to comprehend all the inputs from cameras, lidar, and the multitude of other vehicle sensors. AI can identify object data it has never seen before and still classify it with a high degree of accuracy. This will enable ever more capable and safe self-driving cars, busses, trains, ships, and even planes. All computer-controlled.
The downside to such automated collision avoidance and navigation is, it becomes susceptible to interference, manipulation, and malicious hacking. This is true of all AI innovations. The more we trust, embrace, and enable technology to have control over our lives, the more risk we must be willing to accept or mitigate.
What about the disadvantages when human rights, consent, privacy, and security are concerned?
Many organizations are looking for AI to make sense of tremendous amounts of unstructured data that has been collected about people, transactions, systems, and social connections. It is estimated that over 90% of all data is unstructured and not usable by normal systems. AI holds the potential to find patterns, connections, and derive the value of such telemetry and raw data.
This could provide great beneficial insights into people’s needs, desires, opportunities, and alerts when they are subject to unfair practices. But at the same time, it can aggregate data in ways that build stronger profiles of individuals. Such systems can make connections about very private or personal aspects of people’s lives that those individuals never authorized or want to be known. It could be a confidential relationship, a medical condition, personal fears, economic status, or something they are privately working through. In much of the world, privacy is recognized as an important human right, necessary for people to thrive.
Video surveillance systems are a major concern, with AI now being able to identify and track people from networks of connected camera systems. This will potentially allow governments, who install many cameras to monitor the public, to track where every person is at any time, what they are doing, and whom they are meeting with. I expect such technology will also evolve to process microphone information or effectively read lips to determine what individuals are saying to each other. Such implementations will greatly undermine privacy, free speech, and in some countries could be used to persecute individuals for simply talking about certain topics.
In the world of cybersecurity, both attackers and defenders will leverage the power of AI systems for their respective benefits. Attackers will use AI to launch widespread automated attacks that are customized for individual targets, integrating analysis from disparate systems, and learning from each failure to optimize the chances of socially engineering a person or compromising a technical system. These artificially intelligent cyber attackers will be relentless.
The security folks will use AI to detect anonymous behaviors, trigger mitigation actions, then measure the effectiveness. Such systems will continuously learn and improve to prevent and minimize losses. These AI enhanced capabilities will replace much of the mundane work of security operators, allowing staff to focus on the most interesting issues and be empowered with smart orchestration tools to increase their overall operational capability. The same attack tools will be repurposed by security teams to proactively detect vulnerabilities in people, systems, and processes that must be improved before real attacks occur.
What do you predict the path for AI will be?
The undeniable value of AI will fuel a rapid adoption and further innovation for new use cases. Data analysis against the vast data lakes that exist is appealing to organizations in every sector. This will be a race for enterprising software providers to create new AI enhanced products and quickly maneuver to gain market share. Prioritizing Time-to-Market and Minimum Viable Products, typically sacrifices security, privacy, and may violate ethical practices across the architecture, design, development, implementation, and sustaining operations lifecycle. Rapidly developed AI systems will be the norm, but also a cause for great concern.
What should we be concerned about the most when it comes to AI?
The powerful lure of harnessing the great power of AI to transform digital technology across the globe may blind users to the necessity of mitigating the accompanying risks of unethical use. The ethical ramifications often start with developers asking ‘can we build’ something novel versus ‘should we build’ something that can be misused in terrible ways. The rush to AI solutions has already created many situations where poor design, inadequate security, or architecture bias manifested unintended consequences that were harmful. AI Ethics frameworks are needed to help guide organizations to act consistently and comprehensively when it comes to product design and operation.
Without foresight, proper security controls, and oversight, malicious entities can leverage AI to create entirely new methods of attack which will be far superior to the current defenses. These incidents have the potential to create impacts and losses at a scale matching the benefits AI can bring to society. It is important that AI developers and operators integrate cybersecurity capabilities to predict, prevent, detect, and respond to attacks against AI systems. The goal is to deliver the tremendous benefits AI while managing the risks to acceptable levels. It is a balancing act.
Do you see AI solving human errors and to what extent could it replace humans?
AI is a tremendously powerful tool that is capable of incredible good for mankind, or if used malevolently, could negatively impact society at an equitable level.
Autonomous vehicles hold the promise of nearly eliminating driving accidents, injuries, and fatalities, as most are due to human error. However, if an entire make of cars were hacked and simultaneously forced to crash at high speed, the resulting catastrophe would have momentous impact ripples on society.
In the near term, we will see AI play two roles. First, it will enhance what humans already do now. Supervised driving where AI systems do most of the work, but humans can intervene if necessary. This should provide great benefits while maintaining human safeguards.
Secondly, AI will automate work that is currently not being done, because it is too human-intensive or manually too slow. Real-time data analysis is valuable to almost all industries. Think about better search engines, email filters, personal assistants, conversational foreign language interpreters, helper robots, and understanding how an individual might be swayed to click a link, purchase a product, or believe a narrative.
Like all powerful tools, AI requires human involvement to ensure the architecture, design, and engineering and ethical, effective, possess limits that inhibit unintended usage and are hardened from attack. If we treat AI with respect and understand both the benefits and manage accompanying risks, humankind will have the ability to benefit in unimaginable ways.