Software robots are invading businesses and public institutions. Many companies have forgotten what they told the bots to do. This creates highly risky situations with devastating consequences when not properly managed.
Bot Invasion
We think you should have a broad view on software bots. They represent all software driven business decisions a company or institution makes, whether they are semi-automated, fully automated or machine-learned.
Over the last 60 years we have seen software enabled automation of human activities come in many flavors. Initially, software enabled automation focused on high volume batch processing of repetitive transactions. Later simple personal productivity improvements were added. They further evolved to encompass straight-through processing, enabling end-to-end automation of customer self-service from click to instant result. This has since been supplemented by robotic process automation (RPA) further boosting workplace productivity. Latest developments include artificial intelligence (AI) and machine learning powered by a growing bedrock of data. Today, software enabled automation algorithms can be found everywhere in every industry and public institution, embedded into both old and new technology. The automation approaches seem to differ by technology type and scope, see exhibit 1.
Exhibit 1: Types and scopes of software automation; Source: Oleto Associates experience
There are three key trends in software enabled automation today:
1) From case/semi automation to full automation: Increasingly utilizing bots and algorithms to support complex cases such as reviewing applications (for, e.g., loans, immigration), with algorithms working in unison with representatives to provide accurate results rapidly. In case automation processing, a stage gate approach is commonly used to capture human input at predetermined stages to ensure compliance with outlined objectives. An additional example of case automation can be found in automated application tracking systems (ATS), utilized by most Fortune 500 companies to streamline their HR processes and help recruit key talent. As case handling rules become more explicit they can move towards increasingly full automation.
2) Adding artificial intelligence to full automation: Moving task automation to be conducted more intelligently, with the capability to learn from past experience. When automation of enterprise processes is combined with AI algorithms, it is commonly referred to as intelligent process automation (IPA). The allure of IPA for companies that utilize the technology is the combined benefits derived from automation (e.g., scale capabilities, task efficiency, rapid processing), with the added intelligence inherent within AI (e.g., insights/learning, flexibility).
3) Expanding cognitive networks: Intelligent automation is utilized to collaborate across the network in order to create the extensively autonomous environment businesses need, both today and in the future. Deep neural network (DNN) technology is expanding our capabilities to perform non-linear tasks while operating autonomously in complex and unpredictable environments. An example of DNN use can be found in the emerging field of autonomous driving, where algorithms do not need to be trained for every scenario, and instead can adapt previous learning to future scenarios.
Hidden & Forgotten
Despite the increased reliance on automation there are often cases of this reliance being overlooked. Maintaining control over the application’s logic and the desired applied business rules is more important than ever. The potential impact of failures and misuse of automation tools can be catastrophic to a firm's operations, reputation and bottom line.
We have gathered some examples in exhibit 2 and the likely root causes. What is common across all these cases is a lack of understanding regarding what the robot has actually been taught/told to do and an insufficient level of attention from management.
Exhibit 2: Likely root causes that resulted in automation failures; Source: Oleto Associates experience; Case research
European bank: Recently, a major European bank admitted to collecting outdated or excessive debt from thousands of its customers over a long period. The bank attributes its error to an automated debt collection system and the case is under investigation. Ultimately, even though the management was likely aware of certain aspects of the system malfunction, the bank failed to address the issues timely and adequately. A lack of transparency on the system logic and the absence of clearly defined and traceable business rules may have resulted in losing track of what the "debt collection bot" was doing.
Large tech: Automating various aspects of recruitment is seen as an attractive approach to decreasing overhead and improving talent acquisition processes. A few years ago, a large tech company aimed to pioneer an automated tool to improve the recruitment process for technical roles, only to scrap the program after recognizing that the algorithm had trained itself to be socially biased. The automated software consistently preferred white male candidates, due to the fact that the software was trained by analyzing only one type of candidate and rigorous follow-up testing was ignored. Although the algorithm employed by the large tech company functioned in a logical manner, the result of its selection created unforeseen bias that reinforced imbalanced hiring practices.
Regional telecom: Within telecommunication, robotic process automation has brought greater efficiency to repetitive and/or high volume back-office related tasks, but not without issues for companies that lose track of the hidden and forgotten complexity within the system. A large regional telecom underwent a major technological transformation, which saw a shift in how they processed top-up payments for their thousands of pay-as-you-go customers. Errors in their automated processing of payments resulted in a violation of customer agreements, whereby customers failed to receive call credits for payments made. When the large regional telecom failed to address the issues arisen from the lack of testing, government regulators stepped in and fined the firm.
Social media company: With social media playing such a massive role in our daily lives, it is no surprise that many democratic countries are focusing on the spread of fake news on such platforms. Social media companies employ advanced algorithms to track and influence their users. The commercialization of these social media platforms has enabled interest groups and companies to aggressively target users based on their online habits, connections, preferences, political aspirations and more. As we engage with the platform and provide the algorithms with more information about ourselves, our news feed becomes more and more biased, continuing to supply us with what it believes we want to see. The result, we continue to reinforce our biases while increasing our susceptibility to potentially fake news.
Financial trading institutions: Over the past four decades, the use of robot algorithms in stock market trading has resulted in multiple instances of market disruption, ranging from a near collapse decades ago to a major stock market sell-off recently. The notion of employing algorithms to conduct large-scale trades is now common practice and the consequences of multiple investment banks capable of placing countless orders simultaneously exposes the market to risk. The intention of such algorithms has never been to instigate a stock market crash, rather the objective is to conduct trades rapidly to stay ahead of the competition. With that said, the robot algorithms are highly complex and commonly proprietary intellectual property that is seldom fully understood by all relevant stakeholders within the financial trading institutions.
What Companies Should Do
Over the last decades we have seen the degree of digital automation increase significantly and this trend is bound to continue. Not managing what the whole automation landscape is doing, i.e., what all the bots have been told to do, is comparable to not having control of which policies your employees follow. Companies must ensure that what systems/software robots are doing is completely transparent, fully tested and that issues are brought timely to the management's attention.
Companies should ensure they following steps are taken:
Secure the basic intelligence: Ensure that the business rules and logic used within the system are properly documented and understood by relevant stakeholders. Therefore, maintaining institutional memory of the process and ensuring that the required technical and business talent is available within the company can go a long way in keeping track of the real business decisions made. Furthermore, companies ought to better understand what their software algorithms are capable of prior to unleashing the advanced technology. This should be addressed business domain by business domain.
Professionalize testing: Improve initial testing and regression testing based on real data and the construction of various realistic and uncommon scenarios to better understand the ramification of their automation software. Additionally, with the popularity of global ecosystems and cognitive networks, testing needs to be conducted more broadly to understand the interplay between multiple parties with potentially different agendas utilizing autonomous algorithms simultaneously.
Increase management involvement: Maintain oversight throughout the entire process to ensure risks are mitigated and addressed at every step. This includes setting focus on business areas where automated business rules are not fully understood and need focus.
About the authors: This article was written by a team of consultants from Oleto Associates, a strategy consulting firm based in Denmark. For more information please visit www.oleto.com
September 2020