(This post originally appeared on the FinTech startup – SigmaIQ)

We are in an exciting time for technology as Artificial Intelligence evolves at a rapid rate to help consumers and businesses. Technology is used to make decisions based on data that delivers an outcome. Think of the recommendations by Netflix of what movies to watch or Spotify for which music to listen to. Both of these are driven by decisioning engines that look at a lot of data and make a prediction about what you’d like next.

This article focuses on the comparison between the main methods for building these decisioning engines – Machine Learning technology versus the more traditional rules-based automation technology to help you understand the differences and best uses.

Four Key Pillars of Today’s Technology

The four key pillars of today’s technology advancements are coming together to give consumers and business unprecedented capabilities. The pillars create an environment for innovation in business and problem solving approach. The pillars have been written about quite a bit, but here’s a brief overview:

• The increase in amount of available data gives mathematical models required for predictive analytics, machine learning and AI the juice to deliver outcomes like movie recommendations and the fastest route to you destination.
• Cloud computing/hosted solutions like Software as a Service (SaaS) platforms, and Amazon Web Services/Azure/Google Cloud has made it much easier to spin up a new idea or capability by lowering the startup costs of new, working concepts while decreasing the speed to test.
• The Interconnectedness of digital systems allows innovators to piece together different services like google maps or personalized elements of the website to create new combinations of capabilities. It is becoming easier and easier to stitch the best point solutions into a comprehensive solution.
• We are seeing dramatic improvements in the application of Artificial Intelligence (AI) approaches like Machine Learning (ML) and Natural Language Processing (NLP) to day in/day out problems in unique ways which drives efficiencies and product innovation.

Key Differentiators Between Machine Learning and Traditional Rules Based Systems

Both traditional rules-based and machine learning software systems develop a set of “rules” for analyzing data sets, but there are core differences across the creation, use and maintenance. Here are the key differences:

Source of Knowledge
One of the primary differences between Machine Learning and a Rules Based approach is “where is the knowledge coming from?”. In a rules based system, the knowledge comes from the human analysis, insight and experience of the builder of the rules. While valuable and nuanced, humans are inherently biased and limited in what they can analyze and build rules around.

Machine Learning based systems learn from the data itself. If the data is biased – incomplete, skewed, incorrect – the output will be biased but the development of the algorithm or “rules” is a non-biased algorithmic process driven by the data. Because the machine process is learning from the implicit and explicit relationships in the data, it can find rules in complex data sets much more quickly and accurately than a human can without implicit bias.

Scale
It is very challenging for a human based rules system to move from 1,000 to 1,001 rules due to the need to understand the impact of that one extra element or rule on the current 1,000. Not to mention that to get to 1,000 rules is challenging in and of itself. The impacts of deleting, modifying or adding a human based rule to fix a specific error might cause a cascade effect on the other rules.

Machine Learning models thrive on large amounts of data and are much more capable of creating and understanding complex decision criteria at scale.

The most effective image recognition models (like facial recognition) use close to one BILLION rules. Individual analysts, can’t replicate the pure scale of the understanding of the rules that describe the relationships of the data using traditional rules approach.

Even with help from statistical tools, humans can’t match that level of complexity at scale. A Machine Learning approach to the reconciliation problem really shines when trying to scale not only the amount of data – but the complexity of the data and need for associated rules for matching.

Learning Speed
Rules based systems built by humans can only learn at the speed of humans. As errors or mis-matches are discovered, new rules have to be analyzed, created, tested and implemented while balancing the impact of all of the other rules already in place. This requires specific skills and time that your organization may or may not have (most don’t).

Systems built on Machine Learning can learn in a more iterative way as new data is fed back into the model – aka the data feedback loop. The model might update in real-time if there is a significant flow of data back in and it’s structured correctly, but more likely in a regular batch process when enough data for a particular adjustment is warranted.

Updating and Maintenance
One of the key benefits of Machine Learning over manual rules-based systems is that the entire algorithm can be refreshed with the new data with all considerations of interaction with the other rules, order of rule application and data relationships taken into account without breaking previously defined rules.

You might think of this as a backward compatibility effect in that with a manual rules-based approach, when you add a new rule to address a new situation you need to consider whether it impacts previous rules. This is called regression testing in software but the need is the same in a complicated rules based environment.

Machine Learning solves this by updating the entire model with a complete set of data including the new – and old – data relationships.

The Use Case Drives the Selection of the Solution

There is room for both traditional rules based approaches and machine learning based approach. The selection depends on many factors including the complexity of the problem you are trying to solve, the amount of data you have access to, the structure of the underlying system delivering value and how often the decisioning system needs to be updated.

Thank you for reading this far into the article and I hope it helps in your understanding of what Machine Learning is and how it is different than traditional rules based approaches to decisioning.

Rules-based Technologies Applied to F&A

The rules-based automation world we have lived in until now actually began in the late 1960’s with the development of the first enterprise software applications. However, enterprise applications did not gain significant momentum until the late 1980’s. The 1990’s represented the rapidly accelerating rate of adoption by industry of Enterprise Resource Planning (ERP), Human Resource Information System (HRIS), Supply Chain Management (SCM), and Business Intelligence (BI) applications. This time period also saw the increase in capabilities of Excel – Visual Basic, improved macro support and more advanced functions – which most F&A teams start with in account reconciliation automation.

All the above applications rely on technology based almost exclusively on rules. The notion behind rules is to provide unwavering consistency in business process. Rules may be conditional, Boolean, or mathematical.

Conditional rules may be expressed as, ‘If x, then y’. For example, when setting up a new supplier, most ERPs will allow the user to designate a default General Ledger (GL) account. Accordingly, whenever a transaction is entered for that supplier, the corresponding entry will be posted to the default GL account unless staff overwrites the default account by manually inputting a different account.

Boolean rules are either simplytrueorfalse. A simple example of a Boolean rule is the balancing function of the General Ledger journal entry module. Most systems prevent the posting of unbalanced journal entries. In this case, Boolean logic compares the credit total and debit total. If the two are different by even a penny, the user is unable to post the JE. In this example, the Boolean rule does not guarantee the JE is correct, only that it is balanced.

Mathematical rules utilize a fixed formula to dictate a specific answer. A common use is driver-based budgeting. For example, when budgeting HR expenses at the department level companies will frequently divide total HR expense budget by total headcount as the driver, resulting in an average rate per headcount. Each department’s HR expense budget, therefore, is its number of headcount (driver) multiplied by the average rate.

Rules are Rigid

Conditional, Boolean, and mathematical rules share a common principle, namely rigidity. At the time these rigid rule methods were developed, the main objectives were consistency and efficiency. Solving a majority of transaction volume was still a tremendous benefit at the time, even if substantial human effort remained to address exceptions or transactions that did not fit neatly within the rule structure.

Machine Learning Technologies Applied to F&A

As has been the case with many technologies, the path to capable, production ready Machine Learning was uneven. Only within the last few years did four important elements finally converge: improved algorithm sophistication, cheaper and more powerful CPU processing, lower data storage costs, and the rise of more powerful computational programming languages.

The result being Machine Learning capabilities are finally up to the task of solving far more difficult and complex business problems. Machine Learning is distinct from rules-based methodologies in three important ways: algorithms, its ability to learn or adjust, and its flexibility to handle data variability or uncertainty.

Algorithms determine the best possible answer. The ability to incorporate multiple data fields or sources means there is more information available when the algorithm considers possible answers and can determine the best suited answer. For example, a supplier master record is no longer limited to a single default GL account. Rather, if an invoice arrives with multiple lines, the system would assign the appropriate GL account to each line based on both supplier and line-level information without requiring the set-up of an exhaustive library.

Learning means the algorithm adjusts itself as the system ingests more data over time. Algorithms are designed to be general enough for each broader use case, which allow them to adjust and continue to learn as more data is consumed.

Flexibility allows data variability to be handled effectively. The learning that takes place is more than knowing how to treat a specific situation in each future recurrence. Rather, the system can leverage statistics, such that if a new situation is similar enough to prior ones (even if not exact), it can still effectively disposition an outcome.

Rules-based technologies lack any concept of implicit data relationships. Everything is discrete. When a piece of data arrives, the rule does not consider circumstance or context unless explicitly programmed beforehand. Instead it accepts the data and processes it according to the rule specification.

Machine Learning is both a consumer and creator of data. Instead of a rule dictating a specific outcome, Machine Learning uses prior and tangential data to evaluate the outcome most appropriate for the circumstances. The answer it generates also becomes additional input to the overall data population for future transactions.

Implications for F&A Organizations

In a Machine Learning environment, the role of F&A staff is significantly different than in a rules-based environment. In today’s rules-based organizations, F&A staff spends the vast majority of its time and effort on three activities: generating or inputting data, finding problems, and fixing problems.

In tomorrow’s Machine Learning F&A organization, staff will instead monitor and supervise processes, statistically validate activities of the systems, and greatly reduce their transaction-related effort to a small proportion of their time and effort. They will repurpose those regained hours to focus on higher-level analytical work directly impactful to broader strategic goals.