Decision Engine

November 9, 2016
“The second your customer arrives on your website you need to start making decisions...”

Whether selling via a website or app, you need to respond immediately to each step of the journey. You need to “see” your customer, understand who they are, collect the right data, evaluate their needs, detect anomalous behavior, assess risk / pricing / product / affordability, dynamically adapt the customer journey and make many other decisions... and all this in real-time. Only this way can you optimize your service and profitability. To do all this and more, Zoral has developed one of the world’s most advanced, intelligent decision engines Zoral Decision Engine (ZDE).

ZDE - an essential component for digital products

In many ways selling digitally is no different to selling face to face. You need to understand as much as possible about your customer and ask the right questions. Then, you can continually adapt your approach as you learn more and make consistent, accurate decisions. ZDE is an essential component to solving this problem.

There are many decision engines in use. However, most were designed and developed prior to the mass sale of financial products via the Internet. There was no need to acquire massive amounts of heterogeneous data, (much unstructured), and react instantly, in a complex, non-linear way. Now there is. So Zoral produced ZDE, which was designed and developed from the outset to cope with these issues.

ZDE captures and processes a huge range of data in real-time. This includes application, third party, behavioral, geolocation, MNO digital fingerprint, digitised speech and many other types of data. ZDE uses and enriches this data. It allows the non-technical user to define complex workflows, rules logic and executes real-time decisions. These can be used to adapt the customer journey in real time and progressively react in line with the evolving customer profile.

ZDE is not only essential to making accurate decisions, but is also key to automating previously manual processes. This is vital if your digital product is to scale efficiently and be cost effective.

Decisions are simple but the process is complex

If you are selling financial products via the Internet or an App, you need to make automated, reliable decisions. These are no different to the ones you would make if selling face to face. For example, if you receive a lead or referral to your web site, they might be,

  • do I want to pay for this lead? Is it likely to turn into a sale and a profitable customer?
  • which product should sell?
  • if they don’t qualify for product A, should I sell product B?
  • what is the risk? how should I price? do I underwrite this client?
  • will they pay on time?
  • will they renew or churn?
  • are they who they say they are? is this a fraud?
84%
Reduction in manual effort in underwriting
SME loans
(Finance Corporation, USA)
65%
Reduction in manual effort in fraud management
(Consumer credit Australia)
22%
Reduction in default rate
(Consumer credit UK)

ZDE

Zoral Decision Engine is a sophisticated, multi-functional, fully featured decsioning system.

It is able to execute complex workflows, linear and non-linear functions in real time at ultra high speed.

Its sub-components and functions include:

  • data/metadata interface
  • workflow execution, deployment controls
  • rules and workflow evaluation/optimization
  • interface to analytical services
  • real time decisioning, dynamic customer journey support
  • scheduling
  • testing, development, sampling
  • hosting configurations
  • analytical/modelling environment (batch testing, A/B, champion/challenger etc.)
  • sophisticated modeling support, (e.g. linear and non-linear functions)
  • dashboard reporting (system, model performance, core KPI’s etc.)
  • audit /security controls

ZDE has been specifically designed to capture and automate the “ know how” from business users and data scientists. It is used for a wide range of functions including:

  • intelligent use of predictive data from a wide range of sources, (application, behavioral, MNO, social, unstructured, third party etc.)
  • automation and delivery of digital products
  • real-time, actionable decisioning
  • automation and management of customer/product lifecycle

ZDE is built to address changing business needs, priorities and requirements. It allows you to respond rapidly without the need for costly and time consuming development. For example,

  • ZDE decision workflows can be reconfigured by business users
  • thresholds affecting ZDE decisions can be changed and modeled by business users
  • new decision workflows can be added to address special cases or new products
  • ZDE is simple to customize according to business needs
  • ZDE provides capabilities to design and model changes to rules and models
  • it provides development, experimental and deployment environments

ZDE also has a wide range of facilities for the Technical user, including:

  • complex or resource-critical coded verifications
  • workflow input and workflow output data transfer objects
  • implement and register external data providers with their data transfer objects
  • implement workflow data dependencies
  • create and register workflow runtime configuration
  • Implement new reports or integrate with existing BI
DE Environment

Highly scalable, low latency, adapted for business users

The core of any decision engine is that business users can easily/quickly capture the logic underlying the business’ operation. This is done by creating, testing and managing executable business rules in a human understandable form or graphical notation, without the need of IT involvement. The order of execution and optimization of rules execution is automatically resolved, so is not dependent on business user ability to create optimal logic. Commit or decision boundaries are complex processes. These are also determined automatically in ZDE for rules, models, workflows, heterogeneous data context, and decision cost optimization. ZDE can perform this dynamically, at run time.

DE can handle each client/request as it arrives, or via micro-batches. ZDE does this with very low latency and high throughput for streaming responses / decisions / actions. Additionally, ZDE can automatically distribute clients/requests/work across a clustered, runtime environment. It has a fully extensible and programmatic approach to AI/ML analytics and supports unstructured, device, social, 3rd party, behavioral data, so can deliver a low-latency, stream-decisioning solution.

The ZDE cluster consists of number of components. Each component is designed to perform a specific set of functions. This design separates functions and simplifies the overall system. ZDE components are fault-tolerant and can operate independently of each other. This is important so that, intra-cluster communication failures have minimal impact on data availability and decisioning.

ZDE solves complex, real-time, data analysis, data verification and optimized decisioning problems.

An optional Micro Batch Proxy server can be configured providing a vectorised, ZDE operational mode. The Micro Batch Proxy server component of ZDE can be configured with user defined frequency or default buffer size. The Micro Batch Proxy can periodically flush its buffer to the Decision Engine server instance, while at the same time log the incoming, new ZDE requests to a new buffer. ZDE components can be configured on same server or further scaled on separate server instances as shown above.

Rules, workflows, metadata, and configurations are dynamically read and updated from the ZDE Database, which is replicated to SQL Server, stored in Riak Database and further scaled with in-memory cache to minimize the latency between ZDE and its Database component.

The Data Provider Balancer can be configured as an optional, separate component server instance. It distributes work across a number of different types of ZDE Data Provider components. It can capture different types of data, based on pending/to be evaluated clients/requests/decisions, as inputs into the rules Evaluation Engine. This can include, MNO data, behavioral data, 3rd party data, unstructured data, social data etc.

Data Providers themselves can originate from Big Data platforms, such as behavioral data streaming from Zoral BDW/Hadoop/Spark or from ElasticSearch query output. Each type of Data Provider component can be scaled across one or more servers and optionally vectorised. A similar load balancer deployment scheme can be configured to scale analytical servers that perform model evaluations in R or SPARK or PMML or Python or NLP or C++ (e.g. executing models from the Zoral Model Library), for example:

DE Deployment

ZDE Management Studio

ZDE has a comprehensive management studio where business users can capture/define, view, edit and monitor a wide range of functions, including,

  • Business rules,
  • Models,
  • Workflows,
  • Configurations,
  • Security permissions,
  • Testing,
  • Audits,
  • Reports,
  • and many other functions.

The Management Studio can handle multiple, simultaneous business users and can be scaled across one or more servers. In addition, ZDE can be configured to work with or as extension of a number of enterprise architecture components to include PFM, CRM, Billing, ERP, LMS, Marketing Systems etc.

Decision Workflow Management

ZDE provides powerful, workflow management tools. These are graphical and simple to use for non-technical users. Functions are sophisticated but complexity is handled automatically by ZDE itself rather than requiring deep technical understanding. An example is shadow testing of new decision workflows. Additional versions of the workflow can be executed in “shadow” mode alongside tested, production workflows, without affecting actual decisions.

DE Workflow

Other features and functionality

ZDE interacts securely, synchronously and asynchronously with web/mobile enabled devices / applications including Web/Mobile apps, Email, SMS, IVR, USSD, GPS, etc. This includes internal business ‘CORE’ or ‘Back Office’ processing systems where ZDE can improve and further automate and scale business processes and help reduce operating costs.

ZDE has data/event notification integration points to other enterprise software components, (such as PFM, CRM, LMS, G/L, Billing, ERP, B2C Mobile Apps, Core Banking platforms, etc.). These are API based and are flexible, scalable and bi-directional.

ZDE scales to billions of customer / application/ device interactions per day. It provides low-latency response, (latency is a configurable/granular parameter).

ZDE’s component architecture makes it highly scalable. It is also simple to upgrade/replace sub-components, (e.g. change/add the rules language, rules evaluation engine, execution optimizer, UI, Storage, Scheduler, Analytical Libraries, Data/Event Integration Bus, Data Validation Engine, Workflow Management, Metadata Management, Data Provider Engine, Analytics Engine, Configuration Engine, Monitoring, Alert Engine, etc.)

ZDE integrates with a wide range of 3rd party data providers worldwide. It has re-usable, intelligent 3rd party data/application adapters.

ZDE is capable of handling extremely large, “Big Data” volumes.

ZDE provides traceability, audit and control of decisions made (e.g. for compliance, audit of Government Regulated processes)

ZDE has extensive data validation and verification logic, in support of intelligent STP, including:

  • Identity verification
  • Company verification
  • Credit Card verification
  • Bank details verification
  • Employment/Employee verification
  • Personal information verification
  • Document verification / Image Analysis
  • Financial statements verifications
  • Blacklists usage
  • Digital information verification (device data, click-level behavioral data, for anti-spoofing)
  • Social networks data cross-validation
  • Geolocation data verification

ZDE allows you to define alternative verification processes or models, as there may be multiple ways to perform the same verification or model. Alternative implementations can be used as a backup or a redundancy measure when a primary verification solution fails to run, (e.g. a 3rd party data source is down, an outcome is inconclusive, to improve verification or improve model confidence.)

ZDE architecture is such that it effectively handles attacks such as serial. linked and velocity frauds.

ZDE is designed for use with advanced analytical methods, such as text mining, non-linear AI/ML functions, combining or boosting of models.

It provides the ability to measure, scale and monitor deployed AI/ML (PMML, R, SPARK, Python, NLP, C++ etc.) model performance.

As well as third party language/tool compatibility, ZDE has its own powerful, easy-to-use, rules language that can be utilized by business users to capture their know-how and business intelligence. Version control is efficient and automated. Users can easily build comprehensive business process work flows, incorporating AI/ML models as extensible functions in the rules language.

ZDE can operate in batch, micro-batch or real-time mode.

It allows atomic decisions, (a set of rules), to have their own granular set of conditional thresholds and decision timeouts.

Work flows can be saved as intelligent, version controlled templates to be re-used/modified, across geographies and other business functions.

Users can easily test any part of decisioning logic, including ZDE’s built-in unit testing, batch testing, automated/regression testing, (in support of SDLC best-practice)

ZDE has built in A/B testing for experimentation, digital product innovation and logic optimization.

ZDE allows shadow testing of new decision workflows. Additional versions of the workflow can be executed in a shadow mode alongside tested, production workflows, without affecting the actual decisions.

Leverage a scalable analytical library and analytical tools (e.g. R, PMML, SPARK, NLP, SAS, Python, C++) to enable business KPI optimization.

ZDE also provides analytical reporting and BI, as a component and real-time input source to the enterprise, Big Data architecture.

One of the world’s most advanced decision engines.
AI/ML decisioning for digital products

  • Fault tolerant, highly scalable, low latency
  • Easily customized, user friendly UI
  • Multiple product support
  • Asynchronous/synchronous execution (e.g. to accommodate serial fraud decisioning logic etc.)
  • Credit policy rules verification
  • Decision workflow versioning
  • Designed for use by business, data science and IT/development users
  • Retains decision data context for compliance and analytics
  • Integration with third-party data providers, credit bureaux
  • Data analysis /modeling capabilities, e.g. batch testing and A/B sampling
  • Cost- and performance-based execution optimization for reduced, third party data costs
  • Business intelligence and dashboard
  • Ability to implement a variety of modeling techniques (rules, statistical, AI/ML complex techniques)
  • Handles a wide range of Big Data sources, including behavioral, social, device, unstructured, application, MNO, third party and many others
  • Integration to Zoral sentiment analysis and entity extraction
  • Workflow and thresholds configuration by business users
  • Logical separation of data retrieval from data analysis and parallel data retrieval
  • Versioning of workflows and changes
  • Offline deployment of models
  • Batch runs/simulation based on historical/synthetic data
  • Cloud and on-premises deployment
  • Odata API to provide decision access and runtime data
  • Flexible storage type (SQL Server / NoSQL / Others)
  • Flexible RPC\messaging transport
  • WCF for synchronous calls
  • RabbitMQ for messaging
  • Other protocols/transport easily integrated
  • Integration testing
  • Scalable and fault tolerant architecture
  • Sophisticated rules structure/decisioning logic, (e.g. beyond simple tree structures)
  • Ultra high performance enabling intelligent STP and descisioning
  • Unit and batch testing supporting best-practice decisioning, QA and release management
  • Fully Extensible
  • Easily integrated to UI, LMS, CRM and DW/BI
  • SDLC engineering best-practice toolset: version control, workflow, enterprise enabled, audit-trail, monitoring/reporting, built-in analytics library, encapsulation of decisioning logic,
  • Scales to multiple products, across organizational and geographical locations.
  • Scales across multiple UI platforms and back-end infrastructures
  • Integrates with Zoral Behavioral Data Warehouse, Zoral Model Library and Zoral Platform
  • Long running workflows
  • Dashboard and widgets to monitor KPIs
  • Standard and customized reporting
  • Audit and user management
  • Ability to define custom workflow data model
  • Pre-built and custom data providers
  • Models and scorecards (Zoral Model Library, R models, PMML models, create model directly in Decision Engine etc.)
  • Sub-workflows and sub-verifications to share logic between workflows
  • Configure settings, define new workflows, rules and verifications via UI or XML files, utilizing external version control systems in addition to built-in
  • Create verification by business users
  • Parallel execution of external data providers
  • Cost\time execution optimization for workflows, based on statistics and costs parameters
  • Reusable data providers
  • Different communication patterns for third-party integration

This document is provided by Zoral Limited and its affiliated companies (“Zoral”) for informational purposes only, without representation or warranty of any kind. Zoral shall not be liable for errors or omissions with respect the information contained in this document. Product Specifications are subject to change without notice. The only warranties for Zoral products and services are those that are set forth in the express warranty statements in Zoral’s standard contracts for such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

© Zoral Limited 2016