Menu

Inside Tink: getting top model performance for Income Check

We get into the backstage of Tink tech to learn more about how our teams are tackling challenges and continuously upgrading our solutions – and ways of thinking. Here, Data Scientist Ida Janér explores why income verification is all about top-performing models – and how we improved accuracy by embracing confusion.

TL;DR – Quick summary

  • Tink’s Income Check allows businesses to instantly verify a person's income by analysing their transaction data.

  • For it to work well, having a high-performing classification model (with a high accuracy in machine learning terms) was fundamental.

  • Tinker Ida Janér explains how data scientists in the Risk team worked to improve the model performance, and the key learnings they took from the process.

Inside Tink: getting top model performance for Income Check

By Ida Janér

Earlier this year we launched Income Check, which allows businesses to instantly verify a person's income with secure, real-time data directly from their bank account.

With the help of pattern recognition, incoming transactions to a user's bank account can be grouped and filtered by recurrence and other criteria to extract only the ones that constitute income streams. Each stream is classified and labelled with the type of either salary, pension, benefits, cash deposit or other.

When we launched Income Check we knew from the outset how important it would be to build a high-performing model, with a high accuracy level in machine learning terms.

But even more important would be to put in place a sustainable and scalable way to reach that level, and stay there. So the data scientists in the Risk team got to work, and now Income Check is at 99% accuracy in Sweden. (Meaning in 99% of cases the model understands whether a transaction should be viewed as an income or not.)

So what did it take to get up to that 99% accuracy level?

Building the model

The first step is to observe incoming transactions from real transaction statements and extract keywords from transaction descriptions.

To help us understand which among the transactions actually are an income (and should be considered as such in the context of affordability checks for example), we first analyse the transactions for parameters such as date, frequency, and stability.

Building the model

Based on this initial analysis we train a model to predict whether a given transaction should be viewed as income or not. From then on it’s all about measuring and improving the performance of the model – and in particular its accuracy – when it’s really put into action.

Improving the model

If the last 18 months have taught us anything it’s that circumstances can change quickly. The world is constantly evolving, and so any data model that relies on real world data must be constantly retrained and updated.

Any retraining of the model based on changes to the data and current climate (such as changes in the payout pattern from a pension fund company, for instance) then needs to be validated to ensure they deliver an increase in accuracy, so we always provide the best possible proven model to our customers.

And naturally the changes, and the models, become increasingly tailored to a particular market behaviour as our local knowledge grows.

To reach top model levels of success, embrace confusion

A confusion matrix is a summary of prediction results on a classification problem. The number of correct and incorrect predictions are summarised with count values and broken down by each class. This is the key to the confusion matrix.

The confusion matrix shows the ways in which your classification model is confused when it makes predictions. It gives you insight not only into the errors being made by your classifier but more importantly, the types of errors that are being made.

To reach top model levels of success, embrace confusion

  • True positives (TP): transactions that the model correctly predicts as income

  • True negatives (TN): transactions correctly predicted as not being income

  • False positives (FP): transactions incorrectly predicted as income

  • False negatives (FN): transactions incorrectly predicted as not income

Accuracy is the share of correct predictions.

Accuracy is the share of correct predictions.

Precision tells us how certain we are that the transactions predicted as income actually are income.

Precision tells us how certain we are that the transactions predicted as income actually are income.

Recall tells us how good the model is at identifying all transactions belonging to income.

Recall tells us how good the model is at identifying all transactions belonging to income.

Optimising accuracy – a case study

The team took two different markets (Finland and Sweden) over a six month period and reviewed over 25,000 cases where a transaction history had been analysed and income streams were identified and provided by our Income Check product.

Here are some examples of what our model re-training phase consists of.

Reviewing new data

The new data is reviewed based on some predetermined definitions of income. Part of the data is used for model training, and part is used for model performance evaluation. Splitting the data in this way is important to get an unbiased review of the actual model performance

Revisiting model parameters

We revisit parameters such as keywords and thresholds (setting minimum and maximum cutoffs for different transaction metrics), against which we measure numerical values belonging to the transactions (such as amount, number of occurrences and periodicity).

Analysing specific market behaviour

Since each market is unique, we also need to tailor the model to capture market-specific behaviours. This goes beyond just language differences – different markets also come with specific patterns in payouts for different types of income. This sort of market-specific tuning is done in collaboration with local representatives who can provide a deep local domain knowledge.

Taking this approach and applying a review based on the confusion matrix helped us understand how good our model was in each country in terms of accuracy, precision and recall.

Just going by ‘did we identify an income?’ was no use if it turned out that what we thought was an income actually wasn’t, or if there were income streams we did not spot at all. Not everyone’s income is a stable monthly salary.

Here’s a look at how performance improved in that period.

Performance improvements in Finland

Performance improvements in Sweden

In both markets we saw that analysing performance and retraining accordingly for each class in the confusion matrix saw equivalent gains for both precision and recall.

This means that the model not only classified more transactions as income, but it was also certain that those transactions actually were income. This then is reflected in the significant increase in accuracy in Finland (due to diminishing returns, it was harder to improve from the already high accuracy levels in Sweden).

We improved the salary and benefit classes, as well as being able to look at inconsistencies in periodicity and amounts in Finland, while in Sweden improvements were reached mainly due to us improving pension and benefit intelligence.

But the key learning was how different the approach needed to be in each country in order to achieve the improvements to the model.


What to improve next?

We have talked about why constantly reviewing and retraining a model is necessary in an ever-changing world, especially when they are informing potentially life-changing affordability decisions. For us, success isn’t just about returning a value – but making sure you remove all confusion so you can feel 100% confident with the results.

Since beginning this quality improvement process for Income Check, we won’t stop at Sweden and Finland. Each market and every customer can expect the same approach and diligent focus on improving accuracy – because we simply wouldn’t accept anything else.

TL;DR – Quick summary

  • Tink’s Income Check allows businesses to instantly verify a person's income by analysing their transaction data.

  • For it to work well, having a high-performing classification model (with a high accuracy in machine learning terms) was fundamental.

  • Tinker Ida Janér explains how data scientists in the Risk team worked to improve the model performance, and the key learnings they took from the process.

More in Inside Tink

Inside Tink: changing our approach to outlier detection
2021-08-26 · 8 min read

Inside Tink: changing our approach to outlier detection

Learn how data scientists in Tink’s Enrichment Categorisation team improved categorisation results by changing their approach to detecting outliers. Tinker Eliisabet Hein gets into the technical details and takes us through the change.

Inside Tink
Startup stories
2020-10-22 · 5 min read

Introducing ‘Startup stories’ – and reminiscing on Tink’s own journey

The life of a fledgling startup is rarely glamorous – but it is exhilarating. To celebrate entrepreneurship, we’ll share a few startup stories over the coming months – starting with Tink’s own, as described by an OG Tinker.

Inside Tink
Tink in the DACH region
2020-05-12 · 5 min read

Taking open banking up a notch in the DACH region

Our new Regional Director for DACH has his sights set on making the financial actors in the region the stars of the open banking show - by helping them create the slick digital experiences people are beginning to expect.

Inside Tink

Get started with Tink

Contact our team to learn more about what we can help you build – or create an account to get started right away.

Contact our team to learn more about our premium solutions or create a free account to get started right away.