As a third party offering aggregation and payment initiation, we are required to use the banks’ PSD2 APIs from 14th September onwards. The period between 14th March and this ultimate deadline was meant to give providers like us six months to create and test connections to the banks’ new interfaces. But these sandboxes have not really allowed us to do that testing.
The purpose of the sandboxes has been to give developers a safe place to make mistakes, test connections and try out user journeys. Using dummy data, it should behave in exactly the same way as the production APIs. And the transition from sandbox-to-production should be as simple as sending a slightly different parameter to the bank.
In our analysis of these sandboxes, we have always asked ourselves two questions: is it technically possible for us to integrate and rely on these APIs? And can you build a business on top of these connections? The answer to both is “no” – not with what we’re seeing now.
Today, 14th June, is the deadline for all banks to go live with functional production APIs – if they want to qualify for the exemption from creating a fallback mechanism. To give it an analogy, it’s like the first game of the World Cup. The stadiums are filled, they’re minutes away from blowing the whistle – and no one has had a chance to practice or train.
We're not releasing our findings to bash the banks – we have found the banks to be responsive to our feedback. We are sharing our experiences with the industry as a whole because we do not want to see the mission of PSD2 and open banking get derailed.
Six criteria for a good sandbox
Our integration team has spent thousands of hours reviewing, integrating and testing more than 100 sandbox environments and production APIs across 12 markets.
Although there are a few promising sandboxes that nearly meet our expectations, we have been struggling to test our integrations with the majority of bank APIs because they are broken, missing the full customer journey and lacking the right documentation – if we were even able to get access to the sandbox at all.
At this point, we have identified a sandbox for 97% of the major European banks and requested access – a dramatic improvement since the 14th March deadline when 41% of major banks in 10 markets did not have any.
But 14% have blocked us from sufficiently testing their APIs. And only 65% have functional APIs that can fetch account information (an abysmal 37% excluding the UK).
As an API provider ourselves, we know the criteria that a quality sandbox environment needs to meet to be functional and useful. In the spirit of transparency, the following are the six criteria we used to measure the technical readiness of the APIs and their ability to support business-critical services – as well as the results from our evaluation.
Of the European banks we have tested, the number that meet all six criteria is zero.
1. Available and accessible
The first major criteria is that the testing environment is available and accessible. That sounds ridiculous to even say. But there are banks that do not meet this most basic requirement.
To give third parties enough time to test and use the APIs before September, we need access to the sandboxes. Access should be granted almost instantly through an online form, development portal or an onboarding API.
Every bank is different, but we have found enough of the following situations to be alarmed: some sandboxes appear not to exist despite communication there is one; some have been outsourced to a service provider, but that hasn’t been disclosed publicly; and some banks simply released a website last-minute to claim they had a sandbox ready.
Many access procedures add weeks if not months to an already tight timeline. Some have an online registration form, but nothing happens once you submit. Others take weeks to inform us they’re still processing our request or need more information. And some require notarised copies of our licences – a big surprise because we’ve been trying to access dummy data for testing, not real customer data for production (yet). The worst offenders have rejected us on the basis that we are a foreign third party that did not use a local provider (QTSP) for our eIDAS certificate.
Why it matters:
It appears as if some banks are using the registration process to throttle usage and testing of APIs because they aren’t ready yet. With already tight timelines, the unnecessary hurdles and admin are causing severe delays in terms of our integration efforts – and preventing us from testing APIs according to the set timelines.
The 14th March deadline to release the testing environments was also the cutoff to provide documentation – the instruction manuals third parties need to integrate with the PSD2 APIs.
At a minimum, documentation should include specifications, definitions, as well as the descriptions of things like functions, classes and return types. It should also include tools so that developers can navigate its contents – as well as tutorials, templates and concrete examples of how to execute certain commands. Best practice also includes integration guides and “walkthroughs”.
Most importantly, it needs to be understandable. This means it should be written in a common language like English, and be grammatically sound so it can be read and translated without any misinterpretations.
Most of the documentation we’ve gotten access to is pretty awful, some lacking even the basic description of the available APIs and responses. A significant number of banks in southern Europe do not have English-language documentation, and even when a bank uses its native language, the documentation is often incomplete.
A lot of this we discovered by trial and error, as only 35% of sandboxes provide documentation that’s sufficient enough for us to test.
Why it matters:
Developers will be relying on bank APIs to solve important problems, but they need documentation to do so. It’s the technical instruction manual for how to successfully use APIs. Integrating these APIs without the documentation is like trying to put together a HEMNES bed from IKEA without the instructions. Good luck.
3. Third-party provider identification
PSD2 requires that licensed third parties identify themselves in order to aggregate data or initiate payments. This is done using eIDAS certificates, the standards for electronic identification in the EU. The APIs need to allow third parties to submit these certificates and validate them.
The documentation should contain information on how third parties will be required to use eIDAS certificates in production. Some banks have chosen to require (test) eIDAS certificates in sandboxes. This is useful as long as the bank issues the test certificates or if it can ensure those issued by other providers are not unnecessarily rejected.
There are few banks with the functionality in place to validate eIDAS certificates in their sandboxes. More importantly, these banks have also not provided info on whether this process will be different when they move from a test environment to a production one – leaving third parties in the dark about whether their integration efforts will work in the real world.
Why it matters:
Among the few banks that have documented or implemented eIDAS support, we are seeing significant differences in how they do this. All of this will lead to longer integration and testing processes for third parties.
4. Authentication and user experience
This is arguably one of the most controversial and disputed topics between banks and third parties in the run-up to September. PSD2 introduces the concept of strong customer authentication (SCA) or two-factor authentication. We have no control over which methods the banks choose to use. But what has a great impact on everyone – third parties and end users alike – are the authentication flows.
A quality sandbox has defined and testable flows that accurately reflect the journey a user will take to authenticate themselves in the online or mobile environment after 14th September. This will be a two-step process, and it will vary by bank and market.
The best authentication flows are embedded directly in the third party apps or services or rely on biometrics, making the user journey seamless and similar to the one banks provide today in their online or mobile services.
A large number of bank completely fake most or all authentication steps preventing third parties from testing and experiencing the end-user journey. In other cases, authentication APIs that third parties will use to activate the flows are missing entirely or lack a defined flow. Only 57% of the banks we’ve evaluated support some authentication APIs – a number that drops to 47% when we exclude UK banks.
Among the banks that have defined their flows, nearly all of them are using the worst flow type – the so call web “redirect”. This is a long-disputed type of user journey that forces the app to launch a web browser so a user can authenticate on the bank’s online portal – all from a phone. It offers such a poor and time-consuming user experience that many drop out of the process.
Why it matters:
A lack of technical information about the real authentication journeys third parties can expect will again delay implementation and testing.
But the true issue lies in the “redirect”. It creates such unnecessary friction that doesn’t exist today, as well as phishing concerns, which will scare away many end users. It not only reflects poorly on third-party services, but it prevents them from developing innovative solutions on increasingly ubiquitous “smart” devices that lack web browsers.
In order to provide account information (AIS) or payment initiation (PIS) services, third parties need to have access in the APIs to the functionalities and data that’s available to end users in their mobile or online banking services. It could be support for initiating a full range of payment types, getting access to all payment accounts – all the way to access to end users’ information.
We would love to see all bank APIs and related documentation reflect the terms and conditions around each account or user type, as well as coverage of all account types and payments. Add to that, information that allows third parties to verify users’ identities.
The information about what account types are accessible and what payment types can be initiated is lacking in the documentation, and missing from the sandbox. Only access to production APIs will show us if banks really do give third parties access to all payment accounts, and allow us to initiate all types of payments available to users in their existing services. But what we already see today is that, contrary to the guidance of the European Banking Authority (EBA), a majority of banks are failing to provide identifying information about end users.
Why it matters:
The lack of access to end-user info is a major obstacle for third parties. It limits our ability to build user-friendly services, and it prevents us from fulfilling our obligations to perform anti-money laundering, sanctions and other checks.
For third parties to adequately test connections to the banks, the APIs need to contain ample testing data, support mock credentials and test calls, and offer overall support for a range of test cases. This allows us to test full end-to-end flows before going live.
Banks should invest in offering dummy data and test accounts with mock credentials. This will give third parties a realistic indication of how the APIs perform in a real production environment.
Many banks don’t provide test data, accounts or credentials. When they do, the sandbox often does not provide enough mock transactions to fully test certain functions.
Some banks are also providing only one sample client credential per third party. This means their APIs return the same data set for every request, limiting our ability to test a variety of scenarios before moving into production.
Why it matters:
Testing is critical before moving any code into production mode, as it helps identify issues that require debugging and may point out problems with the bank APIs.
Endangering the open banking movement
If the production APIs released today look the way we anticipate they will, it threatens to deteriorate existing services and negatively impact the experiences of millions of users. It could prevent third parties from innovating, and blunt customer adoption of new and innovative services that PSD2 is supposed to enable. And it could threaten the success of the open banking movement as a whole.
Third parties like Tink are not looking for perfection here. We can bridge the gap when bank APIs are 98% of the way to what they should look like. But we can’t do that for most European banks if they’re just 20% of the way there (these figures are an arbitrary example). It’s the scope of the unreadiness that scares us.
For companies like ours, the only option is to keep working on implementation as quickly as possible.
For banks, we highly recommend they evaluate their sandbox and production environments against the six criteria listed above – and consult with developers on the platform to understand where they can improve.