Introduction
Understanding customer satisfaction and app performance is paramount for financial institutions as mobile apps increasingly conduct banking transactions. In this executive summary, we delve into insights from Touchpoint Group's webinar on banking app performance in the US market for February 2024. Led by Yazad Karkaria, global head of AI and analytics and CX strategy, we analyze key trends and developments affecting major players in the industry.
Analyzing Mobile Banking App Performance
Touchpoint Group regularly reviews the mobile banking app performance of approximately 40 banks operating in the US. By focusing on customers' reviews of Google Play and App Store, they gain valuable insights into user satisfaction and areas for improvement.
Top Performers
Among the top performers in the US market, US Bank and Citibank consistently maintain high ratings, with scores hovering around 4.75 and 4.5, respectively over the past 12 months. Bank of America, while still performing well, experienced a slight dip in February due to design-related feedback on a new app version. Chase Bank has shown significant improvement, climbing from under 3.5 to around the 4 mark over the past year. However, the focus of this analysis is on Capital One, which saw a notable decline in performance in February 2024.
Understanding Capital One's Decline
Capital One's app performance declined from a rating of 4.1 in December and January to 3.8 in February 2024. Using AI-powered analytics, Touchpoint Group identified four foundational pillars crucial for app success: Authentication, Design and UI, App Reliability, and Features and Functionality.
Identifying Issues
The decline in Capital One's performance was primarily attributed to reliability and stability issues, with 17% of users reporting such concerns in February, compared to 10% in January. Authentication-related issues also doubled during this period, indicating significant challenges with the app's functionality.
Digging Deeper into App Versions
Analyzing app version releases over the past few months revealed crucial insights. Notably, rapid releases of versions 6.0.20, 6.1.17, and 6.2.7 in January, led to increased reliability and authentication-related issues. Users reported issues such as the app opening automatically upon phone unlock, slow biometrics, and login failures.
Access to Real-Time Insights
Touchpoint Group's Ipiphany AI analytics platform offers real-time insights to banking clients, enabling proactive monitoring of app performance and rapid response to user feedback. This access empowers institutions to swiftly address issues and enhance the user experience.
Conclusion
In conclusion, the analysis of mobile banking app performance in the US market highlights the critical role of user feedback and data analytics in driving continuous improvement. By leveraging AI-driven insights and real-time monitoring, financial institutions can effectively address user concerns, enhance app functionality, and maintain a competitive edge in the digital banking landscape.
Video Transcript
00:05 - Glenn Marvin: Welcome everybody to yet another Touchpoint Group Banking App Insights Session. This month we are looking at the US market for February 2024. I have with me, as usual now, Yazad Karkaria our global head of AI and Analytics and CX Strategy, and boy, we're already a good way through this first quarter of 2024. What are we looking at in the US market today, Yazad?
00:36 - Yazad Karkaria: Thanks, Glenn. Yes. So, on an ongoing basis, Glenn, we are reviewing the mobile banking app performance of about 40 banks or financial institutes operating in the US, and some of the key players are on our screen over here. So, what we exactly do is we are looking at the Google Play and App Store reviews of these customers of these banks who have used their banking app and left a rating over there. So, for the purpose of this analysis, we focus only on those customers who have left a rating and have explained why they have given this particular rating. This helps us understand why certain customers are happy with certain banks and why are some customers unhappy with others.
Today on my screen, I'll just take a quick walk through the five banks that we have over here before we dive deeper. We have US Bank over here, which is one of the top performers, and I must say it's top consistent performer as we can see, the rating has been quite high up there in the range of about 4.75 throughout the last twelve months. Then, we have Citibank, who's also one of the top performers over here. As we can see, their rating is around the 4.5 mark, as we can see, and in the month of Feb we can see that they have had a slight recovery after a marginal drop in January. So, in January we can see that their score dropped from 4.5 to 4.4 and they have again picked it up and it's at the same level as it was in December.
Next, we are looking at Bank of America over here, and their performance is 3rd out of the current set of banks that we have, around the range of about 4.25, as we can see. They've had a marginal drop over here, as we can see. They were at 4.4 in Jan and now they are at 4.2. This is mainly due to the new version that they've launched. Some of the design related feedback coming from customers.
Then, we also have Chase over here, which is one of the interesting banks, as we can see. About twelve months back, they started off in quite a lower range compared to others, just under 3.5. They were there until the first half of the year, and then they've had some amount of positive movement as we can see, they've moved up around the 4 mark and then, they are consistently in the range of about 3.5 to 4.
But today we'll be focusing on Capital One over here. Now, Capital One is interesting. Their performance gradually dropped after June, as we can see, and then it started picking up around December, November, December. And then, since the last couple of months, we are seeing a drop in their performance over here as well. So, let me just hide the other banks for a moment and just focus on Capital One. Here we can see this particular area. In December, they were at 4.1. In Jan, they were also at 4.1, but they have dropped to 3.8 in Feb 2024, and this is the key area that we'll be looking at today.
03:33 - Glenn: Well, that is quite a significant drop there, Yazad. Why don't you take us beyond the numbers and show us why the user experience of Capital One has declined so drastically.
03:44 - Yazad: Sure, Glenn. And we'll be using our AI powered analytics platform Ipiphany to try and understand what exactly happened with Capital One in this period. And before we get into that, I'll just take a quick moment to explain to our viewers the framework that we would be using to understand this data.
Based on our extensive research and by analysing over a million of user reviews, we have arrived at this framework that consists of four main pillars, that is Authentication, Design and UI, App Reliability and Features and Functionality and for any banking app to perform well or for users to have a good user experience, the app must do well on these four Foundational Pillars. So, we will be using this framework to now understand what happened with Capital One in the last few months.
Now on my screen here, on top we can see how the score for Capital One has moved over the last six months. The drop that we would be investigating here is the drop in Feb. As we can see, it's dropped by 0.3 and on the left hand side over here, you have the four Foundational Pillars that we just saw in the previous visualisation. Here you can see them live 'in the flesh', as we can see here. So here, what do these numbers tell us?
Reliability and Stability says 17% in the month of Feb. We are looking at the Feb results so, what this is telling me is that 17% of Capital One's app users have faced Reliability and Stability related issues. And this is 7.5% more than what it was in the month of Jan, which is quite significant. So, in Jan this number was around 10%. Only 10% users had faced Reliability and Stability related issues, which has now increased to 17% as we can see over here. Now, within each of these Foundational Pillars we have a lot of different subtopics as well to give us that granular view which is required for our banking partners to take the necessary steps or the necessary actions over here to make sure that the app is performing well. So, within Reliability we can see that Technical Issues have increased quite sharply, and also there is some amount of increase in the Authentication related issues as well, about a 2% increase compared to what it was in the month of Jan.
06:10 - Glenn: Well that's very interesting, Yazad. Do we know why these issues have increased?
06:16 - Yazad: Yeah, sure Glenn. Let's do one thing. First, let's take a look at what's been happening with the app in the recent times. We look at the version release cycle for the last two to three months and see what's happening there.
On this particular visualisation, you can see the different weeks across December, January and February and here the different colours represent different app versions which were active and dominant at different time periods. Here, to start off with, we can see that this particular version was the dominant one towards the end of November and beginning of December, which was then replaced by this particular version which remained dominant through most of December and even most of January. Subsequently, the interesting thing is then we had a series of quick versions that got released. Version 6.0.20 got released around mid Jan and then within a few weeks it was pulled back and a new version was released which is 6.1.17, and then, just in a few weeks we had 6.2.7 which was released. So, the interesting thing is that in the month towards the end of Jan, there were few quick releases that we saw over here.
Now, what we are going to do is we are going to understand how these versions have performed across the four Foundational Pillars that we have looked at earlier. So here, each colour over here represents an app version. So here, you have 5.120.35, which was the dominant version before the 6 series was released, and this version was doing relatively well as we can see, because this version has an Engaged Customer Score of 4.2, after which we had the 6 series that started. So, 6.0.2 had an Engaged Customer Score of 4. This is marginally less than the previous version. However, the next version which was launched even had a lower engagement, Customer Engagement Score than the previous one, until 6.2.7 which was launched where we are seeing a partial recovery over here.
Now, the interesting thing and the important thing to understand over here is, we can understand this decline in user experience by looking at our Foundational Pillars. So, when I look at the first version, which was doing well, here I can see that my pain points across the Four Foundational Pillars are relatively lesser compared to others. Then, when version 6.0.20 was released, we can see that there is an increase over here in App Reliability related issues. It's increased from 9.6 to 12.9, and Interestingly, Authentication related issues have doubled over here. As we can see.
The next version, which was released, 6.1.17 over here, we can see that Reliability related issues have further increased from 12.9 to 15.6. And Glenn, over here what we can do is we can actually get into the details of this and go beyond the numbers which are visible on our screen over here. So, what I'll quickly do is I'll click on Reliability related issues for this particular version that we have, and I want to get a deeper understanding of what's going on over here. Straight away, I can see at a very high level what's happening. So, here I can see people are talking about 'open phone', 'opening the app', 'the app not working', etc.
Let's just take a little deeper look in this. Okay, we have something very interesting going on over here. 'Normally, I love this app, but have been having a problem with it opening by itself. Every time I unlock my phone. This is becoming super inconvenient'. 'There's a bug. It starts every time I unlock my phone. It's annoying. Please fix it'. So, there's a very clear issue, which we are seeing over here, Glenn, with this particular version of the app, which has resulted in an increase in Reliability and Stability related issues, and imagine you having to face something like this. It's evident that the score has dropped over here and further, we can see that Reliability related issues have further increased with the newer version as well. And here are a few examples of what's happening on that front. 'The biometrics is not working'. 'It's slow', 'Not allow to operate, not able to log in', which, again, lines up with the increase in Authentication related issues that we are seeing with this version, which we are currently analysing.
So, the numbers that we see over here give us a clear direction in terms of what are the areas where the changes are taking place and then, this is further explained using our advanced text analytics and getting to the root cause, which is looking at the comments straight away itself. So, we have a very clear story in the sense why the user experience has declined over the last few versions of the app.
11:13 - Glenn: Yeah, and just confirming that our clients in the banking sector, if they're using Ipiphany, would have direct access to all of this sort of information that we can provide.
11:26 - Yazad: Absolutely. The clients we work with have access to this information real time. Right now, we are looking at it retrospectively, but they can actually see it happening. So, we also specialise in Version Transition, Consulting and Analysis for our clients, where we work very closely with their product and insights team and help them monitor the performance of the new version on a real time basis, which then helps them to fix any things that need attention really quickly. But unfortunately, quite a few banks are even doing this manually at this stage, that is looking at public reviews data and it does pose many challenges for them and the most important thing is that it slows down the entire process of converting data into insights and then insights into actions. Our clients are really lucky in that sense that they get access to this real time information to make the necessary changes instantly.
12:16 - Glenn: Yeah, that's fantastic insight Yazad and I think especially true for if any banks are planning on a major release, then, this could be a project type engagement. but just as readily we have clients that religiously utilise this technology to constantly monitor performance and that user experience feedback so that they can consistently make those improvements over time. So, thank you yet again. It's amazing what happens even in such a short dive into the data. Imagine if you had access to this all of the time. Feel free to reach out and we will speak to you when you do.