Analyzing Jan 2024 US Banking App Performance with AI Insights

Executive Summary

The Jan 2024 US Banking App Insights Session delved into a comprehensive analysis of the performance of various banking apps within the US market. Led by Yazad Karkaria, Global Head of AI Analytics and CX Strategy at Touchpoint Group, the session utilized Ipiphany, the AI-powered analytics platform, to provide deep insights into the digital performance of banking apps.

The focus of the session was on understanding recent login issues faced by PNC, one of the 40 financial institutions under review. Through the Ipiphany platform, Karkaria presented data based on online reviews from both the App Store and Google Play, emphasizing the importance of the engaged customer score in deciphering user experience trends. 

The ECS amalgamates quantitative ratings and qualitative feedback garnered from users through platforms such as the App Store and Google Play. By synthesizing numerical ratings with textual comments, offering rich insights into the underlying reasons driving user satisfaction or dissatisfaction. Notably, the inclusion of qualitative feedback enables a deeper understanding of user preferences, pain points, and expectations, which are instrumental in shaping product enhancements and feature development strategies.

Key findings highlighted the comparative performance of major players in the market, including US Bank, Citi, Bank of America, Chase, and PNC. While some institutions maintained consistent scores over time, others experienced fluctuations, notably PNC, which witnessed a decline in its engaged customer score from 4.1 to 3.7 within a month.

The session employed a granular approach to investigate the reasons behind PNC's decline, focusing on operating systems, specifically Android and iOS. Analysis revealed a significant drop in the engaged customer score for Android users, indicating a potential area of concern. Further examination pinpointed version 4.43 of the Android app, which exhibited a notable decline in performance from December to January.

Utilizing the pain points analysis framework, specific issues contributing to the decline, notably reliability and authentication-related concerns were identified. Instances of technical glitches and login errors surged in January, correlating with the observed decline in user satisfaction. However, encouragingly, the data also highlighted a subsequent recovery in app performance, signifying the effectiveness of corrective measures implemented by PNC.

The Ipiphany platform's real-time analytics capabilities were underscored, enabling banking institutions to promptly address user-reported issues and maintain agile product management practices. Through AI-powered frameworks, such as those utilized for survey data analysis, institutions can gain strategic insights to enhance customer experience effectively.

In conclusion, the session underscored the critical role of AI analytics in understanding and improving banking app performance. By leveraging platforms like Ipiphany, institutions can act swiftly to proactively address user concerns, mitigate operational challenges, and ultimately enhance customer satisfaction. For banking entities seeking to optimize their digital offerings, engagement with AI-driven analytics solutions presents a transformative opportunity to stay competitive in the evolving landscape of mobile banking.

For further inquiries or to explore the capabilities of the Ipiphany platform, interested parties are encouraged to reach out to the Touchpoint Group team for personalized consultations and demonstrations.

Video Transcription

Hi folks, and welcome back to another Touchpoint group insight session where we are looking at the US market for mobile app. And today we're very lucky to have joining me, Yazid Karkaria, who is, ah, our global head of AI analytics and CX strategy. And uh, he's going to be taking us through doing our deep dives. And I'm really looking to this.

Forward to the session. Now, Yazid, let's get straight into it. I know PNC was in the news recently for some login issues for their digital channels. Is it possible to understand what actually happened there? Sure. Thanks, Glenn. So yes, we are, uh, currently reviewing the banking at performance of about 40 banks or financial institutions operating in the US, and PNC is one of them.

So we are going to be using our aipowered analytics platform called Epiphany to try and understand what happened with PNC recently and why they were in the news. Now, before I jump into, uh, the presentation, I would just like to explain one main concept over here, which is the engage customer score.

So the data that we are currently analysing is based on the online reviews received on App Store and Google Play, and for customers who have given a rating along with a comment, because that comment helps us understand why they have given a particular rating and that helps us understand why the user experience is improving or declining.

So currently on my screen right now, you can see a number of, uh, key players over here in the US. So we have the US bank, we have Citi, bank of America, Chase and PNC. Now, what we are seeing on this screen is the engaged customer score over a period of twelve to 13 months.

And what we observe is a few banks like US Bank, Citi and Bank of America have been performing in a higher range and their, uh, scores have been more or less consistent over time. However, we have chase over here, which is having a score in the lower range, and PNC as well.

Now, as you raised the question for PNC, yes, we can see over here that there has been a, uh, decline in their performance. So to just make it a little easier, let's just focus on PNC over here. So we can see that in December, PNC had an engaged customer score of 4.1, which then dropped to 3.7 in January.

Thanks for that. Is it. Can we have a deeper dive and have a look at the reasons for that decline with PNC? Sure, Glenn, definitely. So let's investigate this step by step to try and understand where is it that the decline is coming from. So firstly, let's look at how both the operating systems have performed.

So here we are looking at the engaged customer score over the last six months for Android and iOS both. Now, the thing to keep in mind is bulk of the feedback coming through for PNC is coming from Android and relatively. Smaller portion is coming from iOS. Now, as we can see here, Android is the one that's dropped from 4.2 to 3.8, whereas iOS is relatively consistent.

So this gives us an area to focus on for our investigation further. So now what we may do is we may go and look at Android more specifically. So here on this screen, what we are doing is we are looking at different versions of Android which have been launched in the recent months.

So on top we have the different months, and on the left hand side I have the different version numbers. So in the month of January, version 4.43 was the most dominant version with about 618 reviews, which from the bulk of the reviews. Now, interestingly, this version was also there in December.

Now, the numbers that you're looking at in this particular, uh, cells are the engaged customer scores. So you can see that the same version had performed better in December with an engaged customer score of 4.2. But then in January, the engaged customer score has dropped to 3.9. So can we see what's happened with that drop from 4.43 down to 3.9 in January?

Yes, definitely, Glenn. So what we'll do is we'll use a number of different visualisations that we have using our aipowered platform to try and get a sense of what is actually happening behind the scenes here. So firstly, I would like to just click on this here and see what is it that people are talking about in December when the score was higher.

So the main themes coming through over here are about easy to use, user friendly, excellent service, easy app, great work, et cetera. Now, if I just move on to January, let's see what comes up. Okay, so here we can see an instant change in terms of the key themes coming up, where users are talking about the app being worse, not being able to open the app, et cetera, and a few other things are coming up.

So this just gives us a very quick sense of the key themes that customers are talking about in the month of Jan when the score actually dropped. Now, this just gives us a quick glimpse, but we can analyse this in a more structured way, Glenn, so let's do that.

Now, one more concept I would like to just explain to all our viewers is that the way we are analysing this data is through a pain points analysis, which means that we are tracking all the negative things that customers have mentioned about using the app or any complaints that they've had.

So we essentially have four pillars over here based on which we are analysing the data. That is how the app performs on reliability and stability, authentication, design and UI, and features and functionality. So here we are comparing the performance of version 4.43 in December versus January. So straight away we can see that in December the incidence of these pain points, especially.

Around reliability and authentication were much lower than what we see in January, which tells us that in December customers experienced fewer pain points around app reliability and authentication. Whereas when we move to January we can see quite a sharp increase in reliability related issues as well as authentication related issues which have jumped to by about 2.5 times.

And this sort of echoes in, into the same, uh, news that you were referring to earlier, Glenn, in terms of login related issues. Now here we are just looking at the four main pillars, but what we can do is we can deep dive into this a little more and try and see specifics over here.

So yes, technical issues have increased and app login issues have almost increased by about four x. As we can see login error, then app password login has increased and a few other pain points related to login. We are seeing uh, quite a sharp increase over here. So straight away we can see that, yes, there have been more login issues in January and this is what is coming up in the database as well.

What I can further do is I can click on this and see a few examples of what customers are actually talking about over here and straight away we can get into the details. So that has to be a big pain for customers. And is there any way that we can cheque to see if this has been resolved?

Yes, definitely. Ghana. That's a really good question because it's not unusual for banking apps to have issues. But the biggest question is how long they take to recover and resolve those issues to restore the customer, uh, experience back to what it was earlier. So in this case, now what we are going to do is let's go and now review the engaged customer score on a weekly basis.

So we are tracking the engaged customer score from week commencing 11th December right until the end of Jan. And as we can see over here, the engaged customer score was in a really, uh, higher range above four in the week commencing 11th and 18th. It's in the week commencing 25th December that it started declining.

And in the week commencing first Jan and eigth Jan we can see that in eigth Jan it's reached right at the bottom and it continued to be at that lower end even in the week commencing 15 Jan. However, we can see a recovery starting 22nd Jan and it's only improved in the week commencing 29th Jan as well.

So yes, to answer your question, Glenn, definitely there has been a recovery and the score has pretty much returned to a similar level to what it was before we encountered this issue. And to understand this better even, let's look at this by our pain points analysis over here. So here again, we are looking at this on a weekly basis, and what I would like you to only focus on mainly is this part which talks about the authentication.

Because we know that that is where the issue mainly was and here we can see that authentication as a pain point. Hardly any customers were talking about it week commencing 18th December, but gradually it has increased and it's peaked over here where it's week commencing eigth Jan and 15 Jan.

Now, incidentally, these are the same weeks where we saw that the engaged customer score was the lowest as well. But then the good news is that, yes, these issues are now starting to drop, which is again in sync with what we have seen in terms of the improving engaged customer score.

Yeah. So if I was a bank and I had access to this platform, I could almost see this happening in real time, couldn't I? Absolutely, Glenn. Indeed. Our clients have real time access to these insights and they can stay on top of any of these issues which are reported by users.

Product teams can remain agile and take corrective actions swiftly. And in fact, we use our AI powered analytics frameworks even to analyse survey data where we work very closely with client insights teams and help them at a strategic and a tactical level as oh well that is fantastic. Thank you so much, hazard.

And if you want to have a look at the epiphany platform and talk to one of our team around how we can help you with your bank or your banking app, feel free to reach out to any of us and we will set up a meeting.