The Current Application Status of Large Models in the Financial Industry: From Strategic Heights to Practical Implementation

The emergence of ChatGPT has caused a huge response in the financial industry. Initially, this technology-focused sector felt a general anxiety, worried about being left behind by the tides of the times. This sentiment even spread to some unexpected places. It is reported that in May this year, one could hear financial practitioners discussing large models in a temple in Dali.

However, over time, this anxiety gradually subsided, and people's thinking became clearer and more rational. Sun Hongjun, the CTO of SoftCom Power Bank, described the evolution of the financial industry's attitude towards large models: widespread anxiety in February to March; teams formed to research in April to May; difficulties encountered in finding direction and implementation in the following months, leading to a more rational approach; and now the focus is on benchmarks, attempting to validate proven application scenarios.

Currently, many financial institutions have begun to strategically emphasize large models. According to incomplete statistics, at least 11 banks among A-share listed companies have clearly stated in their latest semi-annual reports that they are exploring the application of large models, including Industrial and Commercial Bank of China, Agricultural Bank of China, Bank of China, and Bank of Communications. From recent actions, these institutions are conducting clearer thinking and path planning from a strategic and top-level design perspective.

From Enthusiasm to Rational Return

Compared to a few months ago, financial clients' understanding of large models has significantly improved. At the beginning of the year, when ChatGPT first appeared, although enthusiasm was high, there was limited understanding of the essence and application methods of large models.

At this stage, some large banks have taken the lead in launching various "hype" promotions. For example, in March, a certain bank launched a ChatGPT-like large model application, but the industry's evaluation has been mixed. Some believe that this application overemphasizes the chat function and neglects the more important generative capability.

As more domestic technology companies successively release large models, the technology departments of some leading financial institutions have begun to actively discuss the construction of large models with these companies. They generally hope to develop large models independently and inquire about issues such as dataset construction, server configuration, and training methods. A financial technology company under a certain bank even expressed the hope that they could output technology to peers once completed.

After May, the situation began to change. Limited by factors such as the scarcity of computing power resources and high costs, many financial institutions shifted from simply hoping to build their own systems to focusing more on application value. Now, every financial institution is paying attention to the large model application situations and results of other institutions.

Different scales of enterprises have also emerged with two paths. Large financial institutions that possess massive financial data and application scenarios can introduce leading foundational large models, build their own enterprise large models, and simultaneously develop specialized task large models through fine-tuning to quickly empower their business. Meanwhile, small and medium-sized financial institutions can consider ROI and introduce various public cloud APIs or privatized deployment services for large models as needed to directly meet their requirements.

However, due to the high requirements for data compliance, security, and credibility in the financial industry, some industry insiders believe that the progress of large model implementation in this sector is actually slightly slower than expected at the beginning of the year. Sun Hongjun from Softcom Power stated that they initially expected the financial industry to be the first to widely use large models, but the actual situation is that the application progress in the financial sector is not as fast as in industries such as law and recruitment.

Some financial institutions have begun to seek solutions to various constraints in the implementation process of large models.

In terms of computing power, several solutions have emerged in the industry:

1. Directly building computing power is costly but offers strong security, making it suitable for large financial institutions that wish to construct industry or enterprise-level models. It is reported that a large state-owned bank recently purchased a batch of H800 chips for building computing power.

2. Mixed computing power deployment, while ensuring that sensitive data does not leak, uses public cloud large model service interfaces, and simultaneously processes local data through private deployment. This method is cost-effective and suitable for small and medium-sized financial institutions with weaker financial strength and on-demand applications.

3. In response to the GPU card shortage and high prices faced by small and medium-sized institutions, regulatory authorities are exploring the possibility of building shared large model infrastructure for the securities industry, concentrating computing power and general large model resources, allowing small and medium-sized financial institutions to also access large model services and avoid technological lag.

In addition to computing power, many financial institutions have also strengthened data governance in the past six months. An executive from a cloud service provider stated that, in addition to the major banks, an increasing number of medium-sized financial institutions are also beginning to build data middle platforms and data governance systems. He believes that a完善的 data governance system and data lake technology platform will become an important direction for the IT construction of financial institutions in the future.

Some banks are solving data problems through the combination of large models and MLOps. For example, a large bank has established a closed-loop data system using the MLOps model, achieving process automation and unified management and efficient processing of multi-source heterogeneous data, and has currently built a high-quality training dataset of 2.6TB.

Entering from the peripheral scenario

In the past six months, both large model service providers and financial institutions have been actively seeking application scenarios. Fields such as smart office, intelligent development, smart marketing, intelligent customer service, smart investment research, intelligent risk control, and demand analysis have all become key areas of exploration.

As a senior executive of a fintech company said, "Every key link in the financial business chain deserves to be redesigned using large model technology." The company recently launched a large model tailored for the financial industry and collaborated with partners to develop large model products for the financial sector, aiming to create a comprehensive AI business assistant for financial professionals such as wealth advisors, insurance agents, investment researchers, financial marketers, and insurance claim adjusters.

Financial institutions have rich ideas regarding the application of large models. A large bank claims to have deployed applications in over 20 scenarios, another bank stated that it is piloting in over 30 scenarios, and a securities company is exploring the integration of large models with virtual digital human platforms.

However, in the actual implementation process, the general consensus is to apply internally first and then promote externally. After all, the current stage of large model technology is still not mature, with issues such as hallucinations, and the financial industry is a field with strong regulation, high security, and high trust.

The technical leader of a large bank believes that it is not advisable to directly use large models for clients in the short term. Financial institutions should prioritize applying large models to intellectual-intensive scenarios such as financial text and financial image analysis, understanding, and creation, to achieve human-machine collaboration in the form of assistants and improve the work efficiency of business personnel.

Currently, the code assistant has been implemented in multiple financial institutions. For example, a certain bank has built an intelligent research and development system based on a large model, with the code generated by the coding assistant accounting for 40% of the total code volume. In the insurance sector, a certain company has developed a programming assistance plugin based on a large model, directly embedded in internal development tools.

There are also many practical cases in the field of smart office. A certain large model supplier launched a branch Q&A system based on its financial large model, which has been promoted to hundreds of branches after going live at a certain bank, with an answer acceptance rate exceeding 85%. This solution has also been quickly replicated to several other banks and financial institutions.

However, industry insiders believe that these widely implemented scenarios are not yet the core applications of financial institutions, and large models are still some distance away from deeply integrating into the business level of the financial industry.

An executive from an IT service provider stated that marketing, risk control, compliance, and other scenarios are areas where large models could bring about transformation and are also where financial clients have demands. However, currently, these tasks still depend on the improvement of the underlying large model providers' capabilities.

Industry insiders predict that before the end of this year, a batch of projects that will truly apply large models in the core business scenarios of financial institutions will emerge, along with bidding information.

Before this, some top-level design reforms were underway. Experts predict that the entire intelligent and digital system will be rebuilt on the foundation of large models in the future. This requires the financial industry to restructure its systems in the process of promoting the implementation of large models, while also not neglecting the value of traditional small models, and instead allowing large and small models to work together.

This trend has been widely reflected in the financial industry. Currently, financial institutions are piloting large models, mainly adopting a layered approach. Unlike the past where a siloed model was needed to build a platform for each scenario, large models provide financial institutions with an opportunity to start from scratch and plan the overall system more scientifically.

Currently, several leading financial institutions have built a layered system framework based on large models, which includes multiple layers such as the infrastructure layer, model layer, large model service layer, and application layer. These frameworks generally have two characteristics: first, the large model plays a central role, calling upon traditional models as skills; second, the large model layer adopts a multi-model strategy, selecting the optimal effect through internal competition.

In fact, not only financial institutions but also some large model application providers are adopting a multi-model strategy to optimize service effectiveness in the current uncertain landscape. An IT service provider revealed that their underlying model layer integrates a large number of large language models and assembles the optimal responses based on each large model's answers before delivering them to users.

The talent gap is still enormous

The application of large models has begun to pose some challenges and changes to the personnel structure of the financial industry.

A person from a fintech company stated that since the emergence of ChatGPT, his company has laid off more than 300 data analysts from the beginning of this year to the end of May. This has raised concerns for him about his future career development.

A senior financial expert from a large bank also shared the substitution effect of large models on human labor. Originally, the bank had interns summarizing various information for the investment research department every morning, but now this work can be accomplished through large models.

However, some banks do not wish for large models to lead to layoffs. For example, a large bank with 200,000 branch employees has clearly stated that they do not want employees to be replaced by large models, but rather hope that large models can bring new opportunities, improve the quality of service and work efficiency of employees, while freeing some employees to engage in higher-value work.

This considers the stability of personnel and structure on one hand, and on the other hand, it is also because there are still talent gaps in many positions. An executive from an IT service provider stated that large banks have a lot of work that cannot be completed, and some IT project timelines are even pushed to the end of next year. They hope that large models can help employees improve efficiency and speed, rather than leading to staff reductions.

Moreover, the rapid development of large models has led to a mismatch between the supply of scarce talent and the surging demand in a short period of time. This is similar to when the iPhone first came out, and it was difficult to find iOS developers to create applications.

The head of R&D at a large bank summarized the six major challenges faced by the financial industry in applying large model capabilities to core business processes, one of which is the shortage of talent. Among the new hires and graduates they recently recruited, a high proportion studied AI, but there are very few who understand large models.

An executive from an IT service provider had a similar experience recently, having just received a talent support request from a bank client. The bank faced a manpower shortage in its self-built large model team due to someone taking a temporary leave, and had to seek external support for model training work.

Currently, the demand for talent directly applying large models is relatively simple, mainly requiring individuals who can ask questions. However, if one wants to build an industry or enterprise large model, financial institutions will need a competent vertical large model technical team.

A senior executive from a cloud service provider admitted that there is a significant talent gap in the field of AI large models. Leading organizations are currently recruiting professionals related to AI, such as algorithm PhDs. This is because, while financial clients can obtain technical support from large model vendors, they are the end users and innovation leaders. They need a certain accumulation of talent to support the construction of AI large platforms, the planning of various AI applications, and to collaborate with large model vendors during the modeling, tuning, and fine-tuning processes, continually expanding the application scope and effectiveness of AI models.

Some companies have already taken action. A certain technology company has collaborated with a banking laboratory to sort out the personnel transformation practices of large models in corporate applications, designing a series of training courses such as Prompt tuning, fine-tuning, large model operation, etc., and has collaborated with multiple departments to establish joint project teams to promote the enhancement of employee capabilities.

Industry experts point out that large models are currently not mature enough and require the joint efforts of domain experts to develop mature products. The large models from major companies will bring certain enhancements to the existing traditional talent in enterprises, but will not lead to a paradigm shift. A true paradigm shift requires a team within the financial system to deeply integrate internal demands and make significant innovations.

It is worth noting that during this process, the personnel structure of financial institutions will also undergo adjustments and changes. Developers who master the skills of using large models will find it easier to establish themselves in this environment.