Latent Variable Models for Integrated Analysis of Credit and Point Usage History Data on Rewards Credit Card System

Recently, credit cards with point rewards functions (rewards credit cards) are widely used. Credit card companies can collect the users’ usage log data of various stores in multiple industries. The purposes of possessing a credit card varies depending on each user such as to use only the credit function, to use both the credit and point rewards functions, etc. Moreover, credit cards can be used in various situations in users’ lives, and the purchase history of each user is diverse. Focusing on the diversity of both card possessing purposes and purchasing behavior for each user, we propose two latent class models representing these diversities in this research.


Introduce the Problem
In recent years, credit card systems have been introduced, and various credit cards are now being widely used in many countries (Japanese Ministry of Economy, Trade & Industory;2018). With the spread of credit card systems, users are able to purchase items at retail stores, pay utility fees, and so on by simply possessing one card. Also, it is possible for credit card companies to collect purchase history data of various kinds of stores and industries at the same time. This means that the purchase history data of a credit card company contains richer information of users' purchasing behavior than the purchase history data accumulated in a retail store. Therefore, there is a demand for stimulating a user's consumption activities by analyzing the large-scale data accumulated in the credit card companies and utilizing it for proposing marketing measures.
However, compared to other foreign countries, in Japan, users prefer to pay in cash despite the fact that many customers possess credit cards. As a result, there is a low utilization rate of credit cards in Japan (Fuyumoto, 2018). However, if the differences between the preferences of users and the differences between the usage scenarios of users are clarified for a credit card, it may be a good start for solving the problem. Therefore, there is a demand to derive marketing measures that contribute to the increase in the number of users who use the card, by analyzing the large-scale data collected by the credit card companies, and utilizing these results.
They can be used as a credit card at all stores, and in the affiliated stores, they can also be used for point rewards. However, there are no studies which focus on the credit cards with point rewards functions. Moreover, such rewards credit card companies are motivated to increase both users who positively use the credit function and users who positively use the point rewards function at the affiliated stores. Moreover, it is obvious that there are no studies which focus on rewards credit card usage history data considering these two viewpoints simultaneously.

State Hypotheses and Their Correspondence to Research Design
In recent years, rewards credit cards issued by department stores, shopping malls, and EC sites represented by Rakuten cards (Rakuten Card , Lumine shopping cards (Lumine Co., Ltd. 2019), and credit card companies are mainstream (Proc. Japan Marketing Research Association; 2017). They can be used as a credit card at all stores, and in the affiliated stores, they can also be used for point rewards. However, there are no studies which focus on the credit cards with point rewards functions. Moreover, such rewards credit card companies are motivated to increase both users who positively use the credit function and users who positively use the point rewards function at the affiliated stores. Moreover, it is obvious that there are no studies which focus on rewards credit card usage history data considering these two viewpoints simultaneously. Here, the reward credit usage history data is characterized not only by diversity regarding purchasing tendency but also in terms of the usage and possessing purpose of the card, where there are differences between each user. Therefore, when considering measures that contribute to an increase in card users, it is necessary to model the diversity of the purchasing tendencies and the purpose of possessing the cards which exist in the background. From these reasons, in this research, we propose new analysis methods based on the latent class model (Goto & Kobayashi, 2014;Bistore, 2013;Hoffman, 1999) for user behavior analysis, that can effectively model rewards credit cards usage history data (credit history data and point rewards usage data), which consists of groups of different statistical characteristics. In this research, users' card usage patterns and purchasing patterns are extremely diverse. The proposed models make it possible to model complex latent structures of the data, such as the card possession purpose for each user and the preferences of the types of stores that are used by each user. In other words, detailed user segmentation is realized for such data in which users with completely different preferences are mixed.
Finally, we apply the proposed model to the actual data stored on the Odakyu Point Cards system (Odakyu Electric Railway Co., Ltd., 2019a) provided by Odakyu Electric Railway Co., Ltd. (Odakyu Electric Railway Co., Ltd., 2019b) in Japan and suggest the usefulness of the proposed model. As a result, we will clarify the difference between users who use rewards credit cards frequently and users who do not use rewards credit cards frequently and consider the marketing measures to increase the number of rewards credit card users.

Problem Setting
First, the target data in this research is the usage history data of the Odakyu Point (OP) card provided by Odakyu Electric Railway Co., Ltd. The OP card is a rewards credit card that has both a credit function and a point rewards function. The credit function can be used at all stores where credit cards are usable regardless of the Odakyu affiliated stores or other stores. In addition, the point rewards function is a service that can be used in affiliated member stores of the OP card, which include various industries and stores. By using the point rewards function, users can be given points by presenting the card even in the case of purchasing with cash. In addition, the collected points can be used for payments (reduced) by points instead of cash during the next time purchase. To summarize, the target data includes the records with three types of payment, which are credit cards, cash and points.
Based on the background so far, we describe two important features of the target data. First, because the target data is purchase history data collected by the credit card management company, unlike purchase history data collected by ordinary retail stores, the data consists of purchase history across industries and stores. Second, the user can use the OP card not only as a credit card but also as a point rewards card. As a result, the user can possess a card only to use the credit function or only to use the point rewards function. From these points and the basic analysis (stated in appendix), in the latent structure of the rewards credit card data, various types of users exist in various ways, such as different users having different card usage and usage situation.

Latent Class Model
In this research, we propose a method for modeling the target data with latent class models (Goto & Kobayashi, 2014;Bistore, 2013;Hoffman, 1999). The latent class model is a class of stochastically modeling the underlying structure of data assuming that hidden discrete variables exist behind observed variables. And it is widely applied to document classification (Mei & Zhai, 2006), analysis of purchasing history data (Ishigaki, et al., 2011), recommendation system (Adalbjornsson, 2016;Hoffman & Puzicha, 2016) and so on. In the latent class model, it is thought that each observed data always belongs to at least one latent class, and the posterior probability of each data belonging to a latent class is calculated. In other words, the latent class model has a characteristic of which each data belongs not only to a single latent class, but belongs to a plurality of latent classes. This allows the consideration of heterogeneity among observation variables. That is, by using the latent class model, it is possible to probabilistically model the data with a complicated structure. During the training of two models, we estimate the probability of observation variables appearing from each latent class using latent variables (unobserved data), so we use the EM algorithm (Dempster, et al.;1999, McLachlan & Krishnan, 2007Miyagawa, 1987;Ooi, et al., 2015) for parameter estimation.
The rewards credit cards usage history data targeted in this research consists of extremely diverse credit card usage patterns and purchasing patterns of users. Therefore, applying the latent class model to the target data, in which groups with different statistical features contexist should be effective. Thus, in this research, we propose new latent class models for the behavior analysis of users using rewards credit cards. The proposed models make it possible to model complex latent structures of the data, such as the purpose of possession of credit cards for each user and the preference of the store types that are frequently used by each user. In other words, detailed user segmentation is realized for such data in which users with completely different choice of stores are mixed. As a result, the proposed method expresses the diversity of the preferences of OP card users in a probabilistic manner, quantitatively expressing the use purpose of credit cardsand purchasing preferences, thereby supporting the planning of marketing measures to increase the number of users that actually use their credit cards.

Sampling Procedures
Data to be covered in this research has a complicated latent structure compared with purchase history data accumulated in general retail stores. Specifically, we consider user diversity with the following two aspects.
1. diversity in card possession purpose 2. diversity in the preference of stores Therefore, in this research, focusing on these points, we propose two different models that extend latent class models; (1) card possession purpose analysis and (2) purchase store selection analysis that can realize user level analysis. Then theresults obtained from these two models are analyzed together and considered. By combining the analysis results of the two different models, it is possible to find a group of users that are similar in terms of both card possession purpose and the purchasing behavior. This means it becomes possible to express the diversity of users existing in extremely complicated latent structures of the data in more detail. With this method, it is possible to realize detailed user segmentation compared to a method using a simple model. It is also possible to come up with measures customized for each user to improve user satisfaction and stimulate the usage of credit cards.

Card Possession Purpose Analysis Model
Odakyu Point card is given both functions as a credit card and as a point rewards card. Therefore, it is conceivable that the user possesses this card for various purposes. In this research, in order to express them, we propose a latent class model that assumes latent classes between the 13 variables as follows. Here, the "degree of penetration" is defined as the number of industries where items were purchased out of the total 24 industries. And we define the latent class set of the card possession purpose analysis model with the size as = { ∶ 1 ≤ ≤ } and the user set with the size as = { ∶ 1 ≤ ≤ }. Moreover, and are superscripts that signifies credit cards and point cards; onus signifies an affiliated store, and others signifies other stores. The subscript i signifies a user . We propose a latent class model that expresses the details of the card possession purposes of each user, assuming a latent class between the above variables. For each variable, an independent gauss distribution is assumed. At this time, the probabilistic model concerning the user ui is expressed by the following expression (1).
Then, each parameter that maximizes the log-likelihood of the probability model for the entire data is estimated by the EM algorithm.

Purchasing Store Analysis Model
In the purchase history data collected in the credit card management company covered by this research, the user's purchase history data consists of data from various stores and industries. In other words, the purchasing behaviors of users in diverse industries and stores are recorded, and the diversity of purchasing behaviors coexists strongly. Therefore, in this research, we propose another model that expresses the purchasing behavior of each user, by assuming a latent class between each store included in the purchasing history data. Here, the latent class set of the purchasing store analysis model with the size is = { ∶ 1 ≤ ≤ }. In addition, is the -th store out of all stores. We define the variable that takes = 1 when the user ui purchased at the store s and otherwise takes = 0. = 1 , ⋯ , , ⋯ , is a vector and each factor in this vector represents whether the user ui purchased items in each store or not.
In this model, we assume independent binomial distributions for all J shops included in the purchase history data. If the model is designed to analyze the data on a per purchase basis instead of a per store basis, the model will be biasedtowards users that purchased items more frequently. Therefore, in this study, the proposed model will use shop vectors as input data. By focusing the analysis on a per user basis, it becomes possible to prevent such a problem. At this time, the conditional probabilistic model of the purchase store of user ui is defined by the following expression (2).
Here, ( | ) is the conditional probability of purchasing at store given the latent class , ( ̅ | ) is the conditional probability that no purchase will be made at store given . At this time, ( | ) ( ̅ | ) = 1. Then, each parameter that maximizes the log-likelihood of the probability model for the entire data is estimated by the EM algorithm.

Cross Analysis
In this research, by combining the results obtained from both the card possession purpose analysis model and the purchasing store analysis model and by analyzing, it is possible to flexibly express the diversity of users existing in various aspects as a model. Moreover, we propose a method to acquire knowledge that directly leads to marketing measures. Therefore, in this study, the results obtained from proposal two latent class models are analyzed using a cross analysis table (Liu, et al., 2018). In this analysis, the number of users that belong to each segment is calculated by the following equation.
In this case, represents the number of users that belong to the segment . Then, we create a cross table with this . By using this table, it becomes possible to realize the segmentation of users by user characteristics from multiple aspects and to analyze detailed user characteristics. As a result, it becomes possible to consider various marketing measures that lead to the increase in the number of credit card users and to improve user satisfaction.

Data Analysis with Actual Data
By analyzing actual data using the proposed models, we verify the usefulness of the proposed method and observe the obtained results. The target purchase history data was collected by Odakyu Electric Railway Co., Ltd. of users who have the Odakyu Point card. The target period of the data is April 1, 2017 to March 31, 2018. The sample size of purchase records is 64,195,995, the total number of users is 591,409, and the total number of stores is 409,594 (1,295 aliated stores). Both of the latent class sizes (card possession purpose analysis model) and L (purchasing store analysis model) are set to 8 from the viewpoint of AIC, BIC, and ease of interpretation.

Estimation Results of Parameters Obtained from Each Proposal Model
The occurrence probability of each class and the average of each parameter obtained from the card possession purpose analysis model are as Table 2. From this result, for example, like in 1 , , , even among users belonging to a class with similar credit card usage frequencies at a liated stores, clear differences were found in the frequency of credit card usage other than the aliated stores and each point rewards card related variable.
In other words, it became possible to clarify the differences of credit card usage purposes, card possession purposes, the degree of close contact with the daily life of users, consciousness towards the point rewards, etc. Furthermore, the occurrence probability of each latent class obtained from the purchasing store analysis model and stores with the highest affiliation probability to each class are shown in table 3.
From this result, for example, the users that belong to v2 do not use specific areas frequently. On the other hand, users that belong to v6 tend to use OP cards in stores located in Ebina, such as Vinawalk. In this way, the proposed model makes it is possible to clarify the differences of the choice of shop for each user.

Cross Analysis Result and Consideration
In table 4, we show the result of the ratio of the number of users that belong to each segment , which calculated based on from the cross analysis. Here, the card possession purpose analysis model is shown as "model 1" and the purchasing store analysis model is shown as "model 2" due to space limitations. In table 4, each row represents a user group that belongs to each class analyzed by the card possession purpose analysis model, and each column represents a user group that belongs to each class analyzed by the purchasing store analysis model. And from the results obtained from the purchasing store analysis model, 1 and consists of a group of users who do not use specific areas frequently. Also, − consists of a group of users who use specific areas frequently.
Based on the above and from table 4, in 1 and , the proportion of users included in 1 and , which are users who do not use OP cards frequently is relatively high. On the other hand, in (Seijo, Kyodo), (Shitigurigaoka), (Sagami Ono), (Atsugi) and (Sobudai, Sagamiono), the proportion of 1 and is relatively low. This suggests the importance of getting each user to use a specific area frequently. In the field of marketing, the concept of area marketing is well known. The result of this analysis suggest that they should promote area marketing.
In addition, the user group , which mainly uses Ebina is higher in proportion of 1 and than other user groups that use a specific area frequently. In other words, the Odakyu affiliated stores in Ebina tend to have a smaller percentage of active users (users who use OP cards at a high frequency) compared to stores in other areas. These results suggest that there is more space to increase the number of active users considering the current situation and that it is necessary to conduct measures that will encourage the usage of cards.

Interpretation of Analysis Results and Suggesting Marketing Measures
In this section, we consider the measures to increase the number of users and user satisfaction by analyzing the results obtained from the proposed method. In the table 4, users who use the similar shop and card possession purpose belong to the same cell. During the analysis, approaches to solve problems that exist in each segment, and measures to shift each segment closer to the ideal user group should be considered.
For example, the group of users belonging to the 1 segment have low OP card usage at aliated stores throughout the year, and there is a possibility of them becoming a defector. Therefore, we consider measures to grow this segment to an ideal user group. From the table 2, the 1 segment has fewer OP card usage counts at aliates, but there more credit functions usage counts at non-aliated stores. This means that this segment is a group of users that use OP cards to some extent. In addition, from the table 3, the 1 segment is a group of users who frequently use Odakyu aliated stores in Ebina in Japan.
On the other hand, compared to the 1 segment, the segment is a user group where the frequency and the amount of payments using the credit function in non-aliated stores are almost the same. However, it is a user groupwith more frequent usage and larger amounts of payments using the credit function at aliated stores. Moreover, the 1 and the ( segments are user groups that both use stores near Ebina frequently. For these reasons, we assume that the ( ) segment is the ideal user group for the 1 segment. Therefore, for example, measures such as recommending the Vinawalk stores where the ( ) segment often uses, and the 1 segment rarely uses, to the 1 segment, may help promote the usage of credit cards at aliated stores. By such measures, it can be possible to increase the expected number of times the OP cards are used.

Discussion
In this study, with the card possession purpose analysis model, it is possible to analyze users possessing rewards credit cards from the 13 variables in Section 3.2, and to clarify the card usage purpose and possession purpose of each user. As a result, it is possible to clarify the dierences in the amount of money being used, the degree of close contact with the daily life of users, and consciousness towards point rewards, even among users whose card usage frequencies are similar. In addition, it can support the planning of more elective measures for each user. However, other elements that can be analyzed, such as demographic attributes or the number of usage in a specific industry, are also conceivable. Therefore, depending on the needs and objectives of the card management company, we think that it can be applied to a wide range of data by setting co-occurring elements and distributions. Furthermore, in this research, we simply assumed that users with frequent annual number of OP card usage were the active users, and marked them as ideal users.
However, it is possible to customize in detail what kind of user is judged as active, based on the needs and objectives ofthe card management company and the obtained results. As a result, it becomes possible to think of more useful measures, and the effect of the proposed method can improve. The purchasing store analysis model made it possible to clarify the difference in purchasing tendency for each user. At this time, we focused on whether or not the user purchased at each store, and a binomial distribution was assumed for each store. However, for example, in the case of focusing on the number of purchases at each store, it is necessary to assume other distributions. Therefore, the practicality and versatility of the purchasing store analysis model can also be improved further by examining the structure of the latent class model according to the target data and purpose.
By partially differentiating expression (10) and setting it to 0, the estimation expression of each parameter is expressed by the following expression (11)-(13). However, for each of the 13 variables that are co-occurring elements in the proposed model, each parameter is estimated by an expression in which the variable A in the expression (12)-(13) is replaced by each variable. It is expressed due to the space limitation.
# 13 Appendix C Derivation of update formula of EM algorithm -purchasing store analysis model-Since the proposed model contains = { ∶ 1 ≤ ≤ } which can not be observed, parameters are estimated by the EM algorithm. The log-likelihood in the proposed model is defined below. However, in the proposed model, independent binomial distributions are assumed for each element of the shop vector. The parameter is estimated so as to maximize this log-likelihood ℎ . In the E-step, the posterior probability of the latent class under each parameter , is calculated as follows.
Next, in M-step, the value of each parameter when = calculated in E-step is fixed is obtained. In order to derive the updating expression of each parameter that maximizes the log-likelihood, the expression (14) is expanded as follows.
The expansion from the first term to the second term of the right side of the expression (16) depends on Jensen's inequality. Then expression (17) is maximized.
In order to maximize the expression (17), Lagrange's undetermined multiplier method is used. Let, be the undetermined multiplier and define the Lagrangian function as follows.
(1 − ( | ) ( ̅ | ) 1 ) # 18 By partially differentiating expression (18) and setting it to 0, the estimation expression of each parameter is expressed by the following expression (19) In this way, each updating expression of the EM algorithm is derived. Then, each step is repeated until the log-likelihood expressed by the expression (14) converges.

Copyrights
Copyright for this article is retained by the author(s), with first publication rights granted to the journal.
This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).