Skip to main content

TGIN

research paper

Jiang et. al., “Triangle Graph Interest Network for Click-through Rate Prediction”. WSDM, 2022.

Click-through rate prediction is a critical task in online advertising. Currently, many existing methods attempt to extract user potential interests from historical click behavior sequences. However, it is difficult to handle sparse user behaviors or broaden interest exploration. Recently, some researchers incorporate the item-item co-occurrence graph as an auxiliary. Due to the elusiveness of user interests, those works still fail to determine the real motivation of user click behaviors. Besides, those works are more biased towards popular or similar commodities. They lack an effective mechanism to break the diversity restrictions. In this paper, we point out two special properties of triangles in the item-item graphs for recommendation systems: Intra-triangle homophily and Inter-triangle heterophiy. Based on this, we propose a novel and effective framework named Triangle Graph Interest Network (TGIN). For each clicked item in user behavior sequences, we introduce the triangles in its neighborhood of the item-item graphs as a supplement. TGIN regards these triangles as the basic units of user interests, which provide the clues to capture the real motivation for a user clicking an item. We characterize every click behavior by aggregating the information of several interest units to alleviate the elusive motivation problem. The attention mechanism determines users' preference for different interest units. By selecting diverse and relative triangles, TGIN brings in novel and serendipitous items to expand exploration opportunities of user interests. Then, we aggregate the multi-level interests of historical behavior sequences to improve CTR prediction. Extensive experiments on both public and industrial datasets clearly verify the effectiveness of our framework.

Considering the homophily of triangles in the item-item co-occurrence graph, TGIN takes advantage of triangle structures as the basic units of user interests to model complex user behaviors and alleviates the elusive motivation problem. Instead of relying on single clicked items, TGIN can reduce the noise caused by complicated user behaviors, and capture implicit user interests effectively. It uses an efficient triangle extraction and selection mechanism. In the item-item graph, a set of relevant triangles is extracted from the neighborhood of the clicked items, which provides clues to potential user interests. It chooses a few triangles with diversity to bring in serendipitous items for users and expand exploration opportunities. It represent every user click behavior by aggregating the information of several interest units. The attention mechanism determines the user’s preference for different interest units. As the result, TGIN can break diversity restrictions. Based on this, TGIN model the historical behavior sequences including user clicks at different times to learn the user’s profile, which can better capture the trend of user interests over time.

Architecture​

The model architecture of TGIN. Firstly, we introduce diverse and related triangles from the item-item graph to represent potential user interest units. Then, user behavior representations are enriched by intra-triangle and inter-triangle aggregation. Finally, triangles of different orders help to capture multi-levels of user interests, thereby improving the performance of click-through rate prediction. [source](https://arxiv.org/pdf/2202.02698v1.pdf).

The model architecture of TGIN. Firstly, we introduce diverse and related triangles from the item-item graph to represent potential user interest units. Then, user behavior representations are enriched by intra-triangle and inter-triangle aggregation. Finally, triangles of different orders help to capture multi-levels of user interests, thereby improving the performance of click-through rate prediction. source.

Implementation​

Tensorflow​

link

https://github.com/recohut/tgin/blob/main/script/tgin.py

def tgin(ub0_triangle_node, ub0_triangle_score, cand0_triangle_node, cand0_triangle_score,
         ub1_triangle_node, ub1_triangle_score, cand1_triangle_node, cand1_triangle_score,
         pos_aware_tp_fea, user_profile, var_scp, reuse=tf.AUTO_REUSE):
    """
    Args:
        ub0_triangle_node: [B, SEQ_LENGTH, TRIANGLE_NUM*3, H]
        ub_triangle_score: [B, SEQ_LENGTH, TRIANGLE_NUM, 1]
        cand_triangle_node: [B, TRIANGLE_NUM*3, H]
        cand_triangle_score: [B, TRIANGLE_NUM, 1] 
        pos_aware_tp_fea: [B, T, H] 
    Output:
        output: [B, H] 
    """
    with tf.variable_scope(name_or_scope=var_scp, reuse=reuse):
        weight_collects = [ops.GraphKeys.GLOBAL_VARIABLES, ops.GraphKeys.MODEL_VARIABLES]
        hop0_out, ub0_triangle_agg = triangle_net(ub0_triangle_node, ub0_triangle_score, 
                                cand0_triangle_node, cand0_triangle_score, pos_aware_tp_fea, var_scp="tn0")
        hop1_out, ub1_triangle_agg = triangle_net(ub1_triangle_node, ub1_triangle_score, 
                                cand1_triangle_node, cand1_triangle_score, pos_aware_tp_fea, var_scp="tn1")
        multi_seqs = tf.stack([hop0_out, hop1_out], axis=1)
        output = fusion_unit(multi_seqs, user_profile, var_scp="fu")
    if use_negsampling:
        return output, ub0_triangle_agg, ub1_triangle_agg
    else:
        return output