Skip to content

← Research  ·  AAAI 2025

Forecasting Multi-Relational and Recursive Interactions in Temporal Networks

A temporal point process on recursive hyperedges for forecasting higher-order interactions in dynamic networks.

Read the paper

Abstract

Understanding relations arising out of interactions among entities can be very difficult, and predicting them is even more challenging. This problem has many applications in various fields, such as financial networks and e-commerce. These relations can involve much more complexities than just involving more than two entities. One such scenario is evolving recursive relations between multiple entities, and so far, this is still an open problem. This work addresses the problem of forecasting higher-order interaction events that can be multirelational and recursive. We pose the problem in the framework of representation learning of temporal hypergraphs that can capture complex relationships involving multiple entities. The proposed model, Relational Recursive Hyperedge Temporal Point Process (RRHyperTPP) uses an encoder that learns a dynamic node representation based on the historical interaction patterns and then a hyperedge link prediction-based decoder to model the occurrence of interaction events. These learned representations are then used for downstream tasks involving forecasting the type and time of interactions. The main challenge in learning from hyperedge events is that the number of possible hyperedges grows exponentially with the number of nodes in the network. This will make the computation of negative log-likelihood of the temporal point process expensive, as the calculation of survival function requires a summation over all possible hyperedges. In our work, we develop a noise contrastive estimation method to learn the parameters of our model, and we have experimentally shown that our models perform better than previous state-of-the-art methods for interaction forecasting.

Introduction

We use a multi-relational recursive hyperedges structure to incorporate these complex higher-order interactions. Here, hyperedges can act as nodes in other hyperedges (like CCed or receiver groups in email interactions) and contain relation types indicating their role in the interactions. Then, a TPP is defined with these hyperedges as event types with a conditional intensity function parameterized using representations/embeddings of the nodes and relations involved in the interactions.

Furthermore, the previous approaches in TPP-based interaction forecasting use negative sampling to approximate the loss associated with the survival function, which is intractable to calculate due to the huge number of possible event types in the TPP. This can introduce biases in training, especially in these complex interactions. To address these challenges, we propose the model Relational Recursive Hyperedge Temporal Point Process (RRHyperTPP). The following are the contributions of this work:

  • Contrastive learning strategy: We propose a contrastive learning strategy for higher-order interaction for hyperedge events in networks.

    This provides a technique for training the model without maximum likelihood estimation, thereby avoiding the computationally expensive survival function calculation. We establish that the proposed model performs better than previous models that use negative sampling.

  • New deep learning architecture: We propose a new deep learning architecture for learning node representation from higher-order interaction.

    This involves two stages: interaction update and temporal drift. Interaction update is used to revise the node representation when an event involving it occurs by using the features of the interaction. Temporal drift is used to model the evolution of node representation during the interevent period using time projection based on JODIE[1], Neural ODE[2], or time embeddings based on Fourier time features[3].

  • Hyperedge link prediction: We propose a new method for hyperedge link prediction for multi-relational recursive hypergraphs.
  • Extensive experiments: Extensive experiments are conducted to show the advantage of the proposed method over existing state-of-the-art models.
2025 Image
Figure 1: Real world events represented as Relational Recursive Hyperedges. Here, there are source and target hyperedges with three relations inside them and relation types between them to indicate the nature of the interaction.

Problem Statement

Problem. Given $E(t) = \{(e_1, t_1), \ldots, (e_n, t_n)\}$, the historical interactions until time $t > t_n$, we aim to forecast the next interaction $(t_{n+1}, e_{n+1})$, where $t_{n+1} > t > t_n$. Here, we want to estimate the time $t_{n+1}$ and the type of interaction $e_{n+1}$.

This work mainly focuses on higher-order interactions in communications networks and real-world events stored in source and target entities. The higher-order interaction in communications networks is represented using a recursive hyperedge graph of depth one:

$$ h = \{(h_i^0, r_i^0)\}_{i=0}^{k^0} $$

Here, the relations $\{r_i^0\}_{i=0}^{3}$ are used to represent the sender, recipients, and carbon-copied recipients’ addresses in the case of email exchange, and $\{r_i^0\}_{i=0}^{4}$ represent the sender, receiver, retweeter, and retweeted nodes in the Twitter network. The higher-order interactions in real-world events are represented using depth-two hyperedges:

$$ h = \{(h_0^1, r_0^1), (h_1^1, r_1^1)\}, \qquad h_{0/1}^1 = \{(h_i^0, r_i^0)\}_{i=0}^{3} $$

Here, $h_0^1$ and $h_1^1$ are the two entities involved in the interaction, $r_0^1$ and $r_1^1$ are the inverse relation pairs indicating the type of interaction, and $\{r_i^0\}_{i=0}^{3}$ are relations indicating the actor, sector, and country of the entity. Figure 1 shows an example of this type of higher-order interaction.

Relational Recursive Hyperedge Temporal Point Process (RRHyperTPP) Model

The RRHyperTPP model is a temporal point process (TPP) designed for hypergraphs. It defines event probabilities using a conditional intensity function and a survival function. The likelihood function is optimized using Noise Contrastive Estimation (NCE) instead of Maximum Likelihood Estimation (MLE).

Conditional intensity function

The probability of a hyperedge event $h$ occurring at time $t$ is given by:

$$ P_h(t) = \lambda_h(t)\, S_h(t_{p_h}, t) $$

  • $\lambda_h(t)$ is the intensity function.
  • $S_h(t_{p_h}, t)$ is the survival function: the probability that no event has occurred in $[t_{p_h}, t]$.

Likelihood function and optimization

  • The complete likelihood function considers both observed events and survival probabilities.
  • Noise Contrastive Estimation (NCE) is used instead of MLE to enhance computational efficiency.

Noise contrastive estimation (NCE)

  • Introduces noise streams to approximate event distributions.
  • Optimizes a contrastive loss function comparing real event distributions with sampled noise.

Negative sampling

  • Negative hyperedges are generated using a corruption model for better generalization.
  • Diverse negative samples are ensured by replacing parts of real hyperedges.

Dynamic node representation

  • Nodes have time-dependent embeddings $\mathbf{V}(t)$.
  • Captures dynamic node evolution via a continuous-time recurrent neural network (RNN).
  • Two stages:
    • Node update: updates embeddings when an event occurs.
    • Drift: embeddings evolve over time between events.

Training algorithm

  • Uses the AdamW optimizer for parameter updates.
  • Negative samples are incorporated into both event likelihood loss and supervised noise contrastive loss.

Experimental Results

Dataset

We evaluate our model on a real-world dataset of email interactions and Twitter retweet events. The dataset contains information about the sender, receiver, and carbon-copied recipients in the case of email interactions and sender, receiver, retweeter, and retweeted nodes in the case of Twitter retweets.

Publication

Deep Representation Learning for Forecasting Recursive and Multi-Relational Events in Temporal Networks

Tony Gracious, Ambedkar Dukkipati

The Association for the Advancement of Artificial Intelligence, AAAI 2025 [Slides]

@article{gracious2024deep,
title={Deep Representation Learning for Forecasting Recursive and Multi-Relational Events in Temporal Networks},
author={Gracious, Tony and Dukkipati, Ambedkar},
journal={arXiv preprint arXiv:2404.17943},
year={2024}
}

Dynamic Representation Learning with Temporal Point Processes for Higher-Order Interaction Forecasting

Tony Gracious, Ambedkar Dukkipati

The Association for the Advancement of Artificial Intelligence, AAAI 2023 [Slides]

@inproceedings{gracious2023dynamic,
title={Dynamic representation learning with temporal point processes for higher-order interaction forecasting},
author={Gracious, Tony and Dukkipati, Ambedkar},
booktitle={Proceedings of the AAAI conference on artificial intelligence},
volume={37},
number={6},
pages={7748--7756},
year={2023}
}

Code

Our code is available on GitHub.

Authors

  • Tony Gracious
  • Ambedkar Dukkipati

References

  1. S. Kumar, X. Zhang and J. Leskovec. Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks. KDD 2019. arXiv:1908.01207 ↩
  2. R. T. Q. Chen, Y. Rubanova, J. Bettencourt and D. Duvenaud. Neural Ordinary Differential Equations. NeurIPS 2018. arXiv:1806.07366 ↩
  3. D. Xu, C. Ruan, E. Korpeoglu, S. Kumar and K. Achan. Inductive Representation Learning on Temporal Graphs. ICLR 2020. arXiv:2002.07962 ↩