WP2: Graph-Based Methodology

Work Package 2: Graph-Based Methodology Development

Lead: Yiting Liu (University of Twente & BFH) Duration: Months 3-9 Status: Completed

Research Context

Graph Neural Networks (GNNs) have emerged as a powerful paradigm for learning on structured data, achieving state-of-the-art results across diverse domains including social networks, molecular chemistry, and recommendation systems. This work package develops novel GNN methodologies specifically designed for credit risk assessment, addressing the unique challenges of constructing meaningful graphs from tabular financial data.

Theoretical Foundations

The application of GNNs to credit risk builds upon two foundational bodies of literature:

Message Passing Neural Networks: The theoretical framework of message passing (Gilmer et al., 2017) provides the basis for GNN architectures. Nodes iteratively update their representations by aggregating information from neighbors:

Homophily in Networks: The concept of homophily, first formalized by Lazarsfeld and Merton (1954) and extensively studied by McPherson et al. (2001), posits that similarity breeds connection. In credit contexts, this manifests as borrowers with similar characteristics exhibiting correlated default behavior, a pattern we exploit for graph construction.

Objectives

1. Develop novel graph neural network architectures for credit risk
2. Design homophily-guided graph construction methodology
3. Create interpretable credit scoring models suitable for regulatory requirements
4. Benchmark comprehensively against traditional machine learning approaches

Literature Review: GNNs in Finance

Evolution of Graph-Based Methods

The application of graph methods to financial problems has evolved substantially:

Credit Risk Applications

Recent applications of GNNs to credit risk include:

• Fraud Detection: Weber et al. (2019) applied GNNs to transaction networks for anti-money laundering
• Corporate Credit: Cheng et al. (2020) used supply chain networks for SME credit assessment
• P2P Lending: Ma et al. (2021) incorporated social networks in lending platforms

However, these approaches assume pre-existing network structures. Our methodology addresses the fundamental challenge of graph construction from tabular loan data.

Core Innovation: Homophily-Guided Graph Construction

The Graph Construction Problem

Traditional GNN applications benefit from natural graph structures (social networks, molecular bonds). Credit risk data, however, consists primarily of tabular features without inherent relational structure. This creates a fundamental methodological challenge:

How do we construct meaningful graphs from tabular loan data?

Homophily-Guided Approach

Our methodology exploits the homophily principle: borrowers with similar default behavior tend to share observable characteristics. The construction process:

Step 1: Feature Similarity Computation

For each pair of borrowers $(i, j)$, compute multi-metric similarity:

where $\omega_k$ are learnable or cross-validated weights.

Step 2: Homophily-Guided Edge Filtering

During training, edges are retained based on label consistency:

This ensures the constructed graph exhibits high homophily, enabling effective message passing.

Step 3: Inference Graph Construction

For prediction on new borrowers, edges connect to similar training nodes:

The model aggregates information from similar historical borrowers with known outcomes.

Theoretical Justification

The homophily-guided approach is theoretically grounded in:

1. Label Propagation Theory: High-homophily graphs enable effective semi-supervised learning (Zhu et al., 2003)
2. Smoothness Assumption: Connected nodes should have similar labels, which we enforce by construction
3. Information Aggregation: Neighbors provide relevant context when they share similar characteristics and outcomes

GNN Architecture Design

Graph Attention Networks (GAT)

We adopt Graph Attention Networks as the primary architecture due to their interpretability advantages:

Multi-Head Attention Mechanism:

Node Update:

Architecture Components

Complete Architecture

Input: Feature matrix X (n x d), Adjacency matrix A (n x n)
       |
   [Embedding Layer]
   - Continuous: Linear(d_cont, 64) + BatchNorm + ReLU
   - Categorical: Embedding(vocab, 64) + Dropout(0.1)
   - Concatenation: (n x 128)
       |
   [GAT Layer 1]
   - Multi-head attention: 8 heads x 16 dims = 128 dims
   - Activation: ELU
   - Dropout: 0.5
       |
   [GAT Layer 2]
   - Multi-head attention: 8 heads x 16 dims = 128 dims
   - Skip connection from Layer 1
   - Activation: ELU
       |
   [Readout MLP]
   - Linear(128, 64) + ReLU + Dropout(0.3)
   - Linear(64, 1) + Sigmoid
       |
Output: Default probability p (n x 1)

Training Methodology

Loss Function

We employ weighted binary cross-entropy to handle class imbalance:

where $w_1 = \frac{N}{2 \cdot N_1}$ and $w_0 = \frac{N}{2 \cdot N_0}$ balance positive and negative classes.

Optimization

Hyperparameter Optimization

Key hyperparameters optimized via grid search with 5-fold cross-validation:

Experimental Results

Primary Performance Comparison

Comprehensive benchmarking against 12 baseline methods across 5 datasets:

Statistical Significance

Paired t-tests comparing Homophily-GAT to best baseline (TabNet) across 5 datasets:

Ablation Study

Component-wise contribution analysis:

Interpretability Analysis

Attention Weight Interpretation

The attention mechanism provides interpretable credit decisions at multiple levels:

Feature-Level: Attention weights reveal which borrower characteristics the model prioritizes:

Neighbor-Level: For each prediction, the model identifies which similar borrowers influenced the decision:

Example: Borrower #12345 (Predicted: High Risk, p=0.73)
Top Influential Neighbors:
  - Neighbor #8891: Similarity=0.89, Default=Yes, Attention=0.15
  - Neighbor #2234: Similarity=0.85, Default=Yes, Attention=0.12
  - Neighbor #5567: Similarity=0.82, Default=No,  Attention=0.08

Regulatory Compliance

The interpretability features support regulatory requirements:

1. GDPR Right to Explanation: Attention weights provide human-readable decision factors
2. Fair Lending Compliance: Protected attributes can be excluded while monitoring indirect effects
3. Model Documentation: Architecture and training process fully documented for audit

Computational Considerations

Scalability Analysis

Efficiency Optimizations

• Mini-batch Training: GraphSAGE-style sampling for large graphs
• Sparse Operations: Efficient sparse matrix representations
• GPU Acceleration: CUDA-optimized attention computations
• Approximate k-NN: Faiss library for similarity computation

Deliverables

References

• Gilmer, J., et al. (2017). Neural message passing for quantum chemistry. ICML.
• Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. ICLR.
• Velickovic, P., et al. (2018). Graph attention networks. ICLR.
• McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001). Birds of a feather: Homophily in social networks. Annual Review of Sociology.
• Zhu, X., Ghahramani, Z., & Lafferty, J. D. (2003). Semi-supervised learning using Gaussian fields and harmonic functions. ICML.

Related Publications

• Liu, Y., Osterrieder, J., et al. “Credit Risk Prediction via Graph Neural Networks with Homophily-Guided Graph Construction” (JMIS Submission)
• Baals, L.J., Liu, Y., et al. “A Systematic Literature Review on Graph-Based Models in Credit Risk Assessment” (In Preparation)

Next Steps

Methodology validated and ready for WP3: Validation on real-world scenarios.

(c) Joerg Osterrieder 2025