🧬 ClinVar Protein Variant Pathogenicity Prediction

English | 中文

English Version

Machine Learning-based Prediction of Pathogenicity for Protein Variants in ClinVar Database

This project develops machine learning models to predict the pathogenicity of missense mutations in protein-coding regions, using approximately 6.2 million variant records from the ClinVar database.

📖 Project Overview

Background

Predicting the pathogenicity of missense mutations is a critical challenge in precision medicine and genetic disease diagnosis. The ClinVar database contains extensive clinical interpretation of human variants, yet approximately 50% remain classified as "variants of uncertain significance (VUS)," necessitating computational prediction methods.

Objectives

1. Construct a high-quality modeling dataset from ClinVar with 20 biologically meaningful protein mutation features
2. Systematically evaluate 7 imbalanced data handling strategies across 4 machine learning models
3. Develop an interpretable, high-performance pathogenicity prediction model

🎯 Key Contributions

📊 Dataset

Data Source: ClinVar Database (December 2025 release)

The dataset exhibits severe class imbalance (1:19 ratio), reflecting the rarity of pathogenic variants in the human genome.

Train/Test Split: 80%/20% with stratified sampling (random_state=42)

• Training set: 5,009,581 samples (248,882 pathogenic)
• Test set: 1,252,396 samples (62,221 pathogenic)

🔬 Feature Engineering

We extracted 20 protein mutation features across 5 dimensions:

Mutation Pattern Analysis

Key observations:

• Pathogenic variants: Arginine (R) is the most common source amino acid; nonsense mutations (→*) are highly enriched
• Non-pathogenic variants: More dispersed pattern; conservative substitutions (e.g., I↔L, V↔I) are common

Feature Distribution Comparison

The kernel density plots show clear separation between pathogenic (red) and non-pathogenic (green) variants for evolutionary and physicochemical features.

Feature Distribution Comparison (Boxplots)

The boxplots show the distribution comparison of the Top 6 most discriminative features between pathogenic (red) and non-pathogenic (green) variants.

Statistical Effect Size Analysis

We used Mann-Whitney U test (non-parametric, suitable for non-normal distributions) to assess group differences. Given the massive sample size (N > 6 million), p-values are easily driven to near-zero and have limited practical value. Therefore, we report the rank-biserial correlation coefficient (r) as the effect size measure.

Features by Effect Size Category:

Note: Some features (e.g., delta_charge) have extremely small p-values but negligible effect sizes because the two groups have highly overlapping distributions—for example, 58.8% of pathogenic and 70.0% of non-pathogenic samples have delta_charge = 0.

Feature Correlation Matrix

Key correlations:

• BLOSUM62 score vs Grantham distance: r ≈ -0.88 (strong negative)
• delta_volume vs delta_mw: r ≈ 0.91 (strong positive)

BLOSUM62 vs Grantham Distance

The strong negative correlation confirms that evolutionarily conservative substitutions typically involve physicochemically similar amino acids.

🧪 Methods

Models Evaluated

Imbalanced Data Strategies

Ensemble Methods

• Soft Voting: Probability averaging across models
• Stacking: Two-layer meta-learning with LR meta-classifier

📈 Results

28 Model-Strategy Combinations

Key Finding: class_weight and none strategies consistently outperform oversampling methods on this large-scale dataset.

F1 Score Comparison Table:

*Note: XGBoost's poor performance with class_weight is due to excessive scale_pos_weight causing over-prediction of positive class.

Class Weight Sensitivity Analysis

Sensitivity Analysis Results:

• LR & DT: Highly robust to weight parameter changes
• RF: Stable at α ∈ [0.1, 2.5], slight decline when α > 3
• XGBoost: Highly sensitive - performance drops sharply when α > 1

Optimal setting: α = 1.0 (no additional weighting)

Final Model Performance (After Hyperparameter Tuning & Threshold Optimization)

Key Insights:

• XGBoost achieves best F1 score (0.6717) after threshold optimization from 0.50 → 0.30
• Stacking has highest AUC-ROC (0.9108) but marginal F1 improvement over single models
• All models achieve >97% accuracy due to class imbalance, but F1 is the primary metric

ROC and PR Curves

Feature Importance Analysis: RF vs XGBoost vs DT

Top 5 Feature Importance Comparison:

Algorithm-specific Explanations:

• RF's Distributed Importance: Based on average Gini impurity reduction across 400 trees with random feature subsets
• XGBoost's Concentrated Importance: "Gain"-based importance; blosum62_score dominates split decisions (94.67%)
• DT's Extreme Concentration: Limited tree depth (5 layers) concentrates importance on root node feature

Biological Validation:

• Evolutionary conservation features (blosum62_score + grantham_distance) account for >41% of effect size among top 6 features
• Pathogenic variants: BLOSUM62 score mean = -5.73 vs Non-pathogenic = +0.97
• Pathogenic variants: Grantham distance mean = 157 vs Non-pathogenic = 55

Confusion Matrix Analysis

XGBoost (Threshold=0.30) Clinical Interpretation:

The high specificity (99.41%) makes the model suitable for initial screening, with low false positive burden.

🚀 Quick Start

Environment Setup

# Clone the repository
git clone https://github.com/Rain021217/ClinVar-Pathogenicity-Prediction.git
cd ClinVar-Pathogenicity-Prediction

# Create virtual environment (recommended)
python -m venv venv
source venv/bin/activate  # Linux/Mac
# venv\Scripts\activate   # Windows

# Install dependencies
pip install -r requirements.txt

Running the Notebooks

1. Data Processing: notebooks/clinvar_data_processing_v3.ipynb

   • Downloads and processes ClinVar data
   • Extracts 20 protein mutation features
   • Generates train/test splits

2. Model Building: notebooks/model_building_v3.ipynb

   • Trains 4 models with 7 strategies (28 combinations)
   • Hyperparameter tuning with GridSearchCV
   • Ensemble methods (Voting, Stacking)
   • Threshold optimization
   • Feature importance analysis

📁 Project Structure

ClinVar-Pathogenicity-Prediction/
├── README.md                           # Project documentation (bilingual)
├── LICENSE                             # MIT License
├── requirements.txt                    # Python dependencies
├── .gitignore                          # Git ignore rules
│
├── data/                               # Data directory
│   └── README.md                       # Data acquisition guide
│
├── notebooks/                          # Jupyter Notebooks
│   ├── clinvar_data_processing_v3.ipynb   # Data processing pipeline
│   └── model_building_v3.ipynb            # Model training & evaluation
│
└── results/                            # Output results
    └── figures/                        # Visualization figures

Note: Source code utilities (clinvar_utils.py, model_utils_v3.py) are not included in this repository. Please contact the author if needed.

📝 Citation

If this project is helpful for your research, please cite:

@misc{li2026clinvar,
  author = {Jinpeng Li},
  title = {Machine Learning-based Prediction of Pathogenicity for Protein Variants in ClinVar Database},
  year = {2026},
  publisher = {GitHub},
  url = {https://github.com/Rain021217/ClinVar-Pathogenicity-Prediction}
}

📜 License

This project is licensed under the MIT License.

🙏 Acknowledgments

• ClinVar Database - Variant annotation data source
• UniProt - Protein sequence database
• Course: Python Programming for Health Data Science

📧 Contact

• Author: Jinpeng Li (李锦鹏)
• Email: jinpengli@stu.pku.edu.cn
• Affiliation: Medical Technology, Peking University

中文版本

基于机器学习的ClinVar蛋白质变异致病性预测研究

本项目开发了机器学习模型，用于预测蛋白质编码区域错义突变的致病性，使用了来自ClinVar数据库的约620万条变异记录。

📖 项目概述

研究背景

预测错义突变的致病性是精准医学和遗传病诊断的关键挑战。ClinVar数据库包含大量人类变异的临床解读信息，但约50%的变异仍被标记为"意义不明（VUS）"，亟需计算方法辅助预测。

研究目标

1. 从ClinVar构建包含20个具有生物学意义的蛋白质突变特征的高质量建模数据集
2. 系统评估7种不平衡数据处理策略在4种机器学习模型上的效果
3. 开发可解释的高性能致病性预测模型

🎯 主要贡献

📊 数据集

数据来源： ClinVar数据库（2025年12月版本）

数据集呈现严重的类别不平衡（1:19比例），反映了人类基因组中致病变异的罕见性。

训练/测试集划分： 80%/20%分层抽样（random_state=42）

• 训练集：5,009,581个样本（致病性248,882个）
• 测试集：1,252,396个样本（致病性62,221个）

🔬 特征工程

我们从5个维度提取了20个蛋白质突变特征：

突变模式分析

关键观察：

• 致病变异： 精氨酸（R）是最常见的突变起始氨基酸；无义突变（→*）高度富集
• 非致病变异： 模式更分散；保守替换（如I↔L、V↔I）更常见

特征分布对比

核密度图显示致病变异（红色）和非致病变异（绿色）在进化和物化特征上有明显分离。

特征分布对比（箱线图）

箱线图展示了Top 6 最具区分能力特征在致病变异（红色）和非致病变异（绿色）之间的分布对比。

统计效应量分析

我们使用 Mann-Whitney U 检验（非参数检验，适用于非正态分布数据）评估组间差异。由于样本量极大（N > 600万），p值易被驱动至接近零而缺乏实际参考价值。因此，我们报告 rank-biserial 相关系数 (r) 作为效应量指标。

按效应量等级划分的特征：

注意： 部分特征（如 delta_charge）虽然p值极小但效应量可忽略，这是因为两组样本分布高度重叠——例如，58.8%的致病和70.0%的非致病样本的 delta_charge = 0。

特征相关性矩阵

主要相关性：

• BLOSUM62得分 vs Grantham距离：r ≈ -0.88（强负相关）
• delta_volume vs delta_mw：r ≈ 0.91（强正相关）

BLOSUM62 vs Grantham距离

强负相关证实了进化保守的替换通常涉及理化性质相近的氨基酸。

🧪 方法

评估的模型

不平衡数据处理策略

集成方法

• Soft Voting： 跨模型概率平均
• Stacking： 两层元学习，使用LR作为元分类器

📈 结果

28种模型-策略组合

关键发现： class_weight和none策略在这个大规模数据集上始终优于过采样方法。

F1分数对比表：

*注：XGBoost在class_weight下表现差是因为scale_pos_weight过高导致正类过度预测。

类别权重敏感性分析

敏感性分析结果：

• LR & DT： 对权重参数变化高度鲁棒
• RF： 在α ∈ [0.1, 2.5]时稳定，α > 3时略有下降
• XGBoost： 高度敏感 - α > 1时性能急剧下降

最优设置： α = 1.0（无额外加权）

最终模型性能（超参数调优和阈值优化后）

关键洞察：

• XGBoost在阈值从0.50优化到0.30后达到最佳F1分数（0.6717）
• Stacking具有最高AUC-ROC（0.9108），但F1相对单模型提升有限
• 由于类别不平衡，所有模型准确率>97%，但F1是主要评估指标

ROC和PR曲线

特征重要性分析：RF vs XGBoost vs DT

Top 5特征重要性对比：

算法特性解释：

• RF的分布式重要性： 基于400棵树中Gini不纯度减少的平均值
• XGBoost的集中重要性： 基于"增益"的重要性；blosum62_score主导分裂决策（94.67%）
• DT的极端集中： 有限的树深度（5层）将重要性集中在根节点特征上

生物学验证：

• 进化保守性特征（blosum62_score + grantham_distance）在Top 6特征中效应量占比**>41%**
• 致病变异：BLOSUM62得分均值 = -5.73 vs 非致病 = +0.97
• 致病变异：Grantham距离均值 = 157 vs 非致病 = 55

混淆矩阵分析

XGBoost（阈值=0.30）临床解读：

高特异性（99.41%）使该模型适合初步筛查，假阳性负担低。

🚀 快速开始

环境配置

# 克隆仓库
git clone https://github.com/Rain021217/ClinVar-Pathogenicity-Prediction.git
cd ClinVar-Pathogenicity-Prediction

# 创建虚拟环境（推荐）
python -m venv venv
source venv/bin/activate  # Linux/Mac
# venv\Scripts\activate   # Windows

# 安装依赖
pip install -r requirements.txt

运行Notebook

1. 数据处理： notebooks/clinvar_data_processing_v3.ipynb

   • 下载和处理ClinVar数据
   • 提取20个蛋白质突变特征
   • 生成训练/测试集划分

2. 模型构建： notebooks/model_building_v3.ipynb

   • 用7种策略训练4个模型（28种组合）
   • 使用GridSearchCV进行超参数调优
   • 集成方法（Voting、Stacking）
   • 阈值优化
   • 特征重要性分析

📁 项目结构

ClinVar-Pathogenicity-Prediction/
├── README.md                           # 项目文档（双语）
├── LICENSE                             # MIT许可证
├── requirements.txt                    # Python依赖
├── .gitignore                          # Git忽略规则
│
├── data/                               # 数据目录
│   └── README.md                       # 数据获取指南
│
├── notebooks/                          # Jupyter Notebooks
│   ├── clinvar_data_processing_v3.ipynb   # 数据处理流程
│   └── model_building_v3.ipynb            # 模型训练与评估
│
└── results/                            # 输出结果
    └── figures/                        # 可视化图表

注意： 源代码工具函数（clinvar_utils.py、model_utils_v3.py）未包含在此仓库中。如需获取，请联系作者。

📝 引用

如果本项目对您的研究有帮助，请引用：

@misc{li2026clinvar,
  author = {Jinpeng Li},
  title = {Machine Learning-based Prediction of Pathogenicity for Protein Variants in ClinVar Database},
  year = {2026},
  publisher = {GitHub},
  url = {https://github.com/Rain021217/ClinVar-Pathogenicity-Prediction}
}

📜 许可证

本项目采用 MIT许可证 开源。

🙏 致谢

• ClinVar数据库 - 变异注释数据来源
• UniProt - 蛋白质序列数据库
• 课程：健康数据科学的Python语言编程基础

📧 联系方式

• 作者： 李锦鹏 (Jinpeng Li)
• 邮箱： jinpengli@stu.pku.edu.cn
• 单位： 北京大学 医学技术专业

⭐ 如果您觉得这个项目有用，请给它一个星标！