Awesome Knowledge Distillation

Pytorch implementation of various Knowledge Distillation (KD) methods.

Official PyTorch implementation of "A Comprehensive Overhaul of Feature Distillation" (ICCV 2019)

PyContinual (An Easy and Extendible Framework for Continual Learning)

An Extensible Continual Learning Framework Focused on Language Models (LMs)

This repository is mainly dedicated for listing the recent research advancements in the application of Self-Supervised-Learning in medical images computing field

Code and dataset for ACL2018 paper "Exploiting Document Knowledge for Aspect-level Sentiment Classification"

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)

Code and pretrained models for paper: Data-Free Adversarial Distillation

[ECCV2022] Factorizing Knowledge in Neural Networks

A Comprehensive Survey on Knowledge Distillation

[Paper][AAAI 2023] DUET: Cross-modal Semantic Grounding for Contrastive Zero-shot Learning

Knowledge Transfer via Dense Cross-layer Mutual-distillation (ECCV'2020)

PyTorch implementation of (Hinton) Knowledge Distillation and a base class for simple implementation of other distillation methods.

Code for ECML/PKDD 2020 Paper --- Continual Learning with Knowledge Transfer for Sentiment Classification

[NeurIPS'23] Source code of "Data-Centric Learning from Unlabeled Graphs with Diffusion Model": A data-centric transfer learning framework with diffusion model on graphs.

Code for NeurIPS 2020 Paper --- Continual Learning of a Mixed Sequence of Similar and Dissimilar Tasks

KT-BT: A Framework for Knowledge Transfer Through Behavior Trees in Multirobot Systems

[arXiv 2024] PyTorch implementation of RRD: https://arxiv.org/abs/2407.12073

Adaptive Model-based Transfer Evolutionary Algorithm

An activation-based protocol for AI-to-AI knowledge transfer across architectures

This project implements knowledge distillation from DINOv2 (Vision Transformer) to convolutional networks, enabling efficient visual representation learning with reduced computational requirements.

[TPAMI'2024] Multi-sensor Learning Enables Information Transfer across Different Sensory Data and Augments Multi-modality Imaging

Implementation of NAACL 2024 main conference paper: Named Entity Recognition Under Domain Shift via Metric Learning for Life Science

:drum: Teach a newbie how to perform better.

Cross-Cancer Knowledge Transfer in WSI-based Prognosis Prediction (https://arxiv.org/abs/2508.13482)

The default way to fine-tune BERT is wrong. Here is why

Capture, validate, and transfer AI agent expertise. Across sessions, platforms, and teams.

Functional Knowledge Transfer with Self-supervised Representation Learning (ICIP 2023)

Learning in Growing Robots: Knowledge Transfer from Tadpole to Frog Robot