Learning Rate Warmup in PyTorch

optimizer & lr scheduler & loss function collections in PyTorch

Gradient based Hyperparameter Tuning library in PyTorch

A guide that integrates Pytorch DistributedDataParallel, Apex, warmup, learning rate scheduler, also mentions the set-up of early-stopping and random seed.

Polynomial Learning Rate Decay Scheduler for PyTorch

Pytorch cyclic cosine decay learning rate scheduler

Automatic learning-rate scheduler

Warmup learning rate wrapper for Pytorch Scheduler

[EAAI] The official repo for the paper: "A Lightweight Multi-Head Attention Transformer for Stock Price Forecasting".

A learning rate recommending and benchmarking tool.

Keras Callback to Automatically Adjust the learning rate when it stops improving

sharpDARTS: Faster and More Accurate Differentiable Architecture Search

Pytorch implementation of arbitrary learning rate and momentum schedules, including the One Cycle Policy

No description provided.

Code to reproduce the experiments of ICLR2023-paper: How I Learned to Stop Worrying and Love Retraining

Implementation of fluctuation dissipation relations for automatic learning rate annealing.

A method for assigning separate learning rate schedulers to different parameters group in a model.

Comprehensive image classification for training multilayer perceptron (MLP), LeNet, LeNet5, conv2, conv4, conv6, VGG11, VGG13, VGG16, VGG19 with batch normalization, ResNet18, ResNet34, ResNet50, MobilNetV2 on MNIST, CIFAR10, CIFAR100, and ImageNet1K.

(GECCO2023 Best Paper Nomination & ACM TELO) CMA-ES with Learning Rate Adaptation

ABEL implemented in PyTorch

End-to-end Image Classification using Deep Learning toolkit for custom image datasets. Features include Pre-Processing, Training with Multiple CNN Architectures and Statistical Inference Tools. Special utilities for RAM optimization, Learning Rate Scheduling, and Detailed Code Comments are included.

Used different Transformer based and LSTM based models for forecasting rainfall in different areas of Mumbai. Employed different smart training techniques to improve correlation with the true time-series.

Official implementation and experimental results for the paper “Improving Neural Network Training Using Dynamic Learning Rate Schedule for PINNs and Image Classification”.

Master's thesis: Experiments on multistage step size schedulers for first-order optimization in minimax problems

Visualize the progress of the learning rate scheduler graphically.

TVLARS - A Fast Convergence Optimizer for Large Batch Training

SPECTRA: Solar Panel Evaluation through Computer Vision and Advanced Techniques for Reliable Analysis

This project conducts a thorough analysis of weather time series data using diverse statistical and deep learning models. Each model was rigorously applied to the same weather time series data to assess and compare their forecasting accuracy. Detailed results and analyses are provided to delineate the strengths and weaknesses of each approach.

Build from scratch

In this repository, I put into test my newly acquired Deep Learning skills in order to solve the Kaggle's famous Image Classification Problem, called "Dogs vs. Cats".