CPython Memory Tracker
A memory benchmarking and analysis tool for CPython development, designed to track memory usage patterns across different commits and build configurations.
Architecture
This project consists of three main components:
- Backend (
/backend/) - FastAPI application with SQLite database for data storage and API endpoints - Frontend (
/frontend/) - Next.js React application with rich data visualization and analysis tools - Worker (
/worker/) - Python CLI tool for running memory benchmarks on CPython commits
Quick Start
Prerequisites
- Python 3.8+
- Node.js 16+
- CPython source repository (for benchmarking)
Setup & Installation
# Install all dependencies and set up database
make setup
# Or install components separately
make install # Install frontend + backend dependencies
make init-db # Initialize SQLite database
make populate-db # Add mock data for developmentDevelopment
# Start both frontend and backend servers
make dev
# Or start them individually
make dev-frontend # Frontend on http://localhost:9002
make dev-backend # Backend API on http://localhost:8000Development Commands
Testing
npm run lint # Frontend linting (in frontend directory)
npm run typecheck # TypeScript type checkingDatabase Management
make reset-db # Drop and recreate database with fresh data
make populate-db # Add mock benchmark dataProduction
make build # Build frontend for production
make clean # Clean up generated files and cachesWorker Setup
Worker Usage
# Set authentication token
export MEMORY_TRACKER_TOKEN=your_token_here
# List available binaries and environments
memory-tracker list-binaries
memory-tracker list-environments
# Run benchmarks on CPython commits
memory-tracker benchmark /path/to/cpython HEAD~5..HEAD \
--binary-id default \
--environment-id linux-x86_64
# Parallel processing with 4 workers
memory-tracker benchmark /path/to/cpython HEAD~10..HEAD \
--binary-id default \
--environment-id linux-x86_64 \
--max-workers 4
# Local checkout mode (sequential only)
memory-tracker benchmark /path/to/cpython HEAD~5..HEAD \
--binary-id default \
--environment-id linux-x86_64 \
--local-checkoutDocker Support
# Development with hot reload
docker-compose -f docker-compose.dev.yml up
# Production deployment
docker-compose upUsage Examples
Analyzing Memory Trends
- Navigate to
/trendsto view memory usage over time - Filter by specific benchmarks or commit ranges
- Compare different Python build configurations
- Export charts for reports and presentations
Comparing Commits
- Go to
/difffor commit comparison - Select two commits to analyze
- View detailed memory usage differences
- Identify performance regressions or improvements
Running Benchmarks
# Benchmark recent commits with parallel processing
memory-tracker benchmark ~/cpython HEAD~20..HEAD \
--binary-id optimized \
--environment-id linux-x86_64 \
--max-workers 8 \
--batch-size 4
# Force overwrite existing results
memory-tracker benchmark ~/cpython HEAD~10..HEAD \
--binary-id default \
--environment-id linux-x86_64 \
--forceContributing
- Follow the existing code style and conventions
- Run tests before committing:
make test - Use TypeScript for all frontend code
- Follow the repository patterns for new features
- Never commit secrets or authentication tokens
License
This project is licensed under the MIT License - see the LICENSE file for details.
Related Projects
- CPython - The Python programming language
- Memray - Memory profiler for Python
- pyperformance - Python performance benchmarking suite