GitHub Codespace Environment for the Data Engineering Class

User Notes

This codespace is intended for use with the Data Engineering class located SupaeroDataScience/DE, most notably the data computation section.

The class resources are available at the following website.

Start a GitHub Codespace from this repository to get all the resources required to do the hands-on exercises as well as a computing environment pre-configured.

Developer Notes: Dev Container Configuration

The .devcontainer/devcontainer.json configures the Codespace environment. Key documentation:

• Introduction to dev containers
• Universal dev container image
• Available dev container features

Configuration Choices

Base Image: mcr.microsoft.com/devcontainers/universal:2

• GitHub's default Codespace image
• Includes Python, Node.js, Docker, Git, and common dev tools
• See universal image docs for full list

Google Cloud CLI: Installed via install.sh, which runs on every container start (postStartCommand).

• The script downloads and extracts the Google Cloud CLI tarball to /opt/google-cloud-sdk
• This approach was chosen over dev container features for reliability
• Browse available features at containers.dev/features if you want to add other tools

Lifecycle Commands:

We use postStartCommand to run install.sh, which installs the Google Cloud CLI.

VS Code Extensions:

"customizations": {
  "vscode": {
    "extensions": ["ms-toolsai.jupyter", "ms-python.python", "ms-azuretools.vscode-docker"]
  }
}

Updating the Configuration

1. Edit .devcontainer/devcontainer.json
2. Test by rebuilding: Command Palette > "Codespaces: Rebuild Container"
3. For new features, search containers.dev/features

Teacher Notes

The teacher/ directory contains infrastructure for preparing and running the course. Students don't need it. See teacher/README.md for details.

Key things to know:

• This repo is a separate git repo from the course website (isae-data-engineering). The teacher must have push access.
• Student exercise folders (be-*/, tp-*/) are referenced by path in the course docs — don't rename them.
• The teacher/mascots/ directory contains the Flask GIF webapps for the Build-Ship-Run workshop. The teacher/build_push_images.sh script builds and pushes all Docker images in one shot.
• Year-specific values (project IDs, registry paths, bucket names) are hardcoded in several files — see the handover README in isae-data-engineering/src/teacher/README.md Section 4 for the full list and grep commands.