🏗️ Urban Maintenance Scout

Proactively identifying public infrastructure issues with AI.

An intelligent system that uses computer vision and AI to analyze street-view images and automatically detect infrastructure problems like potholes, damaged signs, drainage issues, and safety hazards.

🚀 Features

Automated Street View Analysis: Fetches Google Street View images for any coordinates
AI-Powered Detection: Uses Facebook DETR and optional Grounding DINO for object detection
Infrastructure Issue Identification: Groq LLM analyzes detected objects for maintenance issues
Interactive Dashboard: Streamlit-based web interface with map integration
Cloud Storage: Automatic image upload to Supabase Storage
Historical Tracking: Database storage for all scan results and trend analysis
Severity Classification: Issues categorized as High, Medium, or Low priority
Export Functionality: Download scan results as CSV for reporting

🛠️ Tech Stack

Frontend: Streamlit with Folium maps
Computer Vision: Transformers (Facebook DETR), optional Grounding DINO
AI Analysis: Groq API (Llama 3.1 8B)
Database: Supabase (PostgreSQL)
Storage: Supabase Storage
APIs: Google Street View Static API
Image Processing: PIL, OpenCV

📋 Prerequisites

System Dependencies

# Ubuntu/Debian
sudo apt-get update
sudo apt-get install -y libgl1-mesa-glx libglib2.0-0 libsm6 libxext6 libxrender-dev libgomp1

# CentOS/RHEL  
sudo yum install -y mesa-libGL

Python Requirements

Python 3.8+
pip package manager

🚀 Installation

1. Clone the Repository

git clone https://github.com/LankeSathwik7/urban-maintenance-scout.git
cd urban-maintenance-scout

2. Create Virtual Environment

python -m venv urban_scout_env
source urban_scout_env/bin/activate  # Linux/Mac
# or
urban_scout_env\Scripts\activate  # Windows

3. Install Dependencies

pip install -r requirements.txt

# If you encounter libGL errors, use headless OpenCV:
pip uninstall opencv-python opencv-contrib-python
pip install opencv-python-headless

4. Environment Configuration

Create a .env file in the root directory:

# Google Street View API
STREET_VIEW_API_KEY=your_google_maps_api_key_here

# Groq API for AI Analysis
GROQ_API_KEY=your_groq_api_key_here

# Supabase Configuration
SUPABASE_URL=your_supabase_project_url
SUPABASE_ANON_KEY=your_supabase_anon_key

5. Database Setup

Create a Supabase project at supabase.com
Create the scans table:

CREATE TABLE scans (
  id SERIAL PRIMARY KEY,
  created_at TIMESTAMPTZ DEFAULT NOW(),
  latitude FLOAT8 NOT NULL,
  longitude FLOAT8 NOT NULL,
  image_url TEXT,
  annotated_image_url TEXT,
  detection_results JSONB,
  llm_report TEXT,
  llm_report_structured JSONB
);

Create a storage bucket named street-view-images
Set up RLS policies as needed

6. API Keys Setup

Google Street View API

Go to Google Cloud Console
Enable Street View Static API
Create API credentials
Add your API key to .env

Groq API

Sign up at console.groq.com
Generate an API key
Add to .env file

🎮 Usage

Running the Dashboard

streamlit run app.py

The dashboard will be available at http://localhost:8501

Using the Dashboard

Select Location:
- Click on the map to set coordinates
- Use quick location buttons
- Enter coordinates manually
Run Scan:
- Click "Scan Location" button
- Monitor progress through the pipeline
- View results automatically
Analyze Results:
- Browse historical scans
- View original and annotated images
- Read AI analysis reports
- Export data for reporting

🧩 Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Streamlit     │    │  Google Street   │    │   Computer      │
│   Dashboard     │───▶│  View API        │───▶│   Vision        │
└─────────────────┘    └──────────────────┘    └─────────────────┘
         │                                               │
         ▼                                               ▼
┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Supabase      │◀───│   Groq LLM       │◀───│   Detection     │
│   Database      │    │   Analysis       │    │   Pipeline      │
└─────────────────┘    └──────────────────┘    └─────────────────┘

Key Components

app.py: Main Streamlit dashboard
utils/fetcher.py: Google Street View image fetching
utils/cv_analysis.py: Computer vision pipeline
chains/analyst_chain.py: AI analysis with Groq LLM
utils/database.py: Supabase database operations
utils/storage.py: Image storage management
utils/scan_location.py: Main orchestration logic

🔧 Configuration

Adjusting Detection Sensitivity

Edit utils/cv_analysis.py:

# Lower threshold = more detections (may include false positives)
# Higher threshold = fewer detections (may miss some objects)
detections = analyze_image_combined(image_path, confidence_threshold=0.3)

Customizing AI Analysis

Edit chains/analyst_chain.py prompt template to:

Add new infrastructure categories
Adjust severity criteria
Modify analysis guidelines

Performance Tuning

CPU Usage: Set device=-1 in cv_analysis.py for CPU-only processing
Memory: Adjust batch sizes and image resolution
Speed: Use confidence_threshold=0.5 for faster processing

🔍 Troubleshooting

Common Issues

`libGL.so.1: cannot open shared object file`

# Solution 1: Install system dependencies
sudo apt-get install -y libgl1-mesa-glx

# Solution 2: Use headless OpenCV
pip install opencv-python-headless

# Solution 3: Set environment variables
export QT_QPA_PLATFORM=offscreen
export OPENCV_IO_ENABLE_OPENEXR=0

Groq API Rate Limits

# Add delay between requests in scan_location.py
import time
time.sleep(1)  # Wait 1 second between API calls

Supabase Connection Issues

Verify API keys in .env
Check RLS policies
Ensure storage bucket exists and is accessible

Street View API Errors

Check API key permissions
Verify billing is enabled
Ensure Street View Static API is enabled

Debug Mode

Enable debug logging:

import logging
logging.basicConfig(level=logging.DEBUG)

📊 Data Schema

Scans Table

Column	Type	Description
`id`	SERIAL	Primary key
`created_at`	TIMESTAMPTZ	Scan timestamp
`latitude`	FLOAT8	Location latitude
`longitude`	FLOAT8	Location longitude
`image_url`	TEXT	Original image URL
`annotated_image_url`	TEXT	Annotated image URL
`detection_results`	JSONB	CV detection data
`llm_report`	TEXT	Raw AI analysis
`llm_report_structured`	JSONB	Structured AI report

Detection Results Format

[
  {
    "label": "car",
    "score": 0.95,
    "box": {
      "xmin": 100,
      "ymin": 150,
      "xmax": 200,
      "ymax": 250
    }
  }
]

AI Report Format

{
  "summary": "Infrastructure condition summary",
  "issues": [
    {
      "type": "pothole",
      "severity": "High",
      "description": "Large pothole poses safety risk"
    }
  ]
}

🤝 Contributing

Fork the repository
Create a feature branch
Make your changes
Add tests if applicable
Submit a pull request

Development Setup

# Install development dependencies
pip install -r requirements-dev.txt

# Run tests
python -m pytest tests/

# Code formatting
black .
flake8 .

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

Hugging Face Transformers for computer vision models
Groq for fast LLM inference
Supabase for backend services
Streamlit for the web framework

Made with ❤️ for better urban infrastructure

LankeSathwik7/Urban-Maintenance-Scout