Spring AI Watsonx.ai

Spring AI Watsonx.ai provides Spring AI integration with IBM's Watsonx.ai platform, enabling developers to leverage powerful foundation models for chat, embeddings, content moderation, and document reranking in their applications.

Overview

IBM Watsonx.ai is an enterprise-ready AI platform that provides access to various foundation models including:

Chat Models: IBM Granite, Meta Llama, Mistral AI, and other conversational AI models
Embedding Models: IBM's embedding models for semantic search and similarity analysis
Moderation Models: Content safety detection including HAP, PII, and Granite Guardian detectors
Rerank Models: Document reranking for improved RAG (Retrieval-Augmented Generation) pipelines

This integration brings these capabilities to Spring Boot applications through familiar Spring AI abstractions.

Features

✅ Chat Models: Support for multiple foundation models with streaming capabilities
✅ Embedding Models: Generate embeddings for semantic search and similarity analysis
✅ Moderation Models: Content safety with HAP, PII, and Granite Guardian detectors
✅ Rerank Models: Document reranking for enhanced search relevance in RAG pipelines
✅ Spring Boot Auto-configuration: Zero-configuration setup with Spring Boot
✅ Flexible Configuration: Runtime parameter overrides and multiple model configurations
✅ Function Calling: Connect LLMs with external tools and APIs
✅ Reactive Support: Built-in support for reactive programming with WebFlux

Quick Start

Prerequisites

Create an account at IBM Cloud
Set up a Watsonx.ai service instance
Generate API keys from the IBM Cloud console

Installation

Add the Spring AI Watsonx.ai starter to your project. You can check the Maven Central for the latest version:

Maven:

<dependency>
    <groupId>org.springaicommunity</groupId>
    <artifactId>spring-ai-starter-model-watsonx-ai</artifactId>
    <version>1.X.X</version>
</dependency>

Gradle:

implementation 'org.springaicommunity:spring-ai-starter-model-watsonx-ai:<LATEST VERSION>'

Configuration

Configure your application with Watsonx.ai credentials:

application.yml:

spring:
  ai:
    watsonx:
      ai:
        api-key: ${WATSONX_AI_API_KEY}
        url: ${WATSONX_AI_URL}
        project-id: ${WATSONX_AI_PROJECT_ID}

Environment Variables:

export WATSONX_AI_API_KEY=your_api_key_here
export WATSONX_AI_URL=https://us-south.ml.cloud.ibm.com
export WATSONX_AI_PROJECT_ID=your_project_id_here

Basic Usage

Chat Model

@RestController
public class ChatController {

    private final WatsonxAiChatModel chatModel;

    public ChatController(WatsonxAiChatModel chatModel) {
        this.chatModel = chatModel;
    }

    @GetMapping("/chat")
    public String chat(@RequestParam String message) {
        return chatModel.call(message);
    }

    @GetMapping("/chat/stream")
    public Flux<String> chatStream(@RequestParam String message) {
        return chatModel.stream(new Prompt(message))
            .map(response -> response.getResult().getOutput().getContent());
    }
}

Embedding Model

@RestController
public class EmbeddingController {

    private final WatsonxAiEmbeddingModel embeddingModel;

    public EmbeddingController(WatsonxAiEmbeddingModel embeddingModel) {
        this.embeddingModel = embeddingModel;
    }

    @GetMapping("/embed")
    public List<Double> embed(@RequestParam String text) {
        return embeddingModel.embed(text);
    }
}

Moderation Model

@RestController
public class ModerationController {

    private final WatsonxAiModerationModel moderationModel;

    public ModerationController(WatsonxAiModerationModel moderationModel) {
        this.moderationModel = moderationModel;
    }

    @PostMapping("/moderate")
    public ModerationResponse moderate(@RequestBody String text) {
        ModerationPrompt prompt = new ModerationPrompt(text);
        return moderationModel.call(prompt);
    }
}

Rerank Model

@RestController
public class RerankController {

    private final WatsonxAiDocumentReranker documentReranker;

    public RerankController(WatsonxAiDocumentReranker documentReranker) {
        this.documentReranker = documentReranker;
    }

    @PostMapping("/rerank")
    public List<Document> rerank(@RequestParam String query, @RequestBody List<Document> documents) {
        return documentReranker.rerank(documents, query);
    }
}

Architecture

The Spring AI Watsonx.ai integration consists of three main modules:

Core Modules

watsonx-ai-core: Core implementation with API clients and model classes
spring-ai-autoconfigure-model-watsonx-ai: Spring Boot auto-configuration
spring-ai-starter-model-watsonx-ai: Spring Boot starter for easy integration

Key Components

spring-ai-watsonx-ai/
├── watsonx-ai-core/
│   ├── WatsonxAiChatModel        # Chat model implementation
│   ├── WatsonxAiEmbeddingModel   # Embedding model implementation
│   ├── WatsonxAiModerationModel  # Content moderation implementation
│   ├── WatsonxAiDocumentReranker # Document reranking implementation
│   └── WatsonxAiAuthentication   # IBM Cloud IAM authentication
├── spring-ai-autoconfigure-model-watsonx-ai/
│   └── Auto-configuration classes
└── spring-ai-starter-model-watsonx-ai/
    └── Starter dependencies

Supported Models

A comprehensive list of supported models under the watsonx.ai platform: watsonx.ai Supported Models

Configuration Options

Chat Model Configuration

spring:
  ai:
    watsonx:
      ai:
        chat:
          options:
            model: ibm/granite-13b-chat-v2
            temperature: 0.7
            max-new-tokens: 1024
            top-p: 1.0
            top-k: 50
            repetition-penalty: 1.0

Embedding Model Configuration

spring:
  ai:
    watsonx:
      ai:
        embedding:
          options:
            model: ibm/slate-125m-english-rtrvr
            parameters:
                truncate-input-tokens: true
                return-options:
                    input-text: false

Moderation Model Configuration

spring:
  ai:
    watsonx:
      ai:
        moderation:
          version: "2025-10-01"
          options:
            # HAP (Hate, Abuse, Profanity) detector
            hap:
              threshold: 0.75
            # PII (Personally Identifiable Information) detector
            pii:
              threshold: 0.8
            # Granite Guardian detector
            granite-guardian:
              threshold: 0.6

Rerank Model Configuration

spring:
  ai:
    watsonx:
      ai:
        rerank:
          options:
            model: cross-encoder/ms-marco-minilm-l-12-v2
            top-n: 3
            truncate-input-tokens: true

Advanced Features

Function Calling

Connect your LLMs with external tools and APIs:

@Bean
@Description("Get current weather information")
public Function<WeatherRequest, WeatherResponse> getCurrentWeather() {
    return request -> {
        // Implementation to fetch weather data
        return new WeatherResponse(25.0, "sunny", request.location());
    };
}

Multiple Model Configurations

Configure different models for different use cases:

@Configuration
public class MultiModelConfiguration {

    @Bean("creativeChatModel")
    public WatsonxAiChatModel creativeChatModel(WatsonxAiChatApi chatApi) {
        return new WatsonxAiChatModel(chatApi,
            WatsonxAiChatOptions.builder()
                .withModel("meta-llama/llama-3-70b-instruct")
                .withTemperature(1.2)
                .build());
    }
}

Reactive Streaming

Built-in support for reactive programming:

@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ServerSentEvent<String>> streamResponse(@RequestParam String prompt) {
    return chatModel.stream(new Prompt(prompt))
        .map(response -> response.getResult().getOutput().getContent())
        .map(content -> ServerSentEvent.<String>builder().data(content).build());
}

Content Moderation

The moderation model provides comprehensive content safety detection:

Available Detectors:

HAP (Hate, Abuse, Profanity): Detects hate speech, abusive language, and profanity
PII (Personally Identifiable Information): Identifies sensitive personal information like emails, phone numbers, addresses
Granite Guardian: IBM's comprehensive content moderation detector for harmful content

Example Usage:

@Service
public class ContentModerationService {

    private final WatsonxAiModerationModel moderationModel;

    public ContentModerationService(WatsonxAiModerationModel moderationModel) {
        this.moderationModel = moderationModel;
    }

    public boolean isContentSafe(String userInput) {
        ModerationPrompt prompt = new ModerationPrompt(userInput);
        ModerationResponse response = moderationModel.call(prompt);
        
        // Check if any detector flagged the content
        return !response.getResult().getOutput().getResults().get(0).isFlagged();
    }

    public ContentAnalysis analyzeContent(String text) {
        ModerationPrompt prompt = new ModerationPrompt(text);
        ModerationResponse response = moderationModel.call(prompt);
        
        var result = response.getResult().getOutput().getResults().get(0);
        CategoryScores scores = result.getCategoryScores();
        
        return new ContentAnalysis(
            result.isFlagged(),
            scores.getHate(),
            scores.getHarassment(),
            scores.getSelfHarm(),
            scores.getSexual(),
            scores.getViolence()
        );
    }
}

Response Analysis:

// Get detailed detection information
WatsonxAiModerationResponseMetadata metadata =
    (WatsonxAiModerationResponseMetadata) response.getMetadata();

List<Map<String, Object>> detections = metadata.getDetections();
for (Map<String, Object> detection : detections) {
    String detectionType = (String) detection.get("detectionType"); // "hap", "pii", etc.
    String detectedText = (String) detection.get("text");
    Float confidenceScore = (Float) detection.get("score");
    Integer startPosition = (Integer) detection.get("start");
    Integer endPosition = (Integer) detection.get("end");
    
    System.out.println(String.format(
        "Detected %s: '%s' (score: %.2f) at position %d-%d",
        detectionType, detectedText, confidenceScore, startPosition, endPosition
    ));
}

Examples

Customer Support Chatbot

@Service
public class CustomerSupportService {

    private final WatsonxAiChatModel chatModel;

    public String handleQuery(String customerId, String query) {
        var options = WatsonxAiChatOptions.builder()
            .withModel("ibm/granite-13b-chat-v2")
            .withTemperature(0.3)
            .withFunction("getOrderStatus")
            .withFunction("createSupportTicket")
            .build();

        return chatModel.call(new Prompt(buildContextualPrompt(customerId, query), options));
    }
}

Document Analysis

@Service
public class DocumentAnalysisService {

    private final WatsonxAiChatModel chatModel;
    private final WatsonxAiEmbeddingModel embeddingModel;

    public DocumentAnalysis analyzeDocument(String content) {
        // Generate summary
        String summary = chatModel.call("Summarize: " + content);
        
        // Generate embeddings for similarity search
        List<Double> embeddings = embeddingModel.embed(content);
        
        return new DocumentAnalysis(summary, embeddings);
    }
}

Safe Content Pipeline

@Service
public class SafeContentPipeline {

    private final WatsonxAiModerationModel moderationModel;
    private final WatsonxAiChatModel chatModel;

    public String processUserInput(String userInput) {
        // Step 1: Check content safety
        ModerationPrompt moderationPrompt = new ModerationPrompt(userInput);
        ModerationResponse moderationResponse = moderationModel.call(moderationPrompt);
        
        if (moderationResponse.getResult().getOutput().getResults().get(0).isFlagged()) {
            return "Your input contains inappropriate content. Please revise.";
        }
        
        // Step 2: Process safe content with chat model
        return chatModel.call(userInput);
    }
}

RAG Pipeline with Reranking

@Service
public class RAGPipelineService {

    private final WatsonxAiChatModel chatModel;
    private final WatsonxAiEmbeddingModel embeddingModel;
    private final WatsonxAiDocumentReranker documentReranker;
    private final VectorStore vectorStore;

    public String answerQuestion(String question) {
        // Step 1: Retrieve relevant documents using embeddings
        List<Document> retrievedDocs = vectorStore.similaritySearch(
            SearchRequest.query(question).withTopK(10)
        );
        
        // Step 2: Rerank documents for better relevance
        List<Document> rerankedDocs = documentReranker.rerank(retrievedDocs, question);
        
        // Step 3: Generate answer using top reranked documents
        String context = rerankedDocs.stream()
            .limit(3)
            .map(Document::getContent)
            .collect(Collectors.joining("\n\n"));
        
        String prompt = String.format(
            "Based on the following context, answer the question.\n\nContext:\n%s\n\nQuestion: %s",
            context, question
        );
        
        return chatModel.call(prompt);
    }
}

Documentation

For comprehensive documentation, examples, and API reference, visit:

Building from Source

Prerequisites

Java 17 or later
Maven 3.8.4 or later

Build

git clone https://github.com/spring-ai-community/spring-ai-watsonx-ai.git
cd spring-ai-watsonx-ai
mvn clean install

Run Tests

mvn test

Build Documentation

cd docs
mvn clean package

Contributing

We welcome contributions! Please see our Contributing Guide for details on:

Code of Conduct
Development setup
Submitting pull requests
Reporting issues

Development Setup

Fork the repository
Create a feature branch
Make your changes
Add tests for new functionality
Ensure all tests pass
Submit a pull request

Community

GitHub Discussions - Ask questions and share ideas
Issues - Report bugs and request features

License

This project is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Acknowledgments

Spring AI - The foundational framework
IBM Watsonx.ai - The AI platform
Spring Community - The vibrant community

spring-ai-community/spring-ai-watsonx-ai

Spring AI Watsonx.ai

Overview

Features

Quick Start

Prerequisites

Installation

Configuration

Basic Usage

Chat Model

Embedding Model

Moderation Model

Rerank Model

Architecture

Core Modules

Key Components

Supported Models

Configuration Options

Chat Model Configuration

Embedding Model Configuration

Moderation Model Configuration

Rerank Model Configuration

Advanced Features

Function Calling

Multiple Model Configurations

Reactive Streaming

Content Moderation

Examples

Customer Support Chatbot

Document Analysis

Safe Content Pipeline

RAG Pipeline with Reranking

Documentation

Building from Source

Prerequisites

Build

Run Tests

Build Documentation

Contributing

Development Setup

Community

License

Acknowledgments

On this page

Languages

Contributors

Latest Release