Spring AI with Open Ai

🔄 Updated: September 2026  ·  Rewritten for Spring AI 2.0.1
The starter artifact, the configuration properties and the client class have all changed since 2024
Integrating OpenAI ChatGPT with Spring AI

Spring AI gives Java developers a clean, Spring-idiomatic way to call OpenAI's models — chat, streaming, structured output, tool calling and embeddings — without hand-rolling HTTP clients or JSON mapping. This guide walks through the whole setup, from getting an API key to building endpoints that do real work.

Getting Started: Obtaining an OpenAI API Key

You need an API key before anything else:

  1. Sign up at platform.openai.com (note: the API dashboard, not the ChatGPT consumer site).
  2. Add a payment method and some credit. API access is billed separately from a ChatGPT Plus subscription — having Plus does not give you API credit.
  3. Go to the API Keys page and create a new secret key.
  4. Copy it immediately. OpenAI shows the key exactly once and you cannot retrieve it later.
Keep the key out of your code

Never hardcode the key, never commit it, and never put it in an application.properties file that is tracked in git. Leaked OpenAI keys are actively scraped from public repositories and run up real charges within hours. Export it as an environment variable instead:

export SPRING_AI_OPENAI_API_KEY=<INSERT KEY HERE>

Spring Boot's relaxed binding maps that environment variable onto the spring.ai.openai.api-key property automatically, so nothing else is needed. For production, use your platform's secret manager — AWS Secrets Manager, Vault, Kubernetes secrets — rather than a plain environment variable.

Dependency Management

Enabling Auto-configuration

Add the OpenAI starter. Note the artifact ID — every Spring AI starter was renamed to the spring-ai-starter-model-* pattern in 1.0 GA:

<!-- Maven -->
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>

// Gradle
dependencies {
    implementation 'org.springframework.ai:spring-ai-starter-model-openai'
}

Import the Spring AI BOM so you never specify individual module versions. You no longer need to add the Spring Milestone repository — Spring AI has been on Maven Central since 1.0:

<properties>
    <java.version>21</java.version>
    <spring-ai.version>2.0.1</spring-ai.version>
</properties>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-bom</artifactId>
            <version>${spring-ai.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Configuration

All Spring AI configuration lives under the spring.ai.* prefix. Connection settings sit on spring.ai.openai, chat settings on spring.ai.openai.chat, and retry behaviour on the shared spring.ai.retry namespace.

application.properties


# ---- Connection ----
# Usually supplied via the SPRING_AI_OPENAI_API_KEY environment variable instead
spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.base-url=https://api.openai.com

# ---- Chat model ----
# Pick a current model - see the note below on why this is not hardcoded here
spring.ai.openai.chat.model=gpt-4.1
spring.ai.openai.chat.temperature=0.7
spring.ai.openai.chat.max-tokens=2048

# ---- Retry (shared across all Spring AI models) ----
spring.ai.retry.max-attempts=3
spring.ai.retry.backoff.initial-interval=2s
spring.ai.retry.backoff.multiplier=2
spring.ai.retry.backoff.max-interval=60s

# ---- Enable / disable the chat auto-configuration ----
# 'openai' by default; set to 'none' to switch it off
spring.ai.model.chat=openai

application.yml


spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      base-url: https://api.openai.com
      chat:
        model: gpt-4.1
        temperature: 0.7
        max-tokens: 2048
    retry:
      max-attempts: 3
      backoff:
        initial-interval: 2s
        multiplier: 2
        max-interval: 60s
A note on the .options. prefix

Until Spring AI 2.0, chat settings carried an extra segment: spring.ai.openai.chat.options.model. That form was flattened to spring.ai.openai.chat.model. The old keys are deprecated rather than removed, so existing config still binds — but they will be dropped eventually, and every current example uses the flat form.

A note on model names

Do not treat any model name in a blog post — including this one — as current. OpenAI retires models on a rolling schedule: gpt-3.5-turbo, gpt-4 and gpt-4-turbo all have announced API shutdown dates, and calling a retired model returns a 404 model_not_found. Check the OpenAI models page and the deprecations page before you pick one.

Practically, this means: keep the model name in configuration (never in code), pin a dated snapshot version in production so a silent upgrade cannot change your outputs, and set a calendar reminder to check the deprecation page every few months.

ChatClient: the recommended way to call the model

ChatClient is the high-level fluent API and is what you should reach for in application code. The starter auto-configures a ChatClient.Builder, so you inject that and build from it:


package codeKatha;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class AssistantService {

    private final ChatClient chatClient;

    public AssistantService(ChatClient.Builder builder) {
        this.chatClient = builder
                .defaultSystem("You are a helpful assistant for Java developers. "
                        + "Answer concisely and include code examples where useful.")
                .build();
    }

    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

A system prompt set once via defaultSystem() applies to every call through that client — useful for fixing tone, format and scope in one place rather than repeating instructions in every user prompt.

Sample Controller

A controller with both a blocking and a streaming endpoint:


package codeKatha;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;

@RestController
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/ai/generate")
    public String generate(
            @RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return chatClient.prompt()
                .user(message)
                .call()
                .content();
    }

    /**
     * Streams tokens as the model produces them.
     * TEXT_EVENT_STREAM_VALUE lets the browser render the response progressively
     * instead of waiting for the whole thing.
     */
    @GetMapping(value = "/ai/generateStream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<String> generateStream(
            @RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        return chatClient.prompt()
                .user(message)
                .stream()
                .content();
    }
}

The streaming endpoint needs spring-boot-starter-webflux on the classpath for Flux. Streaming matters more than it looks: a long answer can take fifteen seconds to generate, and users tolerate that far better when text appears immediately.

If you need the raw response — token usage, finish reason, model metadata — drop down to OpenAiChatModel:


package codeKatha;

import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.openai.OpenAiChatModel;
import org.springframework.ai.openai.OpenAiChatOptions;
import org.springframework.stereotype.Service;

@Service
public class LowLevelService {

    private final OpenAiChatModel chatModel;

    public LowLevelService(OpenAiChatModel chatModel) {
        this.chatModel = chatModel;
    }

    public String ask(String question) {
        ChatResponse response = chatModel.call(new Prompt(question));

        // Token usage - worth logging, this is what you get billed on
        var usage = response.getMetadata().getUsage();
        System.out.println("prompt=" + usage.getPromptTokens()
                + " completion=" + usage.getCompletionTokens());

        // NOTE: getText(), not getContent() - renamed in 1.0
        return response.getResult().getOutput().getText();
    }

    public String askPrecisely(String question) {
        // Per-request options override application.properties
        Prompt prompt = new Prompt(question,
                OpenAiChatOptions.builder()
                        .temperature(0.0)
                        .maxTokens(500)
                        .build());

        return chatModel.call(prompt).getResult().getOutput().getText();
    }
}

Structured Outputs

Parsing free-form model text with regex is a losing game. Spring AI can generate a JSON Schema from a Java type, send it with the request, and convert the response straight back into an object:


package codeKatha;

public record MovieReview(
        String title,
        int year,
        int ratingOutOfTen,
        String oneLineVerdict) {
}


MovieReview review = chatClient.prompt()
        .user("Review the movie Inception in one line")
        .call()
        .entity(MovieReview.class);

System.out.println(review.ratingOutOfTen());

That single .entity() call handles schema generation, the API request and deserialisation. It also works with collections:


List<MovieReview> reviews = chatClient.prompt()
        .user("Review three Christopher Nolan films")
        .call()
        .entity(new ParameterizedTypeReference<List<MovieReview>>() {});

This is the feature that makes an LLM usable as a backend component rather than a chat toy — you get a typed object you can validate, persist and test against.

Tool Calling (formerly Function Calling)

Tool calling lets the model ask your application to run a method and then use the result in its answer. The model never executes anything itself — it returns a structured request, Spring AI runs your code, and the result goes back into the conversation.

The terminology changed. What was called "function calling" in 2024 is now "tool calling" throughout Spring AI. FunctionCallback became ToolCallback, .functions(...) became .tools(...), and the old API was removed entirely in 2.0.

Example Implementation

Declare tools as methods annotated with @Tool. The description is not decoration — it is how the model decides whether to call it, so write it carefully:


package codeKatha;

import org.springframework.ai.tool.annotation.Tool;
import org.springframework.ai.tool.annotation.ToolParam;
import java.time.LocalDate;

public class OrderTools {

    private final OrderRepository orderRepository;

    public OrderTools(OrderRepository orderRepository) {
        this.orderRepository = orderRepository;
    }

    @Tool(description = "Get today's date in ISO format")
    public String currentDate() {
        return LocalDate.now().toString();
    }

    @Tool(description = "Look up the delivery status of a customer order by its ID")
    public String orderStatus(
            @ToolParam(description = "The order ID, e.g. ORD-12345") String orderId) {

        return orderRepository.findById(orderId)
                .map(order -> order.getStatus().name())
                .orElse("No order found with that ID");
    }
}

Attach them to the call and Spring AI handles the whole loop — the model requests a tool, your method runs, the result is fed back, and the model produces a final answer:


String answer = chatClient.prompt()
        .user("Where is my order ORD-12345, and how long ago did I place it?")
        .tools(new OrderTools(orderRepository))
        .call()
        .content();

An important 2.0 change: in Spring AI 1.x the tool-execution loop lived inside ChatModel. In 2.0 it was removed from every ChatModel implementation and moved into ChatClient's auto-registered ToolCallingAdvisor. Call chatModel.call(prompt) directly with tools attached and you get back the model's unexecuted request. Use ChatClient and it just works.

Two practical cautions. First, a tool that touches your database is now reachable by anything the model decides to do with user input — validate arguments and scope permissions exactly as you would for a public endpoint. Second, every tool definition is sent with every request and costs tokens, so ten tools attached to a chatbot is ten tools' worth of prompt on every single call.

Chat Memory

The API is stateless: each request knows nothing about the last one. To hold a conversation you must send the history back every time, and Spring AI packages that as an advisor:


package codeKatha;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.stereotype.Service;

@Service
public class ConversationService {

    private final ChatClient chatClient;

    public ConversationService(ChatClient.Builder builder) {
        ChatMemory memory = MessageWindowChatMemory.builder()
                .maxMessages(20)
                .build();

        this.chatClient = builder
                .defaultAdvisors(MessageChatMemoryAdvisor.builder(memory).build())
                .build();
    }

    public String chat(String sessionId, String message) {
        return chatClient.prompt()
                .user(message)
                .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, sessionId))
                .call()
                .content();
    }
}

Note that the conversation ID is required in 2.0 — the old ChatMemory.DEFAULT_CONVERSATION_ID constant was removed, and omitting the parameter throws an IllegalArgumentException. Use the session or user ID.

Watch the cost here: the whole window is resent on every turn, so a 20-message history means you pay for 20 messages' worth of input tokens on each request. That is exactly why MessageWindowChatMemory caps the count rather than keeping everything.

Azure OpenAI users: the module is gone

If your project used spring-ai-azure-openai, that module — along with its starter and auto-configuration — was removed in Spring AI 2.0. Migrate to the standard spring-ai-openai module: drop the Azure prefix from the class names and move your configuration across to the spring.ai.openai.* properties, pointing base-url at your Azure endpoint.

Conclusion

Spring AI turns OpenAI's API into an ordinary Spring dependency — injected, configured through properties, and testable like anything else. The pieces worth building on are structured output, which gives you typed objects instead of strings to parse, and tool calling, which lets the model reach into your own data. Chat and streaming are the easy part.

If you are preparing for interviews, be ready to explain what tool calling actually does (the model requests, your application executes), why the API being stateless forces you to resend history, and where the token cost in an AI feature actually comes from. Those questions come up far more often than framework syntax.

Also Read:  Spring Boot with Ollama

Related categories:

Comments

Post a Comment

Popular Posts on Code Katha

Java Interview Questions for 10 Years Experience

Image
Welcome to the comprehensive guide on "Java Interview Questions for 10 Years Experience." Java has been one of the most popular programming languages for decades, and its widespread adoption means that the demand for skilled Java developers has never been higher. If you are someone who has been working with Java for the last decade, you have accumulated a wealth of experience and knowledge that you can showcase during interviews. Recommended  to read and brush up for your interview: Must know interview questions and concept for any Java Interview Spring boot interview Questions for 10 years experiance These simple and short articles help you to add AI to your resume and enhance the chance of selection: Java Spring Boot AI with Ollama Java Spring Boot AI with Open AI (Chat GPT) This blog post will provide you with a list of commonly asked interview questions for 10 years experienced Java developers. It will also include detailed answers to each question, with explanat...
Read more

Sql Interview Questions for 10 Years Experience

Image
As an experienced SQL professional, you might encounter challenging questions during a job interview. In this blog post, we will explore 10 common SQL interview questions and provide in-depth explanations along with relevant examples. Let's dive in! Note on dialects: the examples below use MySQL syntax by default. Where a feature behaves differently in PostgreSQL, SQL Server or Oracle, it is called out explicitly — interviewers often probe exactly these differences. Contents [ hide ] What is SQL Injection, and how can you prevent it? SQL Injection is a malicious attack where an attacker manipulates input data to execute unintended SQL statements. It can lead to unauthorized access, data breaches, and data manipulation. The root cause is always the same: user input ends up being treated as SQL code instead of as data. To prevent it, follow these practices: 1.1. Use Prepared Statements (the real fix): Prepared statements send the SQL structure and the values to the dat...
Read more

Spring Boot Interview Questions for 10 Years Experience

Image
Welcome to the comprehensive guide on "Spring Boot Interview Questions for 10 Years Experience". Spring Boot has become one of the most popular frameworks for developing enterprise-grade applications, and its widespread adoption means that the demand for skilled Spring Boot developers has never been higher. If you are someone who has been working with Spring Boot for the last decade, you have accumulated a wealth of experience and knowledge that you can showcase during interviews. Recommended  to read and brush up for your interview: Must know interview questions and concept for any Java Interview Java Interview Questions for 10 Years Experience These simple and short articles help you to add AI to your resume and enhance the chance of selection: Spring Boot AI with Ollama Spring Boot AI with Open AI (Chat GPT) This blog post will provide you with a list of commonly asked interview questions for 10 years expe...
Read more

Java interview questions - Must to know concepts

Contents [ hide ] How does Java achieve platform independence, and what is the underlying principle? Java programs can run on diverse operating systems and hardware setups without requiring modification. This feat is primarily achieved through the concept of the Java Virtual Machine (JVM). The Java Virtual Machine (JVM) The JVM acts as a bridge between the Java code you write and the underlying hardware and operating system . When you compile your Java code, it is translated into an intermediate form called bytecode, which is not tied to any specific platform. The JVM then comes into play. It takes this bytecode and dynamically translates it into machine-specific instructions that the host system can execute. This means that a Java program can be written once and executed on any system with the appropriate JVM for that system. Example: Running a Java Program on Different Platforms Consider a simple Java program that calculates the sum of two numbers: package codeKatha...
Read more

Visual Studio Code setup for Java and Spring with GitHub Copilot

Image
Introduction- In this tutorial blog, we will explore the process of setting up and utilising Visual Studio Code for Java programming, as well as incorporating GitHub Copilot as a valuable coding tool. Visual Studio Code is a powerful and versatile code editor that offers a wide range of features and extensions for Java developers. By using Visual Studio Code for Java programming, developers can benefit from features such as code completion, debugging, and live coding. Additionally, with the integration of GitHub Copilot, developers can streamline their coding process with AI-assisted suggestions for code snippets and functions. This blog will provide a comprehensive guide for setting up and using these tools for Java development. Contents [ hide ] Setup Visual Studio Code for Java and Spring Visual Studio Code (VS Code) is an open-source code editor that supports a wide range of programming languages, including Java. It is a powerful tool...
Read more

Spring AI with Ollama

Image
🔄 Updated: September 2026  ·  Rewritten for Spring AI 2.0.1 Artifact IDs, properties and the response API have all changed since this guide was first published in 2024 Introduction to Spring AI with Ollama Welcome to a comprehensive guide on leveraging Spring AI with Ollama to build AI-powered applications in Java — running large language models entirely on your own machine, with no API key and no per-token cost. This tutorial covers installing Ollama, wiring it into a Spring Boot project, and building endpoints for chat, streaming, structured output, reasoning models and tool calling. Contents [ hide ] Setting Up Ollama Installing Ollama Download and install Ollama for your OS from the official download page . It runs as a background service on http://localhost:11434 — no Docker required. The source lives on the official GitHub page . Pulling a model Once Ollama is installed, pull a model. The original version of this guide used mistral , which still...
Read more

Data Structures & Algorithms Tutorial with Coding Interview Questions

Image
About the Course This course offers comprehensive coverage of coding questions that serve as a solid foundation for understanding DSA concepts. These questions are frequently asked in coding interviews, either directly or with some variations, by many companies, including Adobe, Facebook (Meta), Paytm, Amazon, Google, and others. Our primary objective is to enhance your problem-solving skills and enable you to devise optimal solutions. The course includes a wide range of topics, such as Arrays, Strings, Searching and Sorting, Greedy Algorithms, Trees, Graphs, Dynamic Programming, and more. Upon completion, you will possess the necessary skills to ace DSA interviews for typical companies or excel in the initial rounds of product-based companies such as Google, Microsoft and similar others. We are started with Graph data Structure and add others soon. Contents [ hide ] Approach Our course is  organised into chapters ...
Read more

Spring Data JPA

Image
Introduction- We will learn here Spring Data JPA, perform CRUD operation, Clear concept of Relationship mapping like one-to-many, many-to-one with Example. Spring Data JPA is a powerful framework that simplifies the process of working with relational databases in Java applications. It provides an easy-to-use API for working with JPA, allowing developers to focus on their business logic instead of worrying about the details of how JPA works. With Spring Data JPA, you can easily define entities that map to database tables, and then use them to perform CRUD (create, read, update, delete) operations on those tables. In addition, Spring Data JPA provides support for more advanced features like querying and pagination, making it an excellent choice for both small and large projects. Contents [ hide ] In this blog, we will explore Spring Data JPA by building a sample application that uses it to manage users and packs. But, before let...
Read more

Bit Manipulation and Bit Masking Concepts

Image
Introduction- A bit is the fundamental unit of information used in computing and digital communication.We generally worked on decimal numbers having values from 0,1,2.. to 9. Similarly binary system is characterized by two possible values, usually expressed as 0 or 1. In terms of electronic switch we can say 0 denotes an OFF switch and 1 denotes an ON switch. In terms of magnetic domain on a hard drive, we can say 0 denotes an unmagnetized domain, and 1 denotes a magnetized domain. Contents [ hide ] Uses of bit manipulation concepts in our coding Bit manipulation is a technique that can give programmers a significant performance advantage in certain algorithms. It can help to reduce the time complexity. By working directly with binary data using bitwise operators, we can perform operations faster than with traditional arithmetic operations. The reason for this is simple: computers understand binary data. When we perform any operation, it g...
Read more

Topological Sort in Graph

Welcome to our beginner's guide to Topological Sort in Graph Theory! Topological Sort is an essential algorithm used to order the vertices of a directed graph such that for every directed edge uv from vertex u to vertex v, u comes before v in the ordering. Let's explore this important concept together! Contents [ hide ] Understanding the Problem Statement Before delving into Topological Sort, let's understand the problem it addresses. Consider a scenario where tasks depend on one another; for example, in a project where certain tasks must be completed before others can begin. Such dependencies can be modeled using directed graphs, where nodes represent tasks and directed edges represent dependencies between tasks. The goal of Topological Sort is to find a linear ordering of the vertices of the graph such that for every directed edge uv, the vertex u comes before the vertex v in the ordering. This ordering provides a sequence in which the tasks can be perf...
Read more