Spring AI with Ollama

Introduction to Spring AI with Ollama



Welcome to a comprehensive guide on leveraging Spring AI with Ollama to develop AI-driven applications using Java. This tutorial will cover everything from setting up Ollama locally, configuring your development environment, to creating an application that utilizes large language models for text generation.

Setting Up Ollama

Installing Ollama

To use Ollama locally, you must first install it on your machine. This involves downloading the Ollama software from the official repository and configuring it to run without Docker. Follow the installation instructions provided in the Ollama README file available on their official GitHub page.

Ollama Download link

Downloading the Mistral Model

After setting up Ollama, you can download the Mistral model directly from the Ollama management interface. Mistral is designed for a broad range of applications, offering robust text generation capabilities.


// Run this command in cli after downloading
// If running first time, first it download the model before running
// There are many models, let's start with 'mistral' 
ollama run mistral
Can check on browser

Command line



Exploring Other Available Models

Ollama supports several models, each with unique characteristics tailored to different tasks:

  • Orca-mini: Ideal for small-scale, quick-response applications.
  • Llama2: Suitable for more demanding tasks requiring deeper context understanding.
  • Llama3:Meta Llama 3: Next update after Llama2, More capable openly available LLM.
  • Custom models: Developed for specialized tasks; check the Ollama repository for more details.
  • To read more about all available models click this link: Ollama models library

Integrating Ollama with Spring AI: Maven and Gradle Dependencies
Maven Dependency Configuration

To integrate Ollama with Spring AI in a Maven project, you need to add the following dependency to your pom.xml file. This dependency will include the necessary Spring AI Ollama libraries in your project.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
</dependency>
Gradle Dependency Configuration

For Gradle-based projects, you will include the Spring AI Ollama dependency in your build.gradle file. This allows Gradle to manage the library and its associated dependencies.

// Add this to the dependencies block of your build.gradle
dependencies {
    implementation 'org.springframework.ai:spring-ai-ollama-spring-boot-starter'
}
Repository Configuration

Since Spring AI artifacts are typically published in the Spring Milestone and Snapshot repositories, you might need to add these repositories to your build file if the dependencies are not found in Maven Central. Here's how you can do it:

<repository>
    <id>spring-milestones</id>
    <name>Spring Milestones Repository</name>
    <url>https://repo.spring.io/milestone</url>
</repository>

// For Gradle, add this to your build.gradle
repositories {
    maven { url 'https://repo.spring.io/milestone' }
}
Example pom.xml Configuration 
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.3.0</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.codekatha.ai</groupId>
    <artifactId>spring-ai</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>spring-ai</name>
    <description>Spring AI backend</description>
    <properties>
        <java.version>17</java.version>
        <spring-ai.version>1.0.0-M1</spring-ai.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-ollama-spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-bom</artifactId>
                <version>${spring-ai.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
    <repositories>
        <repository>
            <id>spring-milestones</id>
            <name>Spring Milestones</name>
            <url>https://repo.spring.io/milestone</url>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>
</project>

These configurations ensure that your project is set up to utilize the latest Spring AI capabilities with Ollama, allowing you to build and run AI-enhanced applications seamlessly.

Application Properties Configuration

Add an application.properties file under the src/main/resources directory to enable and configure the Ollama chat model:


# Base URL where the Ollama API server is running
spring.ai.ollama.base-url=http://localhost:11434

# The model name to be used for text generation
spring.ai.ollama.chat.options.model=mistral

# The temperature setting controls the creativity of the model's responses
spring.ai.ollama.chat.options.temperature=0.7

Explanation of properties:

  • spring.ai.ollama.base-url: This property specifies the base URL where the Ollama API server is hosted. For local development, it might be http://localhost:11434. Replace this with your actual server URL if different.
  • spring.ai.ollama.chat.options.model: This property sets the model name to be used for generating text responses. In this case, it's set to mistral, which is one of the supported models.
  • spring.ai.ollama.chat.options.temperature: The temperature parameter controls the randomness of the model's output. A value of 0.7 means that the model will produce fairly creative responses without being too random. Lower values make the output more focused and deterministic, while higher values make it more random and creative.

These properties will create an OllamaChatModel implementation that you can inject into your classes and use to generate text responses based on input prompts.

Sample Controller

Here is an example of a simple @RestController class that uses the chat model for text generations.


package codeKatha;

import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.ollama.OllamaChatModel;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
import java.util.Map;

@RestController
public class ChatController {

    // Injecting OllamaChatModel via constructor
    private final OllamaChatModel chatModel;

    @Autowired
    public ChatController(OllamaChatModel chatModel) {
        this.chatModel = chatModel;
    }
    
    /**
     * Endpoint to generate text based on a given prompt.
     * @param prompt the input prompt to generate text
     * @return generated text response from the model
     */
    @GetMapping("/generateText")
    public String generateText(@RequestParam String prompt) {
        // Call the model with the prompt and return the generated content
        return chatModel.call(new Prompt(prompt)).getResult().getOutput().getContent();
    }


    /**
     * For This you may need to add flux dependency
     * Endpoint to generate a streaming response for a given message.
     * @param message the input message to generate a response
     * @return a Flux stream of ChatResponse containing the generated responses
     */
    @GetMapping("/generateStream")
    public Flux<ChatResponse> generateStream(@RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
        // Create a prompt from the user message and return a stream of responses
        Prompt prompt = new Prompt(message);
        return chatModel.stream(prompt);
    }
}

For API /generateText you not need this. But, for API /generateStream you may need to add flux and web dependecy in pom.xml, also need this configuration:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-web</artifactId>
</dependency>

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>


package codeKatha;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestClient;

@Configuration
public class RestClientConfig {

    @Bean
    public RestClient.Builder restClientBuilder() {
        return RestClient.builder();
    }
}

Conclusion

Integrating Spring AI with Ollama offers powerful capabilities for developing AI-powered applications. This guide provides the foundational knowledge needed to implement LLMs in your Java projects, enabling you to harness the potential of AI in your software solutions.

Also Read: Spring Boot with Open AI (Chat Gpt)

Related categories:

Comments

Post a Comment

Popular Posts on Code Katha

Java Interview Questions for 10 Years Experience

Image
Welcome to the comprehensive guide on "Java Interview Questions for 10 Years Experience." Java has been one of the most popular programming languages for decades, and its widespread adoption means that the demand for skilled Java developers has never been higher. If you are someone who has been working with Java for the last decade, you have accumulated a wealth of experience and knowledge that you can showcase during interviews. Recommended  to read and brush up for your interview: Must know interview questions and concept for any Java Interview Spring boot interview Questions for 10 years experiance These simple and short articles help you to add AI to your resume and enhance the chance of selection: Java Spring Boot AI with Ollama Java Spring Boot AI with Open AI (Chat GPT) This blog post will provide you with a list of commonly asked interview questions for 10 years experienced Java developers. It will also include detailed answers to each question, with explanat...
Read more

Sql Interview Questions for 10 Years Experience

Image
As an experienced SQL professional, you might encounter challenging questions during a job interview. In this blog post, we will explore 10 common SQL interview questions and provide in-depth explanations along with relevant examples. Let's dive in! Contents [ hide ] What is SQL Injection, and how can you prevent it? SQL Injection is a malicious attack where an attacker manipulates input data to execute unintended SQL statements. It can lead to unauthorized access, data breaches, and data manipulation. To prevent SQL Injection, follow these best practices: 1.1. Use Prepared Statements: Prepared statements or parameterized queries separate SQL code from data input, preventing attackers from injecting malicious SQL code. Here's an example in PHP: $stmt = $pdo->prepare('SELECT * FROM users WHERE username = :username'); $stmt->bindParam(':username', $username); $stmt->execute(); 1.2. Sanitize User Input: Validate and sanitize user input ...
Read more

Spring Boot Interview Questions for 10 Years Experience

Image
Welcome to the comprehensive guide on "Spring Boot Interview Questions for 10 Years Experience". Spring Boot has become one of the most popular frameworks for developing enterprise-grade applications, and its widespread adoption means that the demand for skilled Spring Boot developers has never been higher. If you are someone who has been working with Spring Boot for the last decade, you have accumulated a wealth of experience and knowledge that you can showcase during interviews. Recommended  to read and brush up for your interview: Must know interview questions and concept for any Java Interview Java Interview Questions for 10 Years Experience These simple and short articles help you to add AI to your resume and enhance the chance of selection: Spring Boot AI with Ollama Spring Boot AI with Open AI (Chat GPT) This blog post will provide you with a list of commonly asked interview questions for 10 years expe...
Read more

Java interview questions - Must to know concepts

Contents [ hide ] How does Java achieve platform independence, and what is the underlying principle? Java programs can run on diverse operating systems and hardware setups without requiring modification. This feat is primarily achieved through the concept of the Java Virtual Machine (JVM). The Java Virtual Machine (JVM) The JVM acts as a bridge between the Java code you write and the underlying hardware and operating system . When you compile your Java code, it is translated into an intermediate form called bytecode, which is not tied to any specific platform. The JVM then comes into play. It takes this bytecode and dynamically translates it into machine-specific instructions that the host system can execute. This means that a Java program can be written once and executed on any system with the appropriate JVM for that system. Example: Running a Java Program on Different Platforms Consider a simple Java program that calculates the sum of two numbers: package codeKatha...
Read more

Visual Studio Code setup for Java and Spring with GitHub Copilot

Image
Introduction- In this tutorial blog, we will explore the process of setting up and utilising Visual Studio Code for Java programming, as well as incorporating GitHub Copilot as a valuable coding tool. Visual Studio Code is a powerful and versatile code editor that offers a wide range of features and extensions for Java developers. By using Visual Studio Code for Java programming, developers can benefit from features such as code completion, debugging, and live coding. Additionally, with the integration of GitHub Copilot, developers can streamline their coding process with AI-assisted suggestions for code snippets and functions. This blog will provide a comprehensive guide for setting up and using these tools for Java development. Contents [ hide ] Setup Visual Studio Code for Java and Spring Visual Studio Code (VS Code) is an open-source code editor that supports a wide range of programming languages, including Java. It is a powerful tool...
Read more

Spring Data JPA

Image
Introduction- We will learn here Spring Data JPA, perform CRUD operation, Clear concept of Relationship mapping like one-to-many, many-to-one with Example. Spring Data JPA is a powerful framework that simplifies the process of working with relational databases in Java applications. It provides an easy-to-use API for working with JPA, allowing developers to focus on their business logic instead of worrying about the details of how JPA works. With Spring Data JPA, you can easily define entities that map to database tables, and then use them to perform CRUD (create, read, update, delete) operations on those tables. In addition, Spring Data JPA provides support for more advanced features like querying and pagination, making it an excellent choice for both small and large projects. Contents [ hide ] In this blog, we will explore Spring Data JPA by building a sample application that uses it to manage users and packs. But, before let...
Read more

Data Structures & Algorithms Tutorial with Coding Interview Questions

Image
About the Course This course offers comprehensive coverage of coding questions that serve as a solid foundation for understanding DSA concepts. These questions are frequently asked in coding interviews, either directly or with some variations, by many companies, including Adobe, Facebook (Meta), Paytm, Amazon, Google, and others. Our primary objective is to enhance your problem-solving skills and enable you to devise optimal solutions. The course includes a wide range of topics, such as Arrays, Strings, Searching and Sorting, Greedy Algorithms, Trees, Graphs, Dynamic Programming, and more. Upon completion, you will possess the necessary skills to ace DSA interviews for typical companies or excel in the initial rounds of product-based companies such as Google, Microsoft and similar others. We are started with Graph data Structure and add others soon. Contents [ hide ] Approach Our course is  organised into chapters ...
Read more

Java interview questions for 5 years experience

Contents [ hide ] What is difference between Heap and Stack Memory in Java, and shed light on their utilization? In Java, memory management plays a crucial role in determining how objects are stored and accessed during program execution . Heap and Stack Memory are two distinct regions where different types of data are managed. Understanding their differences is essential for efficient memory utilisation and managing object lifecycles. Stack Memory Stack Memory is a region used for storing method calls, local variables, and references to objects . It operates in a Last-In-First-Out (LIFO) manner , resembling a stack of items. Each method call creates a new frame in the stack, containing variables specific to that method. Stack Memory is relatively fast for allocation and deallocation because it follows a strict order. However, it has limited space and is typically used for small, short-lived data. Primitive data types and references to objects are often stored here. Ex...
Read more

Elasticsearch Java Spring Boot

Image
Introduction- In this blog, we will learned how to use Elasticsearch with Spring Boot to build a search application. We will created several classes as an example, including ElasticsearchConfig, Book, BookService, BookController, and BookRepository, to demonstrate how to use Elasticsearch with Spring Boot. After this tutorial, you can start building your own search application using Elasticsearch and Spring Boot. Elasticsearch is a powerful search and analytics engine that can be used to build highly scalable and fault-tolerant applications. In this blog, we will learn how to use Elasticsearch with Spring Boot to build a search application. First we will Set up a Spring Boot project with Elasticsearch after that we will see how to make connection with elasticsearch. Once connection setupt is done then we will add data then perform queries and searching on Elasticsearch. Contents [ hide ] Setting up a Spring Boot project with Elas...
Read more