Skip to content

Latest commit

Β 

History

16 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ’¬ Website AI Chatbot

A Multi-Tenant Website Chatbot built with Spring Boot, Spring AI, Gemini, Qdrant, MySQL, Playwright, Flyway, Spring Security & Docker

Java Spring Boot Spring AI Gemini MySQL Qdrant Playwright Flyway Spring Security Docker License


πŸ“– About

A multi-tenant AI website chatbot platform built with Spring Boot and Spring AI.

The system allows different websites to register their site, ingest their website content, store the content as vector embeddings in Qdrant, and provide an AI chatbot that answers questions using the website's own knowledge.

The chatbot uses Google Gemini for language generation and embeddings, Qdrant for vector search, and MySQL for site registration, conversations, administrators, and persistent chat memory.

JavaScript-rendered websites are processed using Playwright, allowing the ingestion system to extract content from modern React, Vue, Angular, and other client-rendered websites.

The project demonstrates practical implementation of modern AI application concepts including RAG, vector databases, embeddings, LLM integration, persistent AI memory, tenant isolation, website ingestion, browser automation, Spring Security, database migrations, and embeddable JavaScript widgets.


✨ Features

  • πŸ€– Google Gemini-powered AI chatbot
  • 🧠 Retrieval-Augmented Generation (RAG)
  • πŸ”Ž Semantic similarity search using Qdrant
  • 🧩 Gemini text embeddings for website knowledge
  • 🌐 Website content ingestion using Playwright
  • βš›οΈ Support for JavaScript-rendered websites
  • βœ‚οΈ Automatic document chunking before embedding
  • 🏒 Multi-site / multi-tenant architecture
  • πŸ” Site-specific public keys for chatbot identification
  • 🌍 Origin-based website authorization
  • πŸ’Ύ Persistent conversation memory using MySQL
  • 🧠 Spring AI MessageChatMemoryAdvisor
  • πŸ“š Spring AI QuestionAnswerAdvisor
  • 🎯 Site-specific Qdrant metadata filtering
  • πŸ‘€ MySQL-backed administrator authentication
  • πŸ”‘ Spring Security Basic Authentication for admin APIs
  • πŸ”’ Role-based admin authorization
  • πŸ—„οΈ MySQL persistence with Spring Data JPA
  • πŸ”„ Flyway database migrations
  • ⚠️ Global API exception handling
  • πŸ’¬ Embeddable JavaScript chatbot widget
  • πŸ“± Responsive chatbot UI
  • 🐳 Docker / Docker Compose support

πŸ—οΈ Application Flow

Main flow

  1. Administrator registers a website.
  2. Application generates a unique siteId and public key.
  3. Website information is stored in MySQL.
  4. Administrator starts website ingestion.
  5. Playwright opens the registered website in a headless Chromium browser.
  6. JavaScript-rendered content is extracted from the page.
  7. Extracted content is converted into Spring AI Document objects.
  8. Documents are split into smaller chunks.
  9. Gemini generates embeddings for each chunk.
  10. Embeddings are stored in Qdrant with the corresponding siteId.
  11. A visitor opens the website chatbot widget.
  12. The widget sends the public key, conversation ID, and user message to the backend.
  13. The backend validates the website origin and resolves the registered site.
  14. Spring AI retrieves conversation memory from MySQL.
  15. Spring AI retrieves relevant website knowledge from Qdrant.
  16. Gemini receives the user question, conversation context, and retrieved website knowledge.
  17. Gemini generates the final response.
  18. The response is returned to the JavaScript chatbot widget.

🧠 RAG Architecture

                         User Question
                               β”‚
                               β–Ό
                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β”‚   ChatController   β”‚
                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
                    Public Key + Origin
                              β”‚
                              β–Ό
                 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                 β”‚ Origin Validation       β”‚
                 β”‚ + Site Resolution       β”‚
                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
                  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                  β”‚                       β”‚
                  β–Ό                       β–Ό
            Chat Memory                Qdrant
               MySQL                Site Knowledge
                  β”‚                       β”‚
                  β”‚                 Similarity Search
                  β”‚                       β”‚
                  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                              β”‚
                              β–Ό
                    QuestionAnswerAdvisor
                              β”‚
                              β–Ό
                         ChatClient
                              β”‚
                              β–Ό
                       Google Gemini
                              β”‚
                              β–Ό
                           Response

🌐 Website Ingestion Flow

          Registered Website URL
                    β”‚
                    β–Ό
             Playwright
                    β”‚
             Chromium Browser
                    β”‚
                    β–Ό
            Execute JavaScript
                    β”‚
                    β–Ό
             Rendered Web Page
                    β”‚
                    β–Ό
               Extract Text
                    β”‚
                    β–Ό
              Spring Document
                    β”‚
                    β–Ό
             TokenTextSplitter
                    β”‚
                    β–Ό
              Text Chunks
                    β”‚
                    β–Ό
          Gemini Embedding Model
                    β”‚
                    β–Ό
           3072-Dimensional Vector
                    β”‚
                    β–Ό
                  Qdrant
                    β”‚
          Metadata: siteId

πŸ”Ž Retrieval Flow

When a user asks a question:

User
 β”‚
 β”‚ "What projects are listed?"
 β–Ό
ChatClient
 β”‚
 β”œβ”€β”€ MessageChatMemoryAdvisor
 β”‚       β”‚
 β”‚       └── MySQL conversation history
 β”‚
 └── QuestionAnswerAdvisor
         β”‚
         └── Qdrant similarity search
                 β”‚
                 └── siteId filter
                         β”‚
                         β–Ό
                  Relevant Documents
                         β”‚
                         β–Ό
                       Gemini
                         β”‚
                         β–Ό
                      Answer

The chatbot does not rely only on Gemini's general knowledge. Website-specific questions are grounded using the indexed website content.


πŸ” Multi-Tenant Architecture

Each registered website receives:

siteId
publicKey
domain

Example:

siteId:
site-c940c842

publicKey:
pk_96a828c3aee54d6b832a7c0c873062dd

domain:
https://www.raguls4.vercel.app

Website knowledge is stored in Qdrant with:

siteId = site-c940c842

When retrieving information, the chatbot applies the corresponding site filter.

This prevents knowledge belonging to one website from being retrieved for another website.


πŸ” Website Origin Validation

The chatbot widget sends:

X-Chatbot-Public-Key: pk_...
Origin: https://example.com

The backend:

Public Key
    ↓
Find Site
    ↓
Compare request Origin
    ↓
Registered Domain
    ↓
Allowed?

Example:

Registered:
https://www.raguls4.vercel.app

Request:
https://www.raguls4.vercel.app

                ↓

             βœ… Allowed

A different website using the same public key is rejected.


πŸ”‘ Admin Authentication Flow

Administrative endpoints are protected using Spring Security and database-backed users.

Admin
 β”‚
 β”‚ Basic Authentication
 β–Ό
Spring Security
 β”‚
 β–Ό
DaoAuthenticationProvider
 β”‚
 β–Ό
CustomUserDetailsService
 β”‚
 β–Ό
MySQL admin_users
 β”‚
 β–Ό
PasswordEncoder
 β”‚
 β–Ό
ROLE_ADMIN
 β”‚
 β–Ό
Admin API

Public chatbot requests do not require administrator credentials.


πŸ’Ύ Conversation Memory

The chatbot stores conversation history using Spring AI's JDBC chat memory repository.

User Message
     β”‚
     β–Ό
MessageChatMemoryAdvisor
     β”‚
     β–Ό
MySQL
     β”‚
     β–Ό
SPRING_AI_CHAT_MEMORY

Conversation identifiers are generated by the widget using UUIDs and remain site-specific on the client side.

Conversation ownership is additionally maintained through the application's conversations table.


πŸ“Œ Results

Website Registration

POST /api/sites
{
  "name": "Ragul Portfolio",
  "domain": "https://raguls4.vercel.app/"
}

Example response:

{
  "siteId": "site-64c1f1ff",
  "publicKey": "pk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "name": "Ragul Portfolio",
  "domain": "https://raguls4.vercel.app/"
}

Website Ingestion

POST /api/sites/{siteId}/ingest

Example response:

{
  "siteId": "site-64c1f1ff",
  "pagesDiscovered": 1,
  "pagesIndexed": 1,
  "chunksCreated": 2,
  "failedPages": 0,
  "status": "COMPLETED"
}

Chat

POST /api/chat

Headers:

X-Chatbot-Public-Key: pk_xxxxxxxxx
Content-Type: application/json

Request:

{
  "conversationId": "550e8400-e29b-41d4-a716-446655440000",
  "message": "What do you know about this website?"
}

Response:

{
  "response": "This website is the personal portfolio of Ragul S..."
}

🌐 REST API

Public APIs

Method Endpoint Description
POST /api/chat Chat with the website-specific AI assistant
GET /widget.js Serve embeddable chatbot widget

Admin APIs

All site-management APIs require ROLE_ADMIN.

Method Endpoint Description
POST /api/sites Register a website
POST /api/sites/{siteId}/ingest Ingest website content

πŸ“₯ Sample Chat Request

{
  "conversationId": "550e8400-e29b-41d4-a716-446655440000",
  "message": "What projects are available?"
}

Header:

X-Chatbot-Public-Key: pk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

πŸ“€ Sample Chat Response

{
  "response": "The website contains several projects including a URL Shortener, Chatter, Digital Banking System, and Secure GatePass Management System."
}

🧩 Embeddable Chatbot Widget

The chatbot can be embedded into a website using a single script:

<script
    src="https://your-chatbot-domain.com/widget.js"
    data-site-key="pk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx">
</script>

The widget automatically:

  • Loads the site's chatbot configuration
  • Creates a site-specific conversation ID
  • Displays the chatbot UI
  • Sends messages to the Spring Boot backend
  • Displays AI responses
  • Preserves the conversation ID across page refreshes

πŸ› οΈ Tech Stack

Category Technology
Language Java 21
Backend Framework Spring Boot 4.1.0
AI Framework Spring AI 2.0.0
AI Model Google Gemini
Embedding Model Gemini Embedding
Vector Database Qdrant
Database MySQL 8
ORM Spring Data JPA / Hibernate
AI Memory Spring AI JDBC Chat Memory
Website Rendering Playwright
Security Spring Security
Authentication HTTP Basic Authentication
Password Hashing BCrypt / Delegating Password Encoder
Database Migration Flyway
Frontend Widget Vanilla JavaScript
Containerization Docker / Docker Compose
Utilities Lombok

πŸ“‚ Project Structure

src
β”œβ”€β”€ main
β”‚   β”œβ”€β”€ java
β”‚   β”‚   └── com.ragul.ChatBot
β”‚   β”‚       β”œβ”€β”€ config
β”‚   β”‚       β”œβ”€β”€ controller
β”‚   β”‚       β”œβ”€β”€ dto
β”‚   β”‚       β”œβ”€β”€ entity
β”‚   β”‚       β”œβ”€β”€ exception
β”‚   β”‚       β”œβ”€β”€ repository
β”‚   β”‚       β”œβ”€β”€ security
β”‚   β”‚       β”œβ”€β”€ service
β”‚   β”‚       └── util
β”‚   β”‚
β”‚   └── resources
β”‚       β”œβ”€β”€ application.yml
β”‚       β”œβ”€β”€ static
β”‚       β”‚   └── widget.js
β”‚       └── db
β”‚           └── migration
β”‚               β”œβ”€β”€ V1__create_application_tables.sql
β”‚               └── V2__create_admin_users.sql
β”‚
β”‚
β”œβ”€β”€ Dockerfile
β”œβ”€β”€ docker-compose.yml
└── pom.xml

πŸ—„οΈ Database

Flyway manages the application's database schema.

Main Tables

sites

Stores registered websites and chatbot configuration.

id
site_id
public_key
name
domain

conversations

Associates conversations with their registered websites.

id
site_id
conversation_id

The pair:

site_id + conversation_id

is unique.


admin_users

Stores administrative users.

id
username
password
role
enabled

Passwords are stored using a password encoder.


SPRING_AI_CHAT_MEMORY

Spring AI's JDBC repository stores conversation messages.

conversation_id
content
type
timestamp
sequence_id

🧠 Vector Database

Qdrant stores embeddings generated from website content.

Collection:

website_knowledge

Each vector contains the associated website identifier as metadata:

siteId

This allows site-specific retrieval.

Example:

Question
   ↓
Embedding
   ↓
Qdrant similarity search
   ↓
siteId filter
   ↓
Relevant website chunks

βš™οΈ Configuration

Configure environment variables:

DB_PASSWORD=password
GEMINI_API_KEY=your-gemini-api-key
ADMIN_USERNAME=admin
ADMIN_PASSWORD=your-admin-password

Never commit real API keys, passwords, or .env files to the repository. Use .env.example for documentation.


⚑ application.yml Example

server:
  port: 8080

spring:
  application:
    name: ChatBot

  datasource:
    url: jdbc:mysql://localhost:3307/chatbot
    username: root
    password: ${DB_PASSWORD}
    driver-class-name: com.mysql.cj.jdbc.Driver

  jpa:
    hibernate:
      ddl-auto: validate
    show-sql: false

  flyway:
    enabled: true
    locations: classpath:db/migration

  ai:
    google:
      genai:
        api-key: ${GEMINI_API_KEY}

        chat:
          model: gemini-3.5-flash
          temperature: 0.7

        embedding:
          api-key: ${GEMINI_API_KEY}

          text:
            model: gemini-embedding-001

    vectorstore:
      qdrant:
        host: localhost
        port: 6334
        collection-name: website_knowledge
        use-tls: false
        initialize-schema: true

    chat:
      memory:
        repository:
          jdbc:
            initialize-schema: always

app:
  security:
    development-origins:
      - http://localhost:5173
      - http://localhost:3000

  admin:
    username: ${ADMIN_USERNAME}
    password: ${ADMIN_PASSWORD}

πŸš€ Running Locally

Clone Repository

git clone https://github.com/<your-username>/WebsiteAIChatbot.git
cd WebsiteAIChatbot

Configure Environment Variables

Create a .env file or configure the variables in your environment:

DB_PASSWORD=your-password
GEMINI_API_KEY=your-gemini-api-key
ADMIN_USERNAME=admin
ADMIN_PASSWORD=your-admin-password

Start MySQL & Qdrant

Make sure Docker is installed and running.

Start the required services:

docker compose up -d

Run Application

Using Maven:

./mvnw spring-boot:run

or:

mvn spring-boot:run

🐳 Docker

Build and start all services:

docker compose up --build

Run in detached mode:

docker compose up -d

View running containers:

docker compose ps

View logs:

docker compose logs -f

Stop containers:

docker compose down

🧩 Docker Services

Service Container Port Purpose
MySQL mysql_chatbot 3307 Application database
Qdrant qdrant_chatbot 6333 / 6334 Vector database
Backend chatbot_backend 8080 Spring Boot API and AI service

πŸ”„ AI Request Architecture

                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                    β”‚    Website    β”‚
                    β”‚    Visitor   β”‚
                    β””β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”˜
                            β”‚
                       widget.js
                            β”‚
                            β–Ό
                  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                  β”‚   Spring Boot    β”‚
                  β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                           β”‚
                  Origin + Public Key
                           β”‚
                           β–Ό
                 β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                 β”‚ Site Validation     β”‚
                 β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β”‚
               β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
               β”‚                         β”‚
               β–Ό                         β–Ό
         MySQL Memory                Qdrant
               β”‚                         β”‚
               β”‚                    Site Filter
               β”‚                         β”‚
               β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                            β–Ό
                     Spring AI
                            β”‚
                 QuestionAnswerAdvisor
                            β”‚
                            β–Ό
                       ChatClient
                            β”‚
                            β–Ό
                      Google Gemini
                            β”‚
                            β–Ό
                         Response

πŸ›‘οΈ Error Handling

The application uses a global exception handler to return consistent API errors.

Example:

{
  "timestamp": "2026-08-18T16:07:12",
  "status": 422,
  "error": "INGESTION_FAILED",
  "message": "Website ingestion failed.",
  "path": "/api/sites/site-c940c842/ingest"
}

Common responses include:

400 β†’ VALIDATION_ERROR
401 β†’ Unauthorized
403 β†’ ORIGIN_NOT_ALLOWED
404 β†’ SITE_NOT_FOUND
422 β†’ INGESTION_FAILED
500 β†’ INTERNAL_SERVER_ERROR

πŸ” Security Model

The application separates public chatbot access from administrative access.

Public chatbot

publicKey
+
Origin
+
conversationId

The backend verifies that the website origin matches the registered site.

Admin APIs

Username
+
Password
+
ROLE_ADMIN

are required for site registration and website ingestion.

This provides two separate security boundaries:

Website Visitor
      ↓
Origin + Public Key

Administrator
      ↓
Spring Security + ADMIN role

πŸ§ͺ Testing

Run the Maven test suite:

./mvnw test

or:

mvn test

The application can also be manually tested through Postman or the embedded chatbot widget.


⚠️ Current Limitations

The current ingestion pipeline processes the configured website URL as a rendered page using Playwright.

The current implementation does not yet provide:

  • Automatic multi-page website crawling
  • Incremental website re-indexing
  • Background ingestion jobs
  • PDF/document ingestion
  • Advanced retrieval reranking
  • Streaming AI responses

These can be added as future improvements.


πŸ“ˆ Future Enhancements

  • πŸ•·οΈ Multi-page website crawling
  • πŸ”„ Incremental knowledge re-indexing
  • πŸ“„ PDF and document ingestion
  • ⚑ Background ingestion jobs
  • πŸ” JWT/OAuth2 admin authentication
  • πŸ‘₯ Multiple administrators with site-level permissions
  • 🎨 More widget customization
  • 🌊 Streaming AI responses
  • πŸ“Š Usage and token analytics
  • πŸ§ͺ Expanded unit and integration test coverage
  • πŸ“š Swagger/OpenAPI documentation
  • πŸ“ˆ Monitoring with Prometheus and Grafana
  • ☁️ Production deployment
  • 🐳 Production container hardening
  • ☸️ Kubernetes deployment

🎯 Project Goals

This project was built to explore practical implementation of:

Spring Boot
     +
Spring AI
     +
Large Language Models
     +
Embeddings
     +
Vector Databases
     +
RAG
     +
AI Memory
     +
Multi-Tenant Architecture
     +
Website Ingestion
     +
Spring Security

The goal is to understand how modern AI capabilities can be integrated into a conventional backend application rather than building an isolated AI demo.


🀝 Contributing

Contributions, suggestions, and improvements are welcome.

  1. Fork the repository
  2. Create a feature branch
  3. Commit your changes
  4. Push the branch
  5. Open a Pull Request

πŸ“„ License

This project is licensed under the MIT License.


⭐ If you found this project useful, consider giving it a star!

Built with ❀️ using Spring Boot, Spring AI, Gemini, Qdrant, MySQL, Playwright, Flyway, Spring Security, and Docker

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages