Integrated Chat-Bot Based on NLP and GPT

"By combining NLP and GPT capabilities, Waverley created a chat-bot that understands users naturally, works with any data source, and can be integrated into any messenger platform."

A forward-thinking enterprise needed a smart chatbot capable of natural human-like interaction through any messenger platform, powered by generative AI and large language models. Waverley created a comprehensive chatbot system that integrates OpenAI's GPT models with locally-running open-source LLMs for cost efficiency and privacy protection. The solution enables businesses to provide customer assistance, internal employee support, or e-commerce guidance through conversational interfaces while maintaining strict control over sensitive data.

Waverley engineered a sophisticated, multi-layered system combining OpenAI's powerful generative models with locally-running open-source alternatives, creating a hybrid architecture that balances capability, cost, privacy, and availability.

Technology Stack

• Backend & Orchestration: GoLang, Python

• Frontend & Integration: React, LangChain

• Vector Databases: Redis

• Data Storage: MongoDB, Amazon S3

• LLM Models: GPT-3.5, GPT-4, Llama, Llama 2, open-source alternatives

Data Collection and Processing Pipeline

The system includes a set of data collectors and microservices (written in GoLang and Python) that gather information from provided databases, files, websites, and other sources, then process it into usable formats. Large documents are split into digestible chunks using ML techniques. Text segments are transformed into embeddings, unique numeric vectors representing meaning, which are stored in vector databases like Redis. This embedding approach enables efficient semantic search without requiring expensive full-text processing on every query.

Context Definition Through Vector Similarity Search

When users interact with the chatbot using natural language, their input is processed by the same ML model and converted to a numeric vector. This vector is used in vector similarity search within the database to find the most relevant text segments or documents. This process specifies the context that will guide OpenAI's response generation, ensuring answers are grounded in actual data rather than fabricated.

Intelligent Caching for Cost Efficiency

To reduce costs on OpenAI API calls, the system performs vector similarity search in a cache database first. If a user's question is semantically similar to a previously-answered question (indicated by similar numeric vectors), the system retrieves the previously-generated response instead of making a new API call. This optimization can dramatically reduce operational costs for high-volume chatbot deployments.

Privacy-First Architecture

Recognizing that many organizations restrict external AI tools due to privacy concerns, Waverley implemented a dual-processing approach: open-source models (Llama, Llama 2, GPT4All) run locally for initial data processing, classification, and summarization. Only processed, appropriately-filtered information is sent to OpenAI. Additionally, the system includes manual classification to mark information as confidential or non-confidential, filtering out sensitive data before it reaches external services. For internal company usage, responses are further filtered based on user role and access level.

Multi-Interface API Gateway

The client-side API Gateway (implemented in GoLang) provides multiple communication interfaces: WebSocket APIs for messenger integrations (e.g., Slack), REST APIs for web interfaces, and gRPC APIs for mobile clients. The gateway maintains local caching and audit logging for troubleshooting and compliance.

Intelligent Chatbot Engine

The core engine performs multiple functions: intent classification to understand user goals and initiate appropriate actions; transformation of user input into embeddings for context retrieval; communication with various storage systems (document databases, S3 buckets) to retrieve data related to embeddings; sentiment analysis to detect user satisfaction and route to human operators when needed; and integration with third-party APIs to automate business operations like purchase completion or support ticket creation through decision-making and action-executor services.

Defining the Context — When a user interacts with the chat-bot, their input is processed by the ML model and transformed into a numeric vector. Vector similarity search finds the most relevant documents, defining the context for OpenAI's natural language response.

Caching — A cache vector database stores prior responses. Similar user queries are matched without sending new requests to OpenAI, reducing costs.

Interface — A GoLang API Gateway provides WebSocket (messengers), REST (web), and gRPC (mobile) interfaces with local cache and audit logs.

Intent classification understands what users want; sentiment analysis detects satisfaction and redirects to operators when needed.

The chat-bot provides the following functionality:

• Recognise the intention of a user

• Identify the context of interaction

• Answer specific questions from available data sources

• Summarise requested documents, webpages, etc.

• Provide a reference to the source of information

• Translate text

• Generate text (emails, posts, etc.) following user instructions

• Integrate with third-party systems (e.g. Jira) for business automation

Waverley developed a production-ready chatbot system relying on OpenAI's capabilities but capable of functioning autonomously using locally-run ML models, providing significant advantages: enhanced information security for privacy-conscious organizations, compliance with data protection regulations, operational independence in regions where OpenAI is restricted, and reduced API costs through intelligent caching and hybrid processing.

The system is adaptable to diverse business needs, offering more control over confidentiality settings compared to direct OpenAI implementation, and proves more cost-efficient at scale. The hybrid approach transforms generative AI from a one-size-fits-all cloud service into a customizable enterprise capability.