Deep Chat JS, semantic search, and OpenAI integration

Building on the foundation laid in the previous article, this post dives into integrating Deep Chat JS component for real-time, context-aware conversations. By leveraging semantic search on your vector database, you can retrieve the most relevant information faster than ever.

Building a robust chat component from the ground up can be both time-consuming and technically challenging, especially if you want to handle real-time interactions, dynamic content, and a smooth user experience. Rather than reinventing the wheel, I decided to explore existing solutions to save development time and reduce complexity. That's when I discovered Deep Chat JS, feature-rich component that provides the functionality and flexibility needed to create a seamless chat interface without starting from scratch.

If we revisit the image from the first article in the series, where we discussed data vectorization using the OpenAI Embedding API, we can see that this article corresponds to the part of the diagram labeled with the number 2.

Given that our data has been converted into vectors (in my case, the pages of this website were vectorized and stored in a Qdrant vector database), we can now perform semantic searches. As shown in the diagram above, the user input—typically a question in a chatbot scenario—also needs to be converted into vectors. In my case, I accomplished this using the OpenAI Embeddings API and a model called text-embedding-3-large, but technically, this could be done with any other closed-source or open-source model.

Once we convert the user input into vectors, we then need to query the vector database. Qdrant provides an API where we pass in the vectorized query and specify how many similar results we want to retrieve. In the world of RAG (Retrieval-Augmented Generation) applications, this parameter is commonly referred to as the Top K parameter.

Once Qdrant returns the results, these documents are the ones closest to the user's query. Here, the search is not based on keywords but rather on proximity within the vector space—meaning that the closer two vectors are, the more similar their meanings. Now that we have these results, we bundle them together with the user's query and any system instructions, then send them to the OpenAI Chat API. This way, the AI can formulate its answer using the specific context we've provided, rather than relying solely on its own internal knowledge base. Essentially, we supply the context from which we expect the LLM to extract and present a well-structured answer.

The OpenAI Chat API will return a response to the user's question, which we then pass to the Deep Chat JS component. It looks like this:

Integrating the Deep Chat JS library into my Drupal website is super simple—just a few parameters and a URL where it should send messages, and that's it.

/**
 * Implements hook_page_bottom().
 */
function ai_chat_page_bottom(array &$page_bottom): void {
  if (!_ai_chat_is_accessible()) {
    return;
  }
  
  $module_path = \Drupal::service('extension.list.module')->getPath('ai_chat');
  
  $page_bottom['ai_chat'] = [
    [
      '#type' => 'inline_template',
      '#template' => '
        <div class="ai-chat-container">
          <div class="ai-chat-header">
            <span class="ai-chat-title" title="{{ ai_chat_hint }}"><span class="ai-chat-bullet"></span> {{ ai_chat_title }}</span>
            <button class="ai-chat-toggle">
              <img src="/' . $module_path . '/icons/chevron-up.png" alt="Toggle chat" height="16" width="16" class="ai-chat-icon" />
            </button>
          </div>
          <div class="ai-chat-window">
            <div id="deep-chat-container">
              <deep-chat
                connect=\'{"url": "/api/ai-chat", "method": "POST", "headers": {"Content-Type": "application/json"}}\'
                textInput=\'{"styles": {"container": {"marginBottom": "25px"}}, "placeholder": {"text": "Ask me anything!"}}\'
                submitButtonStyles=\'{"submit": {"container": {"default": {"marginBottom": "15px"}}}}\'
                style="width: 500px; border-radius: 8px 8px 0 0; box-shadow: 0 4px 12px rgba(0, 0, 0, 0.15); z-index: 9999;"
                introMessage=\'{"text": "{{ intro_message }}"}\'
                requestBodyLimits=\'{"maxMessages": 0}\'
                avatars="true"
              >
              </deep-chat>
            </div>
          </div>
        </div>
      ',
      '#context' => [
        'ai_chat_title' => t('AI Assistant'),
        'ai_chat_hint' => t('Use Ctrl + / to open and close the chat window.'),
        'intro_message' => t('Hi, I am your AI Assistant, here to help you explore articles, code snippets, and everything about the site and the author!'),
      ],
    ],
    '#cache' => [
      'contexts' => ['user.permissions'],
    ],
  ];
}

The chat has a built-in rate limiter to prevent bots from exploiting it. It uses a rolling window rate limit of n requests per n seconds per user IP address.

In conclusion, integrating Deep Chat JS with semantic search and OpenAI's API allowed me to build a responsive, context-aware chatbot with minimal hassle. I found that using vectorized data really helped ensure that each query received a precise and relevant answer. This approach not only enhances the user experience but also opens up exciting possibilities for future innovations in conversational AI. I'm excited to see where these technologies take me next.