Author: Yuri Gee
Date: 12 Aug 2025
10 minute read time
In my previous article, I explored several examples and potential use cases for integrating SugarCRM with AI and natural language prompts. One of the key scenarios outlined was semantic search. In this follow-up, I’ll walk through a simple prototype that demonstrates how semantic search can be integrated with SugarCRM’s Global Search.
AI can be conceptualized as a multi-dimensional function that transforms inputs—such as document content or search queries—into vectors within a latent space. When the latent space is properly aligned using suitable encoders for both the content and the query, it becomes possible to compare their vectors by measuring directional closeness (e.g., cosine similarity), magnitude, or dot products. This enables the identification of semantically relevant matches between queries and documents.
This principle underpins the retrieval step in RAG (Retrieval-Augmented Generation), a technique that enables ranking embedding vectors—derived from external content—based on similarity or other latent-space features, allowing the model to retrieve the most relevant results for downstream generation.
Summary of implementation
To generate embeddings, this prototype uses the Google Gemini Embeddings 001 model. Please note that any indexed content will be processed through this model. If you're working with real data, it's recommended to obfuscate or preprocess the content—using local models or custom code—to extract meaningful, depersonalized information before submitting it to a general-purpose model. That said, many general-purpose models offer configuration options to delete embeddings after generation.
Once embeddings are created for an indexed record (e.g., in the Knowledge Base module), they are stored in a dedicated SearchIndex module, which is custom-built using Module Builder.
During search execution, the user's prompt is converted into an embedding vector using the same model, optimized for query tasks. This query vector is then compared against all stored embeddings in the SearchIndex using cosine similarity—which measures directional differences in latent space. The top-matching vectors are retrieved, and their associated record ID, name, and module (e.g., KB article) are displayed in the search results.
Sample Query, Content, and Corresponding Embeddings
1. How to Schedule a Follow-Up Call with a Lead Learn how to use the CRM calendar to schedule follow-up calls after initial outreach. This guide walks you through selecting a time, adding notes, and setting reminders.
Queries: set reminder to reconnect with lead or organize post-demo engagement
Embedding example converted to base64: G7ihvJrEgTxCzTg74gJ+v...KSLpAxhs6
Embedding example in original dimension (3072 float32 numbers): [ -0.019741109, 0.015840817, ... 0.00059423223 ]
2. Tagging Leads for Targeted Campaigns Discover how to apply tags to leads based on interest or engagement. Tags like “Webinar Attendee” or “Interested in Product X” help you segment your audience for personalized outreach.
Queries: categorize prospects for outreach OR mark contacts for seasonal offer OR group leads by interest level
3. Content: Logging Positive Customer Feedback Capture and record customer compliments or success stories in the CRM. This article shows how to log feedback, link it to customer profiles, and use it for testimonials.
Queries: capture customer praise OR document client success story OR store positive interaction notes
This is how it appears in the UI when using the additional search button within the Global Search layout.
For prototyping purposes, the search index module (/SI_SearchIndex) is enabled in both Studio and List views for created indexes.
This is how a KB record can be indexed, or re-indexed.
Code Prototype
Contains two view controllers, one HBS file for rendering Quick Search and KB Module Index buttons, and two layout files for the respective elements.
./custom/clients/base/views/quicksearch-button2/quicksearch-button2.js
./custom/clients/base/views/quicksearch-button2/quicksearch-button2.hbs
./custom/clients/base/layouts/quicksearch/quicksearch.php
./custom/modules/KBContents/clients/base/views/record/record.js
./custom/modules/KBContents/clients/base/views/record/record.php
/**
* * ./custom/clients/base/views/quicksearch-button2/quicksearch-button2.js
* @class View.Fields.Base.QuicksearchButtonView
* @alias SUGAR.App.view.fields.BaseQuicksearchButtonView
* @extends View.View
*/
({
extendsFrom: 'BaseQuicksearchButtonView',
events: {
'click [data-action=search_icon2]': 'searchIconClickHandler2',
'click [data-action=search_icon]': 'searchIconClickHandler',
'mouseover button': 'onOver',
'mouseout button': 'onOut'
},
apiKey: "",
endpoint: "",
shouldShowResults: true,
/**
* @inheritdoc
*/
initialize: function(options) {
this._super('initialize', [options]);
this.apiKey = 'YOUR_API_KEY';
this.endpoint = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:embedContent';
},
render: function() {
this._super('render');
},
//this should be placed in Common library/backend to be reused across controllers
indexSearchValue: async function (text, task) {
const payload = {
model: 'gemini-embedding-001',
content: { parts: [{ text }] },
taskType: task
};
const res = await fetch(`${this.endpoint}?key=${this.apiKey}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
const data = await res.json();
const embedding = data.embedding?.values;
if (!embedding) throw new Error('No embedding returned');
const floatArray = new Float32Array(embedding);
const byteArray = new Uint8Array(floatArray.buffer);
return btoa(String.fromCharCode(...byteArray));
},
/**
* Handler for clicks on the search icon (or x, depending on state).
*/
searchIconClickHandler2: async function() {
const input = document.querySelector('input.search-query');
if (input && input.value.trim() !== '') {
app.alert.show('searching_popup', {
level: 'process',
title: 'Searching...',
messages: 'Please wait while we process your request.',
autoClose: false
});
try
{
const searchTagIndex = await this.indexSearchValue(input.value.trim(), "RETRIEVAL_QUERY");
app.api.call('read', app.api.buildURL('SI_SearchIndex', null, null, {
fields: 'record_id,name_of_module,record_name,index_value',
max_num: 100
}), null, {
success: (data) => {
let results = [];
if (Array.isArray(data.records) && data.records.length > 0)
{
results = data.records.map(record => ({
moduleName: record.name_of_module,
recordId: record.record_id,
recordName: record.record_name,
index_value: record.index_value
}));
}
let ranked_results = this.rankResults(results, searchTagIndex, 5);
// Pass to your display function
app.alert.dismiss('searching_popup');
this.displayResults(ranked_results);
},
error: function(err) {
app.alert.dismiss('searching_popup');
console.error('Error fetching SearchIndex:', err);
}
});
}
catch (error)
{
app.alert.dismiss('searching_popup');
}
}
},
displayResults: function(results) {
const wrapper = document.querySelector('div.typeahead-wrapper ');
if (wrapper)
{
const existingResults = wrapper.querySelectorAll('.typeahead.dropdown-menu.search-results');
if (this.shouldShowResults) {
existingResults.forEach(el => el.remove());
const ul = document.createElement('ul');
ul.className = 'dropdown-menu search-results typeahead py-1.5';
ul.style.display = 'block';
if (results.length === 0) {
const li = document.createElement('li');
li.className = 'search-result';
li.textContent = 'No search results';
ul.appendChild(li);
} else {
results.forEach(({ moduleName, recordId, recordName }) => {
const li = document.createElement('li');
li.className = 'search-result';
const a = document.createElement('a');
a.className = 'flex-wrap';
a.href = `#${moduleName}/${recordId}`;
a.tabIndex = -1;
a.textContent = `${moduleName} - ${recordName}`;
li.appendChild(a);
ul.appendChild(li);
});
}
wrapper.appendChild(ul);
this.shouldShowResults = false;
} else {
// Explicitly remove any lingering search results
existingResults.forEach(el => el.remove());
this.shouldShowResults = true;
}
}
},
// TO_NOTE: javascript will convert this to Float64
decodeBase64Embedding: function (base64) {
const binary = atob(base64);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
// TO_NOTE: may not need Array.from as Float array can be used for cosineSimilarity
return Array.from(new Float32Array(bytes.buffer));
},
cosineSimilarity: function (a, b) {
let dot = 0, magA = 0, magB = 0;
for (let i = 0; i < a.length; i++) {
dot += a[i] * b[i];
magA += a[i] * a[i];
magB += b[i] * b[i];
}
if (magA === 0 || magB === 0) return 0;
return dot / (Math.sqrt(magA) * Math.sqrt(magB));
},
rankResults: function (results, queryEmbedding, topK) {
const queryVec = this.decodeBase64Embedding(queryEmbedding);
const scored = results.map(r => {
let score = 0;
try {
const docVec = this.decodeBase64Embedding(r.index_value);
score = this.cosineSimilarity(queryVec, docVec);
} catch (error) {
score = 0;
}
return { ...r, score };
});
scored.sort((a, b) => b.score - a.score);
return scored.slice(0, topK);
}
});
{{!--
/*
* ./custom/clients/base/views/quicksearch-button2/quicksearch-button2.hbs
--}}
<button class="quicksearch-button relative" aria-label="{{str 'LBL_GLOBAL_SEARCH_RUN' module}}" data-action="search_icon">
<i class="sicon sicon-search"></i>
</button>
<button class="quicksearch-button relative" aria-label="Search" data-action="search_icon2">
<i class="sicon sicon-search"></i>
</button>
<?php
//./custom/clients/base/layouts/quicksearch/quicksearch.php
$viewdefs['base']['layout']['quicksearch'] = [
'components' => [
[
'view' => 'quicksearch-modulelist',
],
[
'view' => 'quicksearch-taglist',
],
[
'view' => 'quicksearch-bar',
],
[
'view' => 'quicksearch-button2',
],
[
'view' => 'quicksearch-tags',
],
[
'view' => 'quicksearch-results',
],
],
'v2' => true,
];
/**
* * ./custom/modules/KBContents/clients/base/views/record/record.js
*/
({
extendsFrom: 'RecordView',
apiKey: "",
endpoint: "",
initialize: function(options) {
this._super('initialize', [options]);
// Add your custom button to the layout once the view is rendered
this.context.on('button:custom_action:click', this.handleCustomAction, this);
this.apiKey = 'YOUR_API_KEY';
this.endpoint = 'https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:embedContent';
},
//this should be placed in Common library/backend to be reused across controllers
indexSearchValue: async function (text, task) {
const payload = {
model: 'gemini-embedding-001',
content: { parts: [{ text }] },
taskType: task
};
const res = await fetch(`${this.endpoint}?key=${this.apiKey}`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload)
});
const data = await res.json();
const embedding = data.embedding?.values;
if (!embedding) throw new Error('No embedding returned');
const floatArray = new Float32Array(embedding);
const byteArray = new Uint8Array(floatArray.buffer);
return btoa(String.fromCharCode(...byteArray));
},
handleCustomAction: async function() {
const model = this.model;
const indexValue = await this.indexSearchValue(model.get('description') || model.get('kbdocument_body') || '', "RETRIEVAL_DOCUMENT");
const payload = {
name: model.get('id'),
record_id: model.get('id'),
name_of_module: this.module,
record_name: model.has('name') ? model.get('name') : '',
index_value: indexValue
};
app.api.call('read', app.api.buildURL('SI_SearchIndex', null, null, {
fields: ['id'],
filter: [{ name: payload.record_id }],
max_num: 1
}), null, {
success: (data) => {
const method = data.records?.length ? 'update' : 'create';
const url = app.api.buildURL('SI_SearchIndex', data.records?.[0]?.id);
app.api.call(method, url, payload, {
success: () => this.showMessage(`Record ${method === 'create' ? 'indexed' : 'updated'} successfully`),
error: () => this.showMessage(`Error ${method === 'create' ? 'indexing' : 'updating'} record`)
});
},
error: () => this.showMessage('Error checking existing record')
});
},
showMessage: function(text) {
app.alert.show('custom-action-alert', {
level: 'info',
messages: text,
autoClose: true
});
}
});
<?php
/**
* * ./custom/modules/KBContents/clients/base/views/record/record.php
*/
$viewdefs['KBContents']['base']['view']['record'] = [
'buttons' => [
[
'type' => 'rowaction',
'name' => 'custom_action',
'label' => 'Index',
'css_class' => 'btn',
'showOn' => 'view',
'events' => [
'click' => 'button:custom_action:click',
],
'acl_action' => 'view',
],
[
'type' => 'button',
'name' => 'cancel_button',
'label' => 'LBL_CANCEL_BUTTON_LABEL',
'css_class' => 'btn-invisible btn-link',
'showOn' => 'edit',
'events' => [
'click' => 'button:cancel_button:click',
],
],
[
'type' => 'rowaction',
'event' => 'button:save_button:click',
'name' => 'save_button',
'label' => 'LBL_SAVE_BUTTON_LABEL',
'css_class' => 'btn btn-primary',
'showOn' => 'edit',
'acl_action' => 'edit',
],
[
'type' => 'actiondropdown',
'name' => 'main_dropdown',
'primary' => true,
'showOn' => 'view',
'buttons' => [
[
'type' => 'rowaction',
'event' => 'button:edit_button:click',
'name' => 'edit_button',
'label' => 'LBL_EDIT_BUTTON_LABEL',
'acl_action' => 'edit',
],
[
'type' => 'rowaction',
'event' => 'button:create_localization_button:click',
'name' => 'create_localization_button',
'label' => 'LBL_CREATE_LOCALIZATION_BUTTON_LABEL',
'acl_action' => 'edit',
],
[
'type' => 'rowaction',
'event' => 'button:create_revision_button:click',
'name' => 'create_revision_button',
'label' => 'LBL_CREATE_REVISION_BUTTON_LABEL',
'acl_action' => 'edit',
],
[
'type' => 'divider',
],
[
'type' => 'shareaction',
'name' => 'share',
'label' => 'LBL_RECORD_SHARE_BUTTON',
'acl_action' => 'view',
],
[
'type' => 'pdfaction',
'name' => 'download-pdf',
'label' => 'LBL_PDF_VIEW',
'action' => 'download',
'acl_action' => 'view',
],
[
'type' => 'pdfaction',
'name' => 'email-pdf',
'label' => 'LBL_PDF_EMAIL',
'action' => 'email',
'acl_action' => 'view',
],
[
'type' => 'divider',
],
[
'type' => 'rowaction',
'event' => 'button:duplicate_button:click',
'name' => 'duplicate_button',
'label' => 'LBL_DUPLICATE_BUTTON_LABEL',
'acl_module' => 'KBContents',
'acl_action' => 'create',
],
[
'type' => 'rowaction',
'event' => 'button:audit_button:click',
'name' => 'audit_button',
'label' => 'LNK_VIEW_CHANGE_LOG',
'acl_action' => 'view',
],
[
'type' => 'divider',
],
[
'type' => 'rowaction',
'event' => 'button:delete_button:click',
'name' => 'delete_button',
'label' => 'LBL_DELETE_BUTTON_LABEL',
'acl_action' => 'delete',
],
],
],
[
'name' => 'sidebar_toggle',
'type' => 'sidebartoggle',
],
],
'panels' => [
[
'name' => 'panel_header',
'label' => 'LBL_PANEL_HEADER',
'header' => true,
'fields' => [
[
'name' => 'picture',
'type' => 'avatar',
'size' => 'medium',
'dismiss_label' => true,
'readonly' => true,
],
[
'name' => 'name',
'related_fields' => [
'useful',
'notuseful',
'usefulness_user_vote',
'kbdocument_id',
],
],
[
'name' => 'favorite',
'label' => 'LBL_FAVORITE',
'type' => 'favorite',
'dismiss_label' => true,
],
[
'name' => 'follow',
'label' => 'LBL_FOLLOW',
'type' => 'follow',
'readonly' => true,
'dismiss_label' => true,
],
'status' => [
'name' => 'status',
'type' => 'status',
'enum_width' => 'auto',
'dropdown_width' => 'auto',
'dropdown_class' => 'select2-menu-only',
'container_class' => 'select2-menu-only',
'related_fields' => [
'active_date',
'exp_date',
],
],
],
],
[
'name' => 'panel_body',
'label' => 'LBL_RECORD_BODY',
'columns' => 2,
'placeholders' => true,
'fields' => [
[
'name' => 'kbdocument_body',
'type' => 'htmleditable_tinymce',
'dismiss_label' => false,
'span' => 12,
'fieldSelector' => 'kbdocument_body',
'tinyConfig' => [
'toolbar' => 'code | bold italic underline strikethrough | alignleft aligncenter alignright ' .
'alignjustify | forecolor backcolor | fontfamily fontsize blocks | ' .
'cut copy paste pastetext | search searchreplace | bullist numlist | ' .
'outdent indent | ltr rtl | undo redo | link unlink anchor image | subscript ' .
'superscript | charmap | table | hr removeformat | insertdatetime | ' .
'kbtemplate',
],
],
[
'name' => 'tag',
'span' => 12,
],
],
],
[
'name' => 'panel_hidden',
'label' => 'LBL_SHOW_MORE',
'hide' => true,
'columns' => 2,
'placeholders' => true,
'fields' => [
[
'name' => 'attachment_list',
'label' => 'LBL_ATTACHMENTS',
'type' => 'multi-attachments',
'link' => 'attachments',
'module' => 'Notes',
'modulefield' => 'filename',
'bLabel' => 'LBL_ADD_ATTACHMENT',
'span' => 12,
'max_num' => -1,
'related_fields' => [
'filename',
'file_mime_type',
],
'fields' => [
'name',
'filename',
'file_size',
'file_source',
'file_mime_type',
'file_ext',
'upload_id',
],
],
'language' => [
'name' => 'language',
'type' => 'enum-config',
'key' => 'languages',
'readonly' => false,
],
'revision' => [
'name' => 'revision',
'readonly' => true,
],
'category_name' => [
'name' => 'category_name',
'label' => 'LBL_CATEGORY_NAME',
'initial_filter' => 'by_category',
'initial_filter_label' => 'LBL_FILTER_CREATE_NEW',
'filter_relate' => [
'category_id' => 'category_id',
],
],
'active_rev' => [
'name' => 'active_rev',
'type' => 'bool',
],
'viewcount' => [
'name' => 'viewcount',
],
'team_name' => [
'name' => 'team_name',
],
'assigned_user_name' => [
'name' => 'assigned_user_name',
],
'is_external' => [
'name' => 'is_external',
'type' => 'bool',
],
'date_entered' => [
'name' => 'date_entered',
],
'created_by_name' => [
'name' => 'created_by_name',
],
'date_modified' => [
'name' => 'date_modified',
],
'kbsapprover_name' => [
'name' => 'kbsapprover_name',
],
'active_date' => [
'name' => 'active_date',
],
'kbscase_name' => [
'name' => 'kbscase_name',
],
'exp_date' => [
'name' => 'exp_date',
],
],
],
],
'moreLessInlineFields' => [
'usefulness' => [
'name' => 'usefulness',
'type' => 'usefulness',
'span' => 6,
'cell_css_class' => 'pull-right usefulness',
'readonly' => true,
'fields' => [
'usefulness_user_vote',
'useful',
'notuseful',
],
],
],
];
Implementation Highlights and Constraints
The SearchIndex module currently includes the following simplified fields: name, record_id, name_of_module, index_value (embedding), and record_name (for convenience).
For prototyping purposes, indexing and ranking are handled entirely in JavaScript rather than through a dedicated backend service. The JavaScript implementation retrieves a maximum of 100 records from the SearchIndex module. Similarly, model API keys are managed directly in JavaScript. At present, users access the SearchIndex module directly from the frontend, which bypasses proper permission handling, optimized storage and retrieval, and deduplication logic. These aspects should eventually be migrated to a backend service with robust infrastructure.
Indexing is limited to either the description or KB body field—whichever is populated first. However, for more effective indexing, it's important to identify meaningful chunks, summaries, or segments of text to embed. The current retrieval mechanism uses cosine similarity to match queries against indexed content.
Additional Notes
The default embedding dimension is 3072 normalized vectors. For storage and computation optimization, this dimension can be reduced, thanks to Matryoshka Representation Learning (MRL), smaller segments of the embedding—such as 768 or 1536 dimensions—can still retain meaningful semantic information and support effective retrieval tasks. This approach can allows for more efficient indexing and retrieval while minimizing storage and computation overhead.
I hope this example helps! I'd appreciate any feedback you’d like to offer!