Integrating Umbraco CMS with Algolia

Introduction

When you have a lot of content being managed in Umbraco, you want people to easily track down any specific content they want within this vast ocean of data and also be able to do it “pronto.” 

Umbraco does this out-of-the-box through a built-in version of the high-performance open-source search engine Lucene.NET, packed into the Examine library. If you’re unfamiliar with Umbraco’s search engine provider, you can take a moment and walk through the documentation for a better understanding of the concepts.

Relevant, fast, user-friendly search is the perfect partner for any CMS. It can easily produce higher click-through rates, more end-user engagement, and a more accurate understanding of people’s needs or, in the case of e-commerce, of the need to upgrade the customer experience.

Thanks to sites like Google, Amazon, and Netflix, the bar has been raised, and users now have high expectations, looking for relevant, accurate, and rapid results. 

It is a good moment to introduce the concept of Search as a Service, a component of the Software as a Service model intended to provide site search limited to a specific website or related domains through powerful APIs.

Many a customer journey start with the search and this makes the search experience a foundational part of digital experiences. Gartner defined DXP as an “integrated software framework for engaging a broad array of audiences across a broad array of digital touchpoints”, meaning that as customers search more and more, anything related to “touchpoints” is centered around the search experience. If you are building a DXP platform, implementing component technologies that are true “SaaS” can help provide a better overall experience for both developers and users.

Why not just use Google for site search, you might wonder? Well, Google is a great web search engine, but not a great site search, mainly because Google’s (or similar providers') search engine indexing, algorithms, and search results, cannot be customized by individual site owners.

Algolia

Stepping up to the plate is Algolia, a hosted search engine offering search APIs for performing search operations, AI-powered for dynamic experiences that help businesses maximize the speed of search and discovery while solving the pain of relevance tuning through AI. Algolia offers full-text, numerical, and faceted search, capable of delivering results in real-time. The powerful API allows you to quickly and seamlessly implement search within your websites and mobile applications and provides advanced front-end libraries and a crawler (if needed).

You can check out this webinar to get a firsthand look at working with Algolia.

Algolia ecosystem consists of two parts: Algolia Search and Algolia Recommend, each of them built on these three pillars:

1. FAST TO IMPLEMENT
2. FAST TO ADAPT
3. FAST TO RESULTS

The pricing plan offered by Algolia is pay-as-you-go based, but a free tier is also available for a data amount of 10 000 searches + 10 000 recommend requests/month and 10 000 records/month. A detailed overview of the features can be viewed here.

The entire suite of tools for building search can be checked out here, and the source code is open-sourced and available for contributions.

Using Algolia with Umbraco CMS

To get started with the Algolia package, you will need to first make sure that you have a registered account on Algolia. The Algolia dashboard is where you can configure most aspects of Algolia’s products and features.  You can sign-up right away for an account or use this guide for a more personalized and interactive experience.

Using the Algolia interface you will need to create an Application and retrieve the Application ID and Admin API Key.

Integrating Algolia in Umbraco CMS is a matter of installing the NuGet package. Then the following configuration values need to be added to the settings file of your Umbraco site.

Use Cases

- Manage index - user is able to add or update an index by specifying a name and a schema definition. A schema definition is an object containing correlations between content types (no element types or compositions) and their selected properties. The serialized object will then be saved into an Umbraco table called algoliaIndices - no payload is sent to Algolia, only a blank index will be created.
The serialized schema definition has this structure:

- Build Index - content payload is built based on the schema definition of the selected index, and then it is sent to Algolia. The name of the index is the reference used to identify the payload, and the content node Key property is the one to identify the Algolia object. If this is the first build of the index, records will be created in Algolia, otherwise, using the Key property as a reference, objects will be updated.
An Algolia record, based on the index schema definition looks like this:

- Search Index - there are three aspects concerning searching and interpretation of results:

• Searching from Algolia Dashboard
  You can query the content directly from the Algolia dashboard to get an overview of the results:

• Searching from Umbraco Backoffice
  Select your index from the Umbraco dashboard and choose to search over an index. You will have access to an UI interface where you can perform search operations.

I’ve kept the results simple, similar to the ones you will receive performing a search in the Algolia dashboard.

• Extending Algolia Search Engine
  Using the powerful Algolia Search API client, you can index and search very easily, and on top of the basic functionalities included with this integration, you can take full advantage of the API to boost relevance, and improve and extend the search experience. Managing results can be extended as well through the rich palette of tools Algolia is offering.

- Delete Index - remove index reference from the Umbraco database and as well from Algolia.

- Content Notifications - while building an index could be a costly operation, you can also rely on a ContentPublished notification to automatically update data in Algolia if it is part of an index. ContentDeleted and ContentUnpublished notifications will remove the record reference from Algolia.

The Algolia Search Management Dashboard

Current integration provides a custom dashboard called Algolia Search Management, available in the Settings section of the backoffice, where editors can define the content schema they want indexed and push the data payload to Algolia.

How the Integration is Built

Interaction with Algolia can be managed through different paths that offer great flexibility and control to the user in building a great search experience, all of them shaping up the  Algolia ecosystem built around the Search REST API.

While the REST API is the heart of Algolia, the recommendation would be to use an API client or a different integration when carrying out indexing, search, and configuration-related tasks with the REST API. 

The API clients provide a set of convenient methods for interacting with the Algolia APIs and add additional features compared to using the APIs directly:

• Network retry strategy: the client automatically retries connecting to another Algolia server if the connection fails.
• Reindexing: the API clients let you reindex your records in a single operation.
• Automatic batching: when you add records to your index, the API client automatically batches your records to make fewer API requests.

For our Umbraco CMS integration, we have used the .NET API client, which can be explored on GitHub, or you can use the NuGet package to implement an extensive search experience.

With this Umbraco integration, we did not aim to replace the current CMS search engine but to provide insight into using a SaaS solution to improve the search results. You can check out the integration here and use the IAlgoliaSearchService interface to perform simple queries on an index. For a better and more customized searching operation, the .NET API client offers all the tools needed to achieve this.

The first topic that comes up when discussing searching is indexing. Whether it’s a large enterprise-scale site or a small e-commerce store, the first step to adding search to a website is indexing the content through a website crawler or API. Site architecture, schema, or content can all affect indexing. A search index is designed to map search queries to web pages, documents, or other site content.

A search request is a group of one or more search operations sent in a single exchange. Search operations are triggered when a search is performed. In autocomplete and search-as-you-type implementations, a new search request is performed on every keystroke.

Data is indexed in Algolia using the concept of the record. Simply put, a record is an item or object that contains data that someone would search (or get recommendations) for. The total number of records is equal to the sum of records that you have in all of your indices. Each search plan allows you to store a certain number of records at any given time. You can update, delete, or add as many records as you want as long as the total number of the indexed records does not exceed the inclusion of the plan.

Use of UI Library 

The search management dashboard mimics the look and feel of the Examine Management dashboard in the Umbraco backoffice. If you notice something different in the screenshot below, well … you’re right!

The Algolia integration for Umbraco CMS now uses the new UI library, a set of web components that can be used to build Umbraco User Interfaces. The UI Library separates the user interface from Umbraco’s business logic and creates a unified user experience.

The UUI Storybook is a good starting point for working with the UI Library as it offers an overview of all the components, documentation, and different use case scenarios. As a result of this UI architectural change, the current integration requires Umbraco 10.3 or newer, as the UI Library is included.  

But you can also use it outside of an Umbraco context, my colleague Warren Buckley has a great introduction video to Umbraco UI WebComponent Library, and Julia Gruszczynska gives a nice insight into the UUI. And, of course, a big H5YR! to Niels Lyngsø, who helped a lot along the way.

Web components used in the process:

• uui-box
• uui-table
• uui-action-bar

• uui-button

• uui-form + uui-form-layout-item
• uui-ref-node
• uui-card-content-node

• uui-icon-registry-essential
• uui-toast-notification-container

• uui-dialog

If you feel some melancholy for the previous version of the Backoffice UI, here’s a “taste” of how it would have looked like built with the current Backoffice API:

• Dashboard

• Manage indices

Conclusions

I hope you’ve enjoyed this new journey into the cDXP world of Umbraco; As always the code is open-source and available on GitHub. 

You can find the integration on the Umbraco Marketplace, and should you have any feedback, comments, or requests, the issue tracker is open.

2022 has been quite the year for cDXP and integrations for Umbraco products. Not only did we launch a new marketplace, and an Integrations and Tech Partner listing, we also shipped 9 new integrations accompanying all the great work done by our partners and community. Remember to use Umbraco 10.3+ with this integration, and since we are reaching the end of the year, and we aren’t likely to release more integrations until 2023: