How Searching This Website Works

I have been wanting to write about how the search on this website works, but I haven’t had the patience, or the will, to write about it. It’s also not as if anyone asked me about it, so it’s not as if it was a priority. Now I feel like it’s the right time - is there any? - to finally write about it.

Before I started using Hugo again, searching this website was trivial. The website was fully server-side rendered by Eagle, my custom CMS, and all posts were indexed in a PostgreSQL database. I then used a full text search to match the search query.

Now, it is different. The website is, for the most part, statically generated by Hugo, so there are constraints about what I can and can’t do. However, I think I found a reasonably creative solution to solve the problem. An important piece of information is that the website is still being served by Eagle, which adds some custom functionality, like Micropub, Webmentions, etc.

🗂️ Search Index

For the search index itself, I am using MeiliSearch. MeiliSearch is an open-source alternative to Elasticsearch. I learned about it many years ago, and I’ve even used it before using PostgreSQL for this website, so it isn’t completely new.

All posts are indexed in a MeiliSearch index. This is done when Eagle starts up, indexing all existent posts, and also when I push to Git. Then, I try doing an educated guess to only update - or remove - the correct entries from the database. If Eagle can’t make an educated guess, it just reindexes everything. The operation just takes a couple milliseconds, so it’s not something to worry about at this point.

MeiliSearch has the advantage of being made for… search. I could be using a traditional SQL database, but that wouldn’t make sense in this context since the only purpose I’m using it for is search. Additionally, it is a very fast and lightweight software, so the resource consumption is not an issue.

🎨 Rendering

Now you know how the posts are indexed and searched on, but how does the rendering of the search results themselves work? When someone hits the search button on the search page, an HTTP call is done to Eagle, which then gets the results from the MeiliSearch index.

Afterwards, some magic happens. Eagle opens the already rendered file - by Hugo - for the search page and looks for a custom <eagle-page> element. Then, it renders an Eagle-specific template for the search results and replaces the content of the custom HTML element with the search results.

Et voilá! Maybe not that creative, but it works quite well. By default, the search page contains a form which allows users to search my website using DuckDuckGo:

<eagle-page>
  <form id='search-form' class='inline-form' action="https://duckduckgo.com/html/" method="get">
    <input required type='search' name='q' placeholder="Search…" value="site:hacdias.com " />
    <button/>Search</button>
  </form>
</eagle-page>

This means that, if I stop using Eagle, or if I do need to serve my website without Eagle backing it, the search page will just work normally, but now powered by DuckDuckGo. I provide Eagle with a custom template to render the search results, which matches the way I display posts elsewhere.

I use the same technique for error pages, which allows Eagle to provide more suitable errors when it is possible. An example of this is the never ending list of “gone” posts I have. Or if some other error happens for one or other reason - for example if you fail the guestbook anti-bot mechanism - you also get a page showing the error that fits into the rest of the website.