Better search using AI

During a consultation with the Leprosy Foundation, we discussed how AI can support end users in efficiently searching an extensive collection of scientific publications and practical materials. InfoNTD's website contains more than 35,000 documents, which are searchable via Elasticsearch. Although this system is effective, it remains a challenge for users to filter the large amount of documents into relevant results.

We have been exploring for some time how AI can improve our services and those of our clients. So far, AI has mainly been used for conversational chatbots and support tasks. Our next step is to integrate AI user-friendly into our website, in line with our mission to make innovative technologies accessible to end users.

After careful consideration, we see a lot of potential in enhancing the search function with AI. This could support users in various ways to find the right information efficiently.

In this article we will not go too deeply into the technical workings of AI, but focus more on its practical application. Herewith a brief explanation of the terms we use:

• AI (Artificial Intelligence or Artificial Intelligence): The ability of a machine to perform tasks that typically require human intelligence.
• LLM (Large Language Model): A type of AI that specializes in interpreting and processing large amounts of text.
• GPT (Generative Pre-trained Transformer): A specific category of LLMs developed by OpenAI, of which ChatGPT is a well-known example.

We recently collaborated with Kickstart AI, a non-profit organization that aims to increase AI adoption in the Netherlands. During their business presentation, they highlighted the technical opportunities, but also the user experience (UX) challenges in integrating AI applications. We soon decided to jointly organize a hackathon to explore this issue in more detail, figuring out both the technical and UX challenges.

We wanted to avoid a standard conversation-oriented application, as it often does not provide an optimal user experience. Instead, some innovative ideas emerged for the hackathon:

• Natural language search: Users can formulate searches in their own words, without having to set up complex filters.
• Search Assistant: An AI assistant that actively helps users find the right results, for example by providing suggestions.
• Search result summarization: Briefly summarize the content of search results and explain why these items are relevant to the search.
• Personalized suggestions: By better understanding the user, for example through profiling, AI can make more tailored suggestions.
• Improve ranking: Optimize the order of search results to better match the search query.

We were looking for a way to use a Large Language Model (LLM) as part of a system, using AI as an intermediate solution. One challenge here is that the results of an LLM are not always consistent. This can be advantageous for a user who wants to see different suggestions, but consistency is necessary for interaction between two systems.

A practical example is our experience using ChatGPT to analyze, summarize and thematically group user feedback. Although an LLM is well suited for this, we often got inconsistent JSON outputs, which was problematic for our application. With ChatGPT's new "JSON mode," we can now receive consistently valid JSON outputs, which is a big improvement. However, this does not guarantee that the JSON always conforms to a specific schema, so some validation remains necessary.

Moreover, our tests show that carefully formulating the input, known as prompt engineering, remains crucial to obtaining usable responses. Despite suggestions that this should become less important, in our experience it is still essential for obtaining relevant results.

We had only an afternoon to delve into the use case, develop ideas and sketch out solutions. By splitting the group of participants, we were able to develop and test several ideas simultaneously. Although we could not present a complete demo, many puzzle pieces fell into place.

Part of our team focused on the user experience. We analyzed the needs of typical users and a common search query to see how we could use AI to improve the user experience during search. In doing so, we came up with three additions for the search results page (see image):

1. Converting a search term to different filters.
2. Asking follow-up questions for further refinement.
3. Providing a summarized answer based on the search results found.

The other part of the team did two technical explorations. First, we interpreted users' search queries. We provided the AI with context about the users and the content, explained what filters were possible, and gave an example of a typical search query. For example, we translated the search query "I am looking for a training guide on leprosy in Brazil" into a query with filters: "country = br", "subject = leprosy", and "type = practical material".

Then we looked at how to summarize search results. We gave the AI the search query and the articles found to determine if the answer it was looking for was among them. The AI could then indicate how relevant the results were and highlight specific results that may have provided the best answer. If the information sought was not in the articles, the AI could report that and possibly suggest alternatives.

The hackathon with Kickstart AI and the Leprosy Foundation yielded exciting insights, but practical testing is still needed. While the results with standard examples are promising, the real world may turn out differently. User testing seems an essential next step. Some observations emerging from the hackathon:

Speed

An LLM can currently be slow, especially as input and output become large. For typical queries, processing can quickly take several seconds. For real-time applications such as this one (a query), this is not ideal.

This is basically a matter of optimizing the task we give to the LLM: the shorter the task (both the question and the answer), the faster the response. But we also see that the shorter the command is, the less consistent the output is. This is a matter of "prompt engineering," and over time the models and hardware will also become faster.

Cost

Using AI costs money, which is calculated based on the number of (parts of) words and other characters used (also called "tokens"). Costs vary by AI model and depend, among other things, on the complexity of the model, and the cost required to run the model, both on hardware and electricity costs.

Most cloud providers offer a choice of which model to use. Each model has its own specialties, and involves different costs. There are now also many Open Source models, which you can run from the cloud as well as on your own machines.

Ethics

There are important ethical considerations associated with AI. It is far from a "green" solution: the power consumption of applying AI worldwide was compared to the power consumption of Ireland in a paper by the VU Amsterdam. Also, AI development is centralized in large companies, often with unclear methods and intentions. This raises questions about data use and management, as well as the broader societal impact.

But there is, of course, certainly a positive side to this: a paper from MIT shows a possible 40% increase in productivity, scientific research into diseases like Alzheimer's is accelerated, and new Open Source models make AI more accessible to all (which fits well with Kickstart AI's mission).

The hackathon proved to be a successful afternoon in which we were able to complement each other well in terms of techniques used and thoughts on UX. This tastes like more. Many thanks to Kickstart AI for this fun and informative afternoon, and to Leprosy Foundation for thinking along and making their material available.