2023 Comparative Search Review Raw Results

Introduction

The goal of that experiment is to measure (subjectively) the relevancy of 4 search ranking algorithms to retrieve useful content. It does compare the traditional well known algorithms: Okapi BM25 implemented in a most common search engine software, Google ranking (very uniq but the dominant player here and a reference point) and 2 approaches using sentence embeddings: ELSER from elastic.co and all-MiniLM-L6-v2 from sbert.net. Text embeddings combined with vector search and applied to semantic search is an approach intensively discussed in recent times, following the progress of LLM.

Search Queries

3 types of questions:

  1. How to Use Bluetooth in a Suzuki Swift?
  2. What is the best oil type for my Ford Ranger, and is it possible to change the oil myself?
  3. Where are Range Rovers made?
  4. Who owns Rolls Royce?
  5. What is the normal oil for a Honda CR-V and is it an easy DIY job to change?
  6. Is it legal in Victoria to sell a car without a RWC, if so, what are the correct steps?
  7. Why electric cars are not builtin with PV?
  8. What does the acronym LDV mean?
  9. How to change engine oil on my Toyota Prius?
  10. What is the difference between Diesel, Regular Petrol and Unleaded 91, Unleaded E10?
  11. Why my car won’t start?
  12. How to use Bluetooth in a Jeep Willys? ;-)
  13. skoda meaning
  14. bmw x5 diesel problems
  15. best landcruiser engine
  16. ford ranger oil

Results

sumwary

2023-comparative-search-experiment-results.ods

Detailed results

located in /assets/docs/2023-comparative-search-review-results-json/: 3 types: bm25, elser, minilm, for number from 01 to 16. example: /elser.01.json