Learn how to combine multiple sparse, dense and geo fields in custom ways and what is the best way to do it?

How to combine multiple sparse, dense and geo fields in custom ways

Sparse Fields

Dense Fields

Geo Fields

Combining Diverse Fields

Creating "books" Index

Indexing Sample Data

Finding Java Books Near a Tech Hub

Querying for Special Edition Search Technology Books

Searching for Latest IT Books Available in Multiple Languages

Wrap Up

Sparse, dense and geo fields: How to combine them efficiently in Elasticsearch

Combining sparse, dense and geo fields

Learn how to search through documents with multiple kNN fields, and scoring the resultant documents based on multiple kNN vector fields.

Scoring Documents By The Closest One With Multiple kNN Fields

kNN Mechanics

Scoring Documents

Text Embedding and Vector Fields

Creating an Index with kNN Fields

Indexing sample docs

Setting up an inference processor

Indexing the Documents

kNN Search

Approximate Search

Searching for Multiple kNN fields

When do we search with multi-kNN fields

Elasticsearch Scoring: Based on multiple kNN fields

Scoring documents based on the closest ones with multiple kNN fields

Learn strategies for selecting the optimal values for `k` and `num_candidates` parameters in kNN search, illustrated with practical examples.

How to choose the best `k` and `num_candidates` for kNN search?

kNN search query

kNN critical parameters - k and num_candidates

K: Unveiling the closest neighbors

Setting lower K

Setting higher K

The `num_candidates` attribute: Behind the curtain

Setting the lower num_candidates

Setting the higher num_candidates

Balancing precision & performance for kNN parameters

Experimentation to optimize kNN parameters

Practical example: Using kNN for movie recommendations

Manual framework

Creating an inference pipeline

Creating index mappings

Indexing movies

kNN search

kNN query attributes

kNN results

Choosing optimal K value

Choosing the optimal `num_candidates` value

Finding the optimal balance

Wrap up

k refers to the number of global nearest neighbors to retrieve, and num_candidates to the number of candidate neighbors considered for each shard during the search.

What do the k and num_candidates parameters represent in kNN search?

If you prioritize extreme precision with a smaller set of highly relevant results, a lower k with a moderate num_candidates might be ideal. If exploration and discovery are your goals, a higher K with a larger num_candidates could be more suitable. By experimenting with different combinations and monitoring performance, you can achieve the perfect balance between precision, exploration, and speed.

How to choose the optimal value of the K and num_candidates parameters for KNN search?

Elasticsearch kNN search: How to choose the best k and num_candidates

How to choose the best k and num_candidates for kNN search

Author

Madhusudhan Konda

Articles

Combining sparse, dense and geo fields

Scoring documents based on the closest ones with multiple kNN fields

How to choose the best k and num_candidates for kNN search