Recent features bring significant performance gains to Elasticsearch and Lucene vector database.

Elasticsearch and Lucene report card: noteworthy speed and efficiency investments

Harnessing Lucene's Architecture for Multi-Threaded Search

Accelerating Multi-Graph Vector Search

Harnessing Java's Latest Advancements for Ludicrous Speed

Maximizing Memory Efficiency with Scalar Quantization

Achieving Seamless Compression with Minimal Impact on Recall

Paving the Way for Binary Quantization

Multi-Vector Integration in Lucene and Elasticsearch

Wrapping up (for now)

Making Elasticsearch and Lucene the best vector database: up to 8x faster and 32x efficient

Taking Text Embeddings and Vector Similarity Search as the example task, this blog describes the process for getting up and running using deep learning models for Natural Language Processing, and demonstrates vector search capability in Elasticsearch

1. Deploy a text embedding model.

2. Loading initial data

3. Creating pipeline

4. Reindex data

5. Vector Similarity Search

6. Quick Verification

Trying it out

If you want more NLP reads:

How to deploy NLP: Text Embeddings and Vector Search

Explore how the kNN query in Elasticsearch can be used and how it differs from top-level kNN search, including examples.

kNN search as a top-level section

Introducing kNN query in Elasticsearch

kNN query API examples

Elasticsearch kNN Query: An expert way to do kNN search

Introducing kNN Query: An expert way to do kNN search

Sharing information among segment searches in multi-graph vector search allows us to achieve significant search speedups.

The previous state of multi-graph vector search in Lucene

Speeding up multi-graph vector search by sharing information between segment searches

Impact on performance

Impact on recall

Conclusion

Speeding Up Multi-graph Vector Search

Author

Mayya Sharipova

Articles

Making Elasticsearch and Lucene the best vector database: up to 8x faster and 32x efficient

How to deploy NLP: Text Embeddings and Vector Search

Introducing kNN Query: An expert way to do kNN search

Speeding Up Multi-graph Vector Search