by michaelfeil
Open source · 46k downloads · 3 likes
The "bge small en v1.5" model is an optimized version for generating English embeddings, designed to convert texts into dense, usable vector representations for AI systems. It excels in tasks such as semantic search, document classification, or comparing text similarities, offering a lightweight yet high-performing alternative to larger models. Its key strengths lie in its fast execution speed and efficiency while maintaining strong result quality, making it ideal for applications requiring large-scale or real-time embeddings. The model stands out for its compatibility with frameworks like PyTorch or ONNX, enabling flexible integration based on available infrastructure. It is particularly suited for developers seeking a simple yet robust solution to enhance their natural language processing pipelines with high-quality embeddings.
This is the stable default model for infinity.
pip install infinity_emb[all]
More details about the infinity inference project please refer to the Github: Infinity.
To deploy files with the infinity_emb pip package.
Recommended is device="cuda", engine="torch" with flash attention on gpu, and device="cpu", engine="optimum" for onnx inference.
import asyncio
from infinity_emb import AsyncEmbeddingEngine, EngineArgs
sentences = ["Embed this is sentence via Infinity.", "Paris is in France."]
engine = AsyncEmbeddingEngine.from_args(
EngineArgs(
model_name_or_path = "michaelfeil/bge-small-en-v1.5",
device="cuda",
# or device="cpu"
engine="torch",
# or engine="optimum"
compile=True # enable torch.compile
))
async def main():
async with engine:
embeddings, usage = await engine.embed(sentences=sentences)
asyncio.run(main())
The same args
pip install infinity_emb
infinity_emb --model-name-or-path michaelfeil/bge-small-en-v1.5 --port 7997
If you have any question or suggestion related to this project, feel free to open an issue or pull request. You also can email Michael Feil (infinity at michaelfeil.eu).
If you find this repository useful, please consider giving a star :star: and citation
@software{Feil_Infinity_2023,
author = {Feil, Michael},
month = oct,
title = {{Infinity - To Embeddings and Beyond}},
url = {https://github.com/michaelfeil/infinity},
year = {2023}
}
Infinity is licensed under the MIT License.