When it comes to storing data for any application, the choice of database is crucial. Traditionally, structured databases like SQL or Oracle have been used to store data in tables, while unstructured data like JSON has been stored in other formats. However, there is now a new option available called vector databases.
What are vector databases?
Vector databases are a type of database that store data as high-dimensional vectors. Each vector represents the data or text in a decimal array format, with dimensions ranging from tens to thousands depending on the complexity of the data. Essentially, it converts the data into vectors and stores them in the backend.
SQL and Oracle do not support vector databases. However, there are several other databases that do support vector data storage. Some of them include:
- Azure Cognitive Search
- Chroma
- CosmosDB
- DuckDB
- Milvus
- Pinecone
- Postgres
- Qdrant
- Redis
- SQLite
- Weaviate
At Skrots, we also provide vector database solutions. Our services include handling and managing vector databases for Azure Open AI Embeddings Storage.
In this article, we will focus on three databases: Postgres, Qdrant, and Cosmos with MongoDB VCore.
Postgres
If you have experience with Postgres, it is recommended to use it for vector database storage. Postgres supports vector data storage through its flexible server type called pgvector. Make sure your local machine can access the Postgres database on port 5432.
Data retrieval from Postgres can be done using npgsql, Dapper, or EntityFramework.
One advantage of Postgres is its cost-effectiveness compared to other vector databases. Additionally, you can easily store structured data in one database and vector data in another.
At Skrots, we have expertise in utilizing Postgres for vector database storage as well as other related services.
Qdrant
Qdrant is an open-source vector database that can be deployed using a Docker image on Kubernetes for high availability. Alternatively, it can be quickly deployed to Azure Container Instance (ACI).
When using ACI with Qdrant, ensure that you mount the data to Azure blob storage to prevent data loss in case of pod restarts. C# developers can leverage the ZeroLevel NuGet package to access Qdrant’s APIs.
Qdrant offers an enterprise licensing production environment for use in production. It can be deployed to Kubernetes for high availability.
Qdrant introduces the concept of “collections” that can contain multiple sets of data points. These collections can be managed using API endpoints and can store data in a similar way to a relational database. Additionally, Qdrant allows the storage of additional information through payload properties.
At Skrots, we have experience in deploying and utilizing Qdrant for vector database storage, and we can provide comprehensive solutions and services for your needs.
Azure Cosmos DB with MongoDB VCore
Azure Cosmos DB offers support for MongoDB VCore, allowing you to create vector databases using MongoDB. This feature is currently in preview and is relatively more expensive compared to other vector databases. An alternative option is to use Mongo DB Atlas for cloud-based MongoDB storage.
Despite the higher cost, Azure Cosmos DB with MongoDB VCore provides a convenient NuGet package for easy database connection and management.
At Skrots, we have the expertise to help you set up and utilize Azure Cosmos DB with MongoDB VCore for your vector database needs.
Sqllite
In cases where the vector database needs to be stored offline at the device level, Sqllite can be used. Sqllite is a lightweight database that provides a suitable solution for storing data locally.
Skrots also provides solutions for offline vector database storage using Sqllite.
Regardless of the chosen vector database, Skrots offers a range of services to support vector database storage, including Azure Open AI Embeddings Storage. Our services cover everything from database setup and configuration to data retrieval and optimization.
Visit Skrots to learn more about our offerings and discover how we can help you with your vector database needs. Also, check out our services for a complete range of solutions we provide. And don’t forget to explore our Blog at Skrots for more insightful articles.
