Brainvire Develops Web App For Book Vectorization To Enhance Entrepreneur Readership
Tech Stack
- CSS 3
- Django
- Google Cloud Platform
- HTML
- JS
- Pinecone
- Python
About Client
Our client is the forerunner in the world of education material for entrepreneurship. The company shares resources that the current and future leaders need to innovate, manage and build robust businesses.
About Product
The client wanted to develop a web application that takes user input and responds to a query with quoted references from books on entrepreneurship uploaded to Google Bucket. The application would allow users to upload various types of documents about entrepreneurship, extract relevant metadata and generate embeddings for accurate and faster retrieval of relevant book quotes and references. To develop this application, Brainvire developed a scalable and easy-to-use web application for uploading files, extracting text from files, generating embeddings using OpenAI and storing embeddings and associated metadata in a Pinecone index to execute a similarity search.
Scalable Pub/Sub Architecture:
The client desired the application to handle high message volume and concurrency efficiently
Brainvire developed a message queue or Pub/Sub system to act as a buffer for incoming metadata update requests allowing subscribers to consume messages from the published topic
Solution
The Challenges
Result
- Faster Metadata UpdatesWe introduced parallel processing capability for worker instances to reduce time for metadata updation in the Pinecone index
- High availability of the systemThe distributed architecture and the load balancer utilization ensures uninterrupted service by redirecting requests to other available instances or worker
- Real-time updatesBrainvire implemented asynchronous processing and parallel execution for close to real-time updates of Pinecone metadata for enhanced user experience without delay
- Scalable architectureBrainvire designed a scalable architecture for consistent, smooth performances by adding more worker instances and distribute the load with increased metadata update requests
Product Screens
Client Testimonials
They help us coordinate the right skill sets, teams, and activities in order to get the job done. They transferred our platform assets, such as our servers, workstations, and databases, into their managed cloud services environment.
Patricia Walters
Founder and CEO - Creative LLCPowered by Clutch
224 Reviews
“They handle all their work with a personal touch, which encourages a continued long-term partnership. Their deliverable are reliable thanks to vigorous QA testing. I would recommend Brainvire to anyone.”