Case Study | Improvements in Data Retrieval Times
Developing Efficient Cloud Solutions to Improve Data Retrieval Performance
Published: 16 March 2023
Twyn.ai is an artificial intelligence software used to streamline business telecoms and infrastructure. More specifically, this tool allows Telecommunications (Telco) customers to automate sales and commercial back office, track end-to-end visibility of sales and delivery, and manage SLAs.
The Business Challenge
A vital aspect of Twyn.ai’s offering is the logging and tracking of tickets. There are many layers to which tickets are tracked, from outages to service requests. As such, many data points exist that need to be collected and processed to ensure adequate and high-quality services to the customer.
Twyn.ai had been experiencing severe performance degradation due to slow data retrieval from MongoDB. The data store comprised a MongoDB community edition database hosted within an EC2 instance, hampering efforts to monitor and evaluate progress and generate customer-level reports. The current architecture could not effectively scale with growth and was becoming increasingly inefficient.
The challenge presented to Tangent Solutions was to identify and implement a cost-effective and efficient solution that would improve data retrieval performance, scale with growth and provide the ability to generate real-time customer-level reports.
Tangent’s Assessment & Strategy
Tangent’s AWS Cloud team assessed the business environment, and upon initial assessment, it was ascertained that two main factors hindered the current environment.
Namely, the type of database being utilised as well as the method of optimisation for the database. The existing infrastructure posed numerous issues as it did not sufficiently leverage optimisation around high levels of inputs and outputs (i/o’s) or transactions.
MongoDB is a document store type database and, as such, does not leverage the enhancements and optimisations which traditional relational store databases possess. Many transactions were being committed to MongoDB. Although it is easy and fast to spin up a MongoDB database from a development side, it does not handle relational store-type retrieval of said data well, often causing bottlenecks and delays. The database is used for monitoring and evaluation of transactions, and this posed an urgent issue by being unable to meet customer Service Level Agreements (SLA’s).
The Solution
As such, Tangent proposed various approaches to improve performance and meet customer Service Level Agreements (SLA’s). The customer was adamant that MongoDB be installed on an EC2 instance as the best solution for the business needs. Therefore, a detailed and specific approach needed to be followed.
- Evaluate and identify the current indexes within the MongoDB documents.
- Facilitate the understanding (amongst developers) of the benefit of sharding.
- Provide guidance on how best to implement sharding within their environment.
- Propose the benefits of removing the transactions from MongoDB and instead have the highly structured transactions run through another AWS service, namely Redshift.
- Provide guidance on Disaster Recovery best practices and possible costing.
Tangent’s AWS Cloud team suggested the below solutions, of which Twyn.ai agreed to implement the first two. The Redshift Migration will form part of the “next steps” for Twyn.ai.
Level 1 (Short-term solution)
- Implement indexing, ensuring cardinality meets industry standards.
- Implement sharding across all transaction-level documents within MongoDB to optimise retrieval of documents and transactions.
Level 2 (Mid-term solution)
- Upgrade the MongoDB community edition to MongoDB Enterprise edition.
- Implement a Redshift Data Warehouse to collect, aggregate, and report on the transactions.
The Outcome & Next Steps for Twyn.ai
Tangent implemented the recommended optimisations to the MongoDB database, EC2 instance and the underlying infrastructure to ensure that Twyn.ai was able to access the data in a timely and efficient manner.
Tangent successfully assisted Twyn.ai with negotiating the upgrade of MongoDB from the community edition to the Enterprise edition as well as providing Disaster Recovery best practices for the current instance.
After implementation, Twyn.ai experienced significant improvements in data retrieval time, alongside performance increases in monitoring and evaluation processes and customer-level reporting.
In order for Twyn.ai to leverage the full potential of its data, Tangent has suggested the migration from MongoDB to Redshift. Therefore, allowing for improved efficiency and reduced administrative tasks within the database. Tangent is currently in discussion with Twyn.ai to execute the next step of this migration.
Want to learn more
about Tangent?
Take a look at all our Case Studies, Articles, On The Record, and In The News
Contact Us
Partner with Tangent Solutions today and improve data retrieval performance for your business.