Executive Summary
Mission Critical – Production Use-Case for the #1 Digital Television Content streaming service.
Our clients are known for being one of the leading innovators in the new digital content streaming platform space. With a Subscriber base of over 2+ million customers, they seen explosive growth in this area. Some of the brightest technologist and architects work there, but when it comes to Elasticsearch, they lack expert level skills needed to configure Elasticsearch to service content at this scale. When it comes to designing your Elasticsearch cluster to support high loads, massive interactive search in a 99.9% uptime use-case, critical skills are needed. Many times without the right mix of skill sets and experience with Elasticsearch your development teams can come to a complete halt or even make design decisions that affects the overall search performance. It requires a deep understanding of Elasticsearch over many different use-cases, also the knowledge of software systems in general, to tackle such complex types of configuration problems with Elastic. At Weblink Technologies, we not only excel at ElasticSearch professional services but also bring over 20+ years of deep software development and implementation skills on various application stacks. Organizations that engage our Expert Elastic consultant service typically avoids these five common outcomes from ElasticSearch integration
The delayed release of key enhancement or features
Over/Under provisioning of hardware
Purchasing costly software licenses when alternatives exists
Underperformance of your Elastic cluster for any use-cases
Skills gap with the technical team.
Common Design Anti-patterns when building an ElasticSearch based Search API’s
Challenge
Weblink was asked to apply optimizations that meets or exceeds the end customers satisfaction, increase current search response times, and identify areas centered around the existing Elasticsearch Implementation that would enhance ROI and achieve practical results within budget and timeframes.
In one search intensive use-case, at any time 20% of 2+ million users or more could browse digital content from a well-known digital content streaming service.
Digital streaming users expect fast content rendering with fully curated selections made especially for them. Additionally, these digital users typically channel surf and switch between multiple digital channels at any given time, looking at what digital content display cards may peak their interest, in an ultra-fast and prompt manner. In this scenario, Elasticsearch’s ultra fast real-time search handles all incoming user search requests. With the help of Weblink professional services, the technical staff was able to overcome many configuration areas they struggled with. The Weblink team was also able to score bigs wins as they were able to quickly come up with the right configuration mix and correct Elasticsearch cluster configuration to achieve this task.
Goals:
High load search capacity
Search intensive
Up to 400,000++ search/browser users daily
Ultra-fast search, with the right cluster optimization, to support up to 5000 transactions per second (TPS) servicing their 2+ million customers.
Solution -ElasticSearch Expert Consulting package with Rescue and Realign
To achieve such a high transaction per second rate, we offer the ElasticSearch Expert Consulting package with Rescue and Realign services to solve the dilemma. After working on so many Elasticsearch engagements, we have identified many implementation strategies and patterns that are proven to give results to any project. To reach the desired SLA’s, we used 2 of our implementation patterns in Elasticsearch, The Cluster Head Pattern and Shading for speed optimizations pattern. We manage to exceed expectations and deliver well within budget by quickly identifying and removing known Elastic anti-patterns.
With Elastic, It’s easy to get yourself in trouble after you’ve done a lot of development and you have tons of data in your system. (D. Miller)
After every enhancement to the clients configuration and shard pattern, we quickly seen improvements in their performance.
How we work -. Starting in the right configuration really can make a difference when you need to scaling in different ways and overall Cluster performance. When it comes to speed, tuning your indexes to the right size and shard configuration can determine how well your system makes the most of your hardware. This is crucial to achieving the fastest possible spots times. It is also essential to understand the data sets, their size, and structure. Know what things you need and what things you can do without in also important too. Often times clients return all data fields from a query, when they only are using a handful of these fields.
Our approach – “Index Optimizations & Expert level ElasticSearch Cluster tunings“
Intensive streamlining of the datasets can achieve an overall size reduction of response payload by a factor of 50% smaller or more. This superior optimization can easily increase response times by 20% to 30%. There are also addtional side benefits like lower network use. We also applied 3 more intensive tunings (not discussed here) that when combined resulted in an overall 60% improvement in search performance
Results, Return on Investment and Future Plans
The 99th percentile response times in the final configurations were below 400 ms @ 1000+ concurrent users!
Better Quality for Customers – Picks the best content for the client
Greatly improves the overall experience of new and existing customers by displaying contact based on analytics and rules.
Increase screen rendering times by 60%
Saved over $500,000 in (x-pack) software license cost
Reduced overall hardware foot-print, thus resulting in a total saving in overall hardware cost
50% reduction of network payload size
Results Approbation
The efficiency of our techniques and methods are reflected in the positive and thankful attitude of our customers. After applying our extensive experience of work with the Elasticsearch and exceptional optimization patterns the clients are delighted with the results achieved. The grasp of business intent behind the project and passionate attitude to work help our team to resolve the tasks of merely any complexity.
Client Testimonials
Customer Wins: Sling TV/Dish Network in Denver CO
Weblink recently completed a large implementation of a 200+ node cluster for Dish networks in Denver CO. Our cluster design handled ingestion of several Tera-bytes daily with very high data ingestion rates. We success fully architected and design the correct solution for Dish Networks.
Customer Wins: Walgreens in Deerfield IL
Weblink designed and built the Patented (Core Search API) which features an Innovative Relevance GUI which allows for dynamic ingestion of Elasticsearch DSL queries that allows Walgreens to make on the fly search response changes without changing code in the backend.
Contact us at [email protected] to learn more about how we can help you leverage Elastic products for high-performing, easy-to-maintain, and scalable search and analytics solutions.
Boost your ElasticSearch project / Don’t get stuck with your ElasticSearch project
Change your relationship with data. Apply ElasticSearch to detect failures and keep your cluster safer and available. Weblink team has extensive experience in working with clients all over the world and satisfying merely all possible needs. Try our professional services offering, (Elastic-EAAS) to get detailed technical solutions support, including
- Multitenant-capable full-text search
- Logging,
- Search Platforms
- Support of multiple programming languages
- Lower Cost – Monitoring, reporting, and security of data
- Search API’s,
- Query performance
- Large Elasticsearch
- Data Lake (200) + nodes Clusters
- Graph exploration
- Mapping
- Testing
- Kibana setup
- Simple REST-based APIs
- Operational Analytics
- Monitoring Pipeline Design
- Elasticsearch consulting and strategy planning
- Initial Cluster setup and Design and many more.
Contact us at [email protected] or (312) 470-0340 to learn more about how we can help you leverage Elastic products for high-performing, easy-to-maintain, and scalable search and analytics solutions.