Over 35 years of entertainment

Optimising big data queries for large volumes of complex data

22 July 2022 | Noor Khan

Optimising big data queries for large volumes of complex data in Amazon Athena

Key Challenges

Our client's existing system was unable to handle the large volumes of data they were dealing with on a daily basis and required a flexible, robust data reporting engine for time-efficient results.

Key Details

Service

Data Engineering

Technology

Cloud Watch, Cloud Trail, IAM, DynamoDB, SQL, Athena, S3 Data Lake, Angular, API Gateway, Lambda, AWS Batch

Industry

Media

Sector

Broadcasting

Key results

  • Ability to export a large amount of data to Excel
  • Multi-factor authentication to validate users for security purposes
  • Ability to save preferences and default settings for searches increasing efficiency when carrying out future searches
  • Speedy and accurate data results
  • Email alerts to inform users when data results are ready so they do not have to wait around
  • High level of flexibility when carrying out searches with capabilities of combination searches

Multinational broadcasting company

Entertaining and inspiring audiences since 1984

Our client are a well-established, multi-national broadcasting company reaching 300 million subscribers across 150 countries, through 50 channels and in 37 languages. They deal with large sets of complex data audience data ranging from streaming to TV audiences. They need to be able to efficiently carry out searches, for research and sales purposes. Their data sets are sourced from Nielsen AMRLD which is stored in their data warehouse.

Optimising big data queries for large volumes of complex data

Program Search and Market Break Ranker tool

Huge data requires a powerful system

Our client’s existing data search tool performed the search queries extremely slow, frequently timed out because of the large volumes of complex data and did not offer the flexibility required. They needed a tool that would help them gain insights from large amounts of data to help support the research and sales efforts of the business.

Optimising big data queries for large volumes of complex data

Flexible, user-friendly tool that handles data searches on large volumes of complex data

Making big data searches, speedy

The client was looking for an improved data search tool that allowed them to have a Program Search as well as a Market Break Ranker. These two different components were essentially search engines for the two types of data the company dealt with .The Ardent engineering team took on board the requirements and created a tool that optimised big data queries for large volumes of complex data with a user-friendly interface, using API Gateway and Angular technologies. We offered flexibility by adding filters and drop-down menus, allowing the users to select multiple categories for the data results see t.As the client wanted the ability to be able to carry out Boolean logic combination searches, we needed to build a system that could search vast data sets to pull out relevant data results. We were able to provide a solution on Lambda and then moved to AWS Batch to ensure a quick and efficient search of the data.

More on our technology partners.

Optimising big data queries for large volumes of complex data

Ardent's data engineering expertise

Drill down searches with boolean logic

Our expert engineers made recommendations on boolean logic to streamline data searches as we have a vast amount of experience working on a similar scale of data and clients.

Optimised big data queries

Searching huge data sets quickly and easily

Our experienced engineers delivered a fit for purpose data solution, allowing our client to optimise big data queries on large, complex big data with a number of features to make the search process easy and straightforward. This enabled our client to have a quick turnaround time on analysis of the data (household viewing figures), increasing efficiency in the data search process. Overall, they were able to save time, be efficient, and save costs allowing them to focus on other important business goals.

Explore our data engineering services or get in touch to find out how we can help you optimise your big data.


More Success Stories

Automating data collection with OCR technology (1)

Success Story

Automating data collection with OCR technology

Market Research | Retail

Accelerating market research by automating data collection with OCR technology. [...]

Read More... from Over 35 years of entertainment

Ensuring timely data availability for real time mission critical data

Success Story

Monetizing broadcasting data

Media | Media

A market leader, internationally renowned media and broadcasting company Founded in 2002, our client has been around for over two decades and is an internationally known company dealing with broadcasting data for commercial use. With a mission of making high-quality technology and content affordable for everyone, they have established themselves as a market leader. [...]

Read More... from Over 35 years of entertainment

Fine art storage & preservation software

Success Story

Making logistics simple

Logistics | Logistics, Software

Leader logistics software provider Our client is a leading logistics software provider in the UK. With over 3 decades of experience in the industry, they continuously look to innovate with technology. Their range of software products includes a warehouse management system and removal management software. They aim to remove the complexity of software and bring [...]

Read More... from Over 35 years of entertainment

Ardent Insights

Overcoming Data Administration Challenges, and Strategies for Effective Data Management

Businesses face significant challenges to continuously manage and optimise their databases, extract valuable information from them, and then to share and report the insights gained from ongoing analysis of the data. As data continues to grow exponentially, they must address key issues to unlock the full potential of their data asset across the whole business. [...]

Read More... from Over 35 years of entertainment

Are you considering AI adoption? We summarise our learnings, do’s and don’ts from our engagements with leading clients.

How Ardent can help you prepare your data for AI success Data is at the core of any business striving to adopt AI. It has become the lifeblood of enterprises, powering insights and innovations that drive better decision making and competitive advantages. As the amount of data generated proliferates across many sectors, the allure of [...]

Read More... from Over 35 years of entertainment

Why the Market Research sector is taking note of Databricks Data Lakehouse.

Overcoming Market Research Challenges For Market Research agencies, Organisations and Brands exploring insights across markets and customers, the traditional research model of bidding for a blend of large-scale qualitative and quantitative data collection processes is losing appeal to a more value-driven, granular, real-time targeted approach to understanding consumer behaviour, more regular insights engagement and more [...]

Read More... from Over 35 years of entertainment