Reduced CDP Costs by 30% with Revefi Data Operations Cloud

ThoughtSpot is an AI-driven analytics platform that provides companies with insights. The company harnessed cutting-edge data processing technologies to help digital business supervise their data assets and run full-fledged self-service analytics.

The main challenge of keeping ThoughtSpot’s data stack robust is the highly intense, heavy-loaded processing of business users' queries ensuring stable workflows for 600-700 active users.

Working out the weak spots was critical to preserve ThoughtSpot’s output data reliability. Dataset inconsistency and errors could affect end users, who fully trust insights based on the platform’s data. Therefore, ThoughtSpot’s data ops team knew they needed to enhance data observability to address insufficient data governance and management.

The Lowdown

ThoughtSpot’s prime concerns arose from general awareness of gaps in data pipelines and processing. Their data ecosystem has been expanding and growing more complex, which inevitably led to the growing number of data artifacts that could skew metrics calculation. Each new data source integration led to a more complex calculation of corresponding metrics. Those transformations required data integrity and coherence throughout its evolution in ETL and reverse ETL pipelines. 

Pinpointing the origin of those artifacts and outliers was ThoughtSpot’s primary objective in order to keep the outputs consistent and reliable for business users decision-making.

Therefore ThoughtSpot’s main goals were to:

• Reduce time and cost of data issues debugging process
• Automate manual coding to unburden data engineers
• Observe who exactly uses particular data assets and what for
• Identify which missing data or broken pipelines affect downstream users
• Ensure data integrity and cleanliness to get clear and actionable insights from data mining
• Strengthen the trust between the data team and stakeholders from those meaningful insights.

The Deep Dive

The Roadblocks

The key challenge for ThoughtSpot data engineers was to layer data governance and management interface upon the complex data ecosystem they have been operating for years. So, they needed a data monitoring product that would seamlessly fall into the existing architecture and provide informative exploration dashboards.

While researching available data governance solutions, the ThoughtSpot team set their minds on finding the data co-pilot that integrates into the existing data stack instantly. They prioritized the ability to identify and fix data issues without a lengthy debug process. Thus, they chose Revefi as the best-of-breed Data Operations Cloud transforming data observability and ensuring ultimate data governance and management efficiency. This way, ThoughtSpot became an early adopter of Revefi. It’s been using Revefi for about six months now.

Before embracing Revefi, ThoughtSpot:

• faced a lot of flagged data issues and broken data pipelines
• spent a long time manually finding and fixing these problems
• found that the ROI on such efforts was very long.
  ‍

Revefi offered solutions for all ThoughtSpot’s concerns and also helped with the following:

• Data freshness issues
• Ambiguity in understanding the role of data artifacts in the outputs
• Excessive or anomalous data to tackle first-place
• Estimate the ROI of putting the team’s efforts into data issue fixes
• Over-consumption of data warehouse resources.

The Specifics

Revefi-powered top-to-bottom data observability and data governance yielded the ThoughtSpot data ops team tons of benefits. The most note-worthy are:

• Fast and Simple Setup. When ThoughtSpot looked at all the other tools in the data space, they realized that most tools involved a lot of upfront configuration work. The small data team at ThoughtSpot didn’t have the time to contend with that, said Jerin. But with Revefi, Jerin notes, there is no learning curve. ThoughtSpot just went to the setup screen, input its credentials, and clicked connect. Revefi took care of everything else, and it was all automated. The setup process of the Revefi Data Operations Сloud was insanely easy, and the data team received the first data health monitoring alerts right after Revefi consumed metadata.

• Insightful Data Governance and Management. ThoughtSpot’s team obtained a full-fledged tool for identifying the root causes of data issues. They also discovered what unnecessary access permissions provoked the overuse of data artifacts and plugged the loopholes that could endanger data security.

• Enhanced Data Integrity. It became fairly simple for the data quality team to detect and correct outdated and misplaced data points. They finally managed to track the use scenarios of particular assets and decide whether some of them should be terminated, updated, or have discoverability problems.

• Data Freshness. Revefi ensured real-time updates on the freshness of ThoughtSpot’s data. With Revefi, ThoughtSpot’s team gets alerted when something is wrong, such as when artifacts are affected by missing data or a broken pipeline. AI-powered data governance and management eliminated tons of manual effort.

• Automatic, Instant Access to Data Lineage. Within moments, Revefi had all lineage details ready for ThoughtSpot. Before adopting Revefi, it could have taken ThoughtSpot about 24 hours to get that information and solve a data issue. But now they can solve issues in just 15 minutes.

• Improved Trust from the Board Members. Adopting Revefi for data governance and management strategy brought better visibility into all cloud data-related processes and unique insights into data performance & use, thus fostering enterprise-wide acceptance of data strategies alongside with truly data-driven decision making.

• Cost and Spend Management Ensuring Higher Data Warehouse ROI. Owing to optimized data access and data cleansing practices, the company cut 30% of cloud data platform costs while boosting its usage by 35%. Revefi has enhanced ThoughtSpot's awareness of spending patterns, providing insights into processes and workflows that consume the budget.

The Outcome

Revefi is a one-stop solution for data governance, data quality, and cost management. It empowers ThoughtSpot to be more strategic in how it works with and plans for data.  

Revefi automates ThoughtSpot’s systems and makes them work much faster and easier. If there are unwanted users, Revefi points it out. ThoughtSpot experienced such an incident just recently, and after receiving the alert from Revefi, it revoked that user’s permission right away. A bulletin from Revefi recently revealed that ThoughtSpot had exhausted its credits. After receiving this news, ThoughtSpot was able to act to address that issue within five minutes. Revefi also helps companies like ThoughtSpot avoid running out of credits and stay on course. And the 30% in savings that Revefi enabled yielded ThoughtSpot a six-figure cost savings!

With Revefi, Jerin says, he and everybody else at ThoughtSpot sleep peacefully at night.

ThoughtSpot achieved significant business outcomes with Revefi’s automated data observability and, as a result – data governance and management improvements.

On top of that, ThoughtSpot is elated with the way Revefi skyrocketed the productivity of the data team. The 3X less time spent on data issues debugging is an incredible payoff and definitely worth investing in a monthly subscription.

Contact the Revefi team today and get started with the top benefits of data quality monitoring.