What “Data Readiness” Actually Means for GenAI: A Technical Breakdown

The Gap Between a ChatGPT Demo and a Production System

AI looks and feels easy to implement, and that’s intentional. That’s how the AI deployment companies market it, how the AI influencers have risen to fame, and how your organization’s leaders have pressed you to “find ways to use AI.”  

But the reality is far different. You can’t just set up an account, hook it up to a few other systems, and get moving. The distinction between uploading a PDF to ChatGPT to ask questions about it and building a fully functioning chatbot that queries a database of PDFs is huge. Small-scale and production-scale just aren’t the same, and a lot can go wrong in the space between.

By now, you’ve come across the statistic that stops decision-makers in their tracks: a large share of GenAI pilots fail to reach production. Pair that with the rapidly rising token bills, and it’s becoming increasingly obvious that there’s significant risk of failure –– costly failure –– when launching generative AI systems for your business.

Much of this risk comes down to your data and whether it’s ready for AI. After all, “data readiness” isn’t merely a checkbox of tasks to complete. It’s a list of technical questions about where your data lives, how it’s structured, how it changes, and who’s allowed to see it. If you get the answers to these questions wrong, bigger and more expensive models won’t save you. You will need to completely rebuild the foundation.

Ignore the One Big Myth

When starting AI discussions with clients, we often come across the big myth in this work: “If we can get a ChatGPT demo working on a few PDFs, scaling it to production is just a matter of pointing it at more data.”

The gap between a proof of concept and a production-ready AI solution is absolutely bigger than you think. The rest of this post is about why. It’s a massive chasm to be bridged, carefully and methodically.  

Further, it’s data readiness, not the model you move forward with, that determines if you make it to the other side.  

After all, if this process was easy, the failure rate would be significantly below 95%. Execs wouldn’t be getting blindsided by token bills. And our team would be less busy, not solving these very real problems.   

Where Would a Human Look?

The starting point for your data readiness assessment is identifying the data, data locations, and systems a human would use to complete the job eventually meant for the AI system.  

This includes any data you pull from other applications into your main system. For example, if you leverage integrated data from Airtable for your mission-critical FileMaker application, Airtable is a critical data location. If your employees reference documentation that lives in PDFs in Confluence, that’s something to note as well. This supplemental information is what you’d hand a new hire to explain where your data and documentation live.

Once you have mapped out where the relevant data exists, you then determine how to access it, how often it changes, and whether it’s structured or unstructured.

Structured vs. Unstructured Data: Two Very Different Problems

Structured Data

You can access your structured data via traditional SQL-style calls or APIs, and your service provider or database determines the frequency it’s updated. In the past when you’ve done projects with this data, they’re likely closer to a BI project than a traditional machine learning project. You simply query the data and serve it up.

Unstructured Data

Unstructured data usually comes from an existing Confluence page or document folder. Documents are loaded into a vector database, where they’re chunked into smaller pieces, converted into vectors, and stored. When a user asks a question, it gets converted into a vector, and the database looks for the chunk that matches best.

Keeping this data updated and usable is an involved process — and entirely your problem. You must determine how often the source data changes, how it gets updated, and the consequences of using stale data. This is why structured-side pre-processing is only becoming more important: as teams work to reduce token usage (i.e., costs), only ingesting and returning relevant data becomes a competitive necessity. We’re all in a race against the clock as model providers slow down their efforts to subsidize input costs.

Until LLMs, unstructured data was difficult to work with. Older NLP approaches could extract some value, but they were expensive, slow, and required significant expertise. Almost overnight, LLMs made unstructured data usable to the masses.

Why Unstructured Data Is Hard

Consider this thought experiment: When searching through a magazine, how would you define an ad versus everything else? Sit and think about this for a minute. My colleague (who works in marketing) answered, “It’s something that tries to sell you something.” Her answer is correct, but it’s vague and not easy to measure against. How would a computer recognize that? How do you define “trying to sell something”?

I countered with more concrete details: a dollar sign, contact information, image-heavy layouts, terms like “low prices,” “limited time offer,” or “discount code.” Even then, you still might get documents like articles with financial data returned to you.   The reason this is important is that you don’t want advertisements or other miscellaneous information polluting your vector database.  Creating an AI enhanced ingestion pipeline to recognize what’s an ad, what’s an article, and what’s a photo inside a magazine is work almost nobody has done before. Splitting up unstructured documents into meaningful pieces and tagging them isn’t a simple small setup step. It becomes its own project.

Quality, Access, Volume: What Each One Actually Means Technically

Data Quality

You can build the perfect AI solution, but if it queries a messy or incomplete database, you’re going to get messy and incomplete answers returned to you. That’s not a GenAI problem; that’s a data quality problem.

This is where data governance comes in – moving forward, you need standards set for your records (that you follow!). You need to clean up your past records as well. This is no small effort, so plan accordingly.

Labeling, Governance, and Access

GenAI makes it easy for users to access data they shouldn’t ever see. Sometimes these events are accidental; sometimes they’re malicious. Regardless, you’ve got to protect your data from it. This is why permissions matter. Your AI system shouldn’t be able to serve salary information to everyone. Your operations team shouldn’t be able to access the database of HR complaints.  

Don’t skip this part of the assessment. Missteps here can have massive implications down the road. Sometimes, a use case just isn’t worth the risk, and you’ve got to skip implementation and move on to the next one.

Volume and Latency

Now you’ve got to consider latency. Every GenAI solution will have inherent latency; there’s nothing you can do about it; only the model providers have control there.  

As it’s built into your system whether you like it or not, you’ve got to pivot to match the wait time with the expected user experience. For example, the baseline for simple queries is 3-5 seconds. Complex queries, however, with more joins and tables, have a realistic wait time of 30-60++ seconds.  

When designing your system, you’ve got to understand the latency you’re up against so you can match it to the targeted user’s tolerance level. Most people will get frustrated with a chatbot conversation that takes 30 seconds to return answers. For batch processes that run overnight, however, 45 seconds per call is fine.

Most Clients Think Their Data Is Perfect (and They’re Almost Always Wrong)

Our variety of clients provides us with exposure to all kinds of data. And all stages of disarray and disorganization. In fact, a significant part of our job is gently letting our clients know that their data is in bad shape.  

Execs get to see sharp dashboards and finished products, not the data engineer sweating through the cleanup process to get there. When implementing GenAI, all of these raw data problems come to light.  

For example, a past client conducting critical research used an Excel spreadsheet as their “database.” It wasn’t even a shared document; it was emailed back and forth. The versioning issues alone were exhausting.  

The reality is that you’ll be cleaning up your data processes in parallel with development.

Governance and Compliance: Where Your Data Actually Goes

This is a sticky subject. Everyone wants to go the least expensive route – launch a Pro account and go. Avoid expensive token usage as much as possible.  

Unfortunately, for financial services, government organizations, health care, and more, Personally Identifiable Information (PII) becomes a critical security issue when you go that route. They can’t just send data off to model providers, where it is used for model training, or exposed in an unrelated user chat later. Opt-out settings aren’t a fail-safe; we read headlines every day about providers leveraging user inputs.

This is why we use AWS Bedrock for most of our clients’ AI solutions. It functions as a copy of the models but protected within Amazon’s infrastructure. Nothing actually gets sent to the providers. This, paired with Virtual Private Clouds (VPCs) and standard AWS access controls, delivers a better setup for keeping your data locked down.

Sure, you’re technically sharing data with Amazon, but they deliver a stronger, more articulated level of trust, which you can read more about in their Bedrock Data Protection documents.  

Note that agentic systems exacerbate the issue. Hard permission filters become non-negotiable when your AI system can chain queries on your users’ behalf. Things can go awry very quickly and get far down the road before being caught.

Permissions, Documentation, and Metadata: The Overlooked Layer

Permissions

This is often the biggest roadblock, as every database product handles permissions differently. You must get through the difficult process of ensuring your AI solution maps to existing permission groups, one-to-one.

Documentation for Unstructured Data

This is a slightly easier step. Create a folder, point the AI system at it, and you’re done.  

Note: this doesn’t mean your data is good. But the retrieval targets are well-defined. The system can get to the data.  

Metadata: The Black Hole

Your metadata is extremely valuable, but it’s also extremely easy to overdo this step.  

Good metadata lets you narrow a search before the semantic layer even runs. This can save you tokens in the context window (i.e., money), reduces latency, and improves accuracy. However, every filter you add is a filter that can exclude the right answer. Over-filtering on metadata kills the value of semantic search by shrinking the result pool before the semantic layer even runs. The more aggressive you get with metadata, the more you’re relying on your tagging being perfect, which it never will be.

For some clients, metadata doesn’t even exist when we start our AI delivery engagements. Generating this data becomes an entire project on its own to complete first.  

You have to make the call (or work with a partner who can help you make this call) on if you leverage your metadata.

How Long Does Getting Ready Actually Take?

Here’s a very honest take, built on years of prepping client’s data for not just AI solutions but also any application that uses data (all of them): You’ll never solve all your problems, and if you try to do so before starting to build your AI solution, you’ll end up waiting forever.  

We recommend solving 50-70% of the known issues before you start development. You can handle the rest as an ongoing and agile process in parallel to your build. To ensure you catch issues before users see them, keep the feedback loop with your data teams tight.

And remember: perfect datasets don’t exist, especially if they include live update processes or worse, manual human entry.

Tools and Environment: What We Actually Use

Assessing the Data

We’ve got to confess: there’s no one-size-fits-all magic tool we use. Instead, we aim to match our techniques and tooling to the size of the datasets.  

For large datasets (think thousands of rows or more), you can’t use an eyeball technique. Instead, we use quick scripts or SQL queries that surface summary statistics, like the number of null/missing values in each column, the range of these values, outliers, unique values per column, and more. This tells us where a potential mess is without us having to look at every row of data.  

For certain small datasets, we just…open it up and take a look. It sounds overly simple, but sometimes the fastest way to determine data’s quality is for a human to just read through it and identify what’s missing.  

After all, a data readiness assessment isn’t a product you can buy; it’s a judgment call you (or a professional) has to make.  

Our Technological Environments

As for the environment, most of our clients’ data isn’t in AWS, even though that’s where we work. We must pull it from systems like Airtable, Google Sheets, or another cloud provider. This isn’t a bad thing, but it’s not a clean starting point either.   

That means we must help our clients decide if the data should live in Amazon at all. Benefits for this route include better latency, potentially lower costs, and one bill to pay each month. But you also add on time and costs before the project even starts for migration.

Unfortunately, this isn’t a one-size-fits-all situation. You’ve got to make the best call for your organization, your goals for your AI solution, and your users.

So Where Do You Actually Start?

Before you start thinking about (and evaluating) models, take a hard look at your data. The first step should be understanding where it lives, how it’s structured, how it changes, and who’s allowed to access it.  

If you want an experienced AI partner to take a look at your data, or even handle your data readiness assessment, that’s exactly what our Rapid GenAI Assessment is designed for. Contact us to learn more about how we help clients with this critical step.