# AI Training Data: What It Is and Where It Comes From

> AI training data is the text, code and records models learn from. See where it comes from, why rights checks matter, and whether your company's data counts.

Source: https://sellmydata.com/guides/what-is-ai-training-data/
Published: 2026-10-28 · Updated: 2026-10-28

## Key takeaways

- AI training data is the text, code, images and records a model learns from. Training is often described in two stages: pre-training (broad initial learning on large amounts of text) and post-training (later tuning on narrower, higher-quality examples).
- It comes from six broad sources: public web crawls, open datasets, licensed publisher and platform content, synthetic data, human-written or annotated examples, and real business records.
- Public human-written text is finite, and research warns against indiscriminately training on a mix of real and AI-generated content. That makes real, rights-checked, de-identified business data a distinct source.
- Errors in training data can make wrong answers worse, retracted research still turns up in AI answers, and training on unlicensed works has led to litigation such as the $1.5 billion Bartz v. Anthropic settlement. Provenance and rights checks belong before any data is shared.
- Companies can license de-identified business data such as messages, code, documents and tickets, and keep approval over each buyer, the terms and the price.

## What is AI training data? (short answer)

[AI training data](/glossary/ai-training-data/) is the material a model learns from: text, code, documents, images and records. The model studies patterns in that material so it can produce similar output later. What goes in, including its quality, its range and its legal status, shapes what the model can do and what risks come with it.

Training is often described in two stages:

- **Pre-training:** broad initial learning on very large amounts of text, so the model picks up language, facts and general patterns.
- **Post-training:** later tuning on narrower, higher-quality examples, so the model follows instructions or handles specific tasks well.

Data can also play different roles. These are working definitions, not strict categories:

| Type | What it is | What it is used for |
|---|---|---|
| Training data | The main body of material the model learns from | Building general ability |
| Fine-tuning data | A smaller, targeted set used after pre-training | Teaching a specific skill, tone or domain |
| Evaluation data | Examples held back and not used for training | Testing how well the model really performs |

Researchers such as Epoch AI measure text in **tokens**. A token is a small chunk of text, roughly a word or part of a word.

### What AI training data looks like in practice

In practice it looks like ordinary digital work. Here are some examples, described in general terms:

- **Chat threads:** a series of messages with speakers, timestamps and replies.
- **Code repositories:** source files plus their history of changes and review comments.
- **Support tickets:** a customer question, the back-and-forth with staff, and the resolution.
- **Documents:** reports, policies, proposals and specifications.

## Where does AI training data come from?

AI training data comes from six broad sources: public web crawls, open datasets, licensed content, human-written or annotated examples, synthetic data, and real business records. Each source has a different trade-off between scale, quality and rights clarity.

| Source | Strength | Main limit |
|---|---|---|
| Public web crawls | Huge scale, freely available | Uneven quality, bias, rights vary by source |
| Open datasets | Easy to access | Many are derived from the same crawls |
| Licensed publisher and platform content | Agreed terms | Terms may be confidential |
| Human-written and annotated examples | Targeted to a task | Costly at scale |
| Synthetic data | Produced by a model rather than collected | Indiscriminate use linked to model collapse |
| Business records | Real workflows, generally not on the public web | Needs rights checks and de-identification |

### Public web crawls and open datasets

Web crawls provide the scale. [Common Crawl](https://www.mozillafoundation.org/en/research/library/generative-ai-training-data/common-crawl/), a small nonprofit, has built a freely available archive of web crawl data of more than 9.5 petabytes, dating back to 2008. AWS describes it as a corpus of [over 300 billion web pages](https://registry.opendata.aws/commoncrawl/). Since OpenAI published GPT-3 in 2020, Common Crawl has become [one of the most important sources of training data](https://www.mozillafoundation.org/en/research/library/generative-ai-training-data/common-crawl/) for generative AI. [Many pre-training corpora](https://arxiv.org/pdf/2402.18041), the large AI training datasets used in the first stage, are produced by cleaning and filtering Common Crawl data, so open datasets and web crawls often overlap.

The limits are real. The Mozilla Foundation notes that the archive is [far from representing the entire web](https://www.mozillafoundation.org/en/research/library/generative-ai-training-data/common-crawl/), and that assuming otherwise glosses over the shortcomings and biases in the data. Researchers who studied a [1% sample of Common Crawl](https://arxiv.org/pdf/2105.02732) looked for hate speech and adult content. They also tested perplexity-based filtering, which uses a language model to score how typical a piece of text looks and drops text that scores poorly.

The supply also has a ceiling. Epoch AI estimates the total effective stock of human-generated public text at [roughly 300 trillion tokens](https://epoch.ai/publications/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data), with a 90% confidence interval of 100 trillion to 1,000 trillion. The Associated Press reported that Epoch's study projects tech companies will [exhaust publicly available training data](https://www.courthousenews.com/ai-gold-rush-for-chatbot-training-data-could-run-out-of-human-written-text/) for language models sometime between 2026 and 2032. The same report noted that a limited supply could put pressure on companies to tap data now considered private, such as emails or text messages. That is context, not a forecast about any one company. Licensing business records is a separate matter: it is a choice a company makes, covered below.

### Licensed content from publishers and platforms

Some AI developers pay for content under licensing agreements. A license is a contract that grants permission to use data on agreed terms. On [22 February 2024](https://pressgazette.co.uk/platforms/news-publisher-ai-deals-lawsuits-openai-google/), Reuters reported that Reddit struck a deal with Google to make its content available for training Google's AI models. According to one of Reuters' sources, the 2024 deal was [worth about $60 million per year](https://tech.yahoo.com/ai/articles/exclusive-reddit-ai-content-licensing-041012503.html). Reuters itself has confirmed that it [licenses news content to AI developers](https://pressgazette.co.uk/platforms/news-publisher-ai-deals-lawsuits-openai-google/), with terms kept confidential. For a plain-English walkthrough of how these agreements work, see [What Is AI Data Licensing?](/guides/what-is-ai-data-licensing/)

### Human-written and annotated examples

Human-created data is best understood as labor rather than an existing supply. People write examples to order, or annotate them, meaning they label, rank or correct material so a model learns what a good answer looks like. This work feeds post-training, but it is expensive at scale. According to Epoch, as reported by VOA, paying millions of people to write text for AI training [is unlikely to be an economical way](https://www.voanews.com/a/ai-gold-rush-for-chatbot-training-data-could-run-out-of-human-written-text-/7647911.html) to drive better technical performance.

### Synthetic data

Synthetic data is text or other material generated by an AI model rather than by people. It can be produced on demand, but it carries a known risk. A study in [Nature](https://www.nature.com/articles/s41586-024-07566-y), published on 24 July 2024, found that indiscriminately training on a mix of real and AI-generated content can lead to a collapse in a model's ability to produce diverse, high-quality output. The authors concluded that sustaining learning over time requires preserving access to the original data source and keeping data not generated by language models available.

### Real business records

Business data is the workplace material a company produces as it operates: internal messages, code, documents, records and support tickets. Because it sits inside private tools, it is generally not part of public web crawls. Much of it is written by people, which is the kind of data the Nature authors say should stay available.

On our exchange, licensing business records is the company's decision. The company chooses which tools and date ranges to export, and people's names, contact details and identifiers are replaced with consistent placeholders before anything is listed ([de-identification](https://sellmydata.com/de-identification/)).

### Where to get AI training data if you are a buyer

If you are looking to buy AI training data, the same sources apply: open crawl archives and datasets, publishers and platforms that sign licensing deals, annotation and writing services, and companies that license their own de-identified records. Whichever AI training data companies or partners you compare, ask for evidence of where the data came from and what rights the seller holds. [How to Sell Datasets](/guides/how-to-sell-datasets/) covers what buyers typically check.

## What licensed business data adds for buyers

Licensed business data is largely human-written, mostly comes from outside the public web, and is licensed on agreed terms. Those three traits line up with the limits of the other sources described above.

### Real workflows and long histories

Business records show how work actually happens. Examples include a support ticket followed from complaint to fix, a code change together with its review, or years of discussion inside a single team. These records can carry domain language and context that is not on the public web.

### Data quality and model behavior

Data quality is one factor in how a model behaves. A hallucination is a confident but wrong answer from a model. Researchers from OpenAI and Georgia Tech wrote in September 2025 that language models [sometimes guess when uncertain](https://arxiv.org/pdf/2509.04664), producing plausible but incorrect statements instead of admitting uncertainty. Their paper says training data [inevitably contains errors and half-truths](https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf). Even error-free data would not remove hallucinations, because of the objectives models are trained on, and realistic data with errors in it may lead to even higher error rates. In other words, cleaner data does not stop hallucinations, but errors in the data can make them worse.

### Agreed terms and a rights check

The U.S. Copyright Office has urged the market to [develop voluntary licensing solutions](https://devthrottle.com/ai-library/milestones/2025-copyright-office-ai-part-3-training), and licensed data is used on terms both sides agree. Before anything is listed, a [rights check](https://sellmydata.com/rights/) reviews customer contracts, privacy policy, employee communications and third-party content with the seller. This is general information, not legal advice: whether a given use is permitted depends on the specific agreements and the law that applies, so both buyers and sellers should review terms with their own counsel.

## Trust issues: errors, bias and copyright in training data

Errors, bias and copyright questions are best handled before data is shared. The research below shows that flawed material can keep surfacing in AI answers, and that disputes over training on unlicensed works have reached the courts.

### Retracted research can keep turning up

Retracting a paper does not stop AI tools from citing it. MIT Technology Review reported that, according to recent studies, some AI chatbots [rely on flawed research from retracted scientific papers](https://www.technologyreview.com/2025/09/23/1123897/ai-models-are-using-material-from-retracted-scientific-papers/) to answer questions. In one study, ChatGPT (GPT-4o) was asked about 21 retracted medical imaging papers. Its answers referenced retracted papers in five cases but advised caution in only three. A separate [JMIR study](https://www.sciencedirect.com/org/science/article/pii/S1438887126004292) found that none of the AI tools it evaluated consistently handled retracted articles correctly, and no model achieved perfect accuracy across all question sets. These reports describe what the tools said, not exactly where the material came from, but they show why it helps to check what goes into a dataset.

### Bias and a narrow source mix

Bias, in this context, means a model's output systematically leans one way because its data over-represents some views, groups or contexts and under-represents others. A narrow or skewed source mix can contribute. The Mozilla Foundation's warning that web crawl archives carry [inherent shortcomings and biases](https://www.mozillafoundation.org/en/research/library/generative-ai-training-data/common-crawl/) is one example.

### AI training data and copyright

The questions around AI training data and copyright are not settled, and what follows is general information, not legal advice. On 9 May 2025, the U.S. Copyright Office released a pre-publication version of [Part 3 of its AI report](https://authorsguild.org/news/us-copyright-office-ai-report-part-3-what-authors-should-know/), titled Generative AI Training. Fair use is the U.S. legal doctrine that allows some uses of copyrighted work without permission. The report [declined to give a blanket answer on fair use](https://devthrottle.com/ai-library/milestones/2025-copyright-office-ai-part-3-training), stressing that all four statutory factors must be weighed case by case. Instead of recommending new legislation or a compulsory license, it urged the market to develop voluntary licensing solutions. It also [rejected the view](https://uk.practicallaw.thomsonreuters.com/w-046-9134?transitionType=Default&contextData=%28sc.Default%29) that training on a copyrighted work is inherently transformative, while recognizing that many such uses may be. Transformative means the use adds a new purpose or character rather than simply copying.

Courts are also involved. On 20 July 2026, Judge Araceli Martínez-Olguín of the U.S. District Court for the Northern District of California granted [final approval of a $1.5 billion class action settlement](https://authorsguild.org/news/court-grants-final-approval-anthropic-copyright-settlement/) in Bartz v. Anthropic, a case over unauthorized use of authors' works in AI training and development. According to one law firm's summary, the settlement covered works [allegedly torrented and commercially exploited from pirated datasets](https://www.pearlcohen.com/federal-court-approves-1-5-billion-anthropic-copyright-settlement-largest-in-history/). How these developments apply to any particular company depends on its own contracts and jurisdiction, so check with counsel.

### What this means for sellers

For a company, the practical lesson is to do the provenance work before any data leaves your hands. Provenance means a clear record of where data came from and who has rights to it. In practice, that means reviewing your contracts and policies with counsel and removing people's identifying details first.

## How is AI training data stored and prepared?

Before data is used for training or delivered to a buyer, it should be cleaned, filtered and formatted, and people's identifying details should be removed. A sensible practice is to keep raw and prepared versions apart and to tightly limit access to raw exports.

Cleaning can include:

1. **Deduplication:** removing repeated copies of the same content.
2. **Filtering:** dropping low-quality, off-topic or harmful material, for example with perplexity-based scoring.
3. **Formatting:** putting data into a consistent structure, such as one record per conversation or ticket.
4. **Removing personal identifiers:** stripping names, contact details and IDs.

That last step is [de-identification](/glossary/de-identification/). In the approach described on our [de-identification page](https://sellmydata.com/de-identification/), names, contact details and identifiers are replaced with consistent placeholders. The same person becomes the same placeholder throughout, so a conversation still makes sense without revealing who was in it. That page also explains the limits of the method. De-identification is related to, but not the same as, [anonymization](/glossary/anonymization/). Our guide [How to Anonymize Data for AI](/guides/how-to-anonymize-data-for-ai/) covers the methods and, separately, the general legal definitions.

On storage, ask any data partner three questions: where raw exports are kept, who can access them, and when they are deleted. Format and delivery are worth agreeing in writing for each dataset.

## Can your company's data become AI training data?

It can, if it is real business data, your company has the rights to license it, and people's identities are removed first. Whether you hold those rights depends on your customer contracts, policies and other agreements, so check with counsel.

The business data SellMyData works with falls into five types: **messages, code, documents, records and tickets**. We do not buy consumer [personal data](/glossary/pii/), health records tied to people, or payment card data. There is no minimum headcount and no sales call to list.

Here is how the process works:

1. **Get an estimate.** Answer a few questions to see an estimated price range before you share any data. A person reviews every request and replies by email. The [homepage calculator](https://sellmydata.com/) gives an indicative range. It assumes you keep your full history and license it exclusively, and it is not an offer ([how the estimate works](https://sellmydata.com/pricing/#how-we-estimate)). To size your data first, see [How to Check How Much Data You Have in Slack](/guides/how-to-check-slack-data-volume/).
2. **Export what you choose.** You export from the tools and date ranges you pick. Nothing is ever written to your systems. Exports are kept in a restricted folder accessible to the founder only and deleted within 30 days of the de-identified dataset being approved ([security](https://sellmydata.com/security/)).
3. **Cleaning, de-identification and a rights check.** SellMyData cleans and prepares the data, de-identifies it, and runs a [rights check](https://sellmydata.com/rights/) with you (customer contracts, privacy policy, employee communications, third-party content) before anything is listed.
4. **You approve.** You approve the prepared dataset. Buyers see only a redacted sample before purchase, and a licensed lab receives the de-identified dataset, never raw data ([for labs](https://sellmydata.com/labs/)).
5. **You sign off on every deal.** Nothing is licensed without your approval of the buyer, the terms and the price. Scope, format, [exclusivity](/glossary/exclusive-license/) and term are agreed for each dataset, and you can unlist and delete your data at any time ([trust](https://sellmydata.com/trust/)).

Sellers pay nothing up front. We take a commission only when a sale closes, with a target of payment within 7 days of closing. There is no published price list: [price depends](https://sellmydata.com/pricing/) on data type, volume, years of history and current lab demand.

For the full step-by-step, read [How to Sell Your Company's Data to AI Labs](/guides/how-to-sell-company-data-to-ai-labs/). For the wider picture of turning data into revenue, see [How to Monetize Data](/guides/how-to-monetize-data/).

## Frequently asked questions

### What is AI training data?

AI training data is the material a model learns from, such as text, code, documents, images and records. The model studies patterns in that material so it can produce similar output later. Its quality, range and legal status shape what the model can do and the risks that come with it.

### Where does AI training data come from?

It comes from public web crawls and open datasets, content licensed from publishers and platforms, human-written and annotated examples, synthetic data generated by models, and real business records. Web crawls supply scale, while licensed and business sources come with agreed terms and include material that is generally not on the public web.

### What does AI training data look like?

It looks like everyday digital work: chat threads with speakers and timestamps, code repositories with their change history, support tickets from first question to resolution, and documents such as reports and policies. Before use, it should be cleaned, filtered and formatted, with people's identifying details removed.

### How is AI training data stored?

A sound setup stores the raw export separately from the cleaned, prepared dataset used for training, and tightly limits who can open the raw files. If you share data with any partner, ask where raw exports are stored, who can access them and when they are deleted.

### Where can I buy AI training data?

Options include free web crawl archives, open datasets, licensing deals with publishers and platforms, annotation and writing services, and exchanges that license de-identified business data with each seller's approval. Before buying, check where the data came from, what rights the seller holds and how personal details were removed.

### Does AI training data raise copyright issues?

It can, and the questions are not settled. In a May 2025 pre-publication report, the U.S. Copyright Office said fair use must be weighed case by case, and in July 2026 a court approved a $1.5 billion settlement in Bartz v. Anthropic. This is general information, not legal advice, so check your own situation with counsel.

### Can corrected errors stay in AI training data?

Flawed material can keep surfacing in AI answers after it has been corrected at the source. MIT Technology Review reported that some AI chatbots rely on research from retracted scientific papers; in one study, GPT-4o referenced retracted papers in five of 21 cases but advised caution in only three. That is one reason to check what goes into a dataset before it is shared.

### Can my company's data be used as AI training data without exposing people's identities?

Yes, if people's names, contact details and identifiers are removed first, though de-identification has limits (see https://sellmydata.com/de-identification/). In that approach, identifiers are replaced with consistent placeholders. Pair it with a rights check, and approve the final dataset yourself before anything is licensed.

## Sources

- [Common Crawl (Generative AI Training Data research)](https://www.mozillafoundation.org/en/research/library/generative-ai-training-data/common-crawl/)
- [Common Crawl – Registry of Open Data on AWS](https://registry.opendata.aws/commoncrawl/)
- [arXiv paper 2105.02732 (study of a 1% sample of Common Crawl)](https://arxiv.org/pdf/2105.02732)
- [arXiv paper 2402.18041 (survey of pre-training corpora)](https://arxiv.org/pdf/2402.18041)
- [Will We Run Out of Data? Limits of LLM Scaling Based on Human-Generated Data](https://epoch.ai/publications/will-we-run-out-of-data-limits-of-llm-scaling-based-on-human-generated-data)
- [AI 'gold rush' for chatbot training data could run out of human-written text](https://www.courthousenews.com/ai-gold-rush-for-chatbot-training-data-could-run-out-of-human-written-text/)
- [AI 'gold rush' for chatbot training data could run out of human-written text](https://www.voanews.com/a/ai-gold-rush-for-chatbot-training-data-could-run-out-of-human-written-text-/7647911.html)
- [Exclusive: Reddit in AI content licensing deal with Google](https://tech.yahoo.com/ai/articles/exclusive-reddit-ai-content-licensing-041012503.html)
- [Who's suing AI and who's signing: news publisher AI deals and lawsuits tracker](https://pressgazette.co.uk/platforms/news-publisher-ai-deals-lawsuits-openai-google/)
- [AI models collapse when trained on recursively generated data](https://www.nature.com/articles/s41586-024-07566-y)
- [Why Language Models Hallucinate (arXiv 2509.04664)](https://arxiv.org/pdf/2509.04664)
- [Why Language Models Hallucinate](https://cdn.openai.com/pdf/d04913be-3f6f-4d2b-b283-ff432ef4aaa5/why-language-models-hallucinate.pdf)
- [AI models are using material from retracted scientific papers](https://www.technologyreview.com/2025/09/23/1123897/ai-models-are-using-material-from-retracted-scientific-papers/)
- [JMIR study on AI tools' handling of retracted articles (DOI 10.2196/88766)](https://www.sciencedirect.com/org/science/article/pii/S1438887126004292)
- [US Copyright Office AI Report Part 3: What Authors Should Know](https://authorsguild.org/news/us-copyright-office-ai-report-part-3-what-authors-should-know/)
- [2025 Copyright Office AI Report Part 3: Training](https://devthrottle.com/ai-library/milestones/2025-copyright-office-ai-part-3-training)
- [Practical Law note w-046-9134 on Copyright Office Part 3 report](https://uk.practicallaw.thomsonreuters.com/w-046-9134?transitionType=Default&contextData=%28sc.Default%29)
- [Court Grants Final Approval of Anthropic Copyright Settlement](https://authorsguild.org/news/court-grants-final-approval-anthropic-copyright-settlement/)
- [Federal Court Approves $1.5 Billion Anthropic Copyright Settlement, Largest in History](https://www.pearlcohen.com/federal-court-approves-1-5-billion-anthropic-copyright-settlement-largest-in-history/)
- [De-identification](https://sellmydata.com/de-identification/)
- [Rights check](https://sellmydata.com/rights/)
- [Pricing and how we estimate](https://sellmydata.com/pricing/#how-we-estimate)
- [Security](https://sellmydata.com/security/)
- [For AI labs](https://sellmydata.com/labs/)
- [Trust](https://sellmydata.com/trust/)
- [SellMyData homepage and estimate](https://sellmydata.com/)