Data Collection and Scraping Risk Checklist: Sources, Terms, Access, and Output

By Casey Scott McKay · 2025-10-23

Sixteen phases for a data collection program, organized around the source register that replaces a single scraping policy. The early phases classify the material, test whether a gate existed under Van Buren, and determine whether a contract was formed and is enforceable - the sequence that actually decides these disputes. A dedicated phase runs the output analysis, which carries the uncapped statutory damages and is routinely done last or not at all. A worked comparison runs four different sources through the framework to show why one policy cannot cover them. The site-operator phases build the contractual, technical, and evidentiary position that makes a claim available, including seeded records and the API that converts scraping into licensing. Later phases cover the provenance record, retention and deletion, the cease-and-desist exchange from both sides, the injunction, and the governance controls that keep the register current.

IP and Technology > Information Technology | Checklist | Published 23 October 2025 - Updated 13 August 2026 | Casey Scott McKay - marksy.us

Summary. Sixteen phases for a data collection program, organized around the source register that replaces a single scraping policy. The early phases classify the material, test whether a gate existed under Van Buren, and determine whether a contract was formed and is enforceable — the sequence that actually decides these disputes. A dedicated phase runs the output analysis, which carries the uncapped statutory damages and is routinely done last or not at all. A worked comparison runs four different sources through the framework to show why one policy cannot cover them. The site-operator phases build the contractual, technical, and evidentiary position that makes a claim available, including seeded records and the API that converts scraping into licensing. Later phases cover the provenance record, retention and deletion, the cease-and-desist exchange from both sides, the injunction, and the governance controls that keep the register current.

Keywords: source register · material classification · gate analysis · van buren · hiq public data · browsewrap notice · clickwrap assent · account provenance · robots txt · rate limiting · technical measure circumvention · trespass to chattels metrics · output analysis · personal data stripping · biometric exposure · provenance record · retention and deletion · source intake gate · cease and desist response · licensing alternative

What this checklist is for

This is the working document for a scraping or data collection program, on either side. It does not re-teach the doctrine. If you cannot say in one sentence why Feist leaves a dataset unprotected, read Who Owns the Data? first. The reasoning behind each box is in Running or Defending a Data Scraping Program. This document tells you what to do, in order.

Who should use it. In-house counsel at a company whose product depends on collected data; outside counsel advising on a new pipeline or answering a demand letter; site operators building a defensible position against collectors; and diligence counsel who has just been told the target's dataset is "public information."

What you'll need before you start. A list of every source the pipeline touches — from network logs, not from the code, because the two differ; the terms of service for each, captured with a date; the robots.txt for each; the pipeline's ingestion and retention configuration; the account credentials used and how they were obtained; server-side collection volumes; and the product requirements, so you can ask what is actually used.

The worked matter. Wrenfield Labs builds commercial property analytics. It scrapes county assessor records; a commercial listing site behind a free account; a review platform's public pages; and a competitor's published market reports. It has one scraping policy covering all four.

| Phase | What you accomplish | Typical elapsed time | |---|---|---| | 1 | Build the source register | 2-5 weeks | | 2 | Classify the material per source | 1 week | | 3 | Test for a gate | 1 week | | 4 | Test whether a contract was formed | 1-2 weeks | | 5 | Test circumvention and system harm | 3-5 days | | 6 | Run the output analysis | 2-4 weeks | | 7 | Work the four sources to conclusions | 1-2 weeks | | 8 | Segment the pipeline and strip at ingestion | 4-10 weeks | | 9 | Build the provenance record | ongoing | | 10 | Set retention, minimization, and deletion | 4-8 weeks | | 11 | Site operator: build the contractual position | 4-8 weeks | | 12 | Site operator: build technical and evidentiary controls | 4-10 weeks | | 13 | Handle the cease-and-desist, either side | 2-6 weeks | | 14 | Prepare for the injunction motion | 6-10 weeks | | 15 | Price the licensing alternative | 2-8 weeks | | 16 | Install governance and review | ongoing |

Phase 1 — Build the source register

[ ] Enumerate every source from network logs, not from the codebase.
- Why. New sources appear without anyone deciding — a developer added a fallback, an integration changed, a vendor library fetches something on its own.
[ ] Record per source: source name and URL patterns; material type; authentication required; account used and how obtained; terms presented and how; technical measures encountered; robots.txt directives and whether honored; volume and frequency; what is retained; whether personal information is present; the business purpose; and the legal conclusion with its date.
[ ] Replace the single program policy with one row and one conclusion per source.
- Trap. A uniform scraping policy makes every downstream answer wrong for at least one source, and it is the most common structural error in this area.

Phase 2 — Classify the material per source

[ ] Facts — not copyrightable. Feist Publications, Inc. v. Rural Telephone Service Co., 499 U.S. 340, 344-51 (1991).
[ ] Expression — copyrightable: reviews, descriptions, articles, photographs, analyses, and original selection or arrangement of a compilation. 17 U.S.C. § 103.
[ ] Personal information — regulated regardless of whether it is public.
[ ] Trade secrets — only if genuinely secret; material on a public page is not.
[ ] Build the classification into the ingestion code, not into a memo.
- Why. A pipeline that extracts facts and discards expression is in a materially different position from one that stores source documents. That distinction has to be operational.

Phase 3 — Test for a gate

[ ] Ask the Van Buren v. United States, 593 U.S. 374, 389-96 (2021) question: did the collector access an area that was off limits, or access information it was entitled to obtain for a disliked purpose? Only the first is within 18 U.S.C. § 1030.
[ ] Wholly public, no authentication — weak CFAA exposure. hiQ Labs, Inc. v. LinkedIn Corp., 31 F.4th 1180, 1195-1201 (9th Cir. 2022).
[ ] Behind a login — a gate. Record how the account was obtained: ordinary signup, automated account creation, shared credentials, or credentials from third parties. The answer changes the analysis materially.
[ ] Technical barriers defeated — IP blocks, rate limits, CAPTCHAs, bot detection are gates in substance.
[ ] A cease-and-desist letter — contested, and a plaintiff should not build a CFAA case on one alone.
[ ] Check the civil threshold: § 1030(g) requires damage or loss, and "loss" must be proved — investigation and response costs, service interruption — not asserted.

Phase 4 — Test whether a contract was formed

[ ] Identify the presentation: clickwrap (affirmative click adjacent to the terms — generally enforceable); sign-in wrap (referenced at account creation near the button — usually enforceable if conspicuous); browsewrap (footer link — enforceable only on actual or constructive notice).
[ ] Answer, per source: was an account created, by whom and how, and did the flow present terms? Did a human ever see the terms? Did the agent render pages or fetch raw responses? Were the terms changed after collection began, and was notice given?
[ ] Check for an arbitration or forum selection clause, which changes where the dispute goes and is frequently overlooked.
[ ] Note the scraper's argument and its limit: an automated agent that never rendered a page has a real no-formation argument, which weakens sharply once an account is used.
- Why this phase matters most. This is where modern scraping disputes are decided. hiQ itself ended on contract after the CFAA question resolved.

Phase 5 — Test circumvention and system harm

[ ] Anti-circumvention. 17 U.S.C. § 1201 requires the measure to control access to a copyrighted work; where the material is unprotected fact, the claim fails at the threshold. Note the interoperability exemption at § 1201(f).
[ ] Trespass to chattels. Requires actual impairment of function or capacity, evidenced by server metrics. Intel Corp. v. Hamidi, 71 P.3d 296, 300-08 (Cal. 2003).
[ ] Scrape politely and defeat the claim entirely: rate limit well below any plausible threshold; cache aggressively; use conditional requests; run off-peak; identify the agent honestly in the user-agent string.
[ ] Honor robots.txt, crawl-delay, and any explicit statement about automated access.
- Why. None binds you on its own. All appear in the opening paragraph of the complaint. Complying is free.

Phase 6 — Run the output analysis

[ ] Copyright in what was reproduced. Storing expression makes copies; the analysis is infringement plus fair use under 17 U.S.C. § 107, with Andy Warhol Foundation for the Visual Arts, Inc. v. Goldsmith, 598 U.S. 508, 525-33 (2023) tightening factor one and direct competition cutting hard on factor four.
[ ] Personal information. The comprehensive statutes define it broadly and do not exempt publicly available information as expansively as scrapers assume. Cal. Civ. Code §§ 1798.100-1798.199.100.
[ ] Biometric identifiers from images. 740 ILCS 14 — $1,000 negligent, $5,000 intentional, per violation, private right of action, no cap. Also Tex. Bus. & Com. Code § 503.001.
- Why this line matters most. Against a corpus of any size this dwarfs everything else in this checklist.
[ ] Health-adjacent inferences. Wash. Rev. Code ch. 19.373, private right of action.
[ ] FTC unfairness. 15 U.S.C. § 45.
[ ] EU personal data. Regulation (EU) 2016/679 requires a lawful basis and notice to data subjects, which scrapers cannot satisfy in practice.
[ ] Strip at ingestion. Personal information never stored never attaches an obligation.
- Why. One engineering sprint removes the only exposure here with uncapped statutory damages. Nothing else in this checklist has that ratio. See Marketing Privacy Compliance Checklist.

Phase 6A — Training-corpus additions

Where collected data feeds a model, three obligations attach on top of everything in Phase 6, and they should be recorded in the same register.

[ ] Per-source training eligibility. Record whether each source's terms permit use for model training specifically. Many terms now address it expressly, and a source that permitted "use of the data" in 2021 may prohibit training today.
[ ] Separate the training corpus from the operational corpus, so a source that must be removed can be removed from one without unwinding the other.
[ ] Record what was trained on what, and when. Model version, corpus snapshot, and the sources included. Without it, a demand to remove one source's data cannot be answered, and "we cannot tell you what the model was trained on" is an answer that satisfies no regulator, customer, or acquirer.
[ ] Run the output question separately from the training question. Whether training is infringement and whether it is fair use is one analysis under 17 U.S.C. § 107. Whether specific outputs reproduce protected expression is a different, substantial-similarity question about those outputs.
[ ] Run the authorship question separately again. Material lacking human authorship is not copyrightable, which affects what the company can claim to own in its own outputs and what it can represent to a customer. See Who Owns What the Machine Made?; Deploying Generative AI Without Losing Your IP.
[ ] Note the licensing-market factor. Where a source has developed or is developing a training-data licensing market, a use that displaces it is the strongest version of the plaintiff's factor-four argument. Andy Warhol Foundation for the Visual Arts, Inc. v. Goldsmith, 598 U.S. 508, 525-33 (2023).
[ ] Decide the disclosure posture — what the company will tell customers about corpus provenance, and whether it will indemnify. That commitment should be made deliberately by counsel rather than incidentally by a salesperson answering a security questionnaire.

Phase 7 — Work the four sources to conclusions

[ ] County assessor records. Government, factual, published; thin compilation claim in the county's arrangement only. Usually no gate on the public interface. Check bulk-access restrictions and fees, which many jurisdictions impose by statute or ordinance and which are the risk most often missed, because "public record" is heard as "unrestricted." Owner names are personal information, frequently within the government-records exemption — check the exact wording per state. Verdict: lowest risk; confirm bulk rules per county and pay the fee.
[ ] Listing site behind a free account. Listings mix facts with descriptive expression; the site's selection may be protectable. Authentication is a gate. Account creation almost certainly involved assent and terms almost certainly prohibit automated collection. Verdict: stop and license. Not a close call, and the advice should say so.
[ ] Review platform's public pages. Reviews are copyrightable expression written by users; reviewer names are personal information; any facial images implicate the biometric statutes. No gate, so weak CFAA exposure; browsewrap analysis on contract. Verdict: moderate access risk, high content and privacy risk. Extract aggregate sentiment at ingestion; retain no review text, no reviewer identity, no images.
[ ] Competitor's market reports. Copyrighted expression with unprotected underlying facts. Frequently gated behind a form or subscription, which changes everything. Fair use is weak where the use competes directly with the author. Verdict: high risk as executed; extract the facts, discard the expression, do not retain the source documents in the production corpus — as an operational rule, not an aspiration.

Phase 8 — Segment the pipeline and strip at ingestion

[ ] Segment collection, storage, and processing by source, so a defect in one source can be remediated without touching the others.
- Trap. A single undifferentiated data lake means a problem with source three contaminates everything.
[ ] Implement ingestion-time extraction: take the signal, discard the source document.
[ ] Implement ingestion-time stripping of names, contact details, identifiers, and images for every source that does not require them.
[ ] Tag every record with a provenance identifier linking to the Phase 9 record.
[ ] Verify the implementation with a sample audit, not with the code review.

Phase 9 — Build the provenance record

[ ] Record per source and per collection run: the source and URL patterns; the date range; the method, including agent, rate, whether pages were rendered, and whether an account was used; the terms in force at the time, captured with a date, not linked; the robots.txt in force, captured; what was collected, retained, and discarded; the legal analysis, its date, and who performed it; and any communications with the source.
[ ] Keep it contemporaneously.
- Why. Reconstructing it under deadline costs $80,000 to $250,000, produces a worse document, and is the diligence finding an acquirer will raise. Built as you go it is $15,000 to $40,000 a year. See IP Due Diligence Toolkit.
[ ] Keep the provenance record longer than the data.

Phase 10 — Set retention, minimization, and deletion

[ ] Set retention periods per source and per field, enforced by automated deletion.
- Trap. A retention policy with no automated enforcement describes an intention, and regulators and plaintiffs read it as one.
[ ] Handle backups explicitly — the copies everyone forgets and the copies a deletion obligation reaches.
[ ] Plan for a deletion demand: a rights request, a regulator inquiry, or a settlement obligation may require removing an individual's data from the corpus, from derived aggregates, from backups, and from any model trained on it.
- Why it is a design decision. Whether that is technically possible is determined at the beginning. A program that cannot answer it has accepted an obligation it cannot perform.
[ ] Prove both the retention and the deletion, with logs.

Phase 11 — Site operator: build the contractual position

[ ] Clickwrap for every account, with an affirmative click adjacent to the sign-up action.
[ ] API terms with rate limits, field-of-use restrictions, redistribution prohibitions, caching limits, and audit rights.
[ ] Customer agreements addressing what customers may do with data the product produces, aggregate use, and termination.
[ ] Vendor agreements with purpose limitation and no-resale terms.
[ ] Employee and contractor agreements with confidentiality and present assignment.
[ ] Draft the terms to survive: a change-of-terms mechanism with notice, severability, and forum and arbitration provisions.
[ ] Publish an API.
- Why. Most scraping is done by parties who would rather have a supported feed with an SLA. A reasonably priced API removes more scraping than any lawsuit and converts an adversarial relationship into a contractual one on terms you wrote. See Launching a Website or App Without Legal Debt.

Phase 12 — Site operator: build technical and evidentiary controls

[ ] Authentication for anything valuable. Data behind a login is behind a gate; data on a public page is not.
[ ] Rate limiting, documented and enforced.
[ ] Bot detection, with a record of what it blocked and when.
[ ] Internal classification and access controls, so no single employee can export the whole dataset.
[ ] Logging with alerting thresholds that would show a bulk export.
- Why both. The trade secret analysis turns on reasonable measures, 18 U.S.C. § 1839(3), and the CFAA analysis after Van Buren turns on whether a gate existed. Both improve with the same controls. See Building a Trade Secret Program That Survives Litigation.
[ ] Structure the dataset for thin copyright: curated selection on non-obvious criteria, an arrangement reflecting editorial judgment, and original expression in descriptions and annotations. Then register the compilation and register updates. See Registering a Copyright.
[ ] Seed the dataset. Plausible but fictitious records whose appearance elsewhere proves copying. Document the seeding so the evidence is admissible.
[ ] Add watermarking, canary tokens, and per-customer variations.

Phase 13 — Handle the cease-and-desist, either side

[ ] Sending. Complete Phases 2 through 5 first. Identify the specific conduct, terms, and technical measures. State clearly that access is revoked. Include a preservation demand covering collection logs, stored data, pipeline configuration, and communications. Offer the licensed alternative in the same letter.
- Trap. A letter with a deadline invites a declaratory action in the recipient's forum. See Suing First: Declaratory Judgment Actions in Trademark Disputes; Declaratory Judgment Checklist.
[ ] Receiving. Issue a litigation hold. Pause collection from that source while the analysis runs — continuing after notice converts a contested question into a knowing one and drives injunctions. Run Phases 2 through 6 against that source. Assess whether personal information in the corpus creates exposure independent of the sender's claims. Respond substantively with what you collect, what you have stopped, and what you propose.
[ ] Ask about insurance early. Technology errors-and-omissions, cyber, and media liability policies sometimes respond; IP exclusions are common and privacy coverage varies sharply. See Tendering and Winning an IP Coverage Claim.

Phase 14 — Prepare for the injunction motion

[ ] Plaintiff: lead with contract for the irreparable harm framing — loss of platform control, harm to user trust, inability to enforce terms against others. Support with server metrics, the terms and assent record, and the collector's conduct after notice.
[ ] Defendant: attack irreparable harm as compensable, attack the assent record, and press Van Buren and hiQ on any CFAA count. Where the data is public and the harm is competitive rather than technical, say so plainly.
[ ] Both: price the bond under Fed. R. Civ. P. 65(c), which is a real constraint where an injunction would shut down a product.
[ ] Treat the motion as the merits.
- Why. It is decided on an incomplete record in six to ten weeks and the ruling usually ends the commercial dispute regardless of what follows. See eBay Inc. v. MercExchange, L.L.C., 547 U.S. 388 (2006) for the standard on any copyright count.

Phase 15 — Price the licensing alternative

[ ] License it from the source — frequently cheaper than the legal analysis of whether it could be taken.
[ ] Use the published API on its terms.
[ ] Buy from an aggregator with representations and indemnities. The representations are the point.
[ ] Collect first-party, which removes the question.
[ ] Narrow the requirement. Ask what the product actually uses.
- Why. Programs collect far more than they consume, because the pipeline was built to gather everything and nobody went back. Narrowing is free, reduces every risk category at once, and is the recommendation clients accept most readily because it also cuts cost.

Phase 16 — Install governance and review

[ ] A source intake gate. No new source without a completed register row and a Phase 2-6 analysis. A form, a named reviewer, and a two-day turnaround.
- Why the turnaround matters. Engineering routes around a two-week review far more often than a two-day one.
[ ] A named owner for the register, the provenance record, and the retention schedule.
[ ] A quarterly reconciliation of sources the pipeline actually hits — from network logs — against the register.
[ ] A change-of-terms watch. Capture terms per source at intake and re-check annually, because a contract claim rests on the terms in force at the time.
[ ] The escalation rule, written down: any source that requires an account, blocks or rate-limits the agent, sends a notice, or contains images of people goes to counsel before the next run.
- Why these four. They account for nearly all of the exposure in this checklist and they are all machine-detectable.

Phase 15A — Diligence: buying or selling a company whose product runs on collected data

Every issue in this checklist arrives in an acquisition as a representation the seller makes and the buyer accepts. Run this pass from either side.

What the buyer should request.

[ ] The source register, or, if none exists, network logs from which one can be built.
[ ] The provenance record per source: terms captured with dates, robots.txt captured, method, volume, and the legal analysis with its date and author.
[ ] Every cease-and-desist letter, demand, or platform notice received, and the response.
[ ] Every license, API agreement, or data supply agreement, with its field-of-use and redistribution terms.
[ ] The account provenance for any authenticated source: how accounts were created, by whom, and whether terms were accepted.
[ ] The retention schedule and its enforcement mechanism, plus evidence that deletion actually runs.
[ ] The personal information inventory: what categories are in the corpus, from which sources, and whether any biometric or health-adjacent data is present.
[ ] Whether the corpus has been used for model training, and whether any model can be retrained or unwound if a source must be removed.
[ ] Insurance — what policies exist and how they treat data collection claims.

What the buyer should not accept.

[ ] An unqualified representation that the target owns its data. Almost nobody owns data as such, and an unqualified representation is one the law does not support. Ask instead for representations about rights to use, compliance with terms, and absence of claims.
[ ] "It's all public information" as an answer. That is a statement about access, not about content, contract, or output obligations. See Who Owns the Data?.
[ ] A provenance record reconstructed during diligence. Ask when it was created, and treat a record dated after the letter of intent as an assertion rather than evidence.

What the seller should do twelve months before a process.

[ ] Build the register and the provenance record if they do not exist. Reconstructed under deadline this costs $80,000 to $250,000 and produces a worse document than one kept as you go.
[ ] Retire or license the sources that will not survive scrutiny — the authenticated ones, the ones that sent notices, and the ones whose terms plainly prohibit collection.
[ ] Strip and delete personal information the product does not use, and be able to prove it.
[ ] Segment the corpus so a problem source can be removed without unwinding the whole dataset. A buyer that is told "we can't separate it" prices that answer.
[ ] See IP Due Diligence Toolkit; Trademark Due Diligence in Mergers and Acquisitions.

Phase 16A — Cost and timeline

| Task | Elapsed | Cost | |---|---|---| | Source register and classification | 2-5 weeks | $20k-$60k | | Per-source legal analysis | 1-2 weeks each | $8k-$25k each | | Pipeline segmentation and ingestion stripping | 4-10 weeks | $40k-$150k | | Provenance record, contemporaneous | ongoing | $15k-$40k/yr | | Provenance record, reconstructed under deadline | 6-12 weeks | $80k-$250k | | Retention automation and deletion proof | 4-8 weeks | $30k-$90k | | Site-operator contractual and technical build | 6-12 weeks | $50k-$160k | | Cease-and-desist exchange | 2-6 weeks | $10k-$40k | | Preliminary injunction, either side | 6-10 weeks | $150k-$500k | | Through judgment | 18-36 months | $600k-$2.5M | | Biometric class action exposure | — | uncapped statutory | | Licensed feed | 2-8 weeks | negotiated |

[ ] Present two rows against each other: the provenance record at $15k-$40k a year versus the same record reconstructed at $80k-$250k. It is the clearest comparison in this checklist and the one clients act on.
[ ] Note that the biometric row has no number because it has no cap, which is itself the argument for Phase 6.
[ ] Diarize the quarterly reconciliation and the annual terms re-check from Phase 16.

Key Authorities at a Glance

| Authority | What it provides | Phase | |---|---|---| | Feist Publications, Inc. v. Rural Telephone Service Co., 499 U.S. 340 (1991) | Facts unprotected; thin compilation protection | 2 | | 17 U.S.C. § 103 | Compilations | 2, 12 | | 17 U.S.C. § 107 | Fair use | 6 | | 17 U.S.C. § 1201 | Anti-circumvention; interoperability exemption | 5 | | 18 U.S.C. § 1030 | CFAA and the civil action | 3 | | Van Buren v. United States, 593 U.S. 374 (2021) | Gates-up-or-down reading | 3 | | hiQ Labs, Inc. v. LinkedIn Corp., 31 F.4th 1180 (9th Cir. 2022) | Public data likely outside "without authorization" | 3 | | 18 U.S.C. § 1839 | Reasonable measures | 12 | | Intel Corp. v. Hamidi, 71 P.3d 296 (Cal. 2003) | Trespass requires actual harm | 5 | | Andy Warhol Found. v. Goldsmith, 598 U.S. 508 (2023) | Transformative use narrowed | 6 | | 15 U.S.C. § 45 | FTC unfairness | 6 | | 740 ILCS 14 | Biometric identifiers; uncapped damages | 6 | | Tex. Bus. & Com. Code § 503.001 | Texas biometric statute | 6 | | Wash. Rev. Code ch. 19.373 | Consumer health data | 6 | | Cal. Civ. Code §§ 1798.100-1798.199.100 | Obligations over collected personal information | 6 | | Regulation (EU) 2016/679 | Lawful basis and notice | 6 | | Fed. R. Civ. P. 65 | Injunction and bond | 14 | | eBay Inc. v. MercExchange, L.L.C., 547 U.S. 388 (2006) | Injunction standard | 14 |

The five things people get wrong

One policy for every source. Four sources produce four answers, and a uniform policy is wrong for at least one of them — usually the one that generates the claim.

Analyzing access and stopping there. Whether you may collect is half the question. Whether you may hold and use what you collected is the half with the uncapped statutory damages.

Keeping the raw pages "in case we need them." That converts a transient copy into a retained corpus, with every copyright and privacy consequence that follows.

Reconstructing provenance under deadline. Five to ten times the cost, a worse document, and a diligence finding that could have been avoided for $20,000 a year.

Treating "public record" as "unrestricted." Government data frequently carries bulk-access restrictions and fees, and those are contractual or statutory and enforceable.