Minimize sensitive data in prompts, retrieval, logs, and traces.
Privacy work starts by finding all copies and derived forms of user or source data.
Privacy work starts by finding all copies and derived forms of user or source data.
Why this matters: Build a data inventory for an AI workflow.
Problem anchor: a learner pastes a transcript containing names, grades, and health context into a lesson coach. The app sends it to a model, stores a trace, embeds a summary for memory, logs an error, and later includes the example in an eval set. Privacy review must follow all of those copies.
A data inventory names each surface: user input, uploaded files, retrieval chunks, embeddings, model request, model output, trace, analytics event, support ticket, eval dataset, cache, vendor system, and deletion workflow.
Field: learner free-text prompt. Classification: personal data possible, credentials possible, minors data possible depending on deployment. Use: generate coaching response. Stored: trace metadata by default; raw text only with explicit debug mode and redaction. Vendor: approved model route only. Retention: 30 days for raw debug traces, aggregate metrics longer. Deletion: trace, cache, memory, and vector derivatives included.
The strongest privacy control is not collecting or retaining sensitive data unnecessarily.
The strongest privacy control is not collecting or retaining sensitive data unnecessarily.
Why this matters: Classify AI workflow data by sensitivity.
PII detection is useful but imperfect. Privacy governance should first ask whether the data is needed at all. If a lesson coach does not need phone numbers, credentials, exact birth dates, or medical details, block or redact them before storage and avoid sending them to vendors unnecessarily.
Moderation classifies content risk; PII controls classify personal data; governance decides retention, access, consent, deletion, and review authority. Treat them as related layers, not one magic filter.
A school deletes a document containing student names. The source file disappears, but chunks and embeddings remain in the retrieval index. The app can still surface the content indirectly. A real deletion workflow must remove source, chunks, embeddings, cache entries, and trace references according to policy.
| Option | Class | Handling rule | Review trigger | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Public source | Approved public KB note | store provenance, license, and freshness metadata | copyright, stale source, takedown request | Use for retrieval corpora. | Medium | Medium |
| Learner personal data | Names, education records, private context | minimize, redact before logs, restrict access, short retention | school deployment, minors, support access | Use for user prompts and traces. | Medium | Medium |
| Secret or credential | API keys, passwords, tokens | block, do not send to model, rotate if exposed | any detection or attempted tool use | Use at ingress and logs. | Medium | Medium |
AI privacy governance lives in vendor settings, access controls, retention tables, and audit events.
AI privacy governance lives in vendor settings, access controls, retention tables, and audit events.
Why this matters: Design retention by data class.
Decision this forces: Choose the right level of rigor for Privacy and Data Governance: baseline, production design, or advanced optimization.
A model request may leave your infrastructure; a trace may be readable by engineers; an eval set may be shared with contractors; a vector store may outlive the source. Privacy governance assigns owners, access levels, retention periods, and deletion procedures to each copy.
Vendor review should cover data-use settings, retention, subprocessors, region, logging, support access, model training use, incident notification, and deletion support. The product should log policy version and redaction action so later reviewers know what happened.
Raw learner prompts: store only when debug sampling is enabled, redact obvious PII and secrets, retain 30 days, access limited to trust-and-safety reviewers. Trace metadata: retain 180 days for reliability and cost monitoring. Aggregated quality metrics: retain longer without raw text. Vector memory: disabled for school tenants unless explicit agreement covers purpose, retention, and deletion.
A privacy policy becomes real when model calls and persistence paths enforce it.
A privacy policy becomes real when model calls and persistence paths enforce it.
Why this matters: Write a simple data-handling gate.
def handle_prompt(text, tenant_policy): if "sk-" in text or "password=" in text.lower(): return {"action": "block", "reason": "possible secret"} redacted = text.replace("student@example.com", "[email]") audit = { "policy": tenant_policy, "pii_action": "redacted" if redacted != text else "none", "store_raw": False, } return {"action": "send_to_model", "text": redacted, "audit": audit} print(handle_prompt("email student@example.com", "school-privacy-v2"))
Raw prompts can contain sensitive personal data, so default metadata logging is safer unless a justified debug or review workflow exists.
Check that generated plans include a data inventory, classification, minimization, retention, deletion, access control, vendor routing, raw-trace policy, vector-store deletion, eval-set governance, and incident review. Reject plans that say "we redact PII" without explaining when, where, and how failures are handled.
Privacy governance must update when data sources, vendors, logs, memory, or product purpose changes.
Privacy governance must update when data sources, vendors, logs, memory, or product purpose changes.
Why this matters: Recall the privacy governance loop.
Decision this forces: Choose the right level of rigor for Privacy and Data Governance: baseline, production design, or advanced optimization.
Answer this from memory: "How do I govern data in an AI workflow?" Include inventory, classification, minimization, redaction, vendor review, access, retention, deletion, traces, embeddings, evals, audit logs, and incident review.
Pick one AI feature. Draw a table with every data surface, sensitivity class, purpose, vendor, storage location, retention, access group, deletion path, and audit event. Then choose one surface to minimize or eliminate before launch.
From memory, reconstruct AI privacy governance: inventory data, classify sensitivity, minimize before prompts and logs, control vendor and access paths, define retention and deletion, audit actions, and review incidents.
Create a one-page release-ready plan for privacy data-flow map with collection, retention, and deletion rules. Include: the user problem, realistic input, mechanism, design choice, runnable or reviewable check, metric (sensitive fields handled according to policy), failure case (logs keep personal data long after the user request is resolved), owner, and the next rung after this lesson.
Your AI workflow sends user queries to a third-party LLM API and stores the raw responses in a logging database. Which TWO surfaces does this create that require governance treatment? (Pick the answer that names both correctly.)
Any data sent to a third-party vendor is a transfer surface, and any data written to storage is a persistence surface — both require classification, retention rules, and access controls.
Before persisting a user's support-chat transcript to a database, which sequence of controls best reflects the 'classify and minimize before persistence' principle?
Minimization and redaction must happen before persistence so that sensitive data never enters the store in raw form — classifying first tells you which controls to apply.
In one or two sentences, explain why raw LLM traces are a privacy risk even when the original user-facing feature seems low-sensitivity.
Traces record everything passed to and from the model, so sensitive data that 'slips in' through user inputs or system prompts is preserved verbatim, creating exposure that the feature's surface-level design doesn't signal.
Your team updates its data-retention policy to shorten the log retention window from 180 days to 30 days. What must happen in the audit log to make this change governable?
Tying policy versions to audit log entries lets reviewers reconstruct which rules governed data at any point in time — without that link, a retention change is unverifiable.
An AI tool auto-generates a privacy plan for your new feature. You review it and notice it specifies how data is collected but says nothing about deletion schedules or derived data (e.g., embeddings created from user inputs). What is the correct next step?
AI-generated privacy plans routinely omit deletion schedules and derived-data lineage — a human reviewer must verify and add these explicitly, because embeddings and other derivatives carry the same sensitivity as the source data.