Introduction
A provost asks: "What is our four-year graduation rate for first-time, full-time students who started in fall 2022?" Paste that into a general AI assistant and you will get a confident answer. It will almost certainly be wrong, because the assistant does not have your data. It is generating a plausible-sounding number from the public web, from IPEDS averages, or from its training data. None of those are your institution.
This gap shows up in the research. The whitepaper on data request workflows found that 91.2% of institutions report request management and feedback challenges, and that ad hoc requests consume 40 to 60% of IR capacity. A general AI assistant does not close that gap. It answers the same ambiguous question with more confidence and no more data access than the email chain it was supposed to replace.
Where AI stands on campus today
IR teams are not waiting for permission to try AI. In our interviews, offices named NotebookLM, ChatGPT, Gemini, and Claude among the tools already in use, usually for drafting, summarizing, and code help rather than for producing official figures. Institutions are moving in the same direction. In the 2025 EDUCAUSE AI Landscape Study, based on a survey of nearly 800 higher ed professionals in November 2024, 57% said their institution now treats AI as a strategic priority, up from 49% the year before.
The same study shows how far policy lags behind use. Only 9% of respondents said their institution's cybersecurity and privacy policies adequately address AI-related risks. Staff will use AI on institutional data either way. What an IR office can still choose is the tool: one built for governed, sourced answers, or a general assistant and a copy-paste habit.
What general AI gets wrong in higher ed
- It does not have your data. A general assistant can read a file you upload, but it has no governed connection to your SIS, your warehouse, or your IPEDS submissions, and it forgets the context the next time someone asks.
- It does not know the domain. It cannot tell you whether "enrolled student" means headcount or FTE in this context, or whether "graduation rate" means four-year or 150% of normal time.
- It cannot show its source. When it gives you a number, you cannot trace it back to a specific table, year, or calculation method.
- It hallucinates confidently. A number stated with the right cadence is not a number you can put in a dean's memo.
What higher-ed-native AI gets right
- It reads your data. Clema connects to your SIS, warehouse, LMS, and files, alongside the federal datasets you already pull, and answers from those sources.
- It knows the domain. IPEDS definitions, accreditation terms, retention versus graduation, FTE versus headcount, cohort definitions: these are built in, not taught in month one.
- It shows its source. Every figure carries the table, the calculation method, and the year, with an audit trail for compliance.
- It is governed. Role-based access, read-only connections, BAAs, and a query audit trail mean security review is a one-day pack, not a multi-week audit.
General vs higher-ed-native, side by side
| Question an IR office asks | General AI assistant | Higher-ed-native assistant |
|---|---|---|
| Can it reach our data? | Only what someone pastes or uploads in that session | Read-only connections to the SIS, warehouse, and federal datasets |
| Does it know our definitions? | No; it applies a generic meaning of "retention" or "enrolled" | Uses the definitions your office published in its data dictionary |
| Can we trace the number? | Rarely; it paraphrases where it thinks the figure came from | Table, field, calculation, and year on every answer |
| Who can see what? | Whoever has the chat history | Role-based access tied to your identity provider |
| What happens when the data does not exist? | Often a plausible guess | A refusal, or a question back about scope |
A graduation rate example: ChatGPT vs Clema
Ask a general assistant like ChatGPT, "What is our 150% graduation rate for the fall 2018 cohort?" and it will return a plausible-sounding percentage, usually without asking which cohort you mean. It has no way to know that IPEDS reports this figure through the Graduation Rates component (long called the Graduation Rate Survey, or GRS), and that the denominator is the adjusted cohort, not gross headcount at entry. Students who leave for reasons IPEDS allows as exclusions are removed from the base before the rate is computed. A general assistant does not know your adjusted cohort size exists, so it cannot use it.
Ask Clema the same question and it pulls the adjusted cohort and completers from the published IPEDS data for your institution, applies the calculation, and returns the rate with the component name, the collection year, and the math shown. One assistant guesses at a number that sounds like a graduation rate; the other shows you the work behind the number you are about to put in a board memo. The RAG explainer for IR offices walks through why grounding answers in retrieved rows makes that difference.
Where the data goes: FERPA and privacy
Accuracy is one concern. Where student records go when someone pastes them into a chat window is another. FERPA lets an institution share education records with an outside party without consent only under specific conditions. Under the school official exception in 34 CFR 99.31(a)(1)(i)(B), a contractor can be treated as a school official if it performs a service the institution would otherwise use employees for, is under the institution's direct control with respect to the use and maintenance of education records, and is bound by the rules on redisclosure. A personal chatbot account that an analyst signed up for on a Tuesday afternoon meets none of those tests.
To be fair to the general tools, campus-licensed versions exist. OpenAI launched ChatGPT Edu in May 2024 with enterprise security controls for universities, and a campus license is a real improvement over personal accounts. It still leaves the IR-specific problems from the table above: no governed connection to your warehouse, no shared definitions, and no source trail on the figures. Clema is built for that layer. The why Clema page makes the full case, and the security page lists the controls a CISO will ask about: SOC 2 Type II, encryption in transit and at rest, role-based access, and audit logs.
When a general assistant is the right tool
- Drafting and editing. A cover memo, a survey invitation, or a plain-language summary of a policy reads fine from a general assistant, as long as no student records go in.
- Explaining a concept. "What is the difference between retention and persistence?" is a fair question for any assistant; the definitions are public.
- Code help. Debugging a SQL join or an R script on synthetic or de-identified data is a common and reasonable use.
- Anything with a figure that will be cited. That is where the general tool stops being the right one, because the number needs a source and a definition your office can stand behind.
The source test
The fastest way to tell whether an AI is general-purpose or higher-ed-native is the source test. Ask your question, then ask: "Where did that number come from?" A general assistant will give you a paraphrase of where it thinks the number lives in the public record. A higher-ed-native assistant will give you the table, the field, the calculation, and the year, pulled from your actual data.
That distinction is the reason Clema exists. Higher ed answers need to be defensible to accreditors, cabinets, and boards. A sourced answer is defensible; a confident paraphrase is not. We are also clear about the limits: the post on what Clema cannot do yet lists where another tool is the better call. If the source test is what you care about, run it in a live demo on your data.
See a higher-ed-native AI on your data
Book a demo and ask Clema the same question you would ask a general assistant. The source test will tell you which one to trust.
Book a Demo