What to Ask Before You Buy

Four Questions for Any AI Vendor in Permitting

I've watched this happen more than once: a vendor demos something impressive, a director signs, and six months later, the tool sits half-used because nobody thought through oversight before procurement. In April 2025, the Office of Management and Budget issued two memoranda (M-25-21 and M-25-22) that changed how federal agencies buy AI.[1] Documented risk assessments, human oversight, performance tracking, and vendor accountability are all built into contracts awarded after September 2025. The reason these became mandates is that agencies kept buying tools they couldn't oversee.

State environmental agencies aren't bound by OMB memoranda. The failure mode is the same. Standard IT procurement evaluates functionality, security, and cost. AI procurement adds two questions that standard RFPs don't ask: can you explain the output, and who is accountable when it's wrong?

Canada's federal government built a tool that forces those questions to be asked before procurement starts. The Algorithmic Impact Assessment (AIA) is a mandatory, 65-question, open-source, publicly available risk questionnaire that assigns an impact level and determines the required safeguards before any automated decision system is deployed.[2] It makes vendors document their own risk model before a contract discussion begins. Scholars have drawn an explicit comparison between the AIA process and an EIS: both give the agency and the public a structured opportunity to evaluate a proposed system before deployment. The AIA takes what I described in the last post as a locked widget and forces someone to unlock it on paper before the money moves.

Georgia has moved in the same direction at the state level. Its GS-25-002 guidelines, effective since July 2025, require vendors to conduct and share an Algorithmic Impact Assessment, provide documentation on AI logic and model transparency, and, where feasible, complete a pilot phase before full deployment.[3] Georgia's framework is live. It sits in front of every AI contract in the state.

Most state environmental agencies do not have either Canada's mandatory questionnaire or Georgia's vendor requirements. If your agency doesn't have a formal framework yet, four questions cover most of what you need. They're the minimum, not the ceiling. How a vendor responds will tell you more about how their tool was built than any demo will.

The first: Can you explain how it reached that output? Not "is it explainable in principle." Show me, with a specific output from your system, how it got there. If a vendor says "the model is proprietary," that's an answer worth taking seriously. It means the administrative record for any AI-assisted decision will have a gap that your agency, not the vendor, will have to defend.

Second: Who is accountable when it's wrong? Contracts typically disclaim vendor liability for outputs. That's expected. "The agency is accountable" isn't a process, though. Name the official. Document what review they applied and what criteria governed their decision to accept or reject the AI's recommendation. Without that documentation, you have a liability exposure without a safeguard.

Third: Does it integrate with existing workflows, or does it create a parallel one? Across the 320 tools EPIC has tracked, the pattern is consistent: they work in isolation. An AI tool that requires staff to leave their existing system, run a process in a separate interface, and manually re-enter results is a new source of transcription errors. Ask the vendor to show you the handoff point between their system and yours.

Fourth: What does the vendor do with your data? Permitting data includes sensitive project information, tribal consultation records, and proprietary applicant submissions. Where does it go when it enters the vendor's system? Is it used to train the model? Is it retained after the contract ends? Can you get it back? Agencies that can't answer these questions after signing are in a difficult position when a project applicant or a tribal nation asks.

The litigation dimension makes these questions more than bureaucratic prudence. As I noted in the previous post, Beveridge and Diamond published an analysis in late 2025 warning that project opponents will exploit AI use in permitting if the documentation is thin.[4] Courts haven't established how they'll give deference to AI-assisted agency decisions, and the early case law will likely come from the agencies that documented the least. A well-documented AI process protects the agency. A poorly documented one invites challenge.

Minnesota's Pollution Control Agency offers one model for incorporating these requirements into procurement from the start. The MPCA's $780,000 Technology Modernization Fund grant for AI in permitting required governance justification before funding was approved.[5] The procurement filter became a governance mechanism in its own right. The next post describes what that governance infrastructure looks like in practice.

Before the next vendor demo, send these four questions in advance. A vendor who won't answer them beforehand will give you less useful information during the demo. A vendor who has thought seriously about responsible deployment will have answers ready. The difference between those two responses tells you more than any slide deck.

The final post covers what happens after you buy the right tool: the governance infrastructure that determines whether it sticks.

OMB Memorandum M-25-21, "Accelerating Federal Use of AI through Unified Standards." Office of Management and Budget, April 2025. https://www.whitehouse.gov/omb/information-regulatory-affairs/artificial-intelligence/

Government of Canada. Algorithmic Impact Assessment tool. https://www.canada.ca/en/government/system/digital-government/digital-government-innovations/responsible-use-ai/algorithmic-impact-assessment.html

Georgia Department of Administrative Services. GS-25-002 Artificial Intelligence Procurement Guidelines. Effective July 2025. https://doas.georgia.gov

Beveridge & Diamond. "AI in Environmental Permitting: Anticipating the Next Wave of Legal Challenges." 2025. https://www.bdlaw.com/publications/

Minnesota Pollution Control Agency. Technology Modernization Fund AI in Permitting Grant. 2025. https://www.pca.state.mn.us