The State of the Data: Applications and Cascading Effects
How researchers use Tribal drinking water data
EPA’s SABs and SDWIS datasets and IHS’ STARS data create an incomplete picture of Tribal drinking water access. To understand the implications of these limitations, we looked at how researchers use SDWIS data and the effects on the policy and research landscape.
Because IHS drinking water data is confidential, Tribal drinking water researchers looking to evaluate drinking water access among Tribes on a national scale are essentially left with only EPA SDWIS and SABs data. Though these are datasets that reflect EPA’s regulatory scope and objectives, SDWIS’s accessibility to researchers can obfuscate this reality, and the parallel truth that the most pertinent drinking water access barriers may be excluded from the dataset. Likewise, the convenience SDWIS filters that appear to proxy Tribal drinking water access can be enticing to researchers, but fail to capture the full breadth of Tribal drinking water data and leave out pertinent factors that are important to developing meaningful solutions.
A key finding in this literature review was that unclear data definitions and insubstantial descriptions of the limitations in EPA data result in analyses by researchers that are fragmented and thus result in fragmented conclusions. Critical to good Tribal drinking water access and quality research is a strong understanding of the grounding data, an understanding of the unique status of Tribal jurisdiction and governance, and a respect for Tribal sovereignty–particularly as it pertains to data and research. As other researchers have highlighted, “until some effort is devoted to SDWIS, caution is urged against overly broad and speculative conclusions, particularly with respect to guiding public policy,” and this is particularly true when considering Tribal drinking water data.
EPA’s SDWIS Indian country indicator is the best proxy for understanding drinking water quality on Tribal lands using national data, with the caveat that it only captures Tribal lands that meet EPA’s definition of Indian country for the purposes of implementing SDWA–places where either EPA or the Tribe has primacy. The accuracy indicator’s use and entry into the data system is varied, and can depend on the EPA region that serves the Tribe. It’s also important to make clear that this pertains only to the geography of the drinking water system, and does not proxy the demographics of the population served by the systems. Still, it can be a particularly effective proxy when evaluating drinking water where Tribes have the most authority and political influence to improve drinking water access.
The best research using this indicator is careful to identify these limitations in the data’s scope and to clearly describe these limitations in their findings. Writing about these findings is difficult–even the most careful researchers’ conclusion that “violations affected tribal communities at nearly twice the rate of non-tribal customers,” requires an equally careful reader to discern that “communities” refers to location, not community demographics.
Insubstantial EPA data dictionaries have, at times, misled researchers. The general lack of understanding of Tribal sovereignty in drinking water research leads to research that excludes that particular governance levers that Tribes have to address drinking water needs. Seeking to address this, a set of researchers sought to understand how U.S. environmental policy is implemented on Tribal lands and in drinking water systems owned by Tribal governments. They interpreted the Native American owner category as a proxy for this. In another study, researchers seeking to understand PFAS testing on Tribal lands did not use the Indian country indicator, and instead filtered SDWIS data for systems owned by Native American owners. As we have discussed, a Native American-owned system does not mean that it serves Tribal lands or even Tribal citizens. Likewise, it does not allow any sort of meaningful disaggregation, such as separating an individual Tribal citizen-owned water system from one owned by a Tribal government. Its accuracy and use is also greatly varied depending on who is entering the data into the SDWIS system.
An understanding of Tribal sovereignty, and a respect for Tribal data–whether that is for Tribal data protections or different ways of gathering data–is imperative in Tribal drinking water research.
Researchers seeking to understand arsenic in ground and surface water in Arizona used only publicly-available data and excused the omission of Tribal data in their research as an inconvenience, stating “unfortunately, while water quality information exists for some of these Nations for many, if the data exist, it requires strict approval by the various tribal governments to publish them in a public location.” This is a single statement, but it perpetuates the idea among drinking water researchers that it is acceptable to leave Tribes out of research simply because non-Tribal researchers do not want to go through Tribal research approval processes created to protect their people and their lands in response to the dark historical context. Not only is it offensive to exclude Tribes entirely by not even attempting to navigate their processes, but even the most self-interested researcher should understand that it makes research worse to ignore Tribal data entirely.
Cascading Effects of Data Limitations
Our review of EPA and IHS data uncovered the obvious–that federal datasets reflect federal priorities–they capture the world of Tribal drinking water access and quality with a very particular and limited lens. The challenge, then, is in minding the gap.
Right now, national Tribal drinking water access and quality research is limited because a limited number of researchers are interested in understanding Tribal drinking water, and when they are, must understand the applications of Tribal sovereignty in the data. Moreover, EPA, an agency which collects data for communities nationwide, but also operates in a nation-to-nation relationship with Tribes, has not necessarily designed its data collection or dissemination systems to even allow for the reporting of data in a way that reflects Tribal priorities or data needs. The result is an extension of the erasure that Native people face more broadly. Tribes are disproportionately served by drinking water systems not regulated by EPA and thus disproportionately excluded from EPA’s data. Complex regulatory frameworks in Indian country are coupled with insubstantial EPA data dictionaries for Tribal identifiers in EPA data systems, resulting in data entry errors and in data use errors. In sum, the most accessible Tribal drinking water data is harder to find, harder to understand, and leads to research that is less consistent and of poorer quality, and not necessarily relevant to Tribal drinking water priorities.
However, our findings also indicate that these issues can be overcome. IHS STARS is an incredible set of household level drinking water access and quality data that has enormous potential for understanding drinking water quality and access gaps and developing solutions. EPA can add data caveats to help educate data users on the limitations of Tribal data and can improve data dictionaries. Improvements to SDWIS classification typologies, and linking EPA and IHS data are both possible and needed. And respect for and the promotion of Tribal data sovereignty can result in data innovations. We outline these actions in our recommendations.