Working Wonders in Purpose Driven Innovation Labs
The earliest laboratories recorded in English-language texts reference alchemy, the forerunner to modern chemistry chiefly interested in the transformation of base metals into gold and the development of elixirs of eternal life. Environmental innovation labs seek even more valuable - and far more attainable - transformation and immortality: free ideas into better tools for protecting and restoring our life sustaining planet. Ideas enter the lab seeking their form, seeking embodiment into a tool, system, or methodology.
Diving into Details
Labs should take something that is mostly an idea (or marginally a proof of concept) and produce a testable prototype (see our Modular Innovation framework for more on the stages). They achieve this goal under the conditions and through the processes that follow. In an inoculated environment they undergo iterative experiments where they are conceptualized, built, tested, broken, and rebuilt better.
Experimentation - Humans don’t tend to be very good at things the first time we try to do them. Before I was a data and policy nerd, I was a theater director. I’ve carried the succinct and eternally relevant motto of the National Theatre Institute with me: “Risk. Fail. Risk again.” Creative, iterative, experimental processes transform failures into lessons and lessons into progress. To support experimentation, Labs need to source and provide project teams with a few critical supports:
Engineering chops to build and refine prototypes.
Testing environments in which prototypes can run, crash, and be evaluated.
Representative data and practical situations to inform tests.
Access to users willing to test and provide feedback.
Inoculation - Through iterative experimentation the prototype develops responses to known challenges and probable pitfalls. Effectively metering the prototype’s exposure to the “real-world” is the best way to safely and cost effectively weed out bugs and pursue improvements. Simultaneously, insulating development in a lab protects the world from whatever negative consequences each iteration of the prototype causes. The metaphor to medical laboratories is pretty blunt: a mostly controlled environment helps doctors isolate cause and effect in experimental treatments, pathogens are contained during the process, and sometimes when that membrane is crossed something amazing happens. Conversely, when public sector innovators overlook or skip the inoculation stage of laboratory development, we end up with the kind of financially disastrous and mission-damaging rollouts like we saw with the first version of healthcare.gov.
When and Why
Labs are for projects that are primed for practical growth, need skilled support from outside of their initial project team, and aren’t ready for a live environment. Let’s break each of these down:
Primed for Practical Growth
Above, we referred to projects that enter labs as an idea or a proof of concept. The fulcrum point on the spectrum between idea and proof of concept is made up of three elements:
Knowing how to apply the tool in context and, therefore, a fundamental understanding of what data it needs to work.
Ongoing engagement with, and integration of, user perspectives to ensure the implementation is solving a real problem without creating new ones.
Sufficient code to play around with and begin building off of. Labs should be able to support anything from a toy model to a functioning skeleton.
Needs Skilled Support
If the project team is missing any key skillsets, labs can make the difference between stalling out and pushing on. The growth fulcrum described above is a natural state in the development process, innovative and otherwise. A project in this phase with a full capacity development team doesn’t need a lab, it needs money, time, and opportunity (perhaps an Accelerator, but that’s a blog for another day). Personnel infusions serve several functions:
Adding critical skills so the project can continue. Labs should be able to source a wide variety of technical and enabling skillsets to plug gaps and allow the team to be more ambitious.
Providing fresh eyes and independent perspectives. Creative processes benefit from supportive newcomers who provide feedback, propose cuts, explore alternative solutions, and discover novel possibilities.
Expanding the network of champions. Bringing new teammates into a project - even temporarily - increases the number of people who are bought-in to its ongoing success. They become an ongoing resource for advice, and bring with them a network of people and pool of resources that otherwise wouldn’t have been accessible to the core team moving forward.
Not Ready for a Live Environment
Finally, projects entering labs should be primed, but not ready for primetime. While it’s often obvious when a tool is in its toy model phase, the distinction is murkier when it has a few robust operations built out and you’re considering a lab vs a test bed environment. To make that distinction consider the following, the less confident you are in each, the more likely a lab is appropriate:
Can this tool be tested by actual users?
Could you test this tool in your live environment without needing to do an unreasonable amount of extra work to make it safe and functional?
Can you meaningfully evaluate the tool’s performance?
Exemplars
To look at these concepts in practice, we compiled two brief case studies on technology innovation labs at the federal and state levels.
Census Open Innovation Labs (COIL)
COIL’s mission is to “source knowledge and solutions to solve key challenges for the Census Bureau and the public at large through human-centered design, data, creative media, and technology.” Their overarching portfolio is broader than our lab definition, but within it is a repeatedly successful, picture perfect version of a modular innovation lab: The Opportunity Project (TOP). After problem identification and team formation (processes handled in the Portal and Incubator innovation modules) the TOP model runs on 12-14 week product development sprints to deliver a minimum viable product (MVP). Complete details on the model can be found here in toolkit form for adaptation to your own processes. We’re going to focus on Phase 2 of their toolkit “Execute” to describe the core of their approach below:
User Research - TOP sprints spend 2-3 weeks (40-60 hours) solely focused on user research. This is an opportunity for their project teams to engage with communities to understand their perspective on the challenge being addressed, which responses work, which don’t, and what they need from a developing tool. Readers of our blog on Incubators will note that user research is a key element of the incubation process as well. To borrow another line from our friends at COIL, “User research is ongoing throughout the sprint, but this milestone is solely dedicated to reporting out on user research that informs the rest of the sprint.”
Data Exploration - Once teams have deepened their understanding of users’ needs and challenges, they go searching for relevant data. Sometimes this data is already under their control, but more often than not it involves engaging with data stewards and intermediaries who grant access and answer questions about datasets under their control. TOP’s process is specifically focused on leveraging open federal data, which (in contrast to its name) is quite difficult to make truly open and accessible. Regardless of the varying types and sources of data you need, defined data discovery processes are critical for projects combining datasets from across scientific, academic, governmental, civic and (potentially) private sectors.
Product Development - TOP’s development process includes a concept pitch, demos, and iterative redevelopment that incorporates feedback at every stage. Starting points can be very different for different projects, anything from a slide deck to a tool mockup can constitute a “concept pitch.” Beta demos showcase basic functionality and navigation to invite live reactions from users, advisors, and other teams. Teams should manage their time to ensure their engineers have sufficient time to incorporate responses.
Minimum Viable Products - The final two weeks of a TOP sprint hone in on producing an MVP. MVPs differ from betas by being fully testable under actual or nearly actual operating conditions. They are showcased under these conditions in front of user advocates, institutional champions, and team leads. The goal is to prove that the tool is fundamentally functional and to demonstrate how continued investment can achieve even greater returns for the users and institutional goals it is dedicated to
California Office of Data and Innovation (ODI)
ODI is a combined digital service team and enterprise innovation hub driving the adoption of impactful technological tools and innovative best practices across the California state government. Through the State’s Data and Innovation Fund they partner with agencies seeking technical improvements to scope, design, and develop custom-built tools. To understand their approach, let’s explore their collaboration with California’s Department of Drinking Water (DDW) to forecast community water system outages. While COIL’s process exemplifies the innovation concepts of Sprints, User Research, and Iterative Development, the ODI+DDW case is an excellent example of collaboration between technological and domain experts on rapid experimentation. Here’s how they did it:
Clear and tightly scoped problem statement - California has 2,866 community water systems, every year some of them run out of water. DDW wants to identify the factors that are most impactful and consistently predict where interventions are needed.
Domain driven data discovery - DDW scientists provided key processes to model and worked with ODI technologists to identify and obtain data that was relevant, consistent, and comprehensive.
Rapid experimentation - ODI data scientists assessed nearly two dozen datasets and tinkered with a variety of machine learning algorithms and architectures. Combining and adjusting these inputs gave them numerous options to choose from resulting in the easiest, most accurate, most repeatable solution out of the set.
Transparency - By selecting highly interpretable algorithms, open data and code, and public reporting, their work benefits not only those communities directly supported but also other actors in the drought resilience space (EPIC among them!) who can build off of their findings and methods.
On the path from idea to operations, the lab is the workhorse that truly transforms pie-in-the-sky ideas into something tangible and testable. Next up in our series on modular innovation, we’re going to plunge into the module that ensures it really works in the real world: test beds. In the meantime, we’d love to hear your thoughts on innovation labs and where they could be useful to advance environmental tech.