Real Good Research

Illuminating the Black Box

Overview

  • These days, most AIs are powered by neural networks, which are called “black box” models due to researchers’ inability to explain why the model predicts (or generates) what it does. To make matters worse, these models need an unreasonable amount of data and resources to do well.

  • However, we believe there is a way to achieve powerful AI without black boxes. We are convinced using Gaussian process models (or other statistical models) will solve many of the issues with current AI including data and computational efficiency, explainability, and uncertainty quantification. This is why we are working on research that can lead to the development of black box-free AI models!

  • There are a lot of existing data science tools that can really help better our world. However, research problems “for the good of humanity” are often overlooked because they would not make a company money. Therefore, we are happy to use our expertise for the public good by collaborating with other nonprofits and researchers to perform analyses that solve problems vital to humanity.

  • Our research is open source. We are excited to share our results (and failures) with you.

Current Projects: AI without Black Boxes

  • Data Attribution with Surrogate Modeling

    Artists are rightfully concerned about AIs stealing and copying their work. However, the black-box nature of generative AI models makes this hard to prove. That’s why we’re developing methodology that can shed light on the issue by using Gaussian process surrogate models.

  • Helping to Identify when AIs are Wrong

    AI models can be very confidently wrong. Due to their black-box nature, AI models are incapable of quantifying their own uncertainty… in other words, they don’t know when to be confident versus cautious about answering a question. We are developing proof-of-concept models with built-in uncertainty quantification.

  • Showing we don't need Black Boxes

    Neural networks (aka black box models) are the “state-of-the-art” when it comes to AI (booooo). We are working to change that by developing AI frameworks that replace neural networks with Gaussian process models. From diffusion to transformers, we aim to show the world AI can be black box-free.

Can we determine how an AI generated image can be attributed to the training data?

Cody Schultz and Dr. Eric Bell

Artists have been concerned that AIs are infringing on their copyright by blatantly copying their work. However, this is very difficult to prove since neural networks have no explanation for their predictions. We are developing methodology that could help this by breaking apart the architecture and using Gaussian Processes on each layer to attribute the percent reliance on each piece of training data. This could not only help artists protect themselves from theft, but could allow for a system where artists could be properly paid. Further, this could have further implications in simplification of AI models, where the most vital pieces of data are identified.

Selected Projects: AI without Black Boxes

How can we help identify when AIs are lying?

Dr. Eric Bell and Dr. Amanda Muyskens

Because of the “black box” of machine learning algorithms, mistakes that AI’s make are more believable to humans reading them.  This is leading to misinformation and limits the utility of AI to the public. We believe that an AI’s response should be reflective of the confidence of the model in the question it is asked. However, this requires new mathematical methodology, which is exactly what our team is doing. In our proof of concept transformer models, we have eliminated neural networks entirely, and in doing so, we have engineered the model to be able to provide statistics-based uncertainty for each generated token. Additionally, we are pursuing a formal sensitivity analysis to decide how errors in the model can be attributed throughout the architecture of the neural network to try to identify if there is a simpler form of uncertainty that can be defined.

Dr. Mandy discusses why this problem is challenging: the curse of dimensionality.

Selected Projects: Doing Good with Good AI

Can we help nonprofits plan for their future and learn from each other?

Dr. Mayleen Cortez-Rodriguez

An important part of our mission is to help the world with real good data science. We developed a revenue prediction tool for nonprofits. This is a free and simple-to-use tool that helps nonprofits plan for the future, set realistic goals, and learn from others who have been in their shoes. Using open data from the National Center for Charity Statistics, we find other organizations from the same sub-sector that had similar revenue trends in the past and use Gaussian Process modeling to predict an organization’s future.

Learn more

Can we measure the impact of policy on the unhoused population among older adults?

Dr. Imène Goumiri

This research presents a comprehensive, data-driven assessment of Continuum of Care (CoC) effectiveness in supporting unhoused older adults (65+) across California. Leveraging publicly available data, rigorous statistical modeling, and an innovative interpretation methodology, we identify factors impacting the unhoused population. By integrating diverse datasets including unhoused counts, population demographics, and geographical information, we aim to uncover the underlying drivers contributing to housing insecurity in this vulnerable demographic. Our findings highlight the critical role of housing affordability and economic stability, provide a ranking of CoCs based on their effectiveness under controlled conditions, and offer concrete policy recommendations for targeted interventions and best practice dissemination. This study contributes to a more transparent and evidence-based approach to addressing California's escalating unhoused crisis.

Learn more

Research Repository

Articles

AI Without Black Boxes

Doing Good with Good AI

Code

GitHub

Presentations

Coming soon…