Openness and Interpretability in Generative AI

The issue of defining open-source artificial intelligence is multifaceted and merits careful consideration from various perspectives. While many individuals support making AI technologies openly accessible in principle, determining precisely what constitutes “open-source” is challenging given the diverse ways the concept can be interpreted and implemented in practice.

The European Union’s recently adopted Artificial Intelligence Act touches on this nuance to some extent in allowing more flexibility for transparency requirements as they apply to open-source models. However, the legislation does not endeavor to provide comprehensive guidance, acknowledging the complexity involved.

As one academic analysis highlighted, there are numerous potential characteristics of an open-source approach to AI that could be taken into account, ranging from availability of source code and training data to absence of restrictions on use and distribution. Similarly, research conducted by scholars at Radboud University assessed several large language models against fourteen openness criteria, finding that self-described open and open-source systems exhibited variability in terms of code, documentation, and model accessibility provided to outside parties.

The study indicated that even models marketed as open or open-source may in practice only allow access to trained weights with limited inspectability and customizability of the underlying techniques. This prevents full transparency and understanding of efforts to fine-tune the capabilities for specific applications based on human input.

In light of such nuances, the author of the latter study proposed a framework for systematically evaluating the degree to which different AI systems embrace openness. At the same time, whether completely unfettered openness should be the goal in every situation remains an open debate given potential risks if the technologies are abused or misused. Overall, developing consensus around a definition involves weighing technical, ethical, and policy considerations.

The paper can be accessed at: https://dl.acm.org/doi/pdf/10.1145/3630106.3659005