The Stanford Digital Repository: from launch to legacy

The Stanford Digital Repository (SDR) is marking its first two decades with reflections on historical milestones, announcements of recent enhancements, and forecasts of future developments. The service is freely available to students, faculty, postdocs, and staff for the broad circulation and intellectual protection of nearly every type and format of scholarly material. Over 5.5 million digital objects and 1.8 petabytes of data are equipped with persistent URLs that provide long-term citation links and the metadata necessary for discovery by scholars through SearchWorks, the online catalog of Stanford University Libraries, or DataWorks, the Libraries’ new data catalog now in beta release.

“In 2006, we received an inspiring proposal from University Librarian Mike Keller for a long-term archival solution to store key academic materials. This marked the genesis of the SDR, and it was truly exciting,” said Russ Altman, the Kenneth Fong Professor of Bioengineering, Genetics, Medicine, and Biomedical Data Science. Altman, who was the inaugural chair of the SDR’s longstanding faculty advisory committee, added, “At that time—and in many ways still today—it was unprecedented for a university to build such a storehouse. The SDR has become a crucial component for the storage of digital media that would otherwise be lost without mechanisms for preservation.”

“The SDR is critical infrastructure for student papers and local research at the Hopkins Marine Station,” said Amanda Whitmire, Head Librarian & Bibliographer for the Harold A. Miller Library. “For example, we’ve digitized the complete archives of the Stanford Oceanographic Expeditions (1963-1973) and the Stanford component of the California Cooperative Oceanic Fisheries Investigations (1951-1974). Both archives were completely undiscoverable and physically at risk without copies elsewhere.”
 

Advanced Lecturer Robin Elahi, who first used the SDR as a postdoctoral researcher at Stanford to submit data and code for a published article, remarked, “Most importantly, the SDR now provides a local alternative to YouTube for archiving and sharing subtidal video surveys with my students, such as our 2022 transect videos of the kelp forests of the Monterey Peninsula. The streaming video quality of the SDR is excellent.”
 

“One of the foundational principles of open science is elevating research data and code as first-class research outputs,” said Zach Chandler, Director of Open Scholarship Strategy at the Stanford Institute for Human-Centered Artificial Intelligence. He added, “The SDR is independently developed by our in-house team, who have demonstrated foresight by integrating persistent identifiers like ORCID and have proven to be champions of open science practices. Embedding our open science values into the infrastructure layer is how we succeed.”

In the humanities, the SDR has equally compelling applications. Jonathan Berger, the Denning Family Provostial Professor in Music, was introduced to the SDR by PhD candidate Blair Kaneshiro, who discovered novel methods to interpret neurophysiological measures of music engagement and deposited large datasets that continue to be accessed, cited, and replicated today. “Since then, the means of posting have become increasingly simple. I’ve used the SDR to store substantial datasets of acoustic studies from site visits in Italy, Egypt, Peru, and France for my research project, Sound, Space, and the Aesthetics of the Sublime, as well as my own creative work, which includes scores, recordings, performance materials, notes, critical reviews, and more,” said Berger.

At the Graduate School of Education, Senior Lecturer Karin Forssell, director of the Learning Design and Technology program, as well as the school’s innovative Makery and AI Tinkery, teaches her students to apply research from the learning sciences and learning-centered design processes to create effective digital tools. “The SDR provides students with an authentic audience of real-world readers and allows them to put their best foot forward with their capstone projects,” she said.

The SDR serves as the source for the digital showcases in the Libraries’ Spotlight Exhibits. “We assist units across campus in preserving their records, including born-digital content, and in telling their stories,” said University Archivist Josh Schneider. While accessioning records for the Asian American Studies Program and the Department of African & African American Studies, Schneider’s team employed student assistants who utilized the digitized SDR materials to create the online exhibits Asian American Advocacy at Stanford University and Black @ Stanford: An Anthology of Black Activism and Community at Stanford.
 

Stanford’s spatial data infrastructure has been significantly modernized. “Digitized maps and GIS data in the SDR are now supported by standardized, cloud-optimized derivatives designed for faster access, improved web performance, and enhanced programmatic research workflows,” said Stace Maples, Assistant Director of Geospatial Collections & Services. He explained, “Using Embed, an open-source tool created by the Libraries to display repository content, researchers can retrieve and work with specific portions of a spatial dataset rather than downloading and processing the entire file. A striking example of this is a composite image featuring 164 aerial views of San Francisco from 1937 to 1938 from the David Rumsey Map Center. The composite is more than 2 GB in size, yet is rendered quickly and interactively.”
 

Hannah Frost, Associate Director of Digital Library Services, summarized the progress made over twenty years: “After the release of SDR 1.0 in August 2006, the debut of the Electronic Thesis and Dissertation service in 2009 marked our next major milestone. Other milestones were the launch of our SDR web app in 2013, the partnership with Big Local News beginning in 2017, the inclusion of the SDR in the Stanford Open Access Policy in 2020, the adoption of Digital Object Identifiers in 2021, and a major back-end rebuild completed in 2022. This year, we added support for automated GitHub code deposits and a super-fast method for depositing open access articles.”

“Looking ahead,” Frost continued, “the team is considering more enhancements such as are AI-extracted article abstracts and faster downloads for large files. We continue to promote our services for student capstones and are very excited to see SDR datasets available via the beta version of DataWorks.” Frost and her service and operations teammates – Amy Hodge, Andrew Berger and Emma Stanford -- would like to thank all those who have contributed to the growth of the SDR by depositing content and providing valuable feedback.