Building Sustainable Digital Archiving Practices Through Documentation

The Digital Archiving group in the Department of Special Collections and University Archives is responsible for processing, preserving, and providing access to the born-digital archival holdings of the department. As these materials have grown in both importance and complexity, our group has continually looked for ways to scale our capacity to manage them effectively.

One of the primary ways that we have adopted to make our work more efficient is the documentation of our workflows. Moving born-digital materials from their initial state at accession through long-term preservation in the Stanford Digital Repository (SDR), and finally to our access platforms involves numerous sub-workflows, systems, tools, and decision points. Clear and detailed documentation is essential to ensuring that archivists can carry out this work consistently and systematically.

In 2025, we embarked on a project to expand, formalize, and share our documentation with a broader audience of archivists. This effort was inspired by other institutions who had done the same, including at the University of Georgia Libraries, Yale Library, and The New York Public Library. These resources provide invaluable context on how peer institutions approach some of the fundamental questions and common challenges in digital archiving and we hope that our contribution will, in turn, support the wider community.

Developing the Documentation

Before starting this project, we already had documentation on many of the systems, tools, and workflows, but much of it was informal and incomplete. Our group worked collaboratively and iteratively to update, complete, and refine existing documentation and create some entirely new workflows. We each chose specific pieces of documentation, often relating to our processing work at the time, to write and then met regularly to review and finalize the workflows.

This collaborative model proved especially valuable. Drawing on a larger pool of expertise allowed us to identify gaps, clarify assumptions, and incorporate a more diverse set of experiences. The resulting documentation is stronger and more comprehensive because we were able to work in this way.

Preparing Documentation for Publication

Once our documentation reached a stable state, we considered a few potential platforms to publish it, including Consul, Google Drive, and Github. We ultimately determined that Github would offer the most straightforward visual display as well as the necessary capability to provide access to non-Stanford users. Documentation on Github is formatted in Markdown, a markup language used to encode text. In order to publish our documentation, which we created in Google Documents, we first had to encode it in Markdown. Rather than encoding the documents manually, we tried a tool called Pandoc to convert our files to the Markdown format. This turned out to be quite simple and required only minimal manual editing to correct some minor formatting inconsistencies in our resulting documents. Adding our properly formatted documents to our Github repository was a fairly simple process and once complete, our born-digital processing documentation repository was live.

Conclusion and Next Steps

We invite colleagues to explore our documentation and hope that it will be as helpful to others as such efforts have been to us. This repository is intended to be a living document and we plan to update documentation as needed and as our capacity allows.

It is also important to acknowledge that this project was made possible by increased staffing in the department. The addition of two additional full-time Digital Archivists, funded by the Andrew W. Mellon Foundation and the United States Senate Historical Office, provided the critical mass needed to complete a project of this scope. Although these positions are term-limited (with one term recently concluded), the expanded team enabled both the depth of expertise and the time to complete this work in a relatively short timeframe.

We welcome feedback and questions from the community. Sharing documentation is not only about transparency, but about contributing to the ongoing conversation and collaboration in the field of digital archiving.

Acknowledgements

Many thanks to Al Bersch and Annie Schweikert, who led this project with me. We also received invaluable input from the other members of the Born-Digital Processing Working Group, including Chris Doan, Emma Frothingham, Sabrina Gunn, Chloe Pfendler, and Alyssa Tou.