Technology

Repository technology

The SDR is developed and maintained by Stanford Libraries’ Digital Library Systems and Services. The system uses a combination of open source technologies, including:

  • Fedora - serving as a robust, durable repository for managing digital objects. Fedora's design lends itself well to establishing flexible relationships with other components of the digital library. 

  • Solr - used to index information and offers "blazing fast" search of the repository contents. Solr's flexibility can accommodate a diversity of data sources.

  • Blacklight - builds on the Solr index to provide faceted searching, browsing, and tailored views of objects.

In addition to these technologies, Stanford Libraries developed Moab for versioning archived digital objects. Details about this novel approach are available in this 2013 article originally published in the Code4Lib Journal.  Stanford is now actively involved in the specification of the Oxford Common File Layout, which is based on Moab, and plans to implement OCFL in the future.

Beginning in 2017, while working with the community to build on our learnings from use of Fedora, we started to explore alternatives to Fedora 3. Work is now underway to replace it with a database (most likely Postgres) and Cocina, a locally developed data model. In 2022 we expect to complete the migration from Fedora 3.

 

Storage

In 2018, we retired our local LTO-6 tape library and put into operation the ability to replicate all SDR content to cloud storage services, with content stored in three geographic zones by two different service providers. The aim is many copies, geographically dispersed, with multiple vendors and no single point of failure.

In 2020 we completed a 3-year effort to design, plan, and execute a full migration of our preservation storage system from NetApp to Ceph, an open source, distributed storage system. In addition to the access copies readily available online, one preservation copy of SDR content is stored locally on spinning disk. We have twelve local storage servers running Ceph, yielding ~2.2 PB of usable storage.

For more information about the SDR's underlying technology, please contact the SDR team.

accessibilityaccessprivsarrow-circle-rightaskus-chataskus-librarianbarsblogsclosecoffeecomputercomputersulcontactsconversationcopierelectricaloutleteventsexternal-linkfacebook-circlegroupstudyhoursindividualinterlibrarynewsnextoffcampusopenlateoutdoorpeoplepolicypreviousprinterprojectsquietreservesscannersearchstudysupportingtabletourstwitter-circleworking