A call to action: ensuring access and availability of government data and publications

December 19, 2016
Mr. James R. (Librarian) Jacobs
Federal Depository Library Logo

There has been renewed worry and interest from library, academic, and scientific communities, as well as, the general public about the loss of government information with the incoming Trump administration and the need to archive this information. 

Both the New York Times and Washington Post carried articles discussing this issue recently, and I’ve discussed this at length for the library community on my Free Government Information Blog.

There are two national efforts taking place which the Stanford University Libraries is contributing to and we need your HELP!

First, there is the End of Term (EOT) crawl project , which has been taking place since 2008 to capture and document as much of the .gov/.mil domains as possible. Harvested content from previous End of Term Presidential Harvests is already available online.

In addition to federal government websites, the 2016 EOT is also capturing social media content and government information hosted on non-.gov domains (e.g., the St Louis Federal Reserve Bank at www.stlouisfed.org). Librarians and the public are contributing websites, twitter handles, Facebook accounts, etc. so that the EOT team can run several big crawls using the Internet Archive’s Heritrix Web harvester up to and after the Inauguration.

Second is the #DataRefuge Project, a collaboration between the UPenn Program in Environmental Humanities and the Penn Libraries group at University of Pennsylvania. They are building "a refuge for federal climate and environmental data vulnerable under a federal administration which denies the fact of ongoing climate change" and are seeking nominations for Federally created and hosted data sets (see the current list here) as well as people who can take on the stewardship of datasets by downloading, scraping websites and then hosting the data. The EOT project intends to use the #DataRefuge list of data set nominations to collect as much data being served out via FTP servers (as opposed to Http/https Web servers which serve out Web pages, PDFs etc).

The Stanford University Libraries (SUL) is participating in these efforts and wants your input and involvement!  We are asking that you nominate Federal agency Web sites as well as specific datasets or resources that are key to your work and research using this SUL nomination form

Starting this week, we are emailing Stanford faculty, departments, and research institutes and asking them to nominate sites for the EOT Project.  Any readers of this post are also welcome to nominate sites using the SUL form as well. 

We will take care of getting the nominated sites/resources sent to the appropriate group for inclusion in the two national efforts. The EOT and SUL are also concerned with content beyond climate and environmental issues like criminal justice/policing, health, education, oil/gas, food safety and regulation, and campaign finance - so please do include all subject areas of concern.

Here are two examples of resources that have been nominated:

For more information or questions, please get in touch with me, James Jacobs.