About SCMS

A Github repository for documentation about the Beinecke Rare Book and Manuscript Library’s Special Collections Metadata Services Unit (SCMS).

What is SCMS

Special Collections Metadata Services (SCMS) provides a comprehensive, cross-collection, and service-centered approach to enabling and enhancing the discovery of Yale Library’s special collections through metadata services, data reporting and analysis, and special collections systems and tool management.

SCMS Metadata Services

SCMS provides the following services:

  • Metadata management: remediation, creation, and enhancement of special collections metadata

  • Data transformation: migrations, mappings, data re-use

  • Reporting and data analysis

  • Tool and workflow support: automation and process improvement

  • ArchivesSpace support: product ownership, application and user management

  • Training, documentation, consultation

These services are provided in multiple contexts. In some cases our work originates internally - e.g. SCMS staff identify and prioritize data problems and opportunities. In others, services are requested by other SCTS units, special collections departments, or repositories. Our services may also be provided as part of our participation on committees, advisory groups, task forces, or working groups. And finally, SCMS services may be provided to support cross-departmental projects, programs, or initiatives.

Metadata management

Metadata management consists of the bulk remediation, creation, and enhancement of special collections metadata.

Activities

  • Identify and prioritize metadata problems, opportunities, requests

  • Develop strategies for assessing and addressing metadata issues (e.g. data auditing, reporting)

  • Develop metadata project plans

  • Develop ETL-style workflows for managing metadata projects

  • Perform exploratory data analysis and reporting

  • Programmatically manipulate data

  • Develop and execute scripts to update metadata in systems of record

  • Communicate with stakeholders

  • Perform testing and quality assurance activities

  • Maintain and enhance tools, documentation, training materials

Examples

  • Identify and delete expired restriction dates

  • Clean up controlled value lists in ArchivesSpace

  • Update preferred citation notes for Divinity after library name change

  • Add structured dates to ArchivesSpace records which previously only had free-text dates

  • Add SNAC identifiers to agent records

  • Add barcodes to location records

  • Delete archival object records for materials which have been reprocessed

Stakeholders

  • Special collections metadata-creating departments and units: BRBL-TS: ADU, BDU, ACCU; Divinity Library, Music Library, Arts Library, YCBA, Peabody Museum (Archives)

  • DSCA

  • Cross-unit/departmental projects and committees

  • Archives Advisory Group (AAG)

  • YUL MSC

  • YUL Technical Services

  • Library IT

Data transformation

Data transformation services include data migrations and conversions, data re-use projects, and integrations with other systems.

Activities

  • Communicate with stakeholders to identify transformation projects/opportunities

  • Perform mapping/crosswalking activities

  • Develop strategies for handling handling data loss in the transformation process

  • Manage system integrations

  • Create requirements documents for use by developers

  • Act as liaison between stakeholders and developers

  • Create documentation

  • Reporting and exploratory data analysis

  • Develop programmatic data transformation tools

  • Plan and execute database migrations

  • Maintain and update existing data mappings and system integrations

  • Coordinate with product owners of other systems

Examples

  • Migrate YNHH data from Access database and Word documents to ArchivesSpace

  • Migrate Acquisitions DBText database to ArchivesSpace

  • Develop mappings between ArchivesSpace and Lux

  • Develop mappings between ArchivesSpace and Metadata Cloud/DCS

  • Convert architecture project index spreadsheets to ArchivesSpace records

  • Convert GLAD box lists to ArchivesSpace records

  • Convert Proquest-provided dissertation metadata to Elischolar and Voyager records

  • Maintain Excel-to-EAD transformation tool

  • Reassociation of data between Find-It/Ladybird and ArchivesSpace

  • Integration of ArchivesSpace and Quicksearch

  • Integration of ArchivesSpace and DCS

Stakeholders

  • Special collections metadata-creating units: BRBL-TS: ADU, BDU, ACCU, DSCA

  • Cross-unit/departmental projects and committees: UDAG, AAG, CHIT/Lux

  • YUL MSC

  • Library IT

  • YUL Technical Services

  • Public Services and Operations

Reporting and data analysis

Reporting and data analysis activities can stand on their own or be provided as part of another service such as data management and transformation.

Activities

  • Communicate with stakeholders to identify required data elements

  • Develop and execute queries across multiple systems

  • Develop tools to synthesize data from disparate systems

  • Perform exploratory data analysis and manipulation

  • Present requested information to stakeholders

  • Develop and maintain tools enabling staff to self-run reports

  • Perform regular data auditing and quality assurance

  • Leverage data to support decision making

  • Document and share data analysis tools with professional community

Examples

  • Report on Papyri records in order to facilitate migration to DCS, additional physical processing, and patron research

  • Report of University Archives materials stored at off-site storage

  • Report of all acquisitions in the last 5 years

  • Reporting to support Special Collections backlog project

  • Reports plugin: report of all containers in a given location

  • Reports plugin: report of all materials in a given collection that are digitized and in Preservica and/or DCS

Stakeholders

  • BRBL-TS: ADU, BDU, ACCU

  • DSCA

  • BRBL Administration

  • Public Services and Operations

  • YUL Technical Services

  • Library IT

  • Cross-department/unit committees and projects

  • Digital Preservation Services

Tool and workflow support

Activities

  • Communicate with stakeholders re: workflow needs

  • Identify opportunities for workflow improvements or automation tools

  • Planning, testing, and prototyping of tools and workflows

  • Scripting and tool development

  • Write user and technical documentation, train staff on new workflows

  • Maintain and share tools

Examples

  • Workflow for automated processing of electronic dissertations

  • Copy order fulfillment process for previously digitized materials

  • Box and folder labeling process

  • Workflow support for Digital Accessioning Service

  • LSF barcode search tool

  • Reassociation project

Stakeholders

  • BRBL-TS: ADU, BDU, ACCU

  • Public Services and Operations

  • Digital Preservation Services (DPS)

  • DSCA

ArchivesSpace Product Ownership and Support

Activities

  • Policy development surrounding usage of ArchivesSpace (in consultation with advisory groups, MSC, SCSC)

  • Triage staff and patron support requests

  • Administration - manage passwords, permissions, account creation

  • Troubleshooting - reporting and resolving system problems (in collaboration with service owner)

  • Manage parts of the external development process

    • Gather user feedback and prioritize development work

    • Write user stories

    • Meet with external developers

    • Test application upgrades, features, plugins, bug fixes

    • Coordinate and manage implementations

    • Maintain plugins on Yale Github accounts

    • Low effort development work

  • Training and documentation (in collaboration with advisory groups and service owner)

  • Manage hosting (in collaboration with service owner)

    • Recommend and manage infrastructure improvements to the hosting environment

    • Generate Docker images

  • Communicate system changes to stakeholders and provide training

  • Serve as ArchivesSpace representatives on library- or university- wide projects

Stakeholders

  • All ArchivesSpace staff users

  • Public Services and Operations

  • BRBL Tech Services

  • Archives Advisory Group (AAG)

  • YUL Technical Services

  • Library IT

  • Preservation and Conservation Services

  • Lyrasis (hosting provider)

  • Hudson Molonglo (primary contract developers)

How to Update This Documentation

This documentation is generated by Sphinx from scripts, spreadsheets, and various .RST and .MD files within the documentation Github repository.

Spreadsheets

The spreadsheets are stored in the _files directory. The spreadsheets include list of current Yale ArchivesSpace plugins, a list of current DMAC-related Github repositories, lists of completed, active, recurring, and future projects, and a list of ArchivesSpace data that is subject to data auditing protocols.

The Github repository and plugins spreadsheets are generated via the get_repo_data.py script. The project and data auditing spreadsheets are created manually, and are then converted into a table by the Sphinx documentation generator.

Github Readmes

The get_repo_data.py script also pulls in all READMEs from DMAC-related Github repositories, and stores them in the readmes directory of the documentation.

Other Readmes

Separate directories exist in which policy, workflow, and tutorial documentation can be added. These documents should be .MD files.

Each directory has a corresponding .RST file in the main directory, which pulls in all of the files which are in the directories and displays them in the generated documentation.

Building the Docs

To update the documentation, you can edit the files in the repository, and then push your changes to the dmac_data_docs Github repository. The documentation will automatically re-build.