U.S. Geological Survey Manual
U.S. Geological Survey Instructional Memorandum
No: IM OSQI 2015-02
Issuance Date: February 19, 2015
Expiration Date: Retain Until Suspended
Subject: Metadata for Scientific Data, Software, and Other Information Products
1. Purpose. This Instructional Memorandum (IM) provides interim policy on metadata requirements for U.S. Geological Survey (USGS) scientific data, software, and other information products. It also provides guidance for complying with appropriate USGS and other Federal standards, such as the metadata standards endorsed by the Federal Geographic Data Committee (FGDC), the interagency committee that provides metadata guidance for all Federal Government scientific data (geospatial and non-geospatial). This interim policy is issued to allow the time needed for USGS science activities to fully implement the metadata requirements herein and will be retained until superseded by a permanent Bureau Survey Manual (SM) policy chapter.
2. Background. Clear metadata descriptions answer fundamental questions (who, what, when, where, why, and how) and provide critical information intended to promote scientific collaboration, facilitate discovery, enable effective use, promote accurate interpretation of data, document its nature and quality, and augment the inherent value of USGS data and other products. Metadata also help users make effective use of scientific USGS data and information and aid the USGS in tracking its data and publications. Metadata is one of the required lifecycle elements described in IM-OSQI-2015-01 - Scientific Data Management Foundation.
A. Metadata must accompany all USGS scientific data, software, and other information products described in this policy that are approved for release. The content and format of metadata depend upon the type of data, software, or information product being described. When data, software, and other information products are digitally released, metadata must include an appropriate persistent identifier for the product, such as a digital object identifier or DOI, unless access to the product is restricted. Metadata records are to be developed in a standardized way that enables users to evaluate the data, software, or information product’s fitness for use in a research context. Metadata records must be updated to reflect changes and to ensure that links are functioning and continue to point to the intended files. Machine-readable metadata must be generated to enhance the transfer of information between systems and applications. Refer to https://www.usgs.gov/datamanagement/describe.php for additional guidance related to metadata.
B. A minimum of one metadata review by a qualified reviewer is required for all USGS scientific data, software, and other information products approved for release. A metadata review includes both checking for compliance with metadata standards using a recommended metadata validation tool when available, and performing quality checks. The metadata review can be conducted as part of the peer review, data review, editorial review, or separately as appropriate. A report of all metadata reviews (reviewer comments and how they were reconciled) must be included in the package in the internal USGS Information Product Data System (IPDS) that is submitted for Bureau approval.
4. Creating Metadata: General Guidelines.
A. Data. Metadata must be created for all scientific data, prior to approval and release as a USGS information product (refer to IM-OSQI-2015-03 - Review and Approval of Scientific Data for Release). This includes geospatial and non-geospatial datasets, databases, and Web data services that are created, collected, or compiled by USGS employees, volunteers, contractors, or data from other sources that are subsequently made part of a USGS dataset, database, or service. In addition to the records that describe data, metadata records are required to describe databases and Web data services. The metadata record for these data products is updated throughout the project; for example, metadata should be created at the start of data collection, updated during the course of the project, and updated and reviewed at completion of the project, when the dataset or database may be transferred from the active research environment to a system for long-term preservation. A metadata record includes information such as who produced the data and why, methodologies and citations, collection and processing methods used, definitions of entities and attributes, geographic location, and any access or use constraints, all of which facilitate evaluation of the data and information. A metadata record also includes, but is not limited to, authorship, title, abstract and purpose, theme keywords, data quality, temporal extent, and physical location. Refer to additional guidance on developing metadata for datasets and databases on the Data Management Web site (https://www.usgs.gov/datamanagement/describe.php). A checklist that provides guidelines to reviewers of metadata for data is available at (https://www.usgs.gov/datamanagement/documents/MetadataReviewChecklist_2014.pdf).
(1) Metadata records for datasets and databases must comply with one of the following FGDC standards: FGDC Content Standard for Digital Geospatial Metadata or the International Organization for Standardization suite of standards (refer to https://www.fgdc.gov/metadata). Extensions to the standards exist, and those FGDC-approved profiles or extensions that apply must be used. Similarly, any extensions in the ISO suite of standards that apply must be used. In the event an official FGDC metadata standard endorsement transition occurs, all new dataset and database metadata records must comply with that new standard.
(2) Metadata elements must be completed to comply with an applicable current FGDC or ISO standard that is appropriate for the data. There are both mandatory and optional elements in the standards. Mandatory elements apply to all data. Use non-mandatory elements when they apply to the dataset or database to ensure more robust, meaningful, and complete USGS metadata records that increase the ability to understand, use and integrate data. A persistent identifier, specifically a Digital Object Identifier (DOI) for data obtained from the USGS registration agent, is to be included in the metadata unless access to the data is restricted. Refer to additional information and about data DOIs for data (https://www.usgs.gov/datamanagement/preserve/persistentIDs.php).
(3) Metadata need to be shared at particular times in the USGS research process, such as metadata for planned data acquisitions, metadata for fully processed and quality-controlled observations and measurements, and metadata for datasets and databases that are approved for release (refer to IM OSQI-2015-03). These metadata for approved datasets and databases must be deposited in and shared through the USGS Science Data Catalog (https://data.usgs.gov/datacatalog/). Creators of this metadata must ensure that records are developed in a format (for example, xml) accepted by clearinghouses or repositories, such as the USGS Science Data Catalog. The FGDC and the Data Management Websites offer guidance on tools that may be used for creating standardized, well-structured metadata (https://www.fgdc.gov/metadata/geospatial-metadata-tools#availabletools and https://www.usgs.gov/datamanagement/describe/metadata.php).
B. Software. Metadata for scientific software must be embedded in the software code or associated files accompanying the software and, if the software is published on physical media, included on a label physically attached to the media. Metadata documentation includes the software descriptive name; a citation reference with author contact information and, as appropriate, reference to a publication(s) that describes the software functionality, purpose, configuration specifications, operating instructions, and intended use; the name/position title and contact information for one or more individuals or organizations responsible for the software or source code to be published; version or revision numbers and dates or another consistent method of describing currency of the software, technical details of the operating environment for the software, including programming languages and supported operating platforms; and available forms of the software, including executable packages and source code.
C. Other Information Products. Other information products include, but are not limited to, USGS series publications (SM 1100.3), and outside publications (SM 1100.4).
(1) Metadata for these products will contain a complete citation with page numbers, volume, and issue if applicable. For USGS series publications, metadata must include an appropriate persistent identifier to ensure discoverability of the product. USGS series products must also meet the Library of Congress metadata standards related to citations.
(2) Metadata records for publications must be created using the internal USGS IPDS. These metadata records include, but are not limited to, authorship, title, originating office or center, product type and number, and reference to any associated data, if applicable. If other metadata elements apply to the information product, the elements should be included to ensure that metadata records are more complete and useful. Links to available supporting information such as associated data or software should be included in the metadata unless access to these products is restricted. Metadata for products approved through the IPDS are deposited in and shared through the USGS Publications Warehouse. The IPDS Web site also provides internal guidance to USGS employees on developing metadata for these information products.
5. Responsibilities. All USGS employees, contractors, and volunteers engaged in data collection, research, and data development (SM 502.2) activities and in activities related to review, approval, and release of information products (SM 502.4) are responsible for complying with this metadata policy. Specific responsibilities include:
A. Associate Directors and Regional Directors. Associate Directors (ADs) and Regional Directors (RDs) work collaboratively with the entire USGS Executive Leadership Team (ELT), to address issues or take corrective action with regard to metadata policy.
B. Office of Science Quality and Integrity and Core Science Systems. The Office of Science Quality and Integrity (OSQI) and the Core Science Systems (CSS) mission area are responsible for jointly developing USGS metadata policy and collaborating with the ADs and RDs and the entire ELT as needed to resolve issues regarding the implementation of this policy. They work in coordination with managers and staff in USGS science centers and offices to provide Bureau-wide advice, guidance, and procedures related to metadata for the products specified herein. The CSS provides metadata related guidance through the Data Management Web site (https://www.usgs.gov/datamanagement/describe.php). The OSQI maintains and communicates this and other FSP policy documents.
C. Science Center Directors. Science Center Directors ensure compliance of the metadata records for the data, software, and other information products that are produced by authors in their centers or offices and consult with their respective ADs and RDs and CSS and OSQI officials and staff as needed in ensuring compliance.
D. Approving Officials. Approving Officials, including Science Center Directors (or their designee) and Bureau Approving Officials in the OSQI, ensure that the appropriate metadata review has occurred and that documentation about the metadata review is included in the package submitted to them for Bureau approval (SM 205.18).
E. Authors’ Supervisors. Authors’ Supervisors have the first responsibility in the management chain with regard to ensuring a metadata review occurs. They consult with their respective Science Center Directors as needed to ensure the quality and adequate maintenance of metadata records for the data, software, and other information products that are produced by authors they supervise.F. Scientists and Authors. USGS scientists and authors ensure that sufficient metadata records are created for each data, software, and other information product they produce and that these records are consistent, up to date, and verified for accuracy in accordance with the requirements in this policy. This includes ensuring that the appropriate metadata review, peer review, editorial review, and approval for products they produce are obtained.
/s/ Alan D. Thornhill February 19, 2015
Alan D. Thornhill Date
Director, Office of Science Quality and Integrity
Return to Instructional Memoranda Page
Return to Survey Manual Home Page