Home » Managing Data » Q. What file formats will I need to be familiar with?

Q. What file formats will I need to be familiar with?

Part of the DCE Getting Started Guide: Managing Data » Section 2: Identify • Submitted by hbowden (Heather Bowden) on Tue, 05/10/2011 - 10:05
Guide: Managing Data Focus: Learning | Practice File Formats

When curating and preserving research data and digital collections, selecting or normalizing to well-documented, non-proprietary, or widely adopted standard file formats significantly increases the likelihood that digital objects will remain interpretable and reusable over the long term.

Key Sustainability Factors for File Formats

  • Disclosure & Open Documentation: Complete technical specifications are publicly available and standardized (e.g., ISO, W3C, IETF), rather than proprietary or trade-secret formats.
  • Adoption: Widely used across academic, archival, and industry communities with multiple independent software implementations.
  • Transparency: Formats encoded in plain text (such as CSV, XML, or ASCII/UTF-8 text) or uncompressed bitstreams are easier to inspect and migrate than complex binary or encrypted containers.
  • Self-Documentation: Ability to embed descriptive and technical metadata directly within headers (e.g., TIFF tags, Broadcast WAVE chunks, SAS Transport XPORT dataset headers).
  • Absence of Patents / DRM: Free from licensing fees, cryptographic locks, or digital rights management restrictions that prevent preservation copying or migration.

Common File Formats by Content Type

Data / Content Category Preferred Archival / Preservation Formats Common Submission / Exchange Formats
Tabular & Statistical Data Comma-Separated Values (.csv), Tab-Delimited Text (.tab), XML with schema, SAS Transport File Format (XPORT / .xpt), SPSS Portable (.por) Microsoft Excel (.xlsx, .xls), SAS native (.sas7bdat), SPSS (.sav), Stata (.dta), R workspaces
Text & Documents PDF/A (ISO 19005-1 / 19005-2), Plain Text (UTF-8 / ASCII .txt), XML / TEI / DocBook, HTML/XHTML PDF, Microsoft Word (.docx, .doc), OpenDocument Text (.odt), Rich Text Format (.rtf)
Still Images (Raster & Vector) Uncompressed TIFF (.tif), JPEG 2000 Lossless (.jp2), PNG (.png), SVG (.svg) JPEG (.jpg), GIF, Adobe Photoshop (.psd), Camera RAW / DNG
Audio Broadcast WAVE (BWF / .wav Linear PCM), AIFF, FLAC MP3, AAC (.m4a), Windows Media Audio (.wma)
Digital Video / Moving Image Uncompressed or lossless JPEG 2000 / FFV1 in MXF, Matroska (.mkv), or QuickTime (.mov); MPEG-2 / MPEG-4 (H.264) for access MP4, AVI, WMV, AVCHD

Recommended Format Registries & Guidance

  • Library of Congress Sustainability of Digital Formats: Detailed format descriptions (FDDs) evaluating sustainability factors, quality/functionality factors, and file signatures (including the SAS Transport XPORT family, PDF/A, TIFF, WAVE, and MXF).
  • PRONOM Technical Registry (The National Archives, UK): Definitive registry of file format signatures and Persistent Unique Identifiers (PUIDs) used by DROID.
  • UK Data Archive & ICPSR Guidelines: Recommended file formats tables for quantitative and qualitative social science and scientific research data.
  • Unit 13: Characterization of Digital Objects (DigCCurr Slides PDF) — Overview of format identification, validation, and feature extraction.