Understanding Photogrammetric Data: Scope and Complexities

Photogrammetric data encompasses a wide range of digital assets derived from the process of extracting three-dimensional measurements and models from two-dimensional photographs. This data is foundational in fields such as archaeology, civil engineering, environmental monitoring, forensics, and cultural heritage preservation. Typical photogrammetric datasets include high-resolution imagery (often with overlapping coverage), point clouds (in formats like LAS or LAZ), textured 3D meshes (OBJ, FBX, or glTF), orthomosaics, digital elevation models (DEMs), and associated metadata such as camera calibration parameters, ground control points, and coordinate reference system information.

The sheer volume and complexity of photogrammetric data present unique challenges. A single survey can generate hundreds of gigabytes of raw images, and the derived 3D models may require significant computational resources to process and store. Furthermore, many projects involve sensitive or proprietary information—archaeological site locations, critical infrastructure layouts, or classified environmental assessments—making data security a paramount concern. Without safe sharing and archiving practices, organizations risk data loss, unauthorized access, accidental corruption, or legal non-compliance.

Best Practices for Sharing Photogrammetric Data

Secure Transfer Methods and Protocols

When sharing large photogrammetric datasets with collaborators, contractors, or stakeholders, the method of transfer must prioritize both security and data integrity. Unencrypted channels such as basic FTP or unprotected email attachments expose data to interception and tampering. Instead, adopt cryptographic protocols like SFTP (SSH File Transfer Protocol) or FTPS (FTP over SSL/TLS). For cloud-based sharing, use services that enforce encryption in transit and at rest, such as AWS S3 with server-side encryption, or enterprise-grade platforms like Box and Tresorit with granular access controls.

For extremely large datasets (multiple terabytes), consider using physical media shipment (e.g., encrypted hard drives) as a supplement to digital transfers, especially in areas with limited internet bandwidth. Always verify data integrity with checksums (e.g., MD5, SHA-256) before and after transfer to ensure no corruption occurred during transmission.

Access Control and Permission Management

Sharing should always follow the principle of least privilege—grant users only the permissions necessary for their role. Implement role-based access control (RBAC) on file sharing platforms, and regularly audit user lists. For highly sensitive data, enforce multi-factor authentication (MFA) for both internal and external users. Consider using time-limited access links with download restrictions, as many collaborative platforms (e.g., Nextcloud, Egnyte) offer these features. Additionally, avoid sharing raw raw data unless required; instead, share derived products (e.g., downsampled models) when full resolution is not necessary.

Comprehensive Documentation and Metadata

Metadata is often the most overlooked aspect of sharing photogrammetric data, yet it is critical for helping recipients understand, trust, and reuse the data. At minimum, documentation should include:

  • Acquisition parameters: Camera model, lens focal length, sensor size, image resolution, overlap percentage, and flight altitude.
  • Processing workflow: Software used (Agisoft Metashape, Pix4D, RealityCapture, etc.), settings, and version numbers.
  • Coordinate reference system (CRS): The spatial reference system (e.g., WGS 84, UTM zone, state plane) and whether it was applied during processing.
  • Accuracy metrics: Ground sample distance (GSD), root mean square error (RMSE) of ground control points, and reported precision of the model.
  • Purpose and constraints: The intended use of the data (e.g., volumetric calculation, mesh generation for 3D printing) and any known limitations or known artifacts.

Adopt a standardized metadata schema where possible, such as the Open Geospatial Consortium (OGC) standards for geospatial metadata or the FGDC Content Standard for Digital Geospatial Metadata. Embed metadata directly into file headers (e.g., EXIF, XMP) or provide a separate metadata file (e.g., XML, JSON) alongside the dataset.

Data Compression Without Compromising Integrity

Compression reduces transfer time and storage costs, but not all compression methods are appropriate for photogrammetric data. Lossy compression (such as JPEG) degrades image quality and should be avoided for raw source images. Use lossless compression codecs like JPEG 2000 (lossless mode), PNG, or TIFF with LZW compression for imagery. For point clouds, compress using LAZ (lossless LAS compression) or use ZIP for other file types. When creating archives (e.g., TAR.GZ or 7zip), ensure the compression level does not introduce artifacts—always verify with checksums post-transfer.

Collaboration Platforms and Versioning

For ongoing projects, supplement one-time transfers with collaboration platforms that offer version control. Solutions like Dataverse, OSF, or institutional research data repositories allow teams to share iterative improvements without losing oversight of prior states. Alternatively, use version control systems such as Git LFS (Large File Storage) for code-related data, though for massive binary files, purpose-built tools like Pix4Dcloud or Nextcloud with file versioning are more practical.

Archiving Photogrammetric Data Safely

Storage Media and Redundancy

Archives must survive hardware failures, disasters, and technological obsolescence. The 3-2-1 backup rule is a gold standard: maintain at least three copies of the data, on two different media types, with one copy stored off-site. For long-term archiving:

  • Primary storage: Use RAID-configured network-attached storage (NAS) or enterprise-grade cloud object storage (e.g., Amazon S3 Glacier, Google Cloud Storage Nearline) with built-in redundancy.
  • Secondary storage: Maintain offline copies on LTO tape (Linear Tape-Open) or write-once Blu-ray M-Discs for critical data. Tape has a verified lifespan of 30+ years when stored properly.
  • Off-site copy: Replicate to a geographically distant location—physical media vaults or cloud storage in a different region—protecting against fire, flood, or theft.

Regularly test your restoration process. An archive is only as reliable as its ability to be read back. Schedule annual audits where you randomly sample files and verify their integrity.

Version Control and Data Provenance

Photogrammetric data evolves: raw images are aligned, optimized, and exported into various derivatives. Without version control, it becomes impossible to reproduce a specific model or trace errors. Implement an archive structure that distinguishes between raw, intermediate, and final data. Use a naming convention that includes dates, project codes, and version numbers. For example: ProjectA_raw_images_v1.0_20250101. Keep a changelog documenting why and when changes were made. Consider using hash-based content addressing (e.g., IPFS or simple SHA-256 manifest files) to create immutable links.

Metadata Maintenance in Archives

Archived data must remain understandable to future users (including yourself) years later. In addition to the metadata described for sharing, archival metadata should include:

  • Creation dates (original capture, processing, export).
  • Software and hardware dependencies (e.g., "Processed with Agisoft Metashape Professional 2.1.0 on a Windows 10 workstation").
  • Data relationships (e.g., "Point cloud in file points.laz corresponds to the orthomosaic in ortho.tif from the same flight block").
  • License and usage restrictions (e.g., "Creative Commons Attribution-NonCommercial 4.0").

Store metadata both embedded in files and as a separate README text file in plain UTF-8 to ensure human readability even if proprietary software becomes unavailable.

Choosing Sustainable File Formats

Technology evolves rapidly; proprietary formats can become orphaned. For long-term archives, favor open, well-documented, and widely adopted formats:

  • Point clouds: Use LAS (standard) or LAZ (lossless compressed) instead of proprietary .e57 or .rcp. LAS is an open standard maintained by the American Society for Photogrammetry and Remote Sensing (ASPRS).
  • 3D meshes: OBJ, PLY, and STL are widely supported, though OBJ may lack texture support in some contexts. For animated or textured models, consider glTF (GL Transmission Format) which is now an ISO standard (ISO/IEC 12113).
  • Orthomosaics and DEMs: Use GeoTIFF for georeferenced imagery and DEMs. Avoid JPEG 2000 for archives due to ongoing patent issues, though it is acceptable for access copies.
  • Raw images: Retain camera-native RAW files (DNG, CR2, NEF) alongside losslessly compressed copies (TIFF, PNG).

Periodically review your archive and migrate files to newer versions or formats as industry consensus shifts. Automated migration scripts can reduce manual effort.

Checksum Integrity Verification

Bit rot—gradual data corruption caused by physical media degradation—can silently destroy files. Implement systematic checksum verification. Generate checksum manifests (e.g., a .sha256 file listing all files and their hashes) at archive creation time. Schedule periodic integrity scans using tools like Clowder, BagIt, or simple bash scripts. If hashes mismatch, restore the affected files from a redundant copy. For cloud storage, use object-level checksums and enable versioning to recover overwrites.

Data Security: Protecting Sensitive Photogrammetric Information

Encryption at Rest and in Transit

Encrypt photogrammetric data both during storage and transfer. For at-rest encryption, use AES-256 (Advanced Encryption Standard) with securely managed keys. Full-disk encryption (e.g., BitLocker, FileVault) protects laptops and external drives. For cloud storage, consider client-side encryption where the cloud provider cannot access the plaintext. For file-level encryption, tools like GPG (GNU Privacy Guard) or VeraCrypt can secure specific datasets. Data in transit should always use TLS 1.3 or better, with mutual authentication for sensitive exchanges.

Network Security and Access Monitoring

Data sharing should occur over secure internal networks or VPNs when possible. Use firewalls to restrict access to storage servers, and apply network segmentation to isolate photogrammetry workstations from the public internet. Implement intrusion detection systems (IDS) to flag unusual data transfers. Enable detailed access logging on all storage and sharing platforms—track who accessed what, when, and from which IP. Regular log reviews can detect compromised credentials or insider threats.

Personnel Training and Policies

Technology alone cannot prevent human error. Develop a data handling policy that covers classification levels, sharing protocols, and breach reporting. Train all team members—from field technicians to project managers—on secure practices, such as recognizing phishing attempts, using strong passwords, and not leaving sensitive data on unencrypted USB drives. Conduct annual refresher training and include security as a component of project kickoff meetings.

Many jurisdictions have regulations governing geospatial data, especially if it involves critical infrastructure, archaeological sites, or personally identifiable information (PII). For example, the European Union’s General Data Protection Regulation (GDPR) applies if the data includes identifiable individuals. In the U.S., the Archaeological Resources Protection Act (ARPA) restricts distribution of site locations. Understand your legal obligations and include data sharing agreements that specify permitted uses, retention periods, and destruction requirements. Work with your organization’s legal counsel to draft data transfer agreements (DTAs) or memorandums of understanding (MOUs) with external partners.

Lifecycle Management and Future-Proofing

Developing a Data Management Plan (DMP)

Address sharing and archiving from the project’s inception by writing a Data Management Plan. A DMP outlines what data will be created, how it will be stored and backed up during the project, who will have access, which formats will be used, and what happens to the data after the project ends. Many funding agencies (e.g., NSF, NIH, ERC) require a DMP. Use templates from DMPTool or DMPonline to get started.

Migration Strategies for Obsolescence

Software and file formats become obsolete on a scale of years, not decades. Plan for periodic migration every 3–5 years. For example, if you have archived data in a proprietary .rcp format (ReCap project), convert it to open LAS/LAZ when that format becomes dominant. Document the migration process so future archivists understand the chain of transformations. Consider using format registries like PRONOM to check the sustainability of your chosen formats.

Open Data Repositories and FAIR Principles

When safe and appropriate, consider depositing anonymized or non-sensitive photogrammetric data in open repositories that adhere to the FAIR principles (Findable, Accessible, Interoperable, Reusable). Repositories like Zenodo, OpenTopography, or Sketchfab for 3D models provide persistent identifiers (DOIs) and enable long-term preservation. Even for confidential data, a metadata record describing the data’s existence and access conditions (e.g., "Available upon request to qualified researchers") can increase discoverability without compromising security.

Conclusion

Sharing and archiving photogrammetric data safely is not a one-time task but an ongoing commitment that requires thoughtful planning, robust technical infrastructure, and a culture of security awareness. By implementing secure transfer methods, adhering to open standards, maintaining redundant backups, and enforcing access controls, professionals can ensure their valuable datasets remain accurate, accessible, and protected throughout their lifecycle—from field capture to decades-long preservation. Investing in these best practices today will pay dividends by preventing data loss, streamlining collaboration, and enabling future discoveries.