Kopia v0.8

https://github.com/kopia/kopia/releases/v0.8.4.

This version brings many performance, usability and stability improvements and several new and exciting features.

There were over 150 changes by 19 contributors which represents a big growth in community engagement.

New Features:

  • Full Maintenance is now enabled by default with an interval of 24 hours.
  • Added support for configurable Maintenance Safety to speed up cleanups in certain cases.
  • Added support for --json flag to certain CLI commands which will produce machine-readable snasphot, policy, manifest, blob and content responses.
  • Added support for scanning only one filesystem via files policy
  • Major improvements to ignore logic to better follow .gitignore rules.
  • Added support for Actions which allow running custom scripts before and after a directory is snapshotted with lots of useful applications.
  • Added support for ACLs which allow defining custom access control rules for repository server clients.
  • Docker Images are now available
  • Support for write sessions, which ensure garbage collection does not prematurely delete blobs that are in the middle of being uploaded by intermittently-connected clients (e.g. laptops that go to sleep often and can’t checkpoint frequently enough)
  • Added support for stdin snapshots.
  • Added support for user authentication using user profiles stored in the repository (#809)
  • Brand new streaming GRPC protocol for talking to kopia repository server to improve performance and ensure session stickyness, which allows sharding and load-balancing.
  • Reworked error handling when taking snapshots to not fail fast (except when explicitly requested).
  • Switched to new FUSE implementation which supports MacFUSE 4.x
  • Added WebDAV-based mounting alternative for platforms that don’t have working FUSE mechanism.
  • Streamlined flags for caching and improved cache sweep.
  • Major performance improvements when performing lots of index lookups in heavily-fragmented indexes (speeds up snapshot verification by orders of magnitude)
  • Passthrough options to allow other-users access and mounting in empty directory.
  • Don’t run maintenance as part of read-only actions.
  • Implemented caching for contents downloaded from Kopia Repository Server
  • Reduced latency for kopia repository server uploads by performing existence checks
  • Improvements to snapshot verify to make output easier to understand.
  • compress: upgrade klauspost/compress to v1.11.12 (#888)
  • Terminal password prompt - switched to golang.org/x/term (#898)

New KopiaUI features:

  • Added new tab called Tasks which shows the status, progress, logs, and statistics of current and past long-running tasks (restore, estimate, snapshot, maintenance).
  • Streamlined snapshot creation flow to include estimation and initial policy definition.

Provider improvements:

  • S3 client - upgraded minio to v7
  • rclone - testing improvements
  • S3, GCS, Azure, B2 - rewrote retry logic to retry on all but known fatal errors.
  • S3 - support for buckets configured with object locking

Infrastructure:

  • Build for Apple Silicon (M1) is now available. The CLI is now dual-architecture and runs natively on both amd64 and arm64.
  • Switched CI/CD infrastructure to GitHub actions with two self-hosted runners for ARMHF and ARM64
  • Makefile: various tricks to make PR builds faster
  • S3: run tests against private minio deployment in docker (#894)
  • SFTP: fixed retry logic
  • RClone: improved error handling logic

Compatibility Information

  • v0.8 release is compatible with repositories created with v0.7, v0.6 and v0.5.
  • While older clients can read repositories created with v0.8 it’s recommended to use the latest version at earliest convenience.

Breaking Changes

  • When starting kopia server with --htpasswd-file, the UI access is now only granted to a user named kopia (if it exists in the password file). If such user does not exist in the password file, --server-username flag may be specified to indicate which of the users listed in the password file gets UI access (there can be only one). See https://github.com/kopia/kopia/issues/880 for more information.

  • Added explicit --insecure flag to kopia server start to prevent accidentally starting non-secured servers for development. Starting the server without TLS or without authentication now requires this flag.

  • Several low-level maintenance tools have been changed to support individual parameters with --safety=full|none flag instead of specific parameters which were unsafe.