Kopia v0.8

NOTE: Kopia v0.8 is out. You can download it from:

https://github.com/kopia/kopia/releases/latest.

This version brings many performance, usability and stability improvements and several new and exciting features.

There were over 150 changes by 19 contributors which represents a big growth in community engagement.

New Features:

• Full Maintenance is now enabled by default with an interval of 24 hours.
• Added support for configurable Maintenance Safety to speed up cleanups in certain cases.
• Added support for --json flag to certain CLI commands which will produce machine-readable snasphot, policy, manifest, blob and content responses.
• Added support for scanning only one filesystem via files policy
• Major improvements to ignore logic to better follow .gitignore rules.
• Added support for Actions which allow running custom scripts before and after a directory is snapshotted with lots of useful applications.
• Added support for ACLs which allow defining custom access control rules for repository server clients.
• Docker Images are now available
• Support for write sessions, which ensure garbage collection does not prematurely delete blobs that are in the middle of being uploaded by intermittently-connected clients (e.g. laptops that go to sleep often and can’t checkpoint frequently enough)
• Added support for stdin snapshots.
• Added support for user authentication using user profiles stored in the repository (#809)
• Brand new streaming GRPC protocol for talking to kopia repository server to improve performance and ensure session stickyness, which allows sharding and load-balancing.
• Reworked error handling when taking snapshots to not fail fast (except when explicitly requested).
• Switched to new FUSE implementation which supports MacFUSE 4.x
• Added WebDAV-based mounting alternative for platforms that don’t have working FUSE mechanism.
• Streamlined flags for caching and improved cache sweep.
• Major performance improvements when performing lots of index lookups in heavily-fragmented indexes (speeds up snapshot verification by orders of magnitude)
• Passthrough options to allow other-users access and mounting in empty directory.
• Don’t run maintenance as part of read-only actions.
• Implemented caching for contents downloaded from Kopia Repository Server
• Reduced latency for kopia repository server uploads by performing existence checks
• Improvements to snapshot verify to make output easier to understand.
• compress: upgrade klauspost/compress to v1.11.12 (#888)
• Terminal password prompt - switched to golang.org/x/term (#898)

New KopiaUI features:

• Added new tab called Tasks which shows the status, progress, logs, and statistics of current and past long-running tasks (restore, estimate, snapshot, maintenance).
• Streamlined snapshot creation flow to include estimation and initial policy definition.

Provider improvements:

• S3 client - upgraded minio to v7
• rclone - testing improvements
• S3, GCS, Azure, B2 - rewrote retry logic to retry on all but known fatal errors.
• S3 - support for buckets configured with object locking

Infrastructure:

• Build for Apple Silicon (M1) is now available. The CLI is now dual-architecture and runs natively on both amd64 and arm64.
• Switched CI/CD infrastructure to GitHub actions with two self-hosted runners for ARMHF and ARM64
• Makefile: various tricks to make PR builds faster
• S3: run tests against private minio deployment in docker (#894)
• SFTP: fixed retry logic
• RClone: improved error handling logic

Compatibility Information

• v0.8 release is compatible with repositories created with v0.7, v0.6 and v0.5.
• While older clients can read repositories created with v0.8 it’s recommended to use the latest version at earliest convenience.

Breaking Changes

• When starting kopia server with --htpasswd-file, the UI access is now only granted to a user named kopia (if it exists in the password file). If such user does not exist in the password file, --server-username flag may be specified to indicate which of the users listed in the password file gets UI access (there can be only one). See https://github.com/kopia/kopia/issues/880 for more information.

• Added explicit --insecure flag to kopia server start to prevent accidentally starting non-secured servers for development. Starting the server without TLS or without authentication now requires this flag.

• Several low-level maintenance tools have been changed to support individual parameters with --safety=full|none flag instead of specific parameters which were unsafe.