Kopia v0.9

NOTE: Kopia v0.9 is out. You can download it from:

https://github.com/kopia/kopia/releases/latest.

This release focuses on improvements to the repository format, stability and performance.

The new repository index format enables efficient append-only operation where delete permission is not required, removes the need for quick maintenance and enables internal content-level compression.

Kopia now also supports repository password change (new repositories only).

Big thanks to @pawitp for contributing major performance improvement to splitters #1251, which improves large file upload performance by up to 30%.

Upgrading Repository

Repositories created using previous versions can be upgraded to latest format. See the Upgrade Notes.

Providers

  • azure: added support for using SAS Tokens instead of storage keys #1093
  • rclone: wait for rclone transfers to finish when closing #1201
  • sftp: added automatic reconnect and connection pooling #1195
  • sftp: ensure key file and known hosts are absolute paths #1092
  • s3: point-in-time support for s3 versioned stores #1259
  • webdav & and rclone support for atomic writes #1206
  • webdav: disable server-side compression #1100

Repository features

  • new index format supporting internal compression and does not require quick maintenance for performance reasons
  • splitter: major performance improvements #1251
  • implemented epoch-based index manager #1174
  • internal logging - low-level logs are now stored in the repository.
  • support for content-level compression #1076
  • added ‘lz4’ and ‘deflate’ methods #1077
  • added sharding of indexes on write #1176
  • repository password change support #1197
  • limit the duration of kopia.repository caching to 15 minutes #1196
  • encryption: removed old, non-authenticated encryption methods #979
  • big performance improvement for WriteContent with repo server #1182
  • disabled quick maintenance for new repositories

CLI

  • ‘kopia content verify’ performance improvements #1120
  • add a flag to not shallowrestore small files #1126
  • added ‘kopia repo set-parameters’ to override mutable parameters #1148
  • added ‘kopia repository validate-provider` #1205
  • additional ‘index inspect’ flags #1231
  • changed ‘kopia snapshot verify –verify-files-percent’ to float #1210
  • fixed ETA estimation of ‘snapshot verify’ #1213
  • force-hash: Allow fractional percentages #1209
  • include parameters in maintenance info JSON output #981
  • optimized ‘kopia index recover’ by leveraging partial read and parallelism #1094
  • support for shallow restore #725
  • tagging of kopia snapshots and listing of snapshots by tag #1030
  • logging: switched file log timestamps to be UTC by default, otherwise it’s hard to correlate logs #1240

Policy Changes

  • Fix weekly retention labels not being assigned correctly #1211
  • policy: path validation #1006

UI Changes

  • do not attempt running maintenance if the current user is not the maintenance owner, to avoid producing error in the Tasks tab #1010
  • fixed Estimate not honoring the defined policies #1002
  • for read-only repositories start a read-only source manager #1009
  • open .gitignore help in a new window #1016
  • show a spinner when saving/deleting policy #1018

Security

  • security: switched to github.com/golang-jwt/jwt/v4 to fix upstream security issue #1235

Notable internal Changes

  • automatically refresh indexes on read instead of relying on RefreshPeriodically() loop #1243
  • ensure that we always consistently pick content.Info amongst entries with identical time and deleted flag #1239
  • refactored content.Info to be an interface and switched index parsing to be lazy #1008
  • refactored own writes cache and list cache into blob.Storage wrappers #1133
  • few subtle threading bugs uncovered by stress test and rewrote the test to be model-based #1157
  • reworked memory management to avoid allocating large chunks of memory for most buffers
  • ci: Allow to compile Kopia cli on OpenBSD #983
  • upgrade golang to 1.17
  • testing: Refactored most of the CLI tests to run in-process as opposed to using sub-processes #1059
Last modified September 13, 2021: site: 0.9 release notes (#1291) (d5a5ea88)