Announcing etcd v3.7.0

This article is a mirror of the original announcement Today, SIG etcd is releasing etcd v3.7.0, the latest minor release of the popular distributed key-value store and core Kubernetes component. v3.7 ships the long-requested RangeStream feature, delivers several other performance improvements, removes the last remnants of the legacy v2store, and completes a major protobuf overhaul. You can download etcd v3.7.0 here: Source code Binaries Official container images This release also includes new versions of the two core etcd dependencies, bbolt v1.5.0 and raft v3.7.0. For instructions on installing etcd, see the install documentation. For the full list of changes, see the etcd v3.7 changelog. A heartfelt thank you to all the contributors who made this release possible! Major features The most significant changes in v3.7.0 include: RangeStream — stream large result sets in chunks instead of buffering the whole response. Keys-only range requests, faster and more reliable leases, and several other performance improvements. etcd now boots entirely from v3store, eliminating a long-standing dependency on the legacy v2 store A completed protobuf overhaul, replacing outdated protobuf libraries with fully supported ones. etcd v3.7 ships with bbolt v1.5.1 and raft v3.7.0. Features RangeStream In etcd v3.6 and earlier, it is challenging to work with requests that return large result sets. The database would buffer the full result set before sending, leading to unpredictable latency and memory usage, both on the server and the client. The RangeStream RPC lets calling applications accept result sets in chunks, reducing latency and making buffering memory usage more predictable. Instructions on how to use RangeStream in gRPC calls and in etcdctl can be found in the etcd documentation. Users should try it out for their own applications. In coordinated releases, the RangeStream feature will become available to users running the upcoming v1.37 of Kubernetes by enabling the EtcdRangeStr

This article is a mirror of the original announcement Today, SIG etcd is releasing etcd v3.7.0, the latest minor release of the popular distributed key-value store and core Kubernetes component. v3.7 ships the long-requested RangeStream feature, delivers several other performance improvements, removes the last remnants of the legacy v2store, and completes a major protobuf overhaul. You can download etcd v3.7.0 here: Source code Binaries Official container images This release also includes new versions of the two core etcd dependencies, bbolt v1.5.0 and raft v3.7.0. For instructions on installing etcd, see the install documentation. For the full list of changes, see the etcd v3.7 changelog. A heartfelt thank you to all the contributors who made this release possible! Major features The most significant changes in v3.7.0 include: RangeStream — stream large result sets in chunks instead of buffering the whole response. Keys-only range requests, faster and more reliable leases, and several other performance improvements. etcd now boots entirely from v3store, eliminating a long-standing dependency on the legacy v2 store A completed protobuf overhaul, replacing outdated protobuf libraries with fully supported ones. etcd v3.7 ships with bbolt v1.5.1 and raft v3.7.0. Features RangeStream In etcd v3.6 and earlier, it is challenging to work with requests that return large result sets. The database would buffer the full result set before sending, leading to unpredictable latency and memory usage, both on the server and the client. The RangeStream RPC lets calling applications accept result sets in chunks, reducing latency and making buffering memory usage more predictable. Instructions on how to use RangeStream in gRPC calls and in etcdctl can be found in the etcd documentation. Users should try it out for their own applications. In coordinated releases, the RangeStream feature will become available to users running the upcoming v1.37 of Kubernetes by enabling the EtcdRangeStream feature gate. This early and planned adoption is possible thanks to the merger of etcd and Kubernetes development in 2023. Performance improvements v3.7 delivers multiple specific performance improvements, both for the Kubernetes control plane and for other use cases. Kubernetes users should see a significant decrease in overall CPU usage by the etcd members, compared with v3.6. Keys-only range optimization etcd v3.7.0 includes a keys-only Range optimization (#21791: keys-only Range optimization). When processing a keys_only Range request or etcdctl get --keys-only, etcd reads solely from its in-memory index. It returns the matched keys without loading all serialized values from bbolt as it did previously. The only exception where loading from bbolt is still required is when keys_only Range requests must be sorted by value (i.e., when SortTarget is set to VALUE). This reduces unnecessary backend reads and memory use for workloads that only need key names, making large keys-only range requests more efficient. Faster, more reliable etcd leases v3.7 improves lease expiration and renewal: LeaseRevoke requests are now prioritized to ensure timely lease expiration during overload (#20492: stability enhancement during overload conditions). The new FastLeaseKeepAlive feature enables faster lease renewal by skipping the wait for the applied index (#20589: etcdserver: improve linearizable renew lease). Faster find() operations etcd 3.7 improves the performance of concurrent watches on keys by making find() operations faster (#19768: adt: split interval tree by right endpoint on matched left endpoints). Other features Protobuf overhaul v3.7 migrates and replaces multiple outdated protobuf libraries with fully supported dependencies. This includes replacing github.com/golang/protobuf and github.com/gogo/protobuf with the fully-supported google.golang.org/protobuf (#14533: Protobuf: cleanup both golang/protobuf and gogo/protobuf), and migrating grpc-logging to grpc-middleware v2 (#20420: Migrate grpc-logging to grpc-middleware v2). As well as improving security and maintainability, this refactor has been shown to reduce CPU usage by etcd components. While these changes are not expected to directly affect users running etcd via official binaries or container images, they may affect users who depend on etcd Go modules, such as the client SDK or packages under api/ or pkg/. These consumers may need to update their code or dependencies due to protobuf and related API changes introduced in this release. More detailed information is available from the API change tracking issue. Unix socket support etcd now supports Unix socket endpoints (#19760: Add Support for Unix Socket endpoints), enabling local communication without a TCP port. Since this is restricted to single-member clusters, it is mainly aimed at development, testing, and edge device use-cases. Bootstrap from v3store One of the major changes in etcd v3.7 is that the server now bootstraps entirely from the v3 store (#20187 Bootstrap etcdserver from v3store), eliminating its dependency on the legacy v2 store during startup. This milestone is the result of a long-term effort spanning multiple releases, from v3.4 through v3.7. It resolves a long-standing technical debt, significantly simplifies the bootstrap workflow, and lays the foundation for future improvements to etcd. To maintain backward compatibility, etcd v3.7 continues to generate v2 snapshots. As a result, the --snapshot-count flag is also retained in v3.7. This is the last remaining dependency on the legacy v2 store, and both the v2 snapshot generation and the --snapshot-count flag will be removed in v3.8. etcdutl timeouts All etcdutl commands now have a timeout command line argument (#20708: etcdutl: enable timeout functionality for all commands), so offline utility commands no longer block indefinitely when holding a lock. Setting the authentication token directly Client v3 now allows users to set the JWT directly, offerin

This article is a mirror of the original announcement Today, SIG etcd is releasing etcd v3.7.0, the latest minor release of the popular distributed key-value store and core Kubernetes component. v3.7 ships the long-requested RangeStream feature, delivers several other performance improvements, removes the last remnants of the legacy v2store, and completes a major protobuf overhaul. You can download etcd v3.7.0 here: Source code Binaries Official container images This release also includes new versions of the two core etcd dependencies, bbolt v1.5.0 and raft v3.7.0. For instructions on installing etcd, see the install documentation. For the full list of changes, see the etcd v3.7 changelog. A heartfelt thank you to all the contributors who made this release possible! Major features The most significant changes in v3.7.0 include: RangeStream — stream large result sets in chunks instead of buffering the whole response. Keys-only range requests, faster and more reliable leases, and several other performance improvements. etcd now boots entirely from v3store, eliminating a long-standing dependency on the legacy v2 store A completed protobuf overhaul, replacing outdated protobuf libraries with fully supported ones. etcd v3.7 ships with bbolt v1.5.1 and raft v3.7.0. Features RangeStream In etcd v3.6 and earlier, it is challenging to work with requests that return large result sets. The database would buffer the full result set before sending, leading to unpredictable latency and memory usage, both on the server and the client. The RangeStream RPC lets calling applications accept result sets in chunks, reducing latency and making buffering memory usage more predictable. Instructions on how to use RangeStream in gRPC calls and in etcdctl can be found in the etcd documentation. Users should try it out for their own applications. In coordinated releases, the RangeStream feature will become available to users running the upcoming v1.37 of Kubernetes by enabling the EtcdRangeStream feature gate. This early and planned adoption is possible thanks to the merger of etcd and Kubernetes development in 2023. Performance improvements v3.7 delivers multiple specific performance improvements, both for the Kubernetes control plane and for other use cases. Kubernetes users should see a significant decrease in overall CPU usage by the etcd members, compared with v3.6. Keys-only range optimization etcd v3.7.0 includes a keys-only Range optimization (#21791: keys-only Range optimization). When processing a keys_only Range request or etcdctl get --keys-only, etcd reads solely from its in-memory index. It returns the matched keys without loading all serialized values from bbolt as it did previously. The only exception where loading from bbolt is still required is when keys_only Range requests must be sorted by value (i.e., when SortTarget is set to VALUE). This reduces unnecessary backend reads and memory use for workloads that only need key names, making large keys-only range requests more efficient. Faster, more reliable etcd leases v3.7 improves lease expiration and renewal: LeaseRevoke requests are now prioritized to ensure timely lease expiration during overload (#20492: stability enhancement during overload conditions). The new FastLeaseKeepAlive feature enables faster lease renewal by skipping the wait for the applied index (#20589: etcdserver: improve linearizable renew lease). Faster find() operations etcd 3.7 improves the performance of concurrent watches on keys by making find() operations faster (#19768: adt: split interval tree by right endpoint on matched left endpoints). Other features Protobuf overhaul v3.7 migrates and replaces multiple outdated protobuf libraries with fully supported dependencies. This includes replacing github.com/golang/protobuf and github.com/gogo/protobuf with the fully-supported google.golang.org/protobuf (#14533: Protobuf: cleanup both golang/protobuf and gogo/protobuf), and migrating grpc-logging to grpc-middleware v2 (#20420: Migrate grpc-logging to grpc-middleware v2). As well as improving security and maintainability, this refactor has been shown to reduce CPU usage by etcd components. While these changes are not expected to directly affect users running etcd via official binaries or container images, they may affect users who depend on etcd Go modules, such as the client SDK or packages under api/ or pkg/. These consumers may need to update their code or dependencies due to protobuf and related API changes introduced in this release. More detailed information is available from the API change tracking issue. Unix socket support etcd now supports Unix socket endpoints (#19760: Add Support for Unix Socket endpoints), enabling local communication without a TCP port. Since this is restricted to single-member clusters, it is mainly aimed at development, testing, and edge device use-cases. Bootstrap from v3store One of the major changes in etcd v3.7 is that the server now bootstraps entirely from the v3 store (#20187 Bootstrap etcdserver from v3store), eliminating its dependency on the legacy v2 store during startup. This milestone is the result of a long-term effort spanning multiple releases, from v3.4 through v3.7. It resolves a long-standing technical debt, significantly simplifies the bootstrap workflow, and lays the foundation for future improvements to etcd. To maintain backward compatibility, etcd v3.7 continues to generate v2 snapshots. As a result, the --snapshot-count flag is also retained in v3.7. This is the last remaining dependency on the legacy v2 store, and both the v2 snapshot generation and the --snapshot-count flag will be removed in v3.8. etcdutl timeouts All etcdutl commands now have a timeout command line argument (#20708: etcdutl: enable timeout functionality for all commands), so offline utility commands no longer block indefinitely when holding a lock. Setting the authentication token directly Client v3 now allows users to set the JWT directly, offerin