Hello everyone, I am fried fish.
In the past few years, I have been exposed to a bunch of microservice-related components very frequently: grpc + grpc-gateway + etcd + protobuf + protoc-gen-go. At first, everything was fine, and we gradually followed up with new versions.
No, the moth is here. The development leaders (or companies) who wrote these components are not together, and they work independently, each with their own ideas, preferences, and specifications… Therefore, there will be mutual incompatibility, and even bottlenecks.
Various compatibility issues
When etcd is v3.3/v3.4 and grpc > v1.27, you often encounter various compatibility issues that hurt your brain.
At least but not limited to the following scenarios. Just to give some examples of three common compatibility errors.
grpc/naming not found
The google.golang.org/grpc/naming package in the grpc-go library cannot be found. what is the reason? One reason is that go.etcd.io/etcd/client references an experimental package in the grpc-go library.
When go mod tidy, you will encounter the following error:
go: finding module for package google.golang.org/grpc/naming go: finding module for package google.golang.org/grpc/examples/helloworld/helloworld go: found google.golang.org/grpc/examples/helloworld/helloworld in google.golang.org/grpc/examples v0.0.0-20231026203026-8cb98464e599 go: finding module for package google.golang.org/grpc/naming go: git.xxx.cn/xxx/xxx-common/jy imports go.etcd.io/etcd/client tested by go.etcd.io/etcd/client.test imports github.com/coreos/etcd/integration imports github.com/coreos/etcd/proxy/grpcproxy imports google.golang.org/grpc/naming: module google.golang.org/grpc@latest found (v1.59.0), but does not contain package google.golang.org/grpc/naming
See the last but does not contain package google.golang.org/grpc/naming. I thought grpc-go deleted the library randomly and made incompatible changes.
I’m thinking of finding an official solution to the problem. I was very impressed. Someone grpc-go said: My library has long been declared experimental and may be deleted at any time. You should not rely on it. (meaning it will not be supported)
Cannot find etcd/clientv3/balancer/picker
Or it’s because the experimental package of the grpc-go library has been removed in the new version. Will cause etcd v3.3 to appear: undefined: balancer.PickOptions and undefined: resolver.BuildOption related error messages:
$ go get go.etcd.io/etcd/clientv3 # github.com/coreos/etcd/clientv3/balancer/resolver/endpoint ../../go/pkg/mod/github.com/coreos/[email protected] + incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:114:78: undefined: resolver.BuildOption ../../go/pkg/mod/github.com/coreos/[email protected] + incompatible/clientv3/balancer/resolver/endpoint/endpoint.go:182:31: undefined: resolver.ResolveNowOption # github.com/coreos/etcd/clientv3/balancer/picker ../../go/pkg/mod/github.com/coreos/[email protected] + incompatible/clientv3/balancer/picker/err.go:37:44: undefined: balancer.PickOptions ../../go/pkg/mod/github.com/coreos/[email protected] + incompatible/clientv3/balancer/picker/roundrobin_balanced.go:55:54: undefined: balancer.PickOptions
I have to say that for this BUG, I also submitted issues and PR to etcd:
Eventually merged. (But etcd v3.5 was released too slowly that year, so I didn’t wait…)
grpc.SupportPackageIsVersion6 not found
protoc-gen-go is incompatible with grpc version. The following error will appear:
Getting error undefined: grpc.SupportPackageIsVersion6 and undefined: grpc.ClientConnInterface
For this problem itself, you only need to upgrade grpc >= 1.27. But if you use etcd sdk, you will have the previous etcd version dependency problem, and the program will be in trouble, unable to be upgraded or downgraded.
Finally, protoc (protoc-gen-go) was downgraded to v1.3.2, and grpc was kept at v1.26, so that etcd v3.3 dependencies can be used normally.
The reason behind it
Careful students will find that it is ultimately related to etcd v3.3, and grpc cannot be upgraded to v1.27 or above. All other related versions of protoc and grpc-gateway cannot continue to advance.
When you want to use go module for various compatibility management, you will find that etcd v3.3 does not have go module at all…
Although etcd v3.4 has go.mod, it cannot be pulled and used (for details, see: etcd-io/etcd/issues/11154[1]).
The official response from etcd is also not very positive. My guess is that it is a long-standing problem that is difficult to resolve.
Solution
The community has been waiting for many years and now finally has a solution. etcd v3.5 has officially supported go module!
etcd divided the previous modules into areas according to functions, and solved various previous problems such as low version dependencies and circular dependencies.
As shown below:
Split into independent Go modules such as api, client, raft, server, etcdctl, bbolt, etc. There will be no cross-influence like the old version.
If you are a new project, it is recommended to use etcd v3.5 or above without any thought. Never use v3.3/v3.4 or lower!
A bit more helpless
Maybe some students thought everyone was happy? Not really.
His modular transformation success is limited to etcd v3.5. As for historical projects, if you are using etcd v2 store, then I’m sorry.
The old versions of etcd (v3.3/v3.4, etc.) have not changed. In the new version of v3.5 (including the latest documentation), support for etcd v2 is gradually being removed.
If you are still using etcd v2, it is recommended to use v3 for data migration. This can avoid many technical problems.
If migration is inconvenient, except for various replace and lock versions. If the etcd function you use is very basic, you can also implement a simple version of the SDK yourself.
Summary
This historical problem of etcd has existed for several years, and has been dealt with slowly. It even affects some technology selection issues in the Go ecosystem.
A few days ago, some students reported that tidb quoted the cloud.google.com/go/pubsub library, and this library relied on a higher version of grpc. As a result, the original application stuck on grpc v1.26 has another problem. Just looked back.
In the new version, the etcd dependency problem is finally solved. Really, this is worth remembering! Although his old version still hasn’t been processed…
Recommended reading
-
Go project standard layout? Go official guide released
-
The omens and means of layoffs in Internet companies
-
Being slapped in the face by the Go team, the accepted proposal can be overturned with just one sentence! ! !
Reference materials
[1]
etcd-io/etcd/issues/11154: https://github.com/etcd-io/etcd/issues/11154
Follow Hejiajianyu on WeChat,
First-hand information and knowledge will draw you into the technical exchange group
Hello, I am Jianyu. I have published the Go best-selling book “Go Language Programming Journey”, and then won the honor of GOP (the most opinionated expert in the Go field). Click on the blue words to view my book publishing path strong>.
Share high-quality articles every day, output Go interviews, work experience, and architectural design, and join WeChat to attract readers to communicate with everyone!
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge. Cloud native entry-level skills treeHomepageOverview 16,903 people are learning the system