Teleport: tsh login - rpc error in Teleport v5.0.0

Created on 4 Dec 2020  ·  10Comments  ·  Source: gravitational/teleport

Description

What happened:
When running a tsh login --proxy=<my_proxy> on the cli I am getting the following output:

>tsh login --proxy=<my_proxy>
If browser window does not open automatically, open it by clicking on the link:
 http://127.0.0.1:53817/<GUID>
error: rpc error: code = Unimplemented desc = unknown method GetKubeServices for service proto.AuthService

What you expected to happen:
I expected that this would successfully login to teleport

How to reproduce it (as minimally and precisely as possible):
on MacOs -

>brew install teleport
>tsh version
Teleport v5.0.0 git: go1.15.5
>tsh login --proxy=<my_proxy>

Environment

  • Teleport version (use teleport version): Teleport v5.0.0 git: go1.15.5
  • Tsh version (use tsh version): Teleport v5.0.0 git: go1.15.5
  • OS (e.g. from /etc/os-release): macOS Catalina v 10.15.7

  • Where are you running Teleport? (e.g. AWS, GCP, Dedicated Hardware): Server-side v4

Relevant Debug Logs If Applicable

  • tsh login --proxy=<my_proxy> --debug
INFO [CLIENT]    Successful auth with proxy teleport.lacework.net:3023 client/api.go:1672
<redacted>
DEBU [AUTH]      GRPC(CLIENT): keep alive 1m0s count: 3. auth/clt.go:320
DEBU [CLIENT]    Client  is connecting to auth server on cluster "teleport.lacework.net". client/client.go:473

ERROR REPORT:
Original Error: *status.statusError rpc error: code = Unimplemented desc = unknown method GetKubeServices for service proto.AuthService
Stack Trace:
    /private/tmp/teleport-20201127-89185-jfx2dt/teleport-5.0.0/src/github.com/gravitational/teleport/lib/auth/clt.go:2978 github.com/gravitational/teleport/lib/auth.(*Client).GetKubeServices
    /private/tmp/teleport-20201127-89185-jfx2dt/teleport-5.0.0/src/github.com/gravitational/teleport/lib/kube/utils/utils.go:155 github.com/gravitational/teleport/lib/kube/utils.KubeClusterNames
    /private/tmp/teleport-20201127-89185-jfx2dt/teleport-5.0.0/src/github.com/gravitational/teleport/lib/kube/kubeconfig/kubeconfig.go:101 github.com/gravitational/teleport/lib/kube/kubeconfig.UpdateWithClient
    /private/tmp/teleport-20201127-89185-jfx2dt/teleport-5.0.0/src/github.com/gravitational/teleport/tool/tsh/tsh.go:609 main.onLogin
    /private/tmp/teleport-20201127-89185-jfx2dt/teleport-5.0.0/src/github.com/gravitational/teleport/tool/tsh/tsh.go:431 main.Run
    /private/tmp/teleport-20201127-89185-jfx2dt/teleport-5.0.0/src/github.com/gravitational/teleport/tool/tsh/tsh.go:212 main.main
    /usr/local/opt/go/libexec/src/runtime/proc.go:213 runtime.main
    /usr/local/opt/go/libexec/src/runtime/asm_amd64.s:1375 runtime.goexit
User Message:

Any help would be greatly appreciated, as it appears it's not possible to downgrade versions via homebrew at this time.

bug tsh ux

Most helpful comment

I'm reopening the issue to track this as an improvement needed to tsh.

The workaround in the meantime for anyone else who encounters this problem is to downgrade your version of tsh from 5.x.x to 4.4.x.

All 10 comments

The current workaround that I am using is to brew uninstall teleport and manually install tsh v4.4.5 https://get.gravitational.com/tsh-4.4.5.pkg

It appears the issue is down to the use of auth server v4. As explained in slack Teleport v5 it's backwards compatible, but not forwards compatible ( old clients will work with new servers, but new clients may not work with old servers)

We don't have any control over the version packaged in Homebrew. The safest way to make sure nothing breaks due to unexpected upgrades is to install the correct version of our official signed, notarized tsh PKG from https://goteleport.com/teleport/download

Appreciate the response from you and your team @webvictim. Closing as the resolution is to install the teleport package from the above link.

@rmoles / @webvictim my 2p would be that if newer tsh versions aren't backwards compatible then the error message should be improved to point the user to their mistake (client version is newer than remote). I've seen a bunch of our onboarding operators hit this error because the remote endpoint they're trying to connect to is teleport 4.4 but they've just gone and fetched the most recent tsh package from the official download page.

I don't know if fetching the authVersion is an authenticated or non-authenticated API request, but the ideal solution would be if tsh could check the remote version against its own and warn the user that their version is newer than the remote.

FYI installing the MacOS .pkg installer on MacOs is blocked with the following error: “teleport-4.4.5.pkg” cannot be opened because it is from an unidentified developer.
Steps to install anyway can be found here: https://support.apple.com/en-gb/guide/mac-help/mh40616/mac

@rmoles teleport-4.4.5.pkg and tsh-4.4.5.pkg are actually different things.

The teleport-*.pkg files are not signed, because the way that the Teleport binary is built is not currently compatible with MacOS' signing requirements. We have an open issue to fix this here: #3158

The tsh-*.pkg files are signed with an official developer certificate and notarized by Apple, so they will install without any problems.

@dnwe I agree. We do actually already expose the minimum client version via an unauthenticated API call:

$ curl -s https://teleport.example.com:3080/v1/webapi/ping | jq
{
  "auth": {
    "type": "github",
    "second_factor": "u2f",
    "github": {
      "name": "github",
      "display": "Github"
    }
  },
  "proxy": {
    "kube": {},
    "ssh": {
      "listen_addr": "0.0.0.0:3023",
      "tunnel_listen_addr": "0.0.0.0:3080",
      "public_addr": "teleport.example.com:3080",
      "ssh_public_addr": "teleport.example.com:3023",
      "ssh_tunnel_public_addr": "teleport.example.com:3080"
    }
  },
  "server_version": "5.0.0",
  "min_client_version": "3.0.0"
}

I will look into why this isn't triggering logic to warn tsh users that their version won't work.

I'm reopening the issue to track this as an improvement needed to tsh.

The workaround in the meantime for anyone else who encounters this problem is to downgrade your version of tsh from 5.x.x to 4.4.x.

@dnwe I agree. We do actually already expose the minimum client version via an unauthenticated API call:

...

I will look into why this isn't triggering logic to warn tsh users that their version won't work.

In this case it’s a max_client_version we’d need, to warn users that tsh 5.x can’t be used against teleport 4.x server

Was this page helpful?
0 / 5 - 0 ratings

Related issues

kimlisa picture kimlisa  ·  4Comments

cnelson picture cnelson  ·  5Comments

webvictim picture webvictim  ·  4Comments

webvictim picture webvictim  ·  4Comments

Kellel picture Kellel  ·  3Comments