Kubernetes应用系统API用户认证与鉴权 -- 认证篇

SuKai February 13, 2022

前面文章介绍了Kubernetes应用系统用户管理,实现了外部用户存储(LDAP)与Kubernetes用户映射,下面以Kubesphere为参考,一起看一下如何实现Kubernetes应用系统用户的认证。

应用平台用户认证主要实现两个功能:

1,用户身份鉴别

用户身份认证可以通过LDAP进行用户名和密码认证,也可以通过第三方认证服务进行OAuth认证。

2,Token管理

Client端认证通过后,带上获取Token请求API。应用系统根据Token提取用户ID,进行请求权限鉴别。同时应用系统负责Token的生命周期管理。

基本概念:

OpenID Connect(OIDC)是基于OAuth 2.0身份认证协议,增加了OAuth 2.0中末定义的规范,例如scope, Claim用户信息字段。

代码实现

基本认证方式

API路由

// legacy auth API
legacy := &restful.WebService{}
legacy.Path("/aiapis/iam.aiscope/v1alpha2/login").
   Consumes(restful.MIME_JSON).
   Produces(restful.MIME_JSON)
legacy.Route(legacy.POST("").
   To(handler.login).
   Deprecate().
   Doc("Aiscope APIs support token-based authentication via the Authtoken request header. The POST Login API is used to retrieve the authentication token. After the authentication token is obtained, it must be inserted into the Authtoken header for all requests.").
   Reads(LoginRequest{}).
   Returns(http.StatusOK, api.StatusOK, oauth.Token{}).
   Metadata(restfulspec.KeyOpenAPITags, []string{constants.AuthenticationTag}))
container.Add(legacy)

login handler

type LoginRequest struct {
	Username string `json:"username" description:"username"`
	Password string `json:"password" description:"password"`
}

func (h *handler) login(request *restful.Request, response *restful.Response) {
   var loginRequest LoginRequest
   err := request.ReadEntity(&loginRequest)
   if err != nil {
      api.HandleBadRequest(response, request, err)
      return
   }
   h.passwordGrant(loginRequest.Username, loginRequest.Password, request, response)
}

认证和分发Token

func (h *handler) passwordGrant(username string, password string, req *restful.Request, response *restful.Response) {
   authenticated, provider, err := h.passwordAuthenticator.Authenticate(req.Request.Context(), username, password)


   result, err := h.issueTokenTo(authenticated)


   response.WriteEntity(result)
}

OIDC认证方式

这里我们使用Dex为提供OIDC服务,Dex使用LDAP为connector,将我们应用系统注册为Dex的Client。

整个过程为:

1,部署LDAP, Dex服务

2,前端Dex进行用户登录

3,Dex认证通过后重定向到应用系统callback API并提供授权码code

4,交换Token,应用系统通过code请求Dex,获取token

5,应用系统通过Token提取用户信息Claims

6,应用系统发布token给前端Client

Dex配置信息

issuer: http://dex.aiscope.io:32000
storage:
  type: kubernetes
  config:
    inCluster: true
web:
  http: 0.0.0.0:5556
  # tlsCert: /etc/dex/tls/tls.crt
  # tlsKey: /etc/dex/tls/tls.key
connectors:
- type: ldap
  id: ldap
  name: "LDAP"
  config:
    host: openldap.ldap
    startTLS: false
    insecureNoSSL: true
    insecureSkipVerify: true
    bindDN: cn=admin,dc=aiscope,dc=io
    bindPW: adminpassword

    usernamePrompt: Email Address

    userSearch:
      baseDN: ou=Users,dc=aiscope,dc=io
      filter: "(objectClass=person)"
      username: uid
      # "DN" (case sensitive) is a special attribute name. It indicates that
      # this value should be taken from the entity's DN not an attribute on
      # the entity.
      idAttr: DN
      emailAttr: mail
      nameAttr: cn

    groupSearch:
      baseDN: ou=Groups,dc=aiscope,dc=io
      filter: "(objectClass=groupOfNames)"

    userMatchers:
      # A user is a member of a group when their DN matches
      # the value of a "member" attribute on the group entity.
      - userAttr: DN
        groupAttr: member

    # The group name should be the "cn" value.
    nameAttr: cn
oauth2:
  skipApprovalScreen: true

staticClients:
- id: aiscope
  redirectURIs:
  - 'http://api.aiscope.io:9090/oauth/callback/dex'
  name: 'aiscope'
  secret: ZXhhbXBsZS1hcHAtc2VjcmV0

应用系统配置

ldap:
  host: "ldap.aiscope.io:31675"
  managerDN: "cn=admin,dc=aiscope,dc=io"
  managerPassword: "adminpassword"
  userSearchBase: "ou=Users,dc=aiscope,dc=io"
  groupSearchBase: "ou=Groups,dc=aiscope,dc=io"
  initialCap: 10
  maxCap: 100
  poolName: "ldap"
redis:
  host: "redis.aiscope.io"
  port: 6379
  password: "password123"
  db: 1
authentication:
  jwtSecret: "aiscopeSys"
  oauthOptions:
    identityProviders:
      - name: dex
        type: OIDCIdentityProvider
        mappingMethod: auto
        disableLoginConfirmation: true
        provider:
          issuer: "http://dex.aiscope.io:32000"
          clientID: "aiscope"
          clientSecret: "ZXhhbXBsZS1hcHAtc2VjcmV0"
          redirectURL: "http://api.aiscope.io:9090/oauth/callback/dex"
          insecureSkipVerify: true
          getUserInfo: true
          endpoint:
            authURL: "http://dex.aiscope.io:32000/authorize"
            tokenURL: "http://dex.aiscope.io:32000/token"
            userInfoURL: "http://dex.aiscope.io:32000/userinfo"
            jwksURL: "http://dex.aiscope.io:32000/keys"
            endSessionURL: "http://dex.aiscope.io:32000/endsession"
      - name: ldap
        type: LDAPIdentityProvider
        mappingMethod: auto
        disableLoginConfirmation: true
        provider:
          host: "ldap.aiscope.io:31675"
          managerDN: "cn=admin,dc=aiscope,dc=io"
          managerPassword: "adminpassword"
          userSearchBase: "ou=Users,dc=aiscope,dc=io"
          groupSearchBase: "ou=Groups,dc=aiscope,dc=io"
          startTLS: false
          insecureSkipVerify: true
    clients:
      - name: aiscope
        secret: aiscope

API路由

ws.Route(ws.GET("/callback/{callback}").
   Doc("OAuth callback API, the path param callback is config by identity provider").
   Param(ws.QueryParameter("access_token", "The access token issued by the authorization server.").
      Required(true)).
   Param(ws.QueryParameter("token_type", "The type of the token issued as described in [RFC6479] Section 7.1. "+
      "Value is case insensitive.").Required(true)).
   Param(ws.QueryParameter("expires_in", "The lifetime in seconds of the access token.  For "+
      "example, the value \"3600\" denotes that the access token will "+
      "expire in one hour from the time the response was generated."+
      "If omitted, the authorization server SHOULD provide the "+
      "expiration time via other means or document the default value.")).
   Param(ws.QueryParameter("scope", "if identical to the scope requested by the client;"+
      "otherwise, REQUIRED.  The scope of the access token as described by [RFC6479] Section 3.3.").Required(false)).
   Param(ws.QueryParameter("state", "if the \"state\" parameter was present in the client authorization request."+
      "The exact value received from the client.").Required(true)).
   To(handler.oauthCallback).
   Returns(http.StatusOK, api.StatusOK, oauth.Token{}).
   Metadata(restfulspec.KeyOpenAPITags, []string{constants.AuthenticationTag}))

oauthCallback handler

认证和分发token

func (h *handler) oauthCallback(req *restful.Request, response *restful.Response) {
   provider := req.PathParameter("callback")
   authenticated, provider, err := h.oauthAuthenticator.Authenticate(req.Request.Context(), provider, req.Request)


   result, err := h.issueTokenTo(authenticated)


   response.WriteEntity(result)
}

下面具体看一下两种不同认证方式的逻辑

首先定义一个Identity接口,两种不同认证方式认证都返回了这个接口

type Identity interface {
   // GetUserID required
   // Identifier for the End-User at the Issuer.
   GetUserID() string
   // GetUsername optional
   // The username which the End-User wishes to be referred to aiscope.
   GetUsername() string
   // GetEmail optional
   GetEmail() string
}

基本认证

通过用户名和密码查询LDAP,首先查询用户是否存在,如果存在,Bind用户名和密码进行bind操作,bind操作一般用于LDAP认证。

func (l ldapProvider) Authenticate(username string, password string) (identityprovider.Identity, error) {
   conn, err := l.newConn()
   if err != nil {
      klog.Error(err)
      return nil, err
   }

   conn.SetTimeout(time.Duration(l.ReadTimeout) * time.Millisecond)
   defer conn.Close()

   if err = conn.Bind(l.ManagerDN, l.ManagerPassword); err != nil {
      klog.Error(err)
      return nil, err
   }

   filter := fmt.Sprintf("(%s=%s)", l.LoginAttribute, ldap.EscapeFilter(username))
   if l.UserSearchFilter != "" {
      filter = fmt.Sprintf("(&%s%s)", filter, l.UserSearchFilter)
   }
   result, err := conn.Search(&ldap.SearchRequest{
      BaseDN:       l.UserSearchBase,
      Scope:        ldap.ScopeWholeSubtree,
      DerefAliases: ldap.NeverDerefAliases,
      SizeLimit:    1,
      TimeLimit:    0,
      TypesOnly:    false,
      Filter:       filter,
      Attributes:   []string{l.LoginAttribute, l.MailAttribute},
   })
   if err != nil {
      klog.Error(err)
      return nil, err
   }

   if len(result.Entries) == 0 {
      return nil, errors.NewUnauthorized(fmt.Sprintf("ldap: no results returned for filter: %v", filter))
   }

   if len(result.Entries) > 1 {
      return nil, errors.NewUnauthorized(fmt.Sprintf("ldap: filter returned multiple results: %v", filter))
   }

   // len(result.Entries) == 1
   entry := result.Entries[0]
   if err = conn.Bind(entry.DN, password); err != nil {
      if ldap.IsErrorWithCode(err, ldap.LDAPResultInvalidCredentials) {
         klog.V(4).Infof("ldap: %v", err)
         return nil, errors.NewUnauthorized("ldap: incorrect password")
      }
      klog.Error(err)
      return nil, err
   }
   email := entry.GetAttributeValue(l.MailAttribute)
   uid := entry.GetAttributeValue(l.LoginAttribute)
   return &ldapIdentity{
      Username: uid,
      Email:    email,
   }, nil
}

OIDC认证方式

这里主要是完成:

1,code交换成token

2,处理token中Claims用户信息,如果GetUserInfo为True,请求Dex获取用户信息

func (o *oidcProvider) IdentityExchangeCallback(req *http.Request) (identityprovider.Identity, error) {
   //OAuth2 callback, see also https://tools.ietf.org/html/rfc6749#section-4.1.2
   code := req.URL.Query().Get("code")
   ctx := req.Context()
   if o.InsecureSkipVerify {
      client := &http.Client{
         Transport: &http.Transport{
            TLSClientConfig: &tls.Config{
               InsecureSkipVerify: true,
            },
         },
      }
      ctx = context.WithValue(ctx, oauth2.HTTPClient, client)
   }
   token, err := o.OAuth2Config.Exchange(ctx, code)
   if err != nil {
      return nil, fmt.Errorf("oidc: failed to get token: %v", err)
   }
   rawIDToken, ok := token.Extra("id_token").(string)
   if !ok {
      return nil, errors.New("no id_token in token response")
   }
   var claims jwt.MapClaims
   if o.Verifier != nil {
      idToken, err := o.Verifier.Verify(ctx, rawIDToken)
      if err != nil {
         return nil, fmt.Errorf("failed to verify id token: %v", err)
      }
      if err := idToken.Claims(&claims); err != nil {
         return nil, fmt.Errorf("failed to decode id token claims: %v", err)
      }
   } else {
      _, _, err := new(jwt.Parser).ParseUnverified(rawIDToken, &claims)
      if err != nil {
         return nil, fmt.Errorf("failed to decode id token claims: %v", err)
      }
      if err := claims.Valid(); err != nil {
         return nil, fmt.Errorf("failed to verify id token: %v", err)
      }
   }
   if o.GetUserInfo {
      if o.Provider != nil {
         userInfo, err := o.Provider.UserInfo(ctx, oauth2.StaticTokenSource(token))
         if err != nil {
            return nil, fmt.Errorf("failed to fetch userinfo: %v", err)
         }
         if err := userInfo.Claims(&claims); err != nil {
            return nil, fmt.Errorf("failed to decode userinfo claims: %v", err)
         }
      } else {
         resp, err := oauth2.NewClient(ctx, oauth2.StaticTokenSource(token)).Get(o.Endpoint.UserInfoURL)
         if err != nil {
            return nil, fmt.Errorf("failed to fetch userinfo: %v", err)
         }
         data, err := ioutil.ReadAll(resp.Body)
         if err != nil {
            return nil, fmt.Errorf("failed to fetch userinfo: %v", err)
         }
         _ = resp.Body.Close()
         if err := json.Unmarshal(data, &claims); err != nil {
            return nil, fmt.Errorf("failed to decode userinfo claims: %v", err)
         }
      }
   }

   subject, ok := claims["sub"].(string)
   if !ok {
      return nil, errors.New("missing required claim \"sub\"")
   }

   var email string
   emailKey := "email"
   if o.EmailKey != "" {
      emailKey = o.EmailKey
   }
   email, _ = claims[emailKey].(string)

   var preferredUsername string
   preferredUsernameKey := "preferred_username"
   if o.PreferredUsernameKey != "" {
      preferredUsernameKey = o.PreferredUsernameKey
   }

   preferredUsername, _ = claims[preferredUsernameKey].(string)
   if preferredUsername == "" {
      preferredUsername, _ = claims["name"].(string)
   }

   return &oidcIdentity{
      Sub:               subject,
      PreferredUsername: preferredUsername,
      Email:             email,
   }, nil
}

下面具体看一下Token操作

上面看到两种认证方式中,最后都会调用issueTokenTo来分发Token,issueTokenTo会生成accessToken, refreshToken, tokenType, expiresIN.

func (h *handler) issueTokenTo(user user.Info) (*oauth.Token, error) {
   accessToken, err := h.tokenOperator.IssueTo(&token.IssueRequest{
      User:      user,
      Claims:    token.Claims{TokenType: token.AccessToken},
      ExpiresIn: h.options.OAuthOptions.AccessTokenMaxAge,
   })
   if err != nil {
      return nil, err
   }
   refreshToken, err := h.tokenOperator.IssueTo(&token.IssueRequest{
      User:      user,
      Claims:    token.Claims{TokenType: token.RefreshToken},
      ExpiresIn: h.options.OAuthOptions.AccessTokenMaxAge + h.options.OAuthOptions.AccessTokenInactivityTimeout,
   })
   if err != nil {
      return nil, err
   }

   result := oauth.Token{
      AccessToken: accessToken,
      // The OAuth 2.0 token_type response parameter value MUST be Bearer,
      // as specified in OAuth 2.0 Bearer Token Usage [RFC6750]
      TokenType:    "Bearer",
      RefreshToken: refreshToken,
      ExpiresIn:    int(h.options.OAuthOptions.AccessTokenMaxAge.Seconds()),
   }
   return &result, nil
}

调用t.issuer.IssueTo生成Token,缓存token到Redis

func (t *tokenOperator) IssueTo(request *token.IssueRequest) (string, error) {
   tokenStr, err := t.issuer.IssueTo(request)
   if err != nil {
      klog.Error(err)
      return "", err
   }
   if request.ExpiresIn > 0 {
      if err = t.cacheToken(request.User.GetName(), tokenStr, request.ExpiresIn); err != nil {
         klog.Error(err)
         return "", err
      }
   }
   return tokenStr, nil
}

缓存Token

// cacheToken cache the token for a period of time
func (t *tokenOperator) cacheToken(username, token string, duration time.Duration) error {
   key := fmt.Sprintf("aiscope:user:%s:token:%s", username, token)
   if err := t.cache.Set(key, token, duration); err != nil {
      klog.Error(err)
      return err
   }
   return nil
}

IssueTo调用jwt.NewWithClaims根据用户信息生成Token

func (s *issuer) IssueTo(request *IssueRequest) (string, error) {
   issueAt := time.Now().Unix()
   claims := Claims{
      Username:  request.User.GetName(),
      Extra:     request.User.GetExtra(),
      TokenType: request.TokenType,
      StandardClaims: jwt.StandardClaims{
         IssuedAt: issueAt,
         Subject:  request.User.GetName(),
         Issuer:   s.name,
      },
   }

   if len(request.Audience) > 0 {
      claims.Audience = request.Audience
   }
   if request.Name != "" {
      claims.Name = request.Name
   }
   if request.Nonce != "" {
      claims.Nonce = request.Nonce
   }
   if request.Email != "" {
      claims.Email = request.Email
   }
   if request.PreferredUsername != "" {
      claims.PreferredUsername = request.PreferredUsername
   }
   if request.Locale != "" {
      claims.Locale = request.Locale
   }
   if len(request.Scopes) > 0 {
      claims.Scopes = request.Scopes
   }
   if request.ExpiresIn > 0 {
      claims.ExpiresAt = claims.IssuedAt + int64(request.ExpiresIn.Seconds())
   }

   var token string
   var err error
   if request.TokenType == IDToken {
      t := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
      t.Header[headerKeyID] = s.signKey.SigningKey.KeyID
      token, err = t.SignedString(s.signKey.SigningKey.Key)
   } else {
      token, err = jwt.NewWithClaims(jwt.SigningMethodHS256, claims).SignedString(s.secret)
   }
   if err != nil {
      klog.Warningf("jwt: failed to issue token: %v", err)
      return "", err
   }
   return token, nil
}

到这里,客户端完成了用户认证和Token获取的过程,那么当客户端带上Token头来请求API时,应用系统如何通过Token来知道请求的用户是谁呢?

Token中获取用户标识

前面文章有介绍在go-restful框架中构建一个handler链来路由不同的API:应用系统自身的API和Kubernetes API。我们可以利用这个handlerChain增加一个handler,将用户ID从Token中解析出来并保存到context中。

根据Token来解析用户是:匿名用户或真实用户

union.New returns a request authenticator that validates credentials using a chain of authenticator.Request objects. The entire chain is tried until one succeeds. If all fail, an aggregate error is returned.

func (s *APIServer) buildHandlerChain(stopCh <-chan struct{}) {
   requestInfoResolver := &request.RequestInfoFactory{
      APIPrefixes:          sets.NewString("api", "apis"),
   }

   handler := s.Server.Handler
   handler = filters.WithKubeAPIServer(handler, s.KubernetesClient.Config(), &errorResponder{})

   userLister := s.InformerFactory.AIScopeSharedInformerFactory().Iam().V1alpha2().Users().Lister()
   loginRecorder := auth.NewLoginRecorder(s.KubernetesClient.AIScope(), userLister)

   authn := unionauth.New(anonymous.NewAuthenticator(),
      basictoken.New(basic.NewBasicAuthenticator(auth.NewPasswordAuthenticator(
         s.KubernetesClient.AIScope(),
         userLister,
         s.Config.AuthenticationOptions),
         loginRecorder)),
      bearertoken.New(jwt.NewTokenAuthenticator(
         auth.NewTokenOperator(s.CacheClient, s.Issuer, s.Config.AuthenticationOptions),
         userLister)))
   handler = filters.WithAuthentication(handler, authn)

   handler = filters.WithRequestInfo(handler, requestInfoResolver)

   s.Server.Handler = handler
}

处理request中的Token,将user信息保存到http request context中。

func WithAuthentication(handler http.Handler, authRequest authenticator.Request) http.Handler {


	return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
		resp, ok, err := authRequest.AuthenticateRequest(req)

		req = req.WithContext(request.WithUser(req.Context(), resp.User))
		handler.ServeHTTP(w, req)
	})
}

验证token,token中获得用户名,查询用户信息

func (t *tokenAuthenticator) AuthenticateToken(ctx context.Context, token string) (*authenticator.Response, bool, error) {
   verified, err := t.tokenOperator.Verify(token)
   if err != nil {
      klog.Warning(err)
      return nil, false, err
   }

   if verified.User.GetName() == iamv1alpha2.PreRegistrationUser {
      return &authenticator.Response{
         User: verified.User,
      }, true, nil
   }

   u, err := t.userLister.Get(verified.User.GetName())
   if err != nil {
      return nil, false, err
   }

   return &authenticator.Response{
      User: &user.DefaultInfo{
         Name:   u.GetName(),
         Groups: append(u.Spec.Groups, user.AllAuthenticated),
      },
   }, true, nil
}

至此,应用系统完成了用户认证,Token分发,用户信息获取的过程。这样我们可以根据每个API请求的用户信息来进行操作鉴权和审计日志记录。

示例:

创建用户

image-20220207150735652

sukai@SuKai:~$ kubectl -n ldap exec -it openldap-67f848d4d9-m2rfg -- /bin/bash
root@openldap-67f848d4d9-m2rfg:/#
root@openldap-67f848d4d9-m2rfg:/#
root@openldap-67f848d4d9-m2rfg:/# ldapsearch -LLL -x -H ldap://localhost:389 -D "cn=admin,dc=aiscope,dc=io" -w adminpassword -b "ou=Users,dc=aiscope,dc=io" dn
dn: ou=Users,dc=aiscope,dc=io

root@openldap-67f848d4d9-m2rfg:/# ldapsearch -LLL -x -H ldap://localhost:389 -D "cn=admin,dc=aiscope,dc=io" -w adminpassword -b "ou=Users,dc=aiscope,dc=io" dn
dn: ou=Users,dc=aiscope,dc=io

dn: uid=sukai,ou=Users,dc=aiscope,dc=io

root@openldap-67f848d4d9-m2rfg:/#

Dex认证

http://dex.aiscope.io:32000/auth?client_id=aiscope&redirect_uri=http://api.aiscope.io:9090/oauth/callback/dex&response_type=code&scope=openid+email+groups+profile+offline_access

image-20220207150853115

image-20220207150822848

image-20220207151027499

用户名和密码认证

image-20220207151135093