Skip to main content

Gitlab Hydra integration

Gitlab has several OAuth2 related features. The relevant here is the possibility to sign into GitLab with (almost) any OAuth2 provider, in this case Ory Hydra. So, in this guide, we'll connect GitLab's omniauth-connector to Ory Hydra. We'll do that in a docker-based lab-environment in order to investigate the details before you do something like this in production.

Preparation​

Even though we're mostly using Ory Hydra in a docker-container, having the command-line-client available is quite useful. So please install Ory Hydra as explained in the installation-guide. You'll also need docker and docker-compose.

The 5-min-tutorial might be worth checking out upfront. It'll a give a nice quick overview how OAuth2 is working within Ory Hydra with a minimal example. We assume basic knowledge, here.

If you haven't yet the source code of Ory Hydra, which we'll need for the docker-compose yaml-files and the gitlab-configuration, clone the repository:

git clone https://github.com/ory/hydra.git

We will access GitLab via the url http://gitlab.example.com. So we need to map it to localhost. This is done by modifying the hosts-file. On an unixoid system find this file in /etc/hosts, on windows, you should find it in c:\WINDOWS\system32\drivers\etc\hosts. Add this line:

127.0.0.1 gitlab.example.com

As this POC will work with http instead of https, we need to whitelist the above domain-name to allow unencrypted http traffic. So add the following switch to the services.hydra.command-section in the quickstart.yml around line 24 so that the line looks like this:

    serve all --dev

Spin up the instances and logging in​

Use this command to spinup the instances. This will show the logs on the terminal and it will take some time.

docker-compose -f quickstart.yml \
    -f quickstart-postgres.yml -f ./contrib/quickstart/quickstart-gitlab.yml \
    up --build

After this succeeds, you can access the login page sign-in-page. Don't try to log in yet. We have to create the client in Ory Hydra first.

Creating the client in Ory Hydra​

Depending on whether you have the hydra-binary available, you can use it directly or the one in the docker-container.

client=$(hydra create client \
    --endpoint http://127.0.0.1:4445 \
    --format json \
    --grant-type authorization_code,refresh_token \
    --response-type code,id_token, email \
    --scope openid,offline_access,profile,email \
    --redirect-uri http://gitlab.example.com:8000/users/auth/Ory_Hydra/callback \
    --token-endpoint-auth-method client_secret_post)

client_id=$(echo $client | jq -r '.client_id')
client_secret=$(echo $client | jq -r '.client_secret')

or you can use the binary within the docker-container:

docker-compose -f quickstart.yml exec hydra \
    hydra create client \
    --endpoint http://127.0.0.1:4445 \
    --id $client_id \
    --secret $client_secret \
    --grant-type authorization_code,refresh_token \
    --response-type code,id_token,email \
    --scope openid,offline_access,profile,email \
    --redirect-uri http://gitlab.example.com:8000/users/auth/Ory_Hydra/callback \
    --token-endpoint-auth-method client_secret_post

OAuth2 login​

With the first access of your GitLab-instance, you will have to change the root-password. You should see a "Ory Hydra" Login-button. Clicking it will forward you to the hydra-consent-app where you can login with foo@bar.com/foobar similar to the 5-min-tutorial. After that you have to give consent to accessing your email-address. Congratulations, doing that should redirect you directly to your personal GitLab-page. You have logged into GitLab via Ory Hydra.

So now, let's look at the individual pieces and how all of them work together.

Docker-setup​

Gitlab has some documentation about how to use their docker-images. It has also an example for docker-compose. The quickstart-gitlab.yaml file in the contrib directory doesn't contain surprising things:

version: "3"

services:
  gitlab:
    image: gitlab/gitlab-ce:13.0.6-ce.0
    restart: always
    hostname: gitlab.example.com
    environment:
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://gitlab.example.com:8000/'
    ports:
      - "8000:8000" # http
    volumes:
      - "./contrib/quickstart/gitlab/config:/etc/gitlab"
      - "./contrib/quickstart/gitlab/logs:/var/log/gitlab"
      - "./contrib/quickstart/gitlab/data:/var/opt/gitlab"

Other then logs and data, the config directory is already prepopulated and the single most important configuration-file is the gitlab.rb file. GitLab has a mechanism to override values and we use it here to specify the external_url.

So let's move on to gitlab.rb.

GitLab configuration - OAuth 2 setup​

The gitLab-configuration in contrib/quickstart/gitlab/config/gitlab.rb is the original "template" which consists of 2400 lines of comments on how to do stuff. Our relevant configuration starts at line 432 where the corresponding comments about OAuth2 is located as well. It looks like this:

gitlab_rails['omniauth_enabled'] = true
gitlab_rails['omniauth_block_auto_created_users'] = false
gitlab_rails['omniauth_allow_single_sign_on'] = ['Ory_Hydra']
gitlab_rails['omniauth_providers'] = [
  {
    'name' => 'oauth2_generic',
    'app_id' => '<THE-CLIENT-ID-GOES-HERE>',
    'app_secret' => '<THE-CLIENT-SECRET-GOES-HERE>',
    'args' => {
      client_options: {
        'site' => 'http://127.0.0.1:4444', # including port if necessary
        'user_info_url' => 'http://hydra:4444/userinfo',
        'authorize_url' => 'http://127.0.0.1:4444/oauth2/auth',
        'token_url' => 'http://hydra:4444/oauth2/token'
      },
      user_response_structure: {
        root_path: [],
        id_path: 'sub',
        attributes: {
                email: 'sub'
        }
      },
      authorize_params: {
        scope: 'email'
      },
      # optionally, you can add the following two lines to "white label" the display name
      # of this strategy (appears in urls and Gitlab login buttons)
      # If you do this, you must also replace oauth2_generic, everywhere it appears above, with the new name.
      name: 'Ory_Hydra', # display name for this strategy
      #strategy_class: "OmniAuth::Strategies::OAuth2Generic" # Devise-specific config option Gitlab uses to find renamed strategy
    }
  }
]

The documentation for this, other then inside the file, is a bit scattered:

The biggest-source for errors is the clients-options-section. Here we'll specify the details for the OAuth2 flow and where Ory Hydra is located. Two things are important to keep in your mind when looking at configurations which are specifying some flow one way or another:

  • Where's the DNS-name resolved? Sometimes it's on the users browser, sometimes on GitLab or on the hydra-side. In our docker-based POC, it makes a huge difference!
  • Cookies can only be written/read, if they're from the same domain. In that case "127.0.0.1". That would be a different domain than "localhost". Pay attention to that.

These two points in our mind, let's look at the three configurations:

  • 'site' => 'http://127.0.0.1:4444' This is the default for the three URLs later if not specified otherwise.
  • 'authorize_url' => 'http://127.0.0.1:4444/oauth2/auth' this url will be a redirect-target and therefore resolved on the browser of the user. Probably we could omit the scheme, host and port as this is already defined in site.
  • 'token_url' => 'http://hydra:4444/oauth2/token' the token_url will get used on the GitLab-server to get a token after GitLab received the grant. As it's resolved on the GitLab-side, we're using docker-name of the hydra-container which is by default resolvable on the GitLab-container.
  • 'user_info_url' => 'http://hydra:4444/userinfo', same thing for the user_info_url. It's called on the GitLab-container and needs to be resolvable there.

The paths here are by default the same paths which are specified by OpenID connect. The configuration would be simpler if we would use OpenID-Connect (more about that later in the appendix) but in our case we're simply manually specifying the values. So it's not an accident that these pathes here are the very same then what you get from Ory Hydra:

curl http://127.0.0.1:4444/.well-known/openid-configuration | jq .
[...]
{
  "issuer": "http://127.0.0.1:4444/",
  "authorization_endpoint": "http://127.0.0.1:4444/oauth2/auth",
  "token_endpoint": "http://127.0.0.1:4444/oauth2/token",
  "jwks_uri": "http://127.0.0.1:4444/.well-known/jwks.json",
[...]
  "userinfo_endpoint": "http://127.0.0.1:4444/userinfo",
  "scopes_supported": [
    "offline_access",
    "offline",
    "openid"
  ],
  "token_endpoint_auth_methods_supported": [
    "client_secret_post",
    "client_secret_basic",
    "private_key_jwt",
    "none"
  ],
  [...]
}

Also worth noting here is the supported token_endpoint_auth_methods: How does GitLab authenticate against Ory Hydra? So GitLab is using client_secret_post which we needed to specify when we've created the GitLab-client in Ory Hydra.

Some remarks for creating the client. We've created the client like this. The second command shows the created client:

hydra create client \
    --endpoint http://127.0.0.1:4445 \
    --id $client_id \
    --secret $client_secret \
    --grant-type authorization_code,refresh_token \
    --response-type code,id_token, email \
    --scope openid,offline_access,profile,email \
    --redirect-uri http://gitlab.example.com:8000/users/auth/Ory_Hydra/callback \
    --token-endpoint-auth-method client_secret_post

hydra get clienthydra --endpoint http://127.0.0.1:4445
{
        "client_id": "gitlab",
        "created_at": "2020-08-31T08:47:30.000Z",
        "grant_types": [
                "authorization_code",
                "refresh_token"
        ],
        "jwks": {},
        "metadata": {},
        "redirect_uris": [
                "http://gitlab.example.com:8000/users/auth/Ory_Hydra/callback"
        ],
        "response_types": [
                "code",
                "id_token",
                ""
        ],
        "scope": "openid offline_access profile email",
        "subject_type": "public",
        "token_endpoint_auth_method": "client_secret_post",
        "updated_at": "2020-08-31T08:47:30.000Z",
        "userinfo_signed_response_alg": "none"
}
  • The endpoint isn't part of the configuration but it's a command-line-switch telling the hydra-binary to which hydra-instance to talk to
  • id and secret has been specified before in the GitLab-config
  • the token-endpoint-auth-method is by default client_secret_basic but GitLab is using client_secret_post (couldn't find that anywhere in the GitLab-documentation, though)
  • The callback needs to be resolvable on the users-browser. However, originally, the callback-url is created on the GitLab-side. In order to make that resolvable on the client, we set the external_url in the GitLab-configuration. Here that value is just there to cross-check with the generated one. It needs to match.

GitLab user-creation​

Initially, GitLab doesn't have any user but it needs them in order to manage authorisation, no matter how the login is done. This is a common issue and a common solution to this is to create the users on the fly with the first login. So in order to do that, these lines are enabling that:

gitlab_rails['omniauth_block_auto_created_users'] = false
gitlab_rails['omniauth_allow_single_sign_on'] = ['Ory_Hydra']

In order to get the necessary data for the user, gitlab needs to call to hydra's userinfo-endpoint. The most important attribute is the sub-attribute which provides, according to the specification, the ID of a user which is (in this case) the email-address. However, the email-address is also an attribute in the specification but in the implementation of the of this one hardcoded user (foo@bar.com) is empty.

So therefore we're specifying a mapping telling gitlab it should take the sub-field and use it as email:

      user_response_structure: {
        root_path: [],
        id_path: 'sub',
        attributes: {
                email: 'sub'
        }
      },

Whether the attribute "email" is there or not is quite critical here. The Login-ID has the form of an email. So in order to satisfy Gitlab's requirement, we're mapping here the email-attribute to the Login-ID which is represented by "sub". This shouldn't be necessary in a real-world-implementation.

But assuming that it's not doing that mapping, then GitLab would need to ask Ory Hydra on that endpoint the email-address. But is GitLab even allowed to read it? We need consent from the user for that and we configured the client above to be able to ask for that scope. However, we also need to configure GitLab to actually ask for that scope:

     authorize_params: {
           scope: 'email'
      },

Conclusion​

We've successfully integrated GitLab with Ory Hydra. Everything was done as configuration. No code has been created nor has any application been monkey-patched while following this guide (so far).

Troubleshooting​

Client wrong​

After trying to log in, you get a message like this:

Error: invalid_client Description: Client authentication failed (for example, unknown client, no client authentication included, or unsupported authentication method) Hint: The requested OAuth 2.0 Client doesn't exist.

Check your registered clients. Make sure ID and password are correct and matches that of the gitlab.rb:

hydra list clients --endpoint http://127.0.0.1:4445
| CLIENT ID | NAME | RESPONSE TYPES |             SCOPE              |                        REDIRECT URIS                         |           GRANT TYPES            | TOKEN ENDPOINT AUTH METHOD |
|-----------|------|----------------|--------------------------------|--------------------------------------------------------------|----------------------------------|----------------------------|
| gitlab    |      | code,id_token, | openid offline_access profile  | http://gitlab.example.com:8000/users/auth/Ory_Hydra/callback | authorization_code,refresh_token | client_secret_post         |
|           |      |                | email                          |                                                              |                                  |                            |
$

From Hydra: request is missing ... or otherwise malformed​

So after this, clicking the login-button on the sign-in-page will forward to Ory Hydra, which will redirect to the consent-app on port 3000. After the login, you'd get to the granting-page of the consent-app and after you've "allowed access", you'll get redirected back to gitlab which will unfortunately mention:

Couldn't authenticate you from OryHydra because "The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed".

So the message in quotes is from Ory Hydra and not very expressive. Note that it's a bit difficult to expose very meaningful error-messages as this could be used for security-attacks. So in such cases check the hydra-logs on what's wrong.

Ory Hydra logs: redirect URL is using an insecure protocol​

hydra_1          | time=2020-08-24T12:42:36Z level=error msg=An error occurred
audience=application error=map[message:invalid_request reason:Redirect URL is
using an insecure protocol, http is only allowed for hosts with suffix `localhost`,
for example: http://myapp.localhost/. status:Bad Request status_code:400]
http_request=map[headers:map[accept:text/html,application/xhtml+xml,application/xml;
q=0.9,image/webp,*/*;q=0.8 accept-encoding:gzip, deflate accept-language:en-US,en;
q=0.5 cookie:Value is sensitive and has been redacted. To see the value set config
key "log.leak_sensitive_values = true" or environment variable
"LOG_LEAK_SENSITIVE_VALUES=true". referer:http://127.0.0.1:3000/consent?consent_challenge=b695307490fa4732a80d3324f45f5a93
user-agent:Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0]
host:127.0.0.1:4444 method:GET path:/oauth2/auth query:Value is sensitive and has
been redacted. To see the value set config key "log.leak_sensitive_values = true"
or environment variable "LOG_LEAK_SENSITIVE_VALUES=true".
remote:172.19.0.1:47684 scheme:http] service_name= service_version=

The relevant part here is: Redirect URL is using an insecure protocol. Make sure to add the --dev? to the hydra-command as described above.

After that, restart the hydra-container like this:

docker-compose -f quickstart.yml \
    -f quickstart-postgres.yml -f quickstart-gitlab.yml \
    restart hydra

GitLab: signing in ... isn't allowed​

Signing in using your Ory Hydra account without a pre-existing GitLab account isn't allowed. Create a GitLab account first, and then connect it to your Ory Hydra account.

Double-Check the above explanation about user-creation.

GitLab: email can't be blank​

Sign-in failed because Email can't be blank and Notification email can't be blank.

Double-check the user_response_structure and the authorize_params. The attributes need an email-entry.

Appendix: some notes about OpenID Connect (OIDC)​

GitLab is supporting OIDC and Ory Hydra does that as well. Why hasn't that been used in this guide?

OIDC might be the better choice then plain OAuth2. When we tried that, we ran into the issue that the used OIDC implementation doesn't allow HTTP but "only" HTTPS. That's a good thing but not optimal for POCs like this. Whereas Ory Hydra has a switch to whitelist URLs in such cases, the used OIDC doesn't seem to have that. So, here is a reasonable OIDC configuration:

gitlab_rails['omniauth_providers'] = [
  { 'name' => 'openid_connect',
    'label' => 'Ory Hydra',
    # 'icon' => '<custom_provider_icon>',
    'args' => {
      'name' => 'openid_connect',
      'scope' => ['openid'],
      'response_type' => 'code',
      'issuer' => 'http://127.0.0.1:4444/',
      'discovery' => true,
      'client_auth_method' => 'basic',
      'send_scope_to_token_endpoint' => 'false',
      'client_options' => {
        'identifier' => 'gitlab',
        'secret' => 'theSecret',
        'redirect_uri' => 'http://gitlab.example.com:8000/users/auth/openid_connect/callback'
      }
    }
  }
]

In order to make that work which isn't SSL, we need to patch the openid_connect gem. Checkout the details here.

docker-compose -f quickstart.yml  -f quickstart-postgres.yml -f quickstart-gitlab.yml exec gitlab /opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/openid_connect-1.1.8/lib/openid_connect/discovery/provider/config.rb
          def initialize(uri)
            @host = uri.host
            @port = uri.port unless [80, 443].include?(uri.port)
            @path = File.join uri.path, '.well-known/openid-configuration'
            @scheme = uri.scheme
            attr_missing!
          end

          def endpoint
            case scheme
            when "http"
              SWD.url_builder = URI::HTTP
            else
              SWD.url_builder = URI::HTTPS
            end
            SWD.url_builder.build [nil, host, port, path, nil, nil]
          rescue URI::Error => e
            raise SWD::Exception.new(e.message)
          end

In order to avoid that scenario, this guide avoids OIDC.