9229 - Add bearer token auth mechanism for OIDC user with feature flag

conf/keycloak/oidc-keycloak-auth-provider.json

@@ -3,6 +3,6 @@
   "factoryAlias": "oidc",
   "title": "OIDC-Keycloak",
   "subtitle": "OIDC-Keycloak",
-  "factoryData": "type: oidc | issuer: http://localhost:8090/auth/realms/oidc-realm | clientId: oidc-client | clientSecret: ss6gE8mODCDfqesQaSG3gwUwZqZt547E",
+  "factoryData": "type: oidc | issuer: http://keycloak.mydomain.com:8090/realms/oidc-realm | clientId: oidc-client | clientSecret: ss6gE8mODCDfqesQaSG3gwUwZqZt547E",
   "enabled": true
 }

doc/release-notes/9229-bearer-api-auth.md

@@ -0,0 +1 @@
+A feature flag called "api-bearer-auth" has been added. This allows OIDC useraccounts to send authenticated API requests using Bearer Tokens. Note: This feature is limited to OIDC! For more information, see http://preview.guides.gdcc.io/en/develop/installation/config.html#feature-flags

doc/sphinx-guides/source/api/auth.rst

@@ -63,3 +63,20 @@ Resetting Your API Token
 ------------------------
 
 You can reset your API Token from your account page in your Dataverse installation as described in the :doc:`/user/account` section of the User Guide.
+
+.. _bearer-tokens:
+
+Bearer Tokens
+-------------
+
+Bearer tokens are defined in `RFC 6750`_ and can be used as an alternative to API tokens if your installation has been set up to use them (see :ref:`bearer-token-auth` in the Installation Guide).
+
+.. _RFC 6750: https://tools.ietf.org/html/rfc6750
+
+To test if bearer tokens are working, you can try something like the following (using the :ref:`User Information` API endpoint), substituting in parameters for your installation and user.
+
+.. code-block:: bash
+
+  export TOKEN=`curl -s -X POST --location "http://keycloak.mydomain.com:8090/realms/oidc-realm/protocol/openid-connect/token" -H "Content-Type: application/x-www-form-urlencoded" -d "username=kcuser&password=kcpassword&grant_type=password&client_id=oidc-client&client_secret=ss6gE8mODCDfqesQaSG3gwUwZqZt547E" | jq '.access_token' -r | tr -d "\n"`
+
+  curl -H "Authorization: Bearer $TOKEN" http://localhost:8080/api/users/:me

doc/sphinx-guides/source/developers/remote-users.rst

@@ -30,9 +30,13 @@ Now when you go to http://localhost:8080/oauth2/firstLogin.xhtml you should be p
 
 ----
 
+.. _oidc-dev:
+
 OpenID Connect (OIDC)
 ---------------------
 
+STOP! ``oidc-keycloak-auth-provider.json`` was changed from http://localhost:8090 to http://keycloak.mydomain.com:8090 to test :ref:`bearer-tokens`. In addition, ``docker-compose-dev.yml`` in the root of the repo was updated to start up Keycloak. To use these, you should add ``127.0.0.1 keycloak.mydomain.com`` to your ``/etc/hosts file``. If you'd like to use the docker compose as described below (``conf/keycloak/docker-compose.yml``), you should revert the change to ``oidc-keycloak-auth-provider.json``.
+
 If you are working on the OpenID Connect (OIDC) user authentication flow, you do not need to connect to a remote provider (as explained in :doc:`/installation/oidc`) to test this feature. Instead, you can use the available configuration that allows you to run a test Keycloak OIDC identity management service locally through a Docker container.
 
 (Please note! The client secret (``ss6gE8mODCDfqesQaSG3gwUwZqZt547E``) is hard-coded in ``oidc-realm.json`` and ``oidc-keycloak-auth-provider.json``. Do not use this config in production! This is only for developers.)

doc/sphinx-guides/source/installation/config.rst

@@ -330,6 +330,19 @@ As for the "Remote only" authentication mode, it means that:
 - ``:DefaultAuthProvider`` has been set to use the desired authentication provider
 - The "builtin" authentication provider has been disabled (:ref:`api-toggle-auth-provider`). Note that disabling the "builtin" authentication provider means that the API endpoint for converting an account from a remote auth provider will not work. Converting directly from one remote authentication provider to another (i.e. from GitHub to Google) is not supported. Conversion from remote is always to "builtin". Then the user initiates a conversion from "builtin" to remote. Note that longer term, the plan is to permit multiple login options to the same Dataverse installation account per https://github.com/IQSS/dataverse/issues/3487 (so all this talk of conversion will be moot) but for now users can only use a single login option, as explained in the :doc:`/user/account` section of the User Guide. In short, "remote only" might work for you if you only plan to use a single remote authentication provider such that no conversion between remote authentication providers will be necessary.
 
+.. _bearer-token-auth:
+
+Bearer Token Authentication
+---------------------------
+
+Bearer tokens are defined in `RFC 6750`_ and can be used as an alternative to API tokens. This is an experimental feature hidden behind a feature flag.
+
+.. _RFC 6750: https://tools.ietf.org/html/rfc6750
+
+To enable bearer tokens, you must install and configure Keycloak (for now, see :ref:`oidc-dev` in the Developer Guide) and enable ``api-bearer-auth`` under :ref:`feature-flags`.
+
+You can test that bearer tokens are working by following the example under :ref:`bearer-tokens` in the API Guide.
+
 .. _database-persistence:
 
 Database Persistence
@@ -2391,6 +2404,9 @@ please find all known feature flags below. Any of these flags can be activated u
     * - api-session-auth
       - Enables API authentication via session cookie (JSESSIONID). **Caution: Enabling this feature flag exposes the installation to CSRF risks!** We expect this feature flag to be temporary (only used by frontend developers, see `#9063 <https://github.com/IQSS/dataverse/issues/9063>`_) and removed once support for bearer tokens has been implemented (see `#9229 <https://github.com/IQSS/dataverse/issues/9229>`_).
       - ``Off``
+    * - api-bearer-auth
+      - Enables API authentication via Bearer Token for OIDC User Accounts. **Information: This feature works only for OIDC UserAccounts!**
+      - ``Off``
 
 **Note:** Feature flags can be set via any `supported MicroProfile Config API source`_, e.g. the environment variable
 ``DATAVERSE_FEATURE_XXX`` (e.g. ``DATAVERSE_FEATURE_API_SESSION_AUTH=1``). These environment variables can be set in your shell before starting Payara. If you are using :doc:`Docker for development </container/dev-usage>`, you can set them in the `docker compose <https://docs.docker.com/compose/environment-variables/set-environment-variables/>`_ file.
@@ -3837,7 +3853,7 @@ To use the current GDCC version directly:
 :CategoryOrder
 ++++++++++++++
 
-A comma separated list of Category/Tag names defining the order in which files with those tags should be displayed. 
+A comma separated list of Category/Tag names defining the order in which files with those tags should be displayed.
 The setting can include custom tag names along with the pre-defined tags(Documentation, Data, and Code are the defaults but the :ref:`:FileCategories` setting can be used to use a different set of tags).
 The default is category ordering disabled.
 
@@ -3849,7 +3865,7 @@ A true(default)/false option determining whether datafiles listed on the dataset
 :AllowUserManagementOfOrder
 +++++++++++++++++++++++++++
 
-A true/false (default) option determining whether the dataset datafile table display includes checkboxes enabling users to turn folder ordering and/or category ordering (if an order is defined by :CategoryOrder) on and off dynamically. 
+A true/false (default) option determining whether the dataset datafile table display includes checkboxes enabling users to turn folder ordering and/or category ordering (if an order is defined by :CategoryOrder) on and off dynamically.
 
 .. _supported MicroProfile Config API source: https://docs.payara.fish/community/docs/Technical%20Documentation/MicroProfile/Config/Overview.html
 

docker-compose-dev.yml

@@ -12,6 +12,7 @@ services:
       - DATAVERSE_DB_HOST=postgres
       - DATAVERSE_DB_PASSWORD=secret
       - DATAVERSE_DB_USER=${DATAVERSE_DB_USER}
+      - DATAVERSE_FEATURE_API_BEARER_AUTH=1
     ports:
       - "8080:8080" # HTTP (Dataverse Application)
       - "4848:4848" # HTTP (Payara Admin Console)
@@ -98,6 +99,25 @@ services:
     tmpfs:
       - /mail:mode=770,size=128M,uid=1000,gid=1000
 
+  dev_keycloak:
+    container_name: "dev_keycloack"
+    image: 'quay.io/keycloak/keycloak:19.0'
+    hostname: keycloak
+    environment:
+      - KEYCLOAK_ADMIN=kcadmin
+      - KEYCLOAK_ADMIN_PASSWORD=kcpassword
+      - KEYCLOAK_LOGLEVEL=DEBUG
+      - KC_HOSTNAME_STRICT=false
+    networks:
+      dataverse:
+        aliases:
+          - keycloak.mydomain.com #create a DNS alias within the network (add the same alias to your /etc/hosts to get a working OIDC flow)
+    command: start-dev --import-realm --http-port=8090  # change port to 8090, so within the network and external the same port is used
+    ports:
+      - "8090:8090"
+    volumes:
+      - './conf/keycloak/oidc-realm.json:/opt/keycloak/data/import/oidc-realm.json'
+
 networks:
   dataverse:
     driver: bridge

src/main/java/edu/harvard/iq/dataverse/api/auth/BearerTokenAuthMechanism.java

@@ -0,0 +1,116 @@
+package edu.harvard.iq.dataverse.api.auth;
+
+import com.nimbusds.oauth2.sdk.ParseException;
+import com.nimbusds.oauth2.sdk.token.BearerAccessToken;
+import edu.harvard.iq.dataverse.UserServiceBean;
+import edu.harvard.iq.dataverse.authorization.AuthenticationServiceBean;
+import edu.harvard.iq.dataverse.authorization.UserRecordIdentifier;
+import edu.harvard.iq.dataverse.authorization.providers.oauth2.OAuth2Exception;
+import edu.harvard.iq.dataverse.authorization.providers.oauth2.oidc.OIDCAuthProvider;
+import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
+import edu.harvard.iq.dataverse.authorization.users.User;
+import edu.harvard.iq.dataverse.settings.FeatureFlags;
+
+import javax.inject.Inject;
+import javax.ws.rs.container.ContainerRequestContext;
+import javax.ws.rs.core.HttpHeaders;
+import java.io.IOException;
+import java.util.List;
+import java.util.Optional;
+import java.util.logging.Level;
+import java.util.logging.Logger;
+import java.util.stream.Collectors;
+
+public class BearerTokenAuthMechanism implements AuthMechanism {
+    private static final String BEARER_AUTH_SCHEME = "Bearer";
+    public static final String UNAUTHORIZED_BEARER_TOKEN = "Unauthorized bearer token";
+    public static final String INVALID_BEARER_TOKEN = "Could not parse bearer token";
+    public static final String BEARER_TOKEN_DETECTED_NO_OIDC_PROVIDER_CONFIGURED = "Bearer token detected, no OIDC provider configured";
+
+    @Inject
+    protected AuthenticationServiceBean authSvc;
+    @Inject
+    protected UserServiceBean userSvc;
+    private static final Logger logger = Logger.getLogger(BearerTokenAuthMechanism.class.getCanonicalName());
+    @Override
+    public User findUserFromRequest(ContainerRequestContext containerRequestContext) throws WrappedAuthErrorResponse {
+        if (FeatureFlags.API_BEARER_AUTH.enabled()) {
+            Optional<String> bearerToken = getRequestApiKey(containerRequestContext);
+            // No Bearer Token present, hence no user can be authenticated
+            if (!bearerToken.isPresent()) {
+                return null;
+            }
+            //validate and verify provided Bearer Token, and retrieve UserRecordIdentifier
+            UserRecordIdentifier userInfo = verifyOidcBearerTokenAndGetUserIndentifier(bearerToken.get());
+
+            // retrieve Authenticated User from AuthService
+            AuthenticatedUser authUser = authSvc.lookupUser(userInfo);
+            if (authUser != null) {
+                // track the API usage
+                authUser = userSvc.updateLastApiUseTime(authUser);
+                return authUser;
+            } else {
+                // a valid Token was presented, but we have no associated user account.
+                logger.log(Level.WARNING, "Bearer  token detected, OIDC provider {0} validated Token but no linked UserAccount", userInfo.getUserRepoId());
+                return null;
+            }
+        }
+        return null;
+    }
+
+    /**
+     * Verifies the given Bearer token and obtain information about the corresponding user within respective AuthProvider.
+     *
+     * @param token The string containing the encoded JWT
+     * @return
+     */
+    private UserRecordIdentifier verifyOidcBearerTokenAndGetUserIndentifier(String token) throws WrappedAuthErrorResponse {
+        try {
+            BearerAccessToken accessToken = BearerAccessToken.parse(token);
+            // Get list of all authentication providers using Open ID Connect
+            // @TASK: Limited to OIDCAuthProviders, could be widened to OAuth2Providers.
+            List<OIDCAuthProvider> providers = authSvc.getAuthenticationProviderIdsOfType(OIDCAuthProvider.class).stream()
+                    .map(providerId -> (OIDCAuthProvider) authSvc.getAuthenticationProvider(providerId))
+                    .collect(Collectors.toUnmodifiableList());
+            // If not OIDC Provider are configured we cannot validate a Token
+            if(providers.isEmpty()){
+                logger.log(Level.WARNING, "Bearer token detected, no OIDC provider configured");
+                throw new WrappedAuthErrorResponse(BEARER_TOKEN_DETECTED_NO_OIDC_PROVIDER_CONFIGURED);
+            }
+
+            // Iterate over all OIDC providers if multiple. Sadly needed as do not know which provided the Token.
+            for (OIDCAuthProvider provider : providers) {
+                try {
+                    // The OIDCAuthProvider need to verify a Bearer Token and equip the client means to identify the corresponding AuthenticatedUser.
+                    Optional<UserRecordIdentifier> userInfo = provider.getUserIdentifierForValidToken(accessToken);
+                    if(userInfo.isPresent()) {
+                        logger.log(Level.FINE, "Bearer token detected, provider {0} confirmed validity and provided identifier", provider.getId());
+                        return userInfo.get();
+                    }
+                } catch ( IOException| OAuth2Exception e) {
+                    logger.log(Level.FINE, "Bearer token detected, provider " + provider.getId() + " indicates an invalid Token, skipping", e);
+                }
+            }
+        } catch (ParseException e) {
+            logger.log(Level.FINE, "Bearer token detected, unable to parse bearer token (invalid Token)", e);
+            throw new WrappedAuthErrorResponse(INVALID_BEARER_TOKEN);
+        }
+
+        // No UserInfo returned means we have an invalid access token.
+        logger.log(Level.FINE, "Bearer token detected, yet no configured OIDC provider validated it.");
+        throw new WrappedAuthErrorResponse(UNAUTHORIZED_BEARER_TOKEN);
+    }
+
+    /**
+     * Retrieve the raw, encoded token value from the Authorization Bearer HTTP header as defined in RFC 6750
+     * @return An {@link Optional} either empty if not present or the raw token from the header
+     */
+    private  Optional<String> getRequestApiKey(ContainerRequestContext containerRequestContext) {
+        String headerParamApiKey = containerRequestContext.getHeaderString(HttpHeaders.AUTHORIZATION);
+        if (headerParamApiKey != null && headerParamApiKey.toLowerCase().startsWith(BEARER_AUTH_SCHEME.toLowerCase() + " ")) {
+            return Optional.of(headerParamApiKey);
+        } else {
+            return Optional.empty();
+        }
+    }
+}

src/main/java/edu/harvard/iq/dataverse/api/auth/CompoundAuthMechanism.java

@@ -19,9 +19,9 @@ public class CompoundAuthMechanism implements AuthMechanism {
     private final List<AuthMechanism> authMechanisms = new ArrayList<>();
 
     @Inject
-    public CompoundAuthMechanism(ApiKeyAuthMechanism apiKeyAuthMechanism, WorkflowKeyAuthMechanism workflowKeyAuthMechanism, SignedUrlAuthMechanism signedUrlAuthMechanism, SessionCookieAuthMechanism sessionCookieAuthMechanism) {
+    public CompoundAuthMechanism(ApiKeyAuthMechanism apiKeyAuthMechanism, WorkflowKeyAuthMechanism workflowKeyAuthMechanism, SignedUrlAuthMechanism signedUrlAuthMechanism, SessionCookieAuthMechanism sessionCookieAuthMechanism, BearerTokenAuthMechanism bearerTokenAuthMechanism) {
         // Auth mechanisms should be ordered by priority here
-        add(apiKeyAuthMechanism, workflowKeyAuthMechanism, signedUrlAuthMechanism, sessionCookieAuthMechanism);
+        add(apiKeyAuthMechanism, workflowKeyAuthMechanism, signedUrlAuthMechanism, sessionCookieAuthMechanism,bearerTokenAuthMechanism);
     }
 
     public CompoundAuthMechanism(AuthMechanism... authMechanisms) {

src/main/java/edu/harvard/iq/dataverse/authorization/providers/oauth2/oidc/OIDCAuthProvider.java

@@ -29,6 +29,7 @@
 import com.nimbusds.openid.connect.sdk.op.OIDCProviderConfigurationRequest;
 import com.nimbusds.openid.connect.sdk.op.OIDCProviderMetadata;
 import edu.harvard.iq.dataverse.authorization.AuthenticatedUserDisplayInfo;
+import edu.harvard.iq.dataverse.authorization.UserRecordIdentifier;
 import edu.harvard.iq.dataverse.authorization.exceptions.AuthorizationSetupException;
 import edu.harvard.iq.dataverse.authorization.providers.oauth2.AbstractOAuth2AuthenticationProvider;
 import edu.harvard.iq.dataverse.authorization.providers.oauth2.OAuth2Exception;
@@ -272,4 +273,18 @@ Optional<UserInfo> getUserInfo(BearerAccessToken accessToken) throws IOException
             throw new OAuth2Exception(-1, ex.getMessage(), BundleUtil.getStringFromBundle("auth.providers.exception.userinfo", Arrays.asList(this.getTitle())));
         }
     }
+
+    /**
+     * Returns the UserRecordIdentifier corresponding to the given accessToken if valid.
+     * UserRecordIdentifier (same used as in OAuth2UserRecord), i.e. can be used to find a local UserAccount.
+     * @param accessToken
+     * @return Returns the UserRecordIdentifier corresponding to the given accessToken if valid.
+     * @throws IOException
+     * @throws OAuth2Exception
+     */
+    public Optional<UserRecordIdentifier> getUserIdentifierForValidToken(BearerAccessToken accessToken) throws IOException, OAuth2Exception{
+        // Request the UserInfoEndpoint to obtain UserInfo, since this endpoint also validate the Token we can reuse the existing code path.
+        // As an alternative we could use the Introspect Endpoint or assume the Token as some encoded information (i.e. JWT).
+        return  Optional.of(new UserRecordIdentifier(  this.getId(), getUserInfo(accessToken).get().getSubject().getValue()));
+    }
 }

src/main/java/edu/harvard/iq/dataverse/settings/FeatureFlags.java

@@ -30,7 +30,12 @@ public enum FeatureFlags {
      * @since Dataverse 5.14
      */
     API_SESSION_AUTH("api-session-auth"),
-
+    /**
+     * Enables API authentication via Bearer Token.
+     * @apiNote Raise flag by setting "dataverse.feature.api-bearer-auth"
+     * @since Dataverse @TODO:
+     */
+    API_BEARER_AUTH("api-bearer-auth"),
     ;
 
     final String flag;