RANGER-5614 : Performance improvement for role create/update with many users, groups, and sub-roles

[ramackri]

What changes were proposed in this pull request?

Optimized RoleRefUpdater and related role-ref DAOs so role create/update performs fewer database round-trips when a role references many users, groups, and sub-roles. The design mirrors RANGER-3899 / PR #962 (PolicyRefUpdater).

Problem

When creating or updating a role with a large number of principals, Ranger Admin was slow because:

1. N+1 lookups — Users, groups, and sub-roles were resolved with repeated per-name database calls during ref-table maintenance.
2. Full ref-table rebuild on every update — createNewRoleMappingForRefTable() always called cleanupRefTables(), deleting all rows from x_role_ref_user, x_role_ref_group, and x_role_ref_role, then re-inserting every principal even when nothing changed.
3. Monolithic associator — A single RolePrincipalAssociator took three collection parameters and populated one of them inside doAssociate(), which was harder to read and inconsistent with the refactored policy path.

Changes

Performance (RoleRefUpdater)

• Added isRefTableCleanupRequired to createNewRoleMappingForRefTable().
• Create (RoleDBStore, RoleREST): isRefTableCleanupRequired = false — no selective cleanup on empty ref tables.
• Update (RoleDBStore, RoleREST, add/remove users/groups/roles, import update): true — cleanupRefTablesForUpdate() deletes only principals removed from the role; unchanged principals are not re-inserted.
• Batch principal ID resolution via getIdsByUserNames / getIdsByGroupNames / getIdsByRoleNames and batchInsert() for new ref rows.
• cleanupRoleUsers / cleanupRoleGroups / cleanupRoleRoles accept the DAO as a parameter (same pattern as PolicyRefUpdater).

Associator refactor (aligned with PR #962 review)

Replaced RolePrincipalAssociator with three focused inner classes:

Class	getRoleRef()	createRoleRef(Long)
RoleUserAssociator	XXRoleRefUser for batch insert	Direct create on transaction commit
RoleGroupAssociator	XXRoleRefGroup	Direct create on transaction commit
RoleRoleAssociator	XXRoleRefRole	Direct create on transaction commit

Call-site pattern (same as PolicyRoleAssociator / PolicyGroupAssociator / PolicyUserAssociator):

Long userId = userNameIdMap.get(userName);
RoleUserAssociator associator = new RoleUserAssociator(userName, userId, roleId);

if (userId != null) {
    XXRoleRefUser userRef = associator.getRoleRef();
    if (userRef != null) {
        xxRoleRefUsers.add(userRef);
    }
} else if (isCreateNonExistentUGRs) {
    rangerTransactionSynchronizationAdapter.executeOnTransactionCommit(associator);
} else {
    throw restErrorUtil.createRESTException(...);
}

### Manual / functional testing (Done)

Environment: Ranger Admin UI at `http://localhost:6080` (minimal Docker: Postgres + Admin per `dev-support/RANGER-LOCAL-BUILD.md`).

#### Setup — create principals

1. Open **Settings** → **Users/Group/Permission** (or navigate to `http://localhost:6080/index.html#/users/roletab`).
2. **Users** tab — create **5 internal users**:
   - `ramk1`
   - `ramk2`
   - `ramackri1`
   - `ramackri2`
   - `ramackri3`
3. **Groups** tab — create **5 groups**:
   - `example`
   - `ramkGroup1`
   - `ramkGroup2`
   - `ramackrGroup1`
   - `ramackriGroup2`
   - `ramackriGroup3`
4. **Roles** tab — create **5 roles** (empty or minimal members):
   - `ramkRole1`
   - `ramkRole2`
   - `ramackrRole1`
   - `ramackriRole2`
   - `ramackriRole3`

#### Test role — `TestRole5` (create / update / incremental member changes)

5. **Roles** tab → **Create** role **`TestRole5`**.
6. Edit **`TestRole5`** and **add all** principals created above:
   - All 5 users  
   - All 5 groups  
   - All 5 sub-roles  
7. **Save** and confirm the role saves successfully and members appear in the UI.
8. **Update `TestRole5` incrementally** (repeat save after each step; verify UI and behavior):
   - Remove **one user**, save → confirm user removed from role, others unchanged.
   - Remove **one group**, save → same.
   - Remove **one sub-role**, save → same.
   - Add back **one** removed user/group/sub-role, save → confirm only new association is added (no duplicate members).
   - Remove **multiple** users/groups/sub-roles in one update, save → confirm selective cleanup (no full ref-table wipe).
9. **Optional stress check**: With `TestRole5` still holding many members, change membership several times (add/remove mix) and confirm:
   - Updates complete in reasonable time vs baseline (before RANGER-5614).
   - No duplicate entries in **Users / Groups / Roles** lists for `TestRole5`.
   - No errors in Ranger Admin logs (`docker logs ranger`).

#### DB sanity (optional)

10. For role id of `TestRole5`, verify ref tables reflect UI after an update:
    - `x_role_ref_user` — only current users  
    - `x_role_ref_group` — only current groups  
    - `x_role_ref_role` — only current sub-roles  

#### Expected results

- Create/update of roles with many users, groups, and sub-roles completes without timeout or error.
- Removing members updates only removed associations; unchanged members are not deleted and re-inserted.
- Adding members after a partial remove does not create duplicates.

[ramackri]

[Copilot]

Pull request overview

This PR optimizes Ranger Admin role create/update performance for roles with many users, groups, and sub-roles by reducing DB round-trips and avoiding full ref-table rebuilds on update (mirroring the earlier PolicyRefUpdater refactor).

Changes:

• Refactored RoleRefUpdater to support selective ref-table cleanup on update and batch principal resolution + batch inserts.
• Added new JPA named queries and DAO helpers to fetch existing role-ref mappings as name -> refRowId maps for selective deletes.
• Updated RoleStore API + REST/DBStore/tests to pass a new isRefTableCleanupRequired flag.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
security-admin/src/main/java/org/apache/ranger/biz/RoleRefUpdater.java	Selective cleanup + batch insert path; associator refactor.
security-admin/src/main/java/org/apache/ranger/biz/RoleDBStore.java	Updated RoleStore method signatures and wiring for new flag.
security-admin/src/main/java/org/apache/ranger/rest/RoleREST.java	Passes new cleanup flag for create vs update and role-member endpoints.
agents-common/src/main/java/org/apache/ranger/plugin/store/RoleStore.java	RoleStore API updated to include cleanup flag.
security-admin/src/main/java/org/apache/ranger/db/XXRoleRefUserDao.java	Added DAO method to fetch username->refRowId map by roleId.
security-admin/src/main/java/org/apache/ranger/db/XXRoleRefGroupDao.java	Added DAO method to fetch groupName->refRowId map by roleId.
security-admin/src/main/java/org/apache/ranger/db/XXRoleRefRoleDao.java	Added DAO method to fetch subRoleName->refRowId map by roleId.
security-admin/src/main/resources/META-INF/jpa_named_queries.xml	Added named queries backing the new DAO map methods.
security-admin/src/test/java/org/apache/ranger/biz/TestRoleRefUpdater.java	Updated tests for batch behavior + added selective-cleanup test.
security-admin/src/test/java/org/apache/ranger/biz/TestRoleDBStore.java	Updated tests for new RoleStore method signatures.
security-admin/src/test/java/org/apache/ranger/rest/TestRoleREST.java	Updated REST tests for new RoleStore method signatures.
security-admin/src/test/java/org/apache/ranger/biz/TestPolicyRefUpdater.java	Updated mock RoleStore signature usage.
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java	Updated RoleStore call signature when creating roles from policy path.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.