Skip to end of metadata
Go to start of metadata

Document Status:  Draft for public discussion

Summary:   Developing an institutional-scale groups service for the UW is a complex multi-year multi-phase activity.  This document presents a vision for such a service to clarify its benefits, goals, and scope.  It also presents a conceptual architecture to suggest technical components of the service.  It is intended to be useful as a framework for thinking about how the service will evolve, and understanding how service components will fit together, as some may appear and mature sooner than others.


A groups service is a key component of an institutional identity and access management (IAM) program.  The ability to organize entities into groups is a natural extension to managing individual identities, enabling scalable, secure, consistent management in many applications and systems.

As a component of the UW IAM program, the UW groups service shares the goals of that program:  to provide services that are secure, easy to use, easy to integrate, institution-scale, reliable, cost-effective, and support the policy goals of the university.  At a high level, the purpose of the groups service is to support effective groups usage throughout the institution by making it easy to define the right groups, and easy to use those groups in applications and systems.

The use of groups is pervasive in applications, operating systems, and computing environments of all kinds.  Two types of use stand out as very common:

  • Access control:  In an access control expression on a resource or an operation, using a group permits the expression to apply to all group members at once, rather than having to list all members in the expression;
  • Messaging:  A message may be sent (e.g. via email or instant messaging) to all members of a group at once via a group address.

The large number of existing systems of all kinds and the pervasive use of groups in those systems mean that the development of an institution-scale groups service will be incremental, linking and enhancing existing approaches rather than replacing them.  The current state of the groups service is already showing the utility of this approach, and revealing some of the issues that must be dealt with.

The following sections present a vision of a groups service, a view from the outside; and a conceptual architecture, a view from the inside.  While it tries to be comprehensive it is not a requirements or specification document.  A concluding section describes some open questions.

Groups Service Vision:  Features and Benefits

Like any important service for a large organization, the groups service consists of many components and many service interfaces.  This is especially so for a service that is integrating many independent systems and platforms.  It is useful to distinguish several aspects of the groups lifecycle, as each implies different service features.  These aspects are:

  • management:  In this aspect a group is created, named, and modified, along with its membership and metadata.
  • provisioning:  In this aspect group information is moved from one system to another, so as to be accessible in the environment of the destination system.  For example, group information might be moved from a database to an LDAP directory.
  • reference:  In this aspect a group is referred to for some purpose in an application system.  For example, a group name may be entered on an access control list for an application function.
  • evaluation:  In this aspect a group's membership is evaluated to support some application function.  For example, a userid may be checked for group membership to support access to an application function.

In many simple applications these different aspects are combined or not needed.  However, when groups functions are made more modular and are shared across many systems it is important to clarify which aspects are being handled by a particular service component.

The UW groups service includes these features and benefits:

Institutional scale:   The service supports the management and use of hundreds of thousands of groups, by tens of thousands of users, for use in thousands of applications and systems, with performance to meet the needs of large scale applications like UW mailman, MyUW, UWWI, Catalyst, calendaring, etc.

Access for all:  The service supports groups defined as part of, and in support of, institutional business processes such as course enrollment and personnel management, and also a wide range of less formal purposes, including "personal" group management for ordinary community members (e.g. students, alumni).

Distributed management:  Management of groups continues to happen in several systems and venues, because users are familiar with many existing management interfaces, and these provide useful application-oriented service.  Over time, however, a smaller number of more functional, more institutionally-capable management environments dominate, leading to a stable state with anywhere from 5 to 50 groups management venues institution-wide; not hundreds, but not only one.  There will probably be a "central" general-purpose groups management venue, but not necessarily.

Institutional sources:   The sources of groups information include the significant institutional systems of record (HR, student, alumni, medical) which provide groups information derived from business processes:  courses, majors, appointments, payroll units, alumni records, health-care providers, etc.

Global reference:  Regardless of where a group is managed, it can be referenced (and subsequently evaluated) in any application or platform.  This implies the ability to integrate applications and platforms easily with the service, including dealing with issues of group naming, populating pick lists, expanding memberships, etc.

Composability:  Group information from multiple sources is able to be combined during management as well as during reference and evaluation.  For example, a department might use course enrollment groups with local modifications to create a new group for access control.

Transparency and coherence:  A person wishing to reference a group has information available to them about what the group is for, what its members are and what things they can be, who manages the group, when it last changed, any relevant policies, and its "officialness."  The service supports the maintenance, distribution, and display of this information.  Similarly, the system supports well-defined, understandable information flows from producer to consumer of groups information.

Diverse memberships:  Groups may have as members entities of all kinds:  UW NetIDs, person identifiers, email addresses, federated userids, host names, budget numbers, URIs, whatever is useful in common applications.

Support for rich application functionality:   A group by itself is a very simple thing:  an entity with a membership list.  As a primitive data structure, named lists exist in many systems.  In real-world scenarios groups are often elements of more complex data structures.  For example, a course may have distinct groups for its student members and its instructors, as well as much other course-specific data.  The  groups system provides good support of the common features of group use across many disparate systems while making richer functionality possible for some.

Tailored user interfaces:  Users have groups management user interfaces available to them that are suited to their interests and capabilities.  A simple interface is available to the casual user, while a power user (e.g. someone who manages groups for a large organization) can use a more rich and efficient interface.  Interfaces tailored to common roles and tasks, eg instructors managing class lists, are available.

Application user interface integration:  Many applications come with built-in groups management user interfaces, or find it to be a better user experience to offer groups management inside the application rather than externally.  The groups service supports such applications operating against external groups information stores via protocol interfaces.

Integration with privilege management services:   When groups are used for access control there is utility in a clear linkage between groups and privilege management (primarily ASTRA).  The groups service supports the ability of privilege management systems to assign privileges to a group.  Privileges to manage groups are manageable, at some level, via privilege management systems rather than strictly in groups management venues themselves.

Access control:   The system provides the ability to control access to management and provisioning operations to ensure the integrity of groups data.  It also provides the ability to control visibility of groups information to support cases (e.g. student and medical) where the existence and membership of groups may be sensitive.

Low-latency updates:  In most cases groups information will flow through several independent systems from its point of management to its many points of use.  This information flow happens in most cases with low latency (for example less than a minute) assuming appropriate capability in source and destination systems. 

Groups Service:  Conceptual Architecture

The groups service consists of a large number of independent components; it is not a single monolithic service.  Some of the elements below are concrete software services, while others are policies or guidelines.  The architecture is intended to be compatible with Service-Oriented Architecture principles.

Groups Information Sources 

A set of groups information management venues is identified as participants in the service.  Their function is to provide groups information as input to the service.  Each provides a user interface, data store, import/export interfaces, and support some number of groups-management users.  These venues meet defined requirements in order to participate:  stable operation, secure management, regular updates, conformance to the naming plan, etc.  Such a venue exports group information to the rest of the institution, and might also import group information from the institution.  It might export only a subset of its groups information, and import only a subset of the institutional group information.  Examples of such venues are Catalyst, the UW Exchange service, schools and larger departments.  A "central" institutional groups management venue (comparable to the central UW mailman-based mailing list service, for example) is probably useful, but not strictly necessary, and would be one option among many for management venues.

A groups information management venue may offer a protocol interface to its groups management functions, permitting other systems to offer groups management user interfaces that operate against that venue's groups information store.  This is particularly useful for a large or central venue.

Similarly institutional business systems are sources of group information.  This includes HR, student, alumni, medical, and potentially other systems (e.g. ASTRA).

In most cases, for simplicity, a group's information is managed in only one source.  There may be cases where a group could be managed in multiple sources, or moved from one source to another.  It is possible in a management venue to create a new group incorporating by reference a group managed in another source. 

Groups Registry 

An institutional groups registry provides several institution-level functions.  Its primary role is to act as a clearinghouse for groups information to flow from sources to consumers.  Cross-organizational and cross-system groups information flow is strongly encouraged to go through the registry.

Groups information is provisioned into the registry from sources via incremental or batch operations.  Groups information is provisioned out of the registry into groups directories, and in some cases directly into consuming applications or systems.  Groups information may also be transformed in various ways (eg adding identifiers, canonicalizing names) to make it more useful for general consumption; this would be done by agreement with source owners.

Registry administrators are responsible for ensuring that groups information sources are compliant with requirements regarding input data.  They are responsible for assuring the quality and integrity of registry data, including dealing with naming conflicts, data update problems, etc.  Lastly they are responsible for ensuring that groups information is provisioned successfully into groups directories and other consumers.

The Registry has an administration user interface supporting Registry administrators and groups source administrators.

The Registry may also serve as a central groups information management venue.  If so, it is important to maintain the distinction between functions.  For example, editing group entries that are sourced from other sources could be prohibited, or the potential for conflict made clear.  In this case it is important for this function to have a protocol interface supporting management functions so that other systems can offer groups management user interfaces operating against the Registry data store.

Groups directories

Directories provide high-volume access-controlled read-only access to groups information for many applications and platforms.  Groups information is provisioned into the directories from the groups registry in an incremental, low-latency fashion.

Directory access by clients is offered via LDAP.  Access via "web services" (SOAP, REST) is also supported.  Integration with Windows domain management, and access by Windows-domain-based applications, is provided via import of groups into UWWI.

A web-based groups directory user interface is provided, to support groups discovery and inquiry.  Alternative UIs may exist for different purposes or communities.

Metadata definition 

A group entry in the Registry consists of several elements, including both core items (name, membership) and additional information supporting lifecycle, transparency, usability, etc.

  • a name, or names, and other identifiers, including target-platform-specific identifiers if needed
  • information about creation and update, including dates, by-whom, for-what-reason
  • origin, intended use, type, membership criteria, update frequency, etc
  • if the group is managed by a process (eg membership derived from a system of record), the algorithm used
  • access control for various operations (view, update, delete, rename, etc)
  • group membership

The service defines the metadata elements, and supports its evolution to handle new requirements.  It also defines policy including which elements are required or optional.

Group entries from groups information sources must provide required elements (e.g. a name) and are encouraged to provide optional elements.

Groups naming plan

For groups to be globally referenceable they must be uniquely named.  This is supported by an institutional groups naming plan, including both policy (the rules) and administration (enforcement and setup).

Since groups are referenced in heterogeneous environments they may need different names (or other identifiers) appropriate for those environments.

Naming of institutional resources involves some amount of oversight to reduce confusion, conflicts, and inappropriate use.  The naming plan limits the amount of per-group manual review by taking advantage of existing naming schemes, supporting hierarchical delegation, etc.

In some environments (most importantly Windows Active Directory) group names are in the same namespace as (UW NetID based) user names.  At least one naming scheme that avoids conflict with UW NetIDs is provided.  (See UW Group Naming Planfor a proposed scheme).

Access control and privacy

Control of access to groups management operations is the responsibility of the various management venues.  A centrally-operated general-purpose groups management facility (if one exists) offers access to any personal UW NetID holder, and would support delegated access for organization-oriented control.

Control of access to import provisioning interfaces in the groups registry is by arrangement with the groups registry administration.  Update access is controlled to eliminate (or at least limit) the possibility of conflict if multiple sources try to update the same group entry.

Access control is supported in directories and other export provisioning from the Registry, based on access control policies provided by groups information sources.  Per-group policies would also specify privacy controls, i.e. expectations on appropriate use of groups information by parties receiving it.

Hard Problems and Open Questions

Import/export vs remote store

The architecture suggests two means of interaction between a UI and groups store.  In one case a UI interacts with a local groups store, some portion of of which is synchronized with the groups registry later via export.  In the other case a UI does direct operations on a remote store (perhaps the registry store).  This choice affects the overall shape and behavior of the service (more distributed vs more centralized).  Apps/UI developers/deployers will need guidance in choosing between these options.

Authority management

In the proposed distributed environment many groups information stores contribute to the overall groups information space.  Some arrangement is necessary to deal with risks of multiple stores conflicting when managing particular groups.  Various options are possible:  dividing up the namespace by store to prevent conflicts; first-come first-served; a registration scheme; laissez-faire.

Multiple names and naming schemes

Given the wide variety of groups-using environments, some groups consumers will want names in particular formats.  Should these names be managed via the registry?  Does this imply conflict resolution and authority control on all possible name formats?

Inter-institutional federation and groups

Use cases for inter-institutional groups are starting to appear.  Among them is:  a multi-institutional project managing groups in in its own namespace (eg using the UW groups service, possibly in combination with other groups services at other sites.  These scenarios place added pressure on issues of import/export flow and naming authority.

  • No labels