The use of an LDAP directory service as a groups registry limits our present groups service (GWS). One proposed improvement is a new, locally developed registry using a PostgreSQL database. The GWS webservice can work directly with this registry and, by automatic processes, feed the present LDAP directories, which remain unchanged. As the GWS webservice and the LDAP directory service continue to present the same APIs as before this approach has the advantage that no existing clients would be interrupted or affected. As a second, though some might say first, approach the Architect suggests instead that this is a good time to install grouper as the engine for our group registry.
The UW Groups Service consists of a Groups Directory (GDS), a webservice (GWS) providing a user interface for browser clients and a RESTful API for programmed clients, and several automatic feeds of group memberships from student services and administrative sources.
The directory, a cluster of openldap services, provides:
The webservice, directly using the directory as a registry, provides in addition:
Automatic, generally nightly, feeds of group information include:
Grouper is a group management toolkit funded by the NSF Middleware Initiative. It is a sibling project to Shibboleth, although it does not enjoy the same level of developer support and lags behind in its installed base. Grouper provides a user interface, a command line shell, a java API and a webservice API. The shell includes import and export tools and an mechanism to provision an LDAP directory.
The migration plan assumes that we want to keep the user interface of the groups service as much as possible unchanged.
In order to understand the capabilities and challenges of grouper we have installed a working service, complete with the 'u_' and 'uw_' groups from the present GDS (as of 04/02/09). Anyone with access to the existing GWS UI can access and enjoy the corresponding UW Grouper user interface.
For the demonstration site, and probably for any production installation, we provide a fewextensions to the distributed code.
Grouper allows subjects (members, etc.) to come from various sources, one of which is an LDAP directory. Thus we can work with anyone in our PDS directory. However, the LDAP subject provider used by the distribution, the standard JNDI from Sun Microsystems, does not work efficiently with a directory that requires SASL EXTERNAL authentication. In particular it provides neither continuous connections nor connection pooling. We implemented a new subject provider using the ldap library from Virginia Tech. This is also the library used by Shibboleth.
Grouper uses the REMOTE_USER environment variable as the browser user's identity. We are protecting our site with Shibboleth, as that seems likely the mechanism we would use in a production setup. Shibboleth REMOTE_USER values include a domain specification, e.g., "@washington.edu". Our custom subject provider trims these suffixes to match our PDS directlory entries.
We also added a sorting and searching features to the provider.
If we want to allow non-uwnetid membership we will have to come up with an ePPN subject provider. That has not been done.
Our translation tool from GDS to Grouper utilizes the apparent hierarchy of GDS to create a real hierarchy within Grouper.
The translations of 'Departments.C&C.' and 'Admin.' to their 'u_' names are gratuitous and need not be carried forward to a production system.
This export-import operation may have lost a group or member or two. My goal was to capture most groups and members and admins. Any losses are being investigated, but any delay not inhibit other testing of Grouper.
A RESTful API, matching the existing API anu UI of GWS will be developed. There are some issues:
Grouper has an import tool (xmlImport) that can accept group data in a similar format to present day's tegea imported of GWS. However, it may be too slow to be directly useful. Timing test show it to import new groups at a rate of about 500 subjects per minute. It imports existing groups by first deleting the old and then importing the new, at a rate closer to 300 subjects per minute. That's too many hours for a nightly reconciliation of several hundred thousand subjects.
We can write a reconciliation tool that works more efficiently.
As a demonstration and test we ran a couple of trials on a large group (u_subman_ezproxy, 100K members).
Grouper includes provisioning tools that should be able to keep an LDAP directory (or directories) up-to-date. I haven't yet been able to make that work. Expect it to be not too difficult though. Some issues to consider:
