Lolo is a service of UW-IT, developed in collaboration with the UW eScience Institute, providing scalable storage for the UW research community.
Lolo offers two types of service: a shared storage offering known within lolo as "Collaboration" for on-disk storage and exchange of large datasets; and an offering known as "Archive" for long-term storage of large datasets. The two services are described in the UW-IT Service Catalog: Archive Storage Service for Research and Shared Central File System for Research.
The lolo-announce mailman list is a low-volume list used to communicate service changes, maintenance or outages. Subscribe to stay up to date on lolo.
If you have any questions not addressed in this documentation or are having problems with the service, please contact us by sending an e-mail to firstname.lastname@example.org with 'lolo' as the first word in the subject and a brief description of your issue.
Accessing and Using Lolo
Appropriate Use and Warnings
The Collaboration filesystem is for general purpose file storage and access. It is not backed up. Potential uses for this filesystem include sharing data with colleagues on and off campus, and staging data for later transfer to Hyak or other sites.
The Archive filesystem is intended for long term storage of data that changes infrequently, if ever. It is a tape-based system with a small disk buffer to improve write performance. It should NEVER be used for ANY sort of interactive access.
Both filesystems will perform best when used with large files. For this reason, we strongly encourage you to combine collections of small files into tar files before transferring them to the Archive filesystem. To enforce this, inode quotas are in place which limit the number of files that you can store. (See Quotas.) Please be aware of these limits.
The lolo Collaboration filesystem is accessible to Windows, Mac and Unix/Linux users on campus via CIFS (SMB). The Archive filesystem is not CIFS accessible. There is no CIFS access from off-campus.
The fileserver hostname for CIFS mounts is lolo.washington.edu, share name collaboration. Authentication is via username NETID\your_UWNetID and your UWNetID password (i.e. if your UW NetID is jsmith01, enter NETID\jsmith01). If you are logged in to a Windows box as NETID\your_UWNetID, no additional authentication is performed at share mount time. Information for mapping a network drive can be found here http://windows.microsoft.com/en-us/windows-8/create-shortcut-to-map-network-drive.
To connect from a Mac, select Connect to Server from the Go pulldown on the main Finder menu bar, and enter "smb://lolo.washington.edu" in the Server Address field.
The lolo Collaboration and Archive filesystems are available from on-campus or off-campus with SSH or HPNSSH. This includes the ssh, scp and sftp protocols. BBCP and GridFTP are also available via SSH for high speed transfers.
Finding your Data
The lolo data is stored in two different directory structures:
/archive/group-namefor Archive storage
/collaboration/group-namefor Collaboration storage
group-name is the "short" name chosen for your group at account creation time. Do not use your home directory to store files beyond those necessary for your account to function (ssh keys, etc.).
An example command to connect to lolo with ssh:
Transferring files using rsync over ssh is a supported option. There is one important caveat: when using rsync you must always use the -W or --whole-file option. This option disables the rsync transfer checksum algorithm which normally would speed the transfer of changed files by only sending changed bytes. On lolo, this algorithm will cause a tape recall of every file that already exists in order to have the file contents available to calculate the transfer checksum. This recall would be detrimental to lolo archive function and will result in poor performance of transfers. The -W or --whole-file option must always be used.
Using rsync has the benefit of ensuring integrity of transferred files. When rsync transfers a file it always calculates a checksum for the whole file and compares at the completion of the transfer. This works even in whole file (-W, --whole-file) mode.
GridFTP uses SSH for authentication, so you should use the proper syntax for such transfers, i.e. globus-url-copy -vb file:/data/mylocalfile sshftp://lolo.washington.edu/archive/mygroup
The Collaboration filesystem is mounted at /lolo/collaboration and the Archive filesystem at /lolo/archive on the Hyak login nodes. Hyak users have access to an allocation of lolo storage dedicated to Hyak customers. Each hyak group has a directory inside the 'hyak' directory for their use. (See the Hyak User Wiki for more details.) In addition, any Collaboration or Archive storage purchased outside of this Hyak allocation is accessible the same way.
On the Hyak login nodes, normal Unix/Linux commands can be used to move data in and out of the lolo filesystems.
Lolo is implemented with a single large GPFS filesystem for each of Collaboration and Archive. Each Lolo customer purchases storage in chunks of 1TB and this allocation is implemented and enforced as a filesystem byte quota. Byte quotas are set in multiples of 1.0TB.
Both filesystems will perform best when used with large files. To enforce this, inode quotas are in place which limit the number of files that you can store. In Collaboration an 1TB chunk allows 100,000 files and in Archive an 1TB chunk allows 1,000 files. To comply with these file/directory (inode) quotas we recommend that users combine collections of small files into tar (or other archive format) files before transferring them to the Archive filesystem.
The quota and usage for your lolo storage can be viewed by logging in to lolo and viewing the 'usage_report.txt' file in the root of your Lolo allocation. These files are updated hourly.
Viewing Hyak Quota
Hyak users have access to an allocation of lolo storage dedicated to Hyak customers. This allocation is based on number of Hyak nodes. See the Hyak User Wiki for more details.
Hyak users can check their usage of the lolo allocation from either Hyak or lolo with the mmlsquota command:
Problems and Solutions
If you receive a Permission Denied error when reading a file to which you have permission, this means the HSM software is having trouble reading your file from the primary tape copy and your file has to be restored from the secondary tape copy. An administrator has to perform this procedure, so see the Getting Help section above for information on how to contact support.
Account and Group Maintenance - Information for designated customer contacts to manage access to Lolo.