Skip to end of metadata
Go to start of metadata


This document describes a method to configure a Shibboleth Service Provider (SP) to to fetch metadata only for specific IdPs as needed instead of periodically loading the entire InCommon "idp-only" aggregate. This new method is referred to as a per-entity metadata service or MDQ (since it is based on a "Metadata Query" protocol).


Traditionally, federated SPs that relied on InCommon Federation metadata would periodically download a large metadata aggregation file containing all InCommon entities (IdPs and SPs). This quickly grew unwieldy as the federation grew, and soon it was causing very slow start-up times and a large memory footprint for SPs. As an interim measure, InCommon began publishing an "IdP-only" metadata aggregate that was much smaller in size and that was less impactful on SPs. That approach also became unwieldy over time.

Now there is a new option called the InCommon Per-Entity Metadata Distribution Service. This service is analogous to DNS, but instead of looking up an IP address by hostname as needed and caching the result, it looks up InCommon metadata by entityID as needed and caches the result. The per-entity metadata service is also known as MDQ, after the IETF "Metadata Query" protocol that it uses. With MDQ, an SP only looks up IdP metadata when it needs to and it never loads metadata it doesn't need. There is no file to download and verify on restart and no large files to store on disk. If the MDQ service is not available, the SP will use a local copy of IdP metadata that it cached after previous successful MDQ queries.

For additional background please see:


1. Add the MetadataProvider element

Shibboleth SP v3

  • Open your shibboleth2.xml file for editing.
  • Find the section with MetadataProvider elements.
  • Add the current MetadataProvider with the following:

<!-- InCommon Per-Entity Metadata Distribution Service -->
<MetadataProvider type="MDQ" id="incommon" ignoreTransport="true" cacheDirectory="inc-mdq-cache" 
    maxCacheDuration="86400" minCacheDuration="60"
   <MetadataFilter type="Signature" certificate="inc-md-cert-mdq.pem"/>
   <MetadataFilter type="RequireValidUntil" maxValidityInterval="1209600"/>
  • It is recommend that you enable a metadata cache duration of at least one hour, but no longer than one day, in your Shibboleth SP.
  • The first MetadataFilter element requires that the signature on the MDQ metadata provider should be verified using the inc-md-cert-mdq.pem certificate.
  • If you have configured a MetadataProvider for the UW IdP or InCommon metadata aggregate, you should comment it out or delete it.
  • Save your shibboleth2.xml file.

Configuring with multiple metadata providers

If you have more than one metadata provider configured (for example, for IdPs that are not part of InCommon), you will want to put the InCommon Per-Entity Metadata Distribution Service after any other metadata providers. Otherwise, your SP will needlessly attempt to use MDQ to fetch IdP metadata from InCommon before falling back to the correct metadata provider.

Shibboleth SP v2

Shibboleth SP v2 is no longer supported

If you have not upgraded to Shibboleth SP v3 you should do so as soon as possible. Shibboleth SP v2 has already reached end of life and is no longer supported by the Shibboleth Consortium.

The per-entity metadata service works with Shibboleth v2 but there are some limitations:

  • You will need at least Shibboleth SP v2.1 to specify a maximum Cache Duration.
  • You will need at least Shibboleth SP v2.4 to specify a minimum cache duration.

Instructions for SP v2 configuration are the same as for SP v3, except the MetadataProvider section is a little different:

<!-- InCommon Per-Entity Metadata Distribution Service -->
<MetadataProvider type="Dynamic" ignoreTransport="true" maxCacheDuration="86400" minCacheDuration="60">
    <MetadataFilter type="RequireValidUntil" maxValidityInterval="1209600"/>
    <MetadataFilter type="Signature" certificate="inc-md-cert-mdq.pem"/>

2. Install the inc-md-cert-mdq signing certificate

  • Download the MDQ signing certificate.
  • Save the certificate file in the same directory as your shibboleth2.xml file. Name it "inc-md-cert-mdq.pem".

3. Restart the shibd process

  • Restart the shibd process on your SP however you normally do that on your platform.
  • This will cause your SP to re-read the metadata provider configuration in shibboleth2.xml and to take appropriate actions. 

4. Verify success in the logs

  • Actions taken during the restart of shibd are recorded in shibd.log.
  • Open the log file and scroll to near the bottom of the file to find messages from the shibd restart.
  • Look for messages like the following (numbering added for clarity, these are not present in the log file) to verify MDQ was configured on restart:

    1. 2019-12-27 11:32:16 INFO OpenSAML.MetadataProvider.Chaining : building MetadataProvider of type MDQ
    2. 2019-12-27 11:32:16 INFO OpenSAML.MetadataProvider : building MetadataFilter of type Signature
    3. 2019-12-27 11:32:16 INFO XMLTooling.SecurityHelper : loading certificate(s) from file (C:/opt/shibboleth-sp/etc/shibboleth/incommon-mdq.pem)
    4. 2019-12-27 11:32:16 INFO XMLTooling.CredentialResolver.File : no private key resolved, usable for verification/trust only
    5. 2019-12-27 11:32:17 INFO OpenSAML.MetadataProvider.Dynamic [incommon]: cleanup thread started...running every 1800 seconds

  • No labels