Per-Entity Metadata: Tastes Great. Less Filling.
The InCommon Federation started in 2004 with just 10 university participants. From those humble beginnings, InCommon has grown to 957 participants across higher education, government and nonprofit research organizations, and sponsored partners. InCommon participants have registered more than 4,500 SAML entities, and the metadata aggregate (representing not just InCommon, but also our partner federations around the world through eduGAIN) has grown to more than 40 megabytes (or nearly 700,000 lines, if you’re into counting carriage returns).
Regularly downloading, processing, and verifying the integrity of the metadata aggregate takes some non-trivial computing resources: network bandwidth, disk storage, processor cycles, and RAM. As the metadata aggregate has grown, so have the resource demands, and with them the need for a more scalable metadata distribution and consumption solution. Two InCommon community working groups—the Metadata Distribution Working Group, and later the Per-Entity Metadata Working Group—tackled the scalability challenges of metadata distribution and consumption. Thanks to their efforts, the community identified and later piloted a solution called the Metadata Query (MDQ) protocol.
Using the MDQ protocol, participating Identity Providers and Service Providers can request and process metadata for just the specific entities they need (“per-entity metadata”) instead of the entire metadata aggregate. The result is a dramatic reduction in the computing resources needed to consume InCommon metadata.
On the recommendation of the Final Report of the Per-Entity Metadata Working Group, InCommon started a project to implement a fully-automated Per-Entity Metadata service based on the MDQ protocol. This is a complex project involving changes to all aspects of InCommon’s metadata processes, from generation and aggregation, to signing and distribution. Most of the changes will happen behind the scenes and should be mostly invisible to participants. However, we do expect to introduce a new signing key and hostname for the Per-Entity Metadata Service, requiring participants to make adjustments to their Identity Provider and Service Provider configurations. Over the next several weeks, we’ll share more detailed information about the project, the deployment schedule, and updated documentation to help you get ready. In the meantime, please feel free to contact us with any questions or feedback you might have.
Thanks to the working group members, and to the entire InCommon community. This project is made possible by your generous help and support, and I’m looking forward to meeting many of you at the 2017 Technology Exchange, October 15–18 in San Francisco!