Community Attribute Use in Internet2 CPS

	Caren Litvanyi
	lead network engineer
	peering team
	Internet2 NOC
	GigaPoP Geeks BOF
	January 2008
	Honolulu, HawaiÕi

"Quick review of what CPS..."

	Quick review of what CPS is.
	Quick notes on how to connect.
	So if IÕm connected to CPS, how can I use communities to control how my routes are advertised?
	Discussion and feedback from all you GPGÕs.

CPS Background

	CPS is Internet2Õs ÒCommercial Peering ServiceÓ.
	ÒThrough CP Service members can leverage their existing Internet2 Network investments to help serve their commercial Internet needs, thereby saving money on commodity Internet charges.Ó
	CPS is included in the base connection fee, so it is available for Internet2 Network connectors at no additional cost.
	http://www.internet2.edu/network/cp/

CPS Background

	Implemented as an ÒoverlayÓ on the existing Internet2 Layer3 Network.
	Uses MPLS Layer3 VPN, VRF on same T640 routers.
	Currently 4 commercial public peering sites:
		PAIX New York 10GE
		Equinix Chicago 10GE
		PAIX Palo Alto 10GE
		SIX (Seattle) 1GE
	Also PNI (private peerings).

Connecting to CPS - brief

	Call up the Internet2 NOC, open a ticket to connect to Internet2 CPS.
	You will need to enable 802.1Q VLAN encapsulation on your Ethernet connection to Internet2, or frame-relay encapsulation on a SONET connection.  Add an additional VLAN (or DLCI) to carry CPS traffic.
	Assign IP addresses (/30 or /31) in the usual way.  MTU is 1500.
	Supply the NOC with a list of ASs behind you, or reference an AS-SET object you maintain.
	Supply the NOC with a prefix list of what you will advertise to CPS, or agree to use your existing Internet2 prefix lists, or provide a diff.
	Set up the BGP peering - itÕs with AS 11537 (same as R&E network).
	Pad towards your direct commercial providers as desired to shift inbound traffic away from them, letting end commercial networks see the CPS path as ÒbetterÓ.
	Local-pref CPS higher, so your outbound traffic prefers CPS over your direct commodity providers, as desired.

Of course, we didnÕt coverÉ

	Analyzing your current commercial traffic patterns, if any, to get a baseline.
	Checking that your circuit to Internet2 can handle the additional load without affecting R&E traffic.
	Figuring out how, or if, this will interact with your existing bandwidth shapers, firewalls, etc., if any.
	Considering how a circuit failure to Internet2 or one of your other commercial providers will be handled.
	Figuring out how you will distribute this to downstream connectors, how/if it will be measured/chargedÉ
	Educating your downstream connectors.

But I want more control!

	We give you some!  It not perfect, but itÕs pretty easy to understand and implement.
	You can attach certain communities to your prefixes you advertise to Internet2 CPS that in turn, affects how we advertise your prefix to commercial peers. (details next slide)
	You can, if you like, configure your network policy to allow your downstreams to do this themselves.
	Combining this with adjustments to your import policy, gives you better control.
	We also support blackhole routing for up to /24s.

Using communities in CPS

	Inbound traffic, outbound route policy:
		If you do not want CPS peer network X to send traffic to you over Internet2 CPS, you can tag your prefixes with the BGP Community 65000:<foo>  where Ó<foo>" is the BGP ASN of peer network X.
		CPS has an outbound policy specific to each commercial peer that will prevent the advertisement of your prefix to it appropriately, CPS-AS<foo>-OUT.
		Note we do NOT do this Òper locationÓ, eg, Òadvertise my route to Shaw (AS6327) in New York but not SeattleÓ.

Using communities in CPS

	For example:
		Suppose your downstream customer <bar> has called you up to say they donÕt want YouTube to send traffic destined to their dorm network over CPS, ever, not even as a last resort.  Though they want other traffic to come across CPS destined to that network.
		In your BGP policy with CPS, apply policy outbound that tags that dorm network prefix (or prefixes) with 65000:36561, since 36561 is YouTubeÕs AS number.
		When CPS processes what it advertises to AS36561, it will leave out those prefixes.
		YouTube will not have a path across Internet2 CPS to your downstreamÕs dorm network.  Traffic will not come in that way.
		That dorm network prefix will still be advertised to all other CPS commercial peers.

Of course, this is not perfectÉ

	For example:
		YouTube is now moving behind the Google AS (AS15169).
		CPS has peerings with Google and with YouTube.
		Does this mean you should tag that dorm network prefix with 65000:15169 and 65000:36561?
			Maybe, Maybe not.
	Therefore, in some ways, this is better for Òtraffic engineering/balancing/managementÓ than to ÒensureÓ certain traffic doesnÕt come across CPS.
	For example, if you already have a decent path to LimeLight, and you need to keep up a certain minimum bandwidth usage, you might want to tag all your prefixes with 65000:22822 towards CPS.  So CPS will not advertise your prefixes to LimeLight at all.

But, IÕd like CPS as a backup

	Inbound traffic, outbound route policy:
		If you want this ÒgloballyÓ regarding CPS, you can of course simply pad towards CPS in hopes of influencing CPS commercial peers.
		What if you want CPS to be your primary commercial path (for the routes CPS offers), EXCEPT for traffic from peer Y -- for peer Y, you want them to send your traffic across CPS only as a Òlast resortÓ?
		The CPS outbound policy specific to each commercial peer can pad your prefix with the Internet2 AS (AS11537) one, two, or three times, whichever you choose.
		This may cause peer Y to see the path across CPS to your prefix as less desirable, leaving it as a backup.
		Note we do NOT do this Òper locationÓ.

Using communities in CPS

	Specifically, if you want traffic from peer network Y to prefer a different path, but want to use the Internet2 CPS path as a backup, you can tag your prefixes with 65001:<bar>, 65002:<bar>, or 65003:<bar> where Ó<bar>" is the BGP ASN of network Y.
	These communities will cause Internet2 to pad the AS-PATH towards peer network Y 1, 2 or 3 times respectively (using AS11537 for the pad) for those prefixes you tag.

Using communities in CPS

	For example:
		Suppose you see traffic from Akamai is preferring Internet2 CPS.  For whatever reason, youÕd rather they get to you over a different path, leaving the CPS path as a backup.
		In your BGP policy with CPS, apply policy outbound that tags your prefixes with 65001:20940, since 20940 is AkamaiÕs AS number.
		When CPS processes what it advertises to AS20940, it will pad your prefixes with one additional Ò11537Ó in the AS-PATH.
		If you find that is not enough to influence the inbound traffic from Akamai, you can successively try 65002:20940 and 65003:20940.
		Your prefixes will not be padded towards any other CPS peers.
		Of course, this is not perfect either, and additionally peers may not be letting decisions fall to AS-PATH length for their own reasons.

Using communities in CPS

	And thereÕs always the blackhole community:
		We check itÕs your prefix first of course.
		only allowed /32 to /24.
		11537:911.
		sets next-hop to discard

Discussion

	Comments?
	Suggestions?

Thank you!

	http://noc.net.internet2.edu/i2network/commercial-peering-service.html
	http://www.internet2.edu/network/cp/
	litvanyi@grnoc.iu.edu
	network@internet2.edu