Major inbound traffic control problem in real ISP market

Tweet

If you have more than two upstream ISP connections, traffic shaping would be one of major consideration for network administrator. Especially, a company has one big pipe with Tier1 ISP and mid-size connection with Tier2 ISP for redundant purpose. Beside, those ISPs don’t have any peer connection between. See below picture 1 to better understand.
In general, the company wants to use all outbound / inbound traffic on connection with Tier1 ISP. That’s why they are paying big bucks. It’s normal and understandable. However, unexpected happening is happening, mid-size secondary uplink pull(?) some traffic, even though they put higher preference on main connection???

Why? see below There would be few reasons, but main reason is Tier2 ISP’s routing policy. Most Tier2 ISP has many upstream connections as CUSTOMER BASED.

They are fully using a previllage received from upstream provider as customer. More of Tier1 ISPs are providing BGP community strings, and it allows customer to control their inbound traffic within their backbone.

 

 

For example, if ISP(Tier2) are announcing their customer’s IP block with local pref 120 or higher to their upstream providers(in this case ISP_"B", Tier1), the ISP_"B" will choose a path to ISP(Tier2) as the best path for all the block which was tagged with local pref 120 or higher. As we know, local preference won’t be announced thru eBGP peers. Trying AS-path prepending? Even thought end-user prepending AS on their announcement to ISP(Tier2) session, It won’t help due to local-pref attribute value is more prefer than AS prepending attribute.

Tier 2 ISP will pull most of traffic and bring customer’s redundant circuit which end customer doesn’t want.

 

Why they do that? Simple answer would be because of their business. In these days, a lot of competition on ISP / Internet connection market and it leds to worse. One of competitive product in tier2 ISP world is a traffic usage based product. In another words, they only charge how much customer use bandwidth per month or certain period. Sounds cool and fair, but like I mention above example, they will suck all the traffic and push up their subscribed line and charge more. End user might ask them why most traffic route to your (Tier 2) backbone? They would say "’our connection is fast and well optimized, fully covered MPLS netwrok, we care of small company’s traffic not like big ISPs ………..". We know it is not always true.

Most of Tier 1 ISP applies lower local pref for routes which is announcing from their private peer. It means they don’t want to leave customer’s traffic, if the customer has a connection with them. For the business, I understand(trying to).

Only ISP / Verizon Business (Former UUNET) I know doesn’t have any preference on in / outbound traffic between peers. It would be not a best (in these days) for them, but at least good for end users.
I like old UUNET’s routing policy. Verizon Business is still keeping their unique routing policy which let BGP algorithm decides the best path and makes customer easy to control their traffic after it left their network. Of course they will control IGP metric to internal traffic shape, we don’t want nose into their business too deep. If ISPs start modifying a route preference to take advantage on other ISPs, it will eventually bring a disadvantage for all the customers. I hope no more dirty game with pure customer’s traffic.

 

 

Solution:
1. If your ISP (Tier 2) provides a BGP community, use it to prevent the hassle. See our ALL BGP community strings of most ISP in the world.
2. If your ISP(Tier 2) provider doesn’t help you at all, use Conditional BGP announcement. It will give you a bit flexibility. See our other article for "Conditional BGP announcement" on our BGP sample page.

 

I hope this is informative for you.