Monday, June 28, 2010

P802.3ba 40 and 100 Gigabit Ethernet Standard : great ideas, but

P802.3ba 40 and 100 Gigabit Ethernet Standard has been ratified. This is great for data centers world wide who need to shove the equivalent of over 2 encyclopaedia Britanica's (which fits on a DVD, so assume 5GB for the E.B.) per second between systems.

So lets assume that the MTU remains at 1500 bytes, we are talking about 8 millions packets a second being processed on each side. The most I have seen a modern machine do for full default MTU sized packets is about 80,000 and that used multiple cpu's on the receive side. Jumbo frames might reduce the number of packets to around 1 million. The thing is that CPU's are not getting that much faster, they are getting far more cores on the processor which leads me to think 2 things will have to happen before 100 gigabit Ethernet become viable :-

  1. Most/all the TCP and IP processing will need to be done in hardware on the NIC
  2. A lot of TCP stacks will have to be re-written to become far more multi-threaded
Network code is fragile and subtle by its nature, so expect this new wave of network stacks to have new security vulnerabilities and reliability anti-features which existing stacks don't have.

It does make me wonder if TCP is the right protocol to be running in the data centre on to of 40 or 100G?

No comments:

Post a Comment