[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: IPSec still too slow?



"inaccurate" is ... well, "inaccurate."  The numbers are perfectly
correct and represent the actual thruput of the 7140 as configured by an
experienced Cisco-employed engineer at our site.  There is no
inaccuracy. 

It may possibly be true that there are other configurations of the
device which will present different thruput numbers, both faster and
slower.  However, the numbers in that article are by no means
inaccurate.  

It is perhaps a reaffirmation of my point about the complexity and
difficulty of configuring and managing Cisco VPN configurations that the
device, as it comes out of the box, does not offer the end user optimal
performance. 

In short, the numbers are perfectly correct and accurate; they represent
a perfectly correct and valid configuration. 

It may also be true that there are other perfectly correct and valid
configurations which offer greater thruput and have the same security
parameters and functionality. 

The point of the very basic raw testing done in that article was to
offer a baseline apples-to-apples comparison which would let buyers
scale whatever the vendors actually claim versus reality in order to
compare real system thruput.  By offering a baseline from which vendor
numbers can be scaled up or down, all the vendors who participated have
at least put themselves on an equal footing. 

For example, Cisco commonly promises lower throughput than their devices
are capable of doing, probably as a conservative approach to network
engineering and management.  Other vendors offer the highest possible
numbers to make themselves look good.  With the research in that article
at hand, a buyer can properly scale things so that different vendor
performance numbers can at least be put into the same ballpark of
relevence.  

I agree totally that raw throughput numbers are only a part of the whole
picture and that other features of the product will vary performance. 
This is why end user organizations must test products _in situ_ to see
real performance.  Given that of the 13 products tested no two have the
same functionality, it is simply impossible to come up with any more
complex benchmark that is applicable across more than two or three
products.   

jms



Natalie Timms wrote:
> 
> I just wanted to let you all know (sorry to those not particularly interested), that the Cisco 7140 numbers are inaccurate due to a miss-configuration of the device. The 42 Mb/sec quoted is less than half of what this device is capable of. Although this much throughput running process switching only actually rocks, its not what we'd be recommending to customers!
> 
> In our own internal testing we've found that raw throughput tests are somewhat meaningless as they are not indicative of the real world. It would be great to see stats based on other things running on the box - firewalling, NAT, keepalives, routing, etc.....
> 
> -Natalie
> 
> At 09:25 PM 10/5/2001 -0700, Alex Alten wrote:
> 
> >Joel are you willing to post all the raw test results to the IPSEC
> >mailing list?  It's really hard to understand what the numbers really
> >mean in your article.  I for one wouldn't mind if you change vendor/product
> >names to something like vendor A/product X (although some indication of
> >processor speed would help here).
> >
> >What I'm looking for is raw throughput (various packet sizes), latency (SA
> >setup costs), error rates (%dropped packets, lost fragments), timing delays.
> >
> >BTW, did you test with various legacy protocol/apps?  For example did anything
> >break if they didn't get the expected TCP segement handed to them in the app
> >layer?
> >
> >- Alex
> >
> >
> >At 05:20 PM 10/5/2001 -0700, Joel M Snyder wrote:
> >>Alex has misinterpreted what I wrote.  The performance drop I
> >>saw was exclusively with the Microsoft implementation.  I only
> >>brought that statistic out because they have such incredibly good
> >>performance (with $90 Intel NICs) with small numbers of SAs.
> >>
> >>I did not test all the products in that review with hundreds of
> >>SAs, but I have tested Cisco, Nokia, and Netscreen and seen
> >>negligible performance degradation as the size of the SPD/SAD
> >>tables get large.  In this case, I think that we're seeing an
> >>implementation issue with the card---it's just not designed to handle
> >>hundreds of SAs (on the other hand, it costs less than $100, so
> >>it's really an incredible solution for some branch office environments).
> >>
> >>In general, IPSEC performance in labs looks great because we don't
> >>cause fragmentation.  When fragmentation rears its ugly head, performance
> >>can go to hell (or even cause connectivity failure) very quickly. Or not,
> >>depending on the design of your application.  Encryption performance is
> >>generally not the problem.
> >>
> >>I do get pissed off at people who throw around latency claims, though.
> >>One major firewall vendor (who should remain nameless) claims that one
> >>of the big advantages of co-locating the IPSEC and firewall function
> >>inside of the same box is that two boxes add too much latency.  In the
> >>lab, even with the slowest and stupidest of VPN products, I rarely see
> >>more than 1ms latency---completely indistinguishable across the general
> >>purpose Internet.
> >>
> >>http://www.nwfusion.com/reviews/2001/1001rev.html
> >>
> >>jms
> >>
> >>
> >>Joel M Snyder, 1404 East Lind Road, Tucson, AZ, 85719
> >>Phone: +1 520 324 0494 x101 (v) +1 520 324 0495 (FAX)
> >>jms@Opus1.COM    http://www.opus1.com/jms    Opus One
> >>
> >--
> >
> >Alex Alten
> >Alten@Home.Com
> >
> >***********************
> >* Just Say NO to RC4. *
> >***********************
> 
> --------------------------------------------------------------------------------------
> Natalie Timms                   Ph Office : 408 527 5114
> Technical Leader                             Email : ntimms@cisco.com
> Strategic VPN Projects
> VSEC BU                         Pager : 1 800 365 4578 or
> Cisco Systems                   ntimms@epage.cisco.com
> ---------------------------------------------------------------------------------------

--
Joel M Snyder, 1404 East Lind Road, Tucson, AZ, 85719
+1 520 324 0494 x101 (voice)    +1 520 324 0495 (FAX)
jms@Opus1.COM    http://www.opus1.com/jms    Opus One
Electronic mail is always the best way to contact me.


Follow-Ups: References: