[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Closing out the COMPRESSION discussion
All,
First, thanks to all who participated in the compression discussion. The
debate has been lively and useful. I wanted to make sure that the straw
ballot results were available to the list prior to the 3/26 cutoff for
drafts to be released in time for Memphis.
The results of the straw poll indicated: (1) a significant majority of
those who participated in the discussion (there were around 30 total
participants) believe that adding compression as an optional feature to ESP
is a good idea, and (2) of those in that majority, most felt that using the
most-significant bit of the pad length byte as a compressed/not-compressed
indicator was an appropriate technique.
There were a number of concerns raised on the list and I would like to
close out the discussion with some additional insights on these points.
1. Compression belongs at a higher layer than IP
There are a handful of people who feel this way. Some responded to this (as
I did) by stating that there was no universally appropriate higher layer in
which to do the compression. This is indeed true. However, it is certainly
true that if compression was added to a higher layer protocol, say TCP, one
would have the potential advantage of reducing the number of IP packets as
well maintaining compression state across segments. The result would be a
higher compression ratio for the same type of data. While adding
compression to such a higher layer protocol has its advatnages for some
environments, there is *NO* need for it to preclude the OPTIONAL use of
compression in IPSec, where there *IS* gain to be had (leading to the next
point).
2. There's not enough to gain when compressing packet-by-packet
As most of you know (hopefully all of you), the amount of compression you
get depends on the type of data being compressed. I first responded to this
by providing some compression results based on using the LZS algorithm on
some publically available test data. The response to this was "that's not
*real* network data". Since that time, I have had a number of discussions
with people in the networking industry regarding what *real* network data
is. The reality is that one person's *real* network data is another
person's meaningless data. You cannot simply sniff someone's T1 connection
to the internet to get *real* network data. In the context of IPSec, I
would imagine that network data that will be encrypted will look more like
today's LAN traffic (currently privately held data that will now, under
IPSec, be exposed to the internet). Also, as pointed out by Terry Davis
from Boeing, "CAD, GIFs, or other barely compressable objects" will not
compress well, but "email or a few thousand pages of maintenance manual"
can achieve significant gains by compressing.
A second response on this point came from Jim Hughes, who recently stated
that "We also have numbers on the compression ratios in real world, long
run time situations, and even clearing the dictionary on every packet, we
average 2:1 compression of data. Even on most small packets, the overhead
of the encapsulationis removed by what compression there is."
Similarly, a Q&A doc from VPNet states "VPNet's data compression technology
is oriented around a per-packet compensation for security-induced packet
expansion. In current network tests, the performance improvement for very
small packets is small, but for larger packets, network performance gains
are substantial. Improvements between 10 and 40 percent are typical, though
for packets containing highly compressible data, improvements of up to 70
percent are possible."
3. Using the most significant bit (msb) of the pad length will cause
compatibility problems
The proposal involves using the msb of the pad length field to indicate
whether or not a packet is compressed. Note that the use of compression is
intended to be negotiated on a per SA basis at by the key management
protocol. Any current implementation of ESP that does not wish to support
the OPTIONAL compression feature is free not to and will suffer no
compatibility problems when interoperating with those implementations that
support the compression feature. Current implementations will NEVER
negotiate to turn on the compression feature. As a result, compressed data
will NEVER be sent to them. As such, the msb can continue to be interpreted
to mean that there are 128 additional bytes of padding. Implementations
that DO support the compression feature will use a maximum of 127 bytes of
padding, leaving the msb to indicate compressed/not-compressed.
With that said, the working group should expect the next draft of the ESP
specification to include compression as an optional feature of ESP
(negotiated by the KMP) with the msb of the pad length field used as the
packet compressed/not-compressed indicator.
-Bob
Follow-Ups: