mirror of
https://github.com/ApfelTeeSaft/reactos.git
synced 2026-08-29 19:26:33 +00:00
[BZIP2]
* Cleanup this folder. svn path=/trunk/; revision=63294
This commit is contained in:
Amine Khaldi
-59
@@ -1,59 +0,0 @@
|
||||
|
||||
# This Makefile builds a shared version of the library,
|
||||
# libbz2.so.1.0.6, with soname libbz2.so.1.0,
|
||||
# at least on x86-Linux (RedHat 7.2),
|
||||
# with gcc-2.96 20000731 (Red Hat Linux 7.1 2.96-98).
|
||||
# Please see the README file for some important info
|
||||
# about building the library like this.
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# This file is part of bzip2/libbzip2, a program and library for
|
||||
# lossless, block-sorting data compression.
|
||||
#
|
||||
# bzip2/libbzip2 version 1.0.6 of 6 September 2010
|
||||
# Copyright (C) 1996-2010 Julian Seward <[email protected]>
|
||||
#
|
||||
# Please read the WARNING, DISCLAIMER and PATENTS sections in the
|
||||
# README file.
|
||||
#
|
||||
# This program is released under the terms of the license contained
|
||||
# in the file LICENSE.
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
|
||||
SHELL=/bin/sh
|
||||
CC=gcc
|
||||
BIGFILES=-D_FILE_OFFSET_BITS=64
|
||||
CFLAGS=-fpic -fPIC -Wall -Winline -O2 -g $(BIGFILES)
|
||||
|
||||
OBJS= blocksort.o \
|
||||
huffman.o \
|
||||
crctable.o \
|
||||
randtable.o \
|
||||
compress.o \
|
||||
decompress.o \
|
||||
bzlib.o
|
||||
|
||||
all: $(OBJS)
|
||||
$(CC) -shared -Wl,-soname -Wl,libbz2.so.1.0 -o libbz2.so.1.0.6 $(OBJS)
|
||||
$(CC) $(CFLAGS) -o bzip2-shared bzip2.c libbz2.so.1.0.6
|
||||
rm -f libbz2.so.1.0
|
||||
ln -s libbz2.so.1.0.6 libbz2.so.1.0
|
||||
|
||||
clean:
|
||||
rm -f $(OBJS) bzip2.o libbz2.so.1.0.6 libbz2.so.1.0 bzip2-shared
|
||||
|
||||
blocksort.o: blocksort.c
|
||||
$(CC) $(CFLAGS) -c blocksort.c
|
||||
huffman.o: huffman.c
|
||||
$(CC) $(CFLAGS) -c huffman.c
|
||||
crctable.o: crctable.c
|
||||
$(CC) $(CFLAGS) -c crctable.c
|
||||
randtable.o: randtable.c
|
||||
$(CC) $(CFLAGS) -c randtable.c
|
||||
compress.o: compress.c
|
||||
$(CC) $(CFLAGS) -c compress.c
|
||||
decompress.o: decompress.c
|
||||
$(CC) $(CFLAGS) -c decompress.c
|
||||
bzlib.o: bzlib.c
|
||||
$(CC) $(CFLAGS) -c bzlib.c
|
||||
@@ -1,58 +0,0 @@
|
||||
------------------------------------------------------------------
|
||||
This file is part of bzip2/libbzip2, a program and library for
|
||||
lossless, block-sorting data compression.
|
||||
|
||||
bzip2/libbzip2 version 1.0.6 of 6 September 2010
|
||||
Copyright (C) 1996-2010 Julian Seward <[email protected]>
|
||||
|
||||
Please read the WARNING, DISCLAIMER and PATENTS sections in the
|
||||
README file.
|
||||
|
||||
This program is released under the terms of the license contained
|
||||
in the file LICENSE.
|
||||
------------------------------------------------------------------
|
||||
|
||||
bzip2-1.0.6 should compile without problems on the vast majority of
|
||||
platforms. Using the supplied Makefile, I've built and tested it
|
||||
myself for x86-linux and amd64-linux. With makefile.msc, Visual C++
|
||||
6.0 and nmake, you can build a native Win32 version too. Large file
|
||||
support seems to work correctly on at least on amd64-linux.
|
||||
|
||||
When I say "large file" I mean a file of size 2,147,483,648 (2^31)
|
||||
bytes or above. Many older OSs can't handle files above this size,
|
||||
but many newer ones can. Large files are pretty huge -- most files
|
||||
you'll encounter are not Large Files.
|
||||
|
||||
Early versions of bzip2 (0.1, 0.9.0, 0.9.5) compiled on a wide variety
|
||||
of platforms without difficulty, and I hope this version will continue
|
||||
in that tradition. However, in order to support large files, I've had
|
||||
to include the define -D_FILE_OFFSET_BITS=64 in the Makefile. This
|
||||
can cause problems.
|
||||
|
||||
The technique of adding -D_FILE_OFFSET_BITS=64 to get large file
|
||||
support is, as far as I know, the Recommended Way to get correct large
|
||||
file support. For more details, see the Large File Support
|
||||
Specification, published by the Large File Summit, at
|
||||
|
||||
http://ftp.sas.com/standards/large.file
|
||||
|
||||
As a general comment, if you get compilation errors which you think
|
||||
are related to large file support, try removing the above define from
|
||||
the Makefile, ie, delete the line
|
||||
|
||||
BIGFILES=-D_FILE_OFFSET_BITS=64
|
||||
|
||||
from the Makefile, and do 'make clean ; make'. This will give you a
|
||||
version of bzip2 without large file support, which, for most
|
||||
applications, is probably not a problem.
|
||||
|
||||
Alternatively, try some of the platform-specific hints listed below.
|
||||
|
||||
You can use the spewG.c program to generate huge files to test bzip2's
|
||||
large file support, if you are feeling paranoid. Be aware though that
|
||||
any compilation problems which affect bzip2 will also affect spewG.c,
|
||||
alas.
|
||||
|
||||
AIX: I have reports that for large file support, you need to specify
|
||||
-D_LARGE_FILES rather than -D_FILE_OFFSET_BITS=64. I have not tested
|
||||
this myself.
|
||||
Vendored
-454
@@ -1,454 +0,0 @@
|
||||
.PU
|
||||
.TH bzip2 1
|
||||
.SH NAME
|
||||
bzip2, bunzip2 \- a block-sorting file compressor, v1.0.6
|
||||
.br
|
||||
bzcat \- decompresses files to stdout
|
||||
.br
|
||||
bzip2recover \- recovers data from damaged bzip2 files
|
||||
|
||||
.SH SYNOPSIS
|
||||
.ll +8
|
||||
.B bzip2
|
||||
.RB [ " \-cdfkqstvzVL123456789 " ]
|
||||
[
|
||||
.I "filenames \&..."
|
||||
]
|
||||
.ll -8
|
||||
.br
|
||||
.B bunzip2
|
||||
.RB [ " \-fkvsVL " ]
|
||||
[
|
||||
.I "filenames \&..."
|
||||
]
|
||||
.br
|
||||
.B bzcat
|
||||
.RB [ " \-s " ]
|
||||
[
|
||||
.I "filenames \&..."
|
||||
]
|
||||
.br
|
||||
.B bzip2recover
|
||||
.I "filename"
|
||||
|
||||
.SH DESCRIPTION
|
||||
.I bzip2
|
||||
compresses files using the Burrows-Wheeler block sorting
|
||||
text compression algorithm, and Huffman coding. Compression is
|
||||
generally considerably better than that achieved by more conventional
|
||||
LZ77/LZ78-based compressors, and approaches the performance of the PPM
|
||||
family of statistical compressors.
|
||||
|
||||
The command-line options are deliberately very similar to
|
||||
those of
|
||||
.I GNU gzip,
|
||||
but they are not identical.
|
||||
|
||||
.I bzip2
|
||||
expects a list of file names to accompany the
|
||||
command-line flags. Each file is replaced by a compressed version of
|
||||
itself, with the name "original_name.bz2".
|
||||
Each compressed file
|
||||
has the same modification date, permissions, and, when possible,
|
||||
ownership as the corresponding original, so that these properties can
|
||||
be correctly restored at decompression time. File name handling is
|
||||
naive in the sense that there is no mechanism for preserving original
|
||||
file names, permissions, ownerships or dates in filesystems which lack
|
||||
these concepts, or have serious file name length restrictions, such as
|
||||
MS-DOS.
|
||||
|
||||
.I bzip2
|
||||
and
|
||||
.I bunzip2
|
||||
will by default not overwrite existing
|
||||
files. If you want this to happen, specify the \-f flag.
|
||||
|
||||
If no file names are specified,
|
||||
.I bzip2
|
||||
compresses from standard
|
||||
input to standard output. In this case,
|
||||
.I bzip2
|
||||
will decline to
|
||||
write compressed output to a terminal, as this would be entirely
|
||||
incomprehensible and therefore pointless.
|
||||
|
||||
.I bunzip2
|
||||
(or
|
||||
.I bzip2 \-d)
|
||||
decompresses all
|
||||
specified files. Files which were not created by
|
||||
.I bzip2
|
||||
will be detected and ignored, and a warning issued.
|
||||
.I bzip2
|
||||
attempts to guess the filename for the decompressed file
|
||||
from that of the compressed file as follows:
|
||||
|
||||
filename.bz2 becomes filename
|
||||
filename.bz becomes filename
|
||||
filename.tbz2 becomes filename.tar
|
||||
filename.tbz becomes filename.tar
|
||||
anyothername becomes anyothername.out
|
||||
|
||||
If the file does not end in one of the recognised endings,
|
||||
.I .bz2,
|
||||
.I .bz,
|
||||
.I .tbz2
|
||||
or
|
||||
.I .tbz,
|
||||
.I bzip2
|
||||
complains that it cannot
|
||||
guess the name of the original file, and uses the original name
|
||||
with
|
||||
.I .out
|
||||
appended.
|
||||
|
||||
As with compression, supplying no
|
||||
filenames causes decompression from
|
||||
standard input to standard output.
|
||||
|
||||
.I bunzip2
|
||||
will correctly decompress a file which is the
|
||||
concatenation of two or more compressed files. The result is the
|
||||
concatenation of the corresponding uncompressed files. Integrity
|
||||
testing (\-t)
|
||||
of concatenated
|
||||
compressed files is also supported.
|
||||
|
||||
You can also compress or decompress files to the standard output by
|
||||
giving the \-c flag. Multiple files may be compressed and
|
||||
decompressed like this. The resulting outputs are fed sequentially to
|
||||
stdout. Compression of multiple files
|
||||
in this manner generates a stream
|
||||
containing multiple compressed file representations. Such a stream
|
||||
can be decompressed correctly only by
|
||||
.I bzip2
|
||||
version 0.9.0 or
|
||||
later. Earlier versions of
|
||||
.I bzip2
|
||||
will stop after decompressing
|
||||
the first file in the stream.
|
||||
|
||||
.I bzcat
|
||||
(or
|
||||
.I bzip2 -dc)
|
||||
decompresses all specified files to
|
||||
the standard output.
|
||||
|
||||
.I bzip2
|
||||
will read arguments from the environment variables
|
||||
.I BZIP2
|
||||
and
|
||||
.I BZIP,
|
||||
in that order, and will process them
|
||||
before any arguments read from the command line. This gives a
|
||||
convenient way to supply default arguments.
|
||||
|
||||
Compression is always performed, even if the compressed
|
||||
file is slightly
|
||||
larger than the original. Files of less than about one hundred bytes
|
||||
tend to get larger, since the compression mechanism has a constant
|
||||
overhead in the region of 50 bytes. Random data (including the output
|
||||
of most file compressors) is coded at about 8.05 bits per byte, giving
|
||||
an expansion of around 0.5%.
|
||||
|
||||
As a self-check for your protection,
|
||||
.I
|
||||
bzip2
|
||||
uses 32-bit CRCs to
|
||||
make sure that the decompressed version of a file is identical to the
|
||||
original. This guards against corruption of the compressed data, and
|
||||
against undetected bugs in
|
||||
.I bzip2
|
||||
(hopefully very unlikely). The
|
||||
chances of data corruption going undetected is microscopic, about one
|
||||
chance in four billion for each file processed. Be aware, though, that
|
||||
the check occurs upon decompression, so it can only tell you that
|
||||
something is wrong. It can't help you
|
||||
recover the original uncompressed
|
||||
data. You can use
|
||||
.I bzip2recover
|
||||
to try to recover data from
|
||||
damaged files.
|
||||
|
||||
Return values: 0 for a normal exit, 1 for environmental problems (file
|
||||
not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
|
||||
compressed file, 3 for an internal consistency error (eg, bug) which
|
||||
caused
|
||||
.I bzip2
|
||||
to panic.
|
||||
|
||||
.SH OPTIONS
|
||||
.TP
|
||||
.B \-c --stdout
|
||||
Compress or decompress to standard output.
|
||||
.TP
|
||||
.B \-d --decompress
|
||||
Force decompression.
|
||||
.I bzip2,
|
||||
.I bunzip2
|
||||
and
|
||||
.I bzcat
|
||||
are
|
||||
really the same program, and the decision about what actions to take is
|
||||
done on the basis of which name is used. This flag overrides that
|
||||
mechanism, and forces
|
||||
.I bzip2
|
||||
to decompress.
|
||||
.TP
|
||||
.B \-z --compress
|
||||
The complement to \-d: forces compression, regardless of the
|
||||
invocation name.
|
||||
.TP
|
||||
.B \-t --test
|
||||
Check integrity of the specified file(s), but don't decompress them.
|
||||
This really performs a trial decompression and throws away the result.
|
||||
.TP
|
||||
.B \-f --force
|
||||
Force overwrite of output files. Normally,
|
||||
.I bzip2
|
||||
will not overwrite
|
||||
existing output files. Also forces
|
||||
.I bzip2
|
||||
to break hard links
|
||||
to files, which it otherwise wouldn't do.
|
||||
|
||||
bzip2 normally declines to decompress files which don't have the
|
||||
correct magic header bytes. If forced (-f), however, it will pass
|
||||
such files through unmodified. This is how GNU gzip behaves.
|
||||
.TP
|
||||
.B \-k --keep
|
||||
Keep (don't delete) input files during compression
|
||||
or decompression.
|
||||
.TP
|
||||
.B \-s --small
|
||||
Reduce memory usage, for compression, decompression and testing. Files
|
||||
are decompressed and tested using a modified algorithm which only
|
||||
requires 2.5 bytes per block byte. This means any file can be
|
||||
decompressed in 2300k of memory, albeit at about half the normal speed.
|
||||
|
||||
During compression, \-s selects a block size of 200k, which limits
|
||||
memory use to around the same figure, at the expense of your compression
|
||||
ratio. In short, if your machine is low on memory (8 megabytes or
|
||||
less), use \-s for everything. See MEMORY MANAGEMENT below.
|
||||
.TP
|
||||
.B \-q --quiet
|
||||
Suppress non-essential warning messages. Messages pertaining to
|
||||
I/O errors and other critical events will not be suppressed.
|
||||
.TP
|
||||
.B \-v --verbose
|
||||
Verbose mode -- show the compression ratio for each file processed.
|
||||
Further \-v's increase the verbosity level, spewing out lots of
|
||||
information which is primarily of interest for diagnostic purposes.
|
||||
.TP
|
||||
.B \-L --license -V --version
|
||||
Display the software version, license terms and conditions.
|
||||
.TP
|
||||
.B \-1 (or \-\-fast) to \-9 (or \-\-best)
|
||||
Set the block size to 100 k, 200 k .. 900 k when compressing. Has no
|
||||
effect when decompressing. See MEMORY MANAGEMENT below.
|
||||
The \-\-fast and \-\-best aliases are primarily for GNU gzip
|
||||
compatibility. In particular, \-\-fast doesn't make things
|
||||
significantly faster.
|
||||
And \-\-best merely selects the default behaviour.
|
||||
.TP
|
||||
.B \--
|
||||
Treats all subsequent arguments as file names, even if they start
|
||||
with a dash. This is so you can handle files with names beginning
|
||||
with a dash, for example: bzip2 \-- \-myfilename.
|
||||
.TP
|
||||
.B \--repetitive-fast --repetitive-best
|
||||
These flags are redundant in versions 0.9.5 and above. They provided
|
||||
some coarse control over the behaviour of the sorting algorithm in
|
||||
earlier versions, which was sometimes useful. 0.9.5 and above have an
|
||||
improved algorithm which renders these flags irrelevant.
|
||||
|
||||
.SH MEMORY MANAGEMENT
|
||||
.I bzip2
|
||||
compresses large files in blocks. The block size affects
|
||||
both the compression ratio achieved, and the amount of memory needed for
|
||||
compression and decompression. The flags \-1 through \-9
|
||||
specify the block size to be 100,000 bytes through 900,000 bytes (the
|
||||
default) respectively. At decompression time, the block size used for
|
||||
compression is read from the header of the compressed file, and
|
||||
.I bunzip2
|
||||
then allocates itself just enough memory to decompress
|
||||
the file. Since block sizes are stored in compressed files, it follows
|
||||
that the flags \-1 to \-9 are irrelevant to and so ignored
|
||||
during decompression.
|
||||
|
||||
Compression and decompression requirements,
|
||||
in bytes, can be estimated as:
|
||||
|
||||
Compression: 400k + ( 8 x block size )
|
||||
|
||||
Decompression: 100k + ( 4 x block size ), or
|
||||
100k + ( 2.5 x block size )
|
||||
|
||||
Larger block sizes give rapidly diminishing marginal returns. Most of
|
||||
the compression comes from the first two or three hundred k of block
|
||||
size, a fact worth bearing in mind when using
|
||||
.I bzip2
|
||||
on small machines.
|
||||
It is also important to appreciate that the decompression memory
|
||||
requirement is set at compression time by the choice of block size.
|
||||
|
||||
For files compressed with the default 900k block size,
|
||||
.I bunzip2
|
||||
will require about 3700 kbytes to decompress. To support decompression
|
||||
of any file on a 4 megabyte machine,
|
||||
.I bunzip2
|
||||
has an option to
|
||||
decompress using approximately half this amount of memory, about 2300
|
||||
kbytes. Decompression speed is also halved, so you should use this
|
||||
option only where necessary. The relevant flag is -s.
|
||||
|
||||
In general, try and use the largest block size memory constraints allow,
|
||||
since that maximises the compression achieved. Compression and
|
||||
decompression speed are virtually unaffected by block size.
|
||||
|
||||
Another significant point applies to files which fit in a single block
|
||||
-- that means most files you'd encounter using a large block size. The
|
||||
amount of real memory touched is proportional to the size of the file,
|
||||
since the file is smaller than a block. For example, compressing a file
|
||||
20,000 bytes long with the flag -9 will cause the compressor to
|
||||
allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560
|
||||
kbytes of it. Similarly, the decompressor will allocate 3700k but only
|
||||
touch 100k + 20000 * 4 = 180 kbytes.
|
||||
|
||||
Here is a table which summarises the maximum memory usage for different
|
||||
block sizes. Also recorded is the total compressed size for 14 files of
|
||||
the Calgary Text Compression Corpus totalling 3,141,622 bytes. This
|
||||
column gives some feel for how compression varies with block size.
|
||||
These figures tend to understate the advantage of larger block sizes for
|
||||
larger files, since the Corpus is dominated by smaller files.
|
||||
|
||||
Compress Decompress Decompress Corpus
|
||||
Flag usage usage -s usage Size
|
||||
|
||||
-1 1200k 500k 350k 914704
|
||||
-2 2000k 900k 600k 877703
|
||||
-3 2800k 1300k 850k 860338
|
||||
-4 3600k 1700k 1100k 846899
|
||||
-5 4400k 2100k 1350k 845160
|
||||
-6 5200k 2500k 1600k 838626
|
||||
-7 6100k 2900k 1850k 834096
|
||||
-8 6800k 3300k 2100k 828642
|
||||
-9 7600k 3700k 2350k 828642
|
||||
|
||||
.SH RECOVERING DATA FROM DAMAGED FILES
|
||||
.I bzip2
|
||||
compresses files in blocks, usually 900kbytes long. Each
|
||||
block is handled independently. If a media or transmission error causes
|
||||
a multi-block .bz2
|
||||
file to become damaged, it may be possible to
|
||||
recover data from the undamaged blocks in the file.
|
||||
|
||||
The compressed representation of each block is delimited by a 48-bit
|
||||
pattern, which makes it possible to find the block boundaries with
|
||||
reasonable certainty. Each block also carries its own 32-bit CRC, so
|
||||
damaged blocks can be distinguished from undamaged ones.
|
||||
|
||||
.I bzip2recover
|
||||
is a simple program whose purpose is to search for
|
||||
blocks in .bz2 files, and write each block out into its own .bz2
|
||||
file. You can then use
|
||||
.I bzip2
|
||||
\-t
|
||||
to test the
|
||||
integrity of the resulting files, and decompress those which are
|
||||
undamaged.
|
||||
|
||||
.I bzip2recover
|
||||
takes a single argument, the name of the damaged file,
|
||||
and writes a number of files "rec00001file.bz2",
|
||||
"rec00002file.bz2", etc, containing the extracted blocks.
|
||||
The output filenames are designed so that the use of
|
||||
wildcards in subsequent processing -- for example,
|
||||
"bzip2 -dc rec*file.bz2 > recovered_data" -- processes the files in
|
||||
the correct order.
|
||||
|
||||
.I bzip2recover
|
||||
should be of most use dealing with large .bz2
|
||||
files, as these will contain many blocks. It is clearly
|
||||
futile to use it on damaged single-block files, since a
|
||||
damaged block cannot be recovered. If you wish to minimise
|
||||
any potential data loss through media or transmission errors,
|
||||
you might consider compressing with a smaller
|
||||
block size.
|
||||
|
||||
.SH PERFORMANCE NOTES
|
||||
The sorting phase of compression gathers together similar strings in the
|
||||
file. Because of this, files containing very long runs of repeated
|
||||
symbols, like "aabaabaabaab ..." (repeated several hundred times) may
|
||||
compress more slowly than normal. Versions 0.9.5 and above fare much
|
||||
better than previous versions in this respect. The ratio between
|
||||
worst-case and average-case compression time is in the region of 10:1.
|
||||
For previous versions, this figure was more like 100:1. You can use the
|
||||
\-vvvv option to monitor progress in great detail, if you want.
|
||||
|
||||
Decompression speed is unaffected by these phenomena.
|
||||
|
||||
.I bzip2
|
||||
usually allocates several megabytes of memory to operate
|
||||
in, and then charges all over it in a fairly random fashion. This means
|
||||
that performance, both for compressing and decompressing, is largely
|
||||
determined by the speed at which your machine can service cache misses.
|
||||
Because of this, small changes to the code to reduce the miss rate have
|
||||
been observed to give disproportionately large performance improvements.
|
||||
I imagine
|
||||
.I bzip2
|
||||
will perform best on machines with very large caches.
|
||||
|
||||
.SH CAVEATS
|
||||
I/O error messages are not as helpful as they could be.
|
||||
.I bzip2
|
||||
tries hard to detect I/O errors and exit cleanly, but the details of
|
||||
what the problem is sometimes seem rather misleading.
|
||||
|
||||
This manual page pertains to version 1.0.6 of
|
||||
.I bzip2.
|
||||
Compressed data created by this version is entirely forwards and
|
||||
backwards compatible with the previous public releases, versions
|
||||
0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and above, but with the following
|
||||
exception: 0.9.0 and above can correctly decompress multiple
|
||||
concatenated compressed files. 0.1pl2 cannot do this; it will stop
|
||||
after decompressing just the first file in the stream.
|
||||
|
||||
.I bzip2recover
|
||||
versions prior to 1.0.2 used 32-bit integers to represent
|
||||
bit positions in compressed files, so they could not handle compressed
|
||||
files more than 512 megabytes long. Versions 1.0.2 and above use
|
||||
64-bit ints on some platforms which support them (GNU supported
|
||||
targets, and Windows). To establish whether or not bzip2recover was
|
||||
built with such a limitation, run it without arguments. In any event
|
||||
you can build yourself an unlimited version if you can recompile it
|
||||
with MaybeUInt64 set to be an unsigned 64-bit integer.
|
||||
|
||||
|
||||
|
||||
.SH AUTHOR
|
||||
Julian Seward, jsewardbzip.org.
|
||||
|
||||
http://www.bzip.org
|
||||
|
||||
The ideas embodied in
|
||||
.I bzip2
|
||||
are due to (at least) the following
|
||||
people: Michael Burrows and David Wheeler (for the block sorting
|
||||
transformation), David Wheeler (again, for the Huffman coder), Peter
|
||||
Fenwick (for the structured coding model in the original
|
||||
.I bzip,
|
||||
and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
|
||||
(for the arithmetic coder in the original
|
||||
.I bzip).
|
||||
I am much
|
||||
indebted for their help, support and advice. See the manual in the
|
||||
source distribution for pointers to sources of documentation. Christian
|
||||
von Roques encouraged me to look for faster sorting algorithms, so as to
|
||||
speed up compression. Bela Lubkin encouraged me to improve the
|
||||
worst-case compression performance.
|
||||
Donna Robinson XMLised the documentation.
|
||||
The bz* scripts are derived from those of GNU gzip.
|
||||
Many people sent patches, helped
|
||||
with portability problems, lent machines, gave advice and were generally
|
||||
helpful.
|
||||
-399
@@ -1,399 +0,0 @@
|
||||
bzip2(1) bzip2(1)
|
||||
|
||||
|
||||
|
||||
NNAAMMEE
|
||||
bzip2, bunzip2 − a block‐sorting file compressor, v1.0.6
|
||||
bzcat − decompresses files to stdout
|
||||
bzip2recover − recovers data from damaged bzip2 files
|
||||
|
||||
|
||||
SSYYNNOOPPSSIISS
|
||||
bbzziipp22 [ −−ccddffkkqqssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ]
|
||||
bbuunnzziipp22 [ −−ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ]
|
||||
bbzzccaatt [ −−ss ] [ _f_i_l_e_n_a_m_e_s _._._. ]
|
||||
bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e
|
||||
|
||||
|
||||
DDEESSCCRRIIPPTTIIOONN
|
||||
_b_z_i_p_2 compresses files using the Burrows‐Wheeler block
|
||||
sorting text compression algorithm, and Huffman coding.
|
||||
Compression is generally considerably better than that
|
||||
achieved by more conventional LZ77/LZ78‐based compressors,
|
||||
and approaches the performance of the PPM family of sta
|
||||
tistical compressors.
|
||||
|
||||
The command‐line options are deliberately very similar to
|
||||
those of _G_N_U _g_z_i_p_, but they are not identical.
|
||||
|
||||
_b_z_i_p_2 expects a list of file names to accompany the com
|
||||
mand‐line flags. Each file is replaced by a compressed
|
||||
version of itself, with the name "original_name.bz2".
|
||||
Each compressed file has the same modification date, per
|
||||
missions, and, when possible, ownership as the correspond
|
||||
ing original, so that these properties can be correctly
|
||||
restored at decompression time. File name handling is
|
||||
naive in the sense that there is no mechanism for preserv
|
||||
ing original file names, permissions, ownerships or dates
|
||||
in filesystems which lack these concepts, or have serious
|
||||
file name length restrictions, such as MS‐DOS.
|
||||
|
||||
_b_z_i_p_2 and _b_u_n_z_i_p_2 will by default not overwrite existing
|
||||
files. If you want this to happen, specify the −f flag.
|
||||
|
||||
If no file names are specified, _b_z_i_p_2 compresses from
|
||||
standard input to standard output. In this case, _b_z_i_p_2
|
||||
will decline to write compressed output to a terminal, as
|
||||
this would be entirely incomprehensible and therefore
|
||||
pointless.
|
||||
|
||||
_b_u_n_z_i_p_2 (or _b_z_i_p_2 _−_d_) decompresses all specified files.
|
||||
Files which were not created by _b_z_i_p_2 will be detected and
|
||||
ignored, and a warning issued. _b_z_i_p_2 attempts to guess
|
||||
the filename for the decompressed file from that of the
|
||||
compressed file as follows:
|
||||
|
||||
filename.bz2 becomes filename
|
||||
filename.bz becomes filename
|
||||
filename.tbz2 becomes filename.tar
|
||||
filename.tbz becomes filename.tar
|
||||
anyothername becomes anyothername.out
|
||||
|
||||
If the file does not end in one of the recognised endings,
|
||||
_._b_z_2_, _._b_z_, _._t_b_z_2 or _._t_b_z_, _b_z_i_p_2 complains that it cannot
|
||||
guess the name of the original file, and uses the original
|
||||
name with _._o_u_t appended.
|
||||
|
||||
As with compression, supplying no filenames causes decom
|
||||
pression from standard input to standard output.
|
||||
|
||||
_b_u_n_z_i_p_2 will correctly decompress a file which is the con
|
||||
catenation of two or more compressed files. The result is
|
||||
the concatenation of the corresponding uncompressed files.
|
||||
Integrity testing (−t) of concatenated compressed files is
|
||||
also supported.
|
||||
|
||||
You can also compress or decompress files to the standard
|
||||
output by giving the −c flag. Multiple files may be com
|
||||
pressed and decompressed like this. The resulting outputs
|
||||
are fed sequentially to stdout. Compression of multiple
|
||||
files in this manner generates a stream containing multi
|
||||
ple compressed file representations. Such a stream can be
|
||||
decompressed correctly only by _b_z_i_p_2 version 0.9.0 or
|
||||
later. Earlier versions of _b_z_i_p_2 will stop after decom
|
||||
pressing the first file in the stream.
|
||||
|
||||
_b_z_c_a_t (or _b_z_i_p_2 _‐_d_c_) decompresses all specified files to
|
||||
the standard output.
|
||||
|
||||
_b_z_i_p_2 will read arguments from the environment variables
|
||||
_B_Z_I_P_2 and _B_Z_I_P_, in that order, and will process them
|
||||
before any arguments read from the command line. This
|
||||
gives a convenient way to supply default arguments.
|
||||
|
||||
Compression is always performed, even if the compressed
|
||||
file is slightly larger than the original. Files of less
|
||||
than about one hundred bytes tend to get larger, since the
|
||||
compression mechanism has a constant overhead in the
|
||||
region of 50 bytes. Random data (including the output of
|
||||
most file compressors) is coded at about 8.05 bits per
|
||||
byte, giving an expansion of around 0.5%.
|
||||
|
||||
As a self‐check for your protection, _b_z_i_p_2 uses 32‐bit
|
||||
CRCs to make sure that the decompressed version of a file
|
||||
is identical to the original. This guards against corrup
|
||||
tion of the compressed data, and against undetected bugs
|
||||
in _b_z_i_p_2 (hopefully very unlikely). The chances of data
|
||||
corruption going undetected is microscopic, about one
|
||||
chance in four billion for each file processed. Be aware,
|
||||
though, that the check occurs upon decompression, so it
|
||||
can only tell you that something is wrong. It can’t help
|
||||
you recover the original uncompressed data. You can use
|
||||
_b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files.
|
||||
|
||||
Return values: 0 for a normal exit, 1 for environmental
|
||||
problems (file not found, invalid flags, I/O errors, &c),
|
||||
2 to indicate a corrupt compressed file, 3 for an internal
|
||||
consistency error (eg, bug) which caused _b_z_i_p_2 to panic.
|
||||
|
||||
|
||||
OOPPTTIIOONNSS
|
||||
−−cc ‐‐‐‐ssttddoouutt
|
||||
Compress or decompress to standard output.
|
||||
|
||||
−−dd ‐‐‐‐ddeeccoommpprreessss
|
||||
Force decompression. _b_z_i_p_2_, _b_u_n_z_i_p_2 and _b_z_c_a_t are
|
||||
really the same program, and the decision about
|
||||
what actions to take is done on the basis of which
|
||||
name is used. This flag overrides that mechanism,
|
||||
and forces _b_z_i_p_2 to decompress.
|
||||
|
||||
−−zz ‐‐‐‐ccoommpprreessss
|
||||
The complement to −d: forces compression,
|
||||
regardless of the invocation name.
|
||||
|
||||
−−tt ‐‐‐‐tteesstt
|
||||
Check integrity of the specified file(s), but don’t
|
||||
decompress them. This really performs a trial
|
||||
decompression and throws away the result.
|
||||
|
||||
−−ff ‐‐‐‐ffoorrccee
|
||||
Force overwrite of output files. Normally, _b_z_i_p_2
|
||||
will not overwrite existing output files. Also
|
||||
forces _b_z_i_p_2 to break hard links to files, which it
|
||||
otherwise wouldn’t do.
|
||||
|
||||
bzip2 normally declines to decompress files which
|
||||
don’t have the correct magic header bytes. If
|
||||
forced (‐f), however, it will pass such files
|
||||
through unmodified. This is how GNU gzip behaves.
|
||||
|
||||
−−kk ‐‐‐‐kkeeeepp
|
||||
Keep (don’t delete) input files during compression
|
||||
or decompression.
|
||||
|
||||
−−ss ‐‐‐‐ssmmaallll
|
||||
Reduce memory usage, for compression, decompression
|
||||
and testing. Files are decompressed and tested
|
||||
using a modified algorithm which only requires 2.5
|
||||
bytes per block byte. This means any file can be
|
||||
decompressed in 2300k of memory, albeit at about
|
||||
half the normal speed.
|
||||
|
||||
During compression, −s selects a block size of
|
||||
200k, which limits memory use to around the same
|
||||
figure, at the expense of your compression ratio.
|
||||
In short, if your machine is low on memory (8
|
||||
megabytes or less), use −s for everything. See
|
||||
MEMORY MANAGEMENT below.
|
||||
|
||||
−−qq ‐‐‐‐qquuiieett
|
||||
Suppress non‐essential warning messages. Messages
|
||||
pertaining to I/O errors and other critical events
|
||||
will not be suppressed.
|
||||
|
||||
−−vv ‐‐‐‐vveerrbboossee
|
||||
Verbose mode ‐‐ show the compression ratio for each
|
||||
file processed. Further −v’s increase the ver
|
||||
bosity level, spewing out lots of information which
|
||||
is primarily of interest for diagnostic purposes.
|
||||
|
||||
−−LL ‐‐‐‐lliicceennssee ‐‐VV ‐‐‐‐vveerrssiioonn
|
||||
Display the software version, license terms and
|
||||
conditions.
|
||||
|
||||
−−11 ((oorr −−−−ffaasstt)) ttoo −−99 ((oorr −−−−bbeesstt))
|
||||
Set the block size to 100 k, 200 k .. 900 k when
|
||||
compressing. Has no effect when decompressing.
|
||||
See MEMORY MANAGEMENT below. The −−fast and −−best
|
||||
aliases are primarily for GNU gzip compatibility.
|
||||
In particular, −−fast doesn’t make things signifi
|
||||
cantly faster. And −−best merely selects the
|
||||
default behaviour.
|
||||
|
||||
−−‐‐ Treats all subsequent arguments as file names, even
|
||||
if they start with a dash. This is so you can han
|
||||
dle files with names beginning with a dash, for
|
||||
example: bzip2 −‐ −myfilename.
|
||||
|
||||
−−‐‐rreeppeettiittiivvee‐‐ffaasstt ‐‐‐‐rreeppeettiittiivvee‐‐bbeesstt
|
||||
These flags are redundant in versions 0.9.5 and
|
||||
above. They provided some coarse control over the
|
||||
behaviour of the sorting algorithm in earlier ver
|
||||
sions, which was sometimes useful. 0.9.5 and above
|
||||
have an improved algorithm which renders these
|
||||
flags irrelevant.
|
||||
|
||||
|
||||
MMEEMMOORRYY MMAANNAAGGEEMMEENNTT
|
||||
_b_z_i_p_2 compresses large files in blocks. The block size
|
||||
affects both the compression ratio achieved, and the
|
||||
amount of memory needed for compression and decompression.
|
||||
The flags −1 through −9 specify the block size to be
|
||||
100,000 bytes through 900,000 bytes (the default) respec
|
||||
tively. At decompression time, the block size used for
|
||||
compression is read from the header of the compressed
|
||||
file, and _b_u_n_z_i_p_2 then allocates itself just enough memory
|
||||
to decompress the file. Since block sizes are stored in
|
||||
compressed files, it follows that the flags −1 to −9 are
|
||||
irrelevant to and so ignored during decompression.
|
||||
|
||||
Compression and decompression requirements, in bytes, can
|
||||
be estimated as:
|
||||
|
||||
Compression: 400k + ( 8 x block size )
|
||||
|
||||
Decompression: 100k + ( 4 x block size ), or
|
||||
100k + ( 2.5 x block size )
|
||||
|
||||
Larger block sizes give rapidly diminishing marginal
|
||||
returns. Most of the compression comes from the first two
|
||||
or three hundred k of block size, a fact worth bearing in
|
||||
mind when using _b_z_i_p_2 on small machines. It is also
|
||||
important to appreciate that the decompression memory
|
||||
requirement is set at compression time by the choice of
|
||||
block size.
|
||||
|
||||
For files compressed with the default 900k block size,
|
||||
_b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To
|
||||
support decompression of any file on a 4 megabyte machine,
|
||||
_b_u_n_z_i_p_2 has an option to decompress using approximately
|
||||
half this amount of memory, about 2300 kbytes. Decompres
|
||||
sion speed is also halved, so you should use this option
|
||||
only where necessary. The relevant flag is ‐s.
|
||||
|
||||
In general, try and use the largest block size memory con
|
||||
straints allow, since that maximises the compression
|
||||
achieved. Compression and decompression speed are virtu
|
||||
ally unaffected by block size.
|
||||
|
||||
Another significant point applies to files which fit in a
|
||||
single block ‐‐ that means most files you’d encounter
|
||||
using a large block size. The amount of real memory
|
||||
touched is proportional to the size of the file, since the
|
||||
file is smaller than a block. For example, compressing a
|
||||
file 20,000 bytes long with the flag ‐9 will cause the
|
||||
compressor to allocate around 7600k of memory, but only
|
||||
touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the
|
||||
decompressor will allocate 3700k but only touch 100k +
|
||||
20000 * 4 = 180 kbytes.
|
||||
|
||||
Here is a table which summarises the maximum memory usage
|
||||
for different block sizes. Also recorded is the total
|
||||
compressed size for 14 files of the Calgary Text Compres
|
||||
sion Corpus totalling 3,141,622 bytes. This column gives
|
||||
some feel for how compression varies with block size.
|
||||
These figures tend to understate the advantage of larger
|
||||
block sizes for larger files, since the Corpus is domi
|
||||
nated by smaller files.
|
||||
|
||||
Compress Decompress Decompress Corpus
|
||||
Flag usage usage ‐s usage Size
|
||||
|
||||
‐1 1200k 500k 350k 914704
|
||||
‐2 2000k 900k 600k 877703
|
||||
‐3 2800k 1300k 850k 860338
|
||||
‐4 3600k 1700k 1100k 846899
|
||||
‐5 4400k 2100k 1350k 845160
|
||||
‐6 5200k 2500k 1600k 838626
|
||||
‐7 6100k 2900k 1850k 834096
|
||||
‐8 6800k 3300k 2100k 828642
|
||||
‐9 7600k 3700k 2350k 828642
|
||||
|
||||
|
||||
RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS
|
||||
_b_z_i_p_2 compresses files in blocks, usually 900kbytes long.
|
||||
Each block is handled independently. If a media or trans
|
||||
mission error causes a multi‐block .bz2 file to become
|
||||
damaged, it may be possible to recover data from the
|
||||
undamaged blocks in the file.
|
||||
|
||||
The compressed representation of each block is delimited
|
||||
by a 48‐bit pattern, which makes it possible to find the
|
||||
block boundaries with reasonable certainty. Each block
|
||||
also carries its own 32‐bit CRC, so damaged blocks can be
|
||||
distinguished from undamaged ones.
|
||||
|
||||
_b_z_i_p_2_r_e_c_o_v_e_r is a simple program whose purpose is to
|
||||
search for blocks in .bz2 files, and write each block out
|
||||
into its own .bz2 file. You can then use _b_z_i_p_2 −t to test
|
||||
the integrity of the resulting files, and decompress those
|
||||
which are undamaged.
|
||||
|
||||
_b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam
|
||||
aged file, and writes a number of files
|
||||
"rec00001file.bz2", "rec00002file.bz2", etc, containing
|
||||
the extracted blocks. The output filenames are
|
||||
designed so that the use of wildcards in subsequent pro
|
||||
cessing ‐‐ for example, "bzip2 ‐dc rec*file.bz2 > recov
|
||||
ered_data" ‐‐ processes the files in the correct order.
|
||||
|
||||
_b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2
|
||||
files, as these will contain many blocks. It is clearly
|
||||
futile to use it on damaged single‐block files, since a
|
||||
damaged block cannot be recovered. If you wish to min
|
||||
imise any potential data loss through media or transmis
|
||||
sion errors, you might consider compressing with a smaller
|
||||
block size.
|
||||
|
||||
|
||||
PPEERRFFOORRMMAANNCCEE NNOOTTEESS
|
||||
The sorting phase of compression gathers together similar
|
||||
strings in the file. Because of this, files containing
|
||||
very long runs of repeated symbols, like "aabaabaabaab
|
||||
..." (repeated several hundred times) may compress more
|
||||
slowly than normal. Versions 0.9.5 and above fare much
|
||||
better than previous versions in this respect. The ratio
|
||||
between worst‐case and average‐case compression time is in
|
||||
the region of 10:1. For previous versions, this figure
|
||||
was more like 100:1. You can use the −vvvv option to mon
|
||||
itor progress in great detail, if you want.
|
||||
|
||||
Decompression speed is unaffected by these phenomena.
|
||||
|
||||
_b_z_i_p_2 usually allocates several megabytes of memory to
|
||||
operate in, and then charges all over it in a fairly ran
|
||||
dom fashion. This means that performance, both for com
|
||||
pressing and decompressing, is largely determined by the
|
||||
speed at which your machine can service cache misses.
|
||||
Because of this, small changes to the code to reduce the
|
||||
miss rate have been observed to give disproportionately
|
||||
large performance improvements. I imagine _b_z_i_p_2 will per
|
||||
form best on machines with very large caches.
|
||||
|
||||
|
||||
CCAAVVEEAATTSS
|
||||
I/O error messages are not as helpful as they could be.
|
||||
_b_z_i_p_2 tries hard to detect I/O errors and exit cleanly,
|
||||
but the details of what the problem is sometimes seem
|
||||
rather misleading.
|
||||
|
||||
This manual page pertains to version 1.0.6 of _b_z_i_p_2_. Com
|
||||
pressed data created by this version is entirely forwards
|
||||
and backwards compatible with the previous public
|
||||
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
|
||||
1.0.2 and above, but with the following exception: 0.9.0
|
||||
and above can correctly decompress multiple concatenated
|
||||
compressed files. 0.1pl2 cannot do this; it will stop
|
||||
after decompressing just the first file in the stream.
|
||||
|
||||
_b_z_i_p_2_r_e_c_o_v_e_r versions prior to 1.0.2 used 32‐bit integers
|
||||
to represent bit positions in compressed files, so they
|
||||
could not handle compressed files more than 512 megabytes
|
||||
long. Versions 1.0.2 and above use 64‐bit ints on some
|
||||
platforms which support them (GNU supported targets, and
|
||||
Windows). To establish whether or not bzip2recover was
|
||||
built with such a limitation, run it without arguments.
|
||||
In any event you can build yourself an unlimited version
|
||||
if you can recompile it with MaybeUInt64 set to be an
|
||||
unsigned 64‐bit integer.
|
||||
|
||||
|
||||
|
||||
|
||||
AAUUTTHHOORR
|
||||
Julian Seward, jsewardbzip.org.
|
||||
|
||||
http://www.bzip.org
|
||||
|
||||
The ideas embodied in _b_z_i_p_2 are due to (at least) the fol
|
||||
lowing people: Michael Burrows and David Wheeler (for the
|
||||
block sorting transformation), David Wheeler (again, for
|
||||
the Huffman coder), Peter Fenwick (for the structured cod
|
||||
ing model in the original _b_z_i_p_, and many refinements), and
|
||||
Alistair Moffat, Radford Neal and Ian Witten (for the
|
||||
arithmetic coder in the original _b_z_i_p_)_. I am much
|
||||
indebted for their help, support and advice. See the man
|
||||
ual in the source distribution for pointers to sources of
|
||||
documentation. Christian von Roques encouraged me to look
|
||||
for faster sorting algorithms, so as to speed up compres
|
||||
sion. Bela Lubkin encouraged me to improve the worst‐case
|
||||
compression performance. Donna Robinson XMLised the docu
|
||||
mentation. The bz* scripts are derived from those of GNU
|
||||
gzip. Many people sent patches, helped with portability
|
||||
problems, lent machines, gave advice and were generally
|
||||
helpful.
|
||||
|
||||
|
||||
|
||||
bzip2(1)
|
||||
Vendored
-391
@@ -1,391 +0,0 @@
|
||||
|
||||
NAME
|
||||
bzip2, bunzip2 - a block-sorting file compressor, v1.0.6
|
||||
bzcat - decompresses files to stdout
|
||||
bzip2recover - recovers data from damaged bzip2 files
|
||||
|
||||
|
||||
SYNOPSIS
|
||||
bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ]
|
||||
bunzip2 [ -fkvsVL ] [ filenames ... ]
|
||||
bzcat [ -s ] [ filenames ... ]
|
||||
bzip2recover filename
|
||||
|
||||
|
||||
DESCRIPTION
|
||||
bzip2 compresses files using the Burrows-Wheeler block
|
||||
sorting text compression algorithm, and Huffman coding.
|
||||
Compression is generally considerably better than that
|
||||
achieved by more conventional LZ77/LZ78-based compressors,
|
||||
and approaches the performance of the PPM family of sta-
|
||||
tistical compressors.
|
||||
|
||||
The command-line options are deliberately very similar to
|
||||
those of GNU gzip, but they are not identical.
|
||||
|
||||
bzip2 expects a list of file names to accompany the com-
|
||||
mand-line flags. Each file is replaced by a compressed
|
||||
version of itself, with the name "original_name.bz2".
|
||||
Each compressed file has the same modification date, per-
|
||||
missions, and, when possible, ownership as the correspond-
|
||||
ing original, so that these properties can be correctly
|
||||
restored at decompression time. File name handling is
|
||||
naive in the sense that there is no mechanism for preserv-
|
||||
ing original file names, permissions, ownerships or dates
|
||||
in filesystems which lack these concepts, or have serious
|
||||
file name length restrictions, such as MS-DOS.
|
||||
|
||||
bzip2 and bunzip2 will by default not overwrite existing
|
||||
files. If you want this to happen, specify the -f flag.
|
||||
|
||||
If no file names are specified, bzip2 compresses from
|
||||
standard input to standard output. In this case, bzip2
|
||||
will decline to write compressed output to a terminal, as
|
||||
this would be entirely incomprehensible and therefore
|
||||
pointless.
|
||||
|
||||
bunzip2 (or bzip2 -d) decompresses all specified files.
|
||||
Files which were not created by bzip2 will be detected and
|
||||
ignored, and a warning issued. bzip2 attempts to guess
|
||||
the filename for the decompressed file from that of the
|
||||
compressed file as follows:
|
||||
|
||||
filename.bz2 becomes filename
|
||||
filename.bz becomes filename
|
||||
filename.tbz2 becomes filename.tar
|
||||
filename.tbz becomes filename.tar
|
||||
anyothername becomes anyothername.out
|
||||
|
||||
If the file does not end in one of the recognised endings,
|
||||
.bz2, .bz, .tbz2 or .tbz, bzip2 complains that it cannot
|
||||
guess the name of the original file, and uses the original
|
||||
name with .out appended.
|
||||
|
||||
As with compression, supplying no filenames causes decom-
|
||||
pression from standard input to standard output.
|
||||
|
||||
bunzip2 will correctly decompress a file which is the con-
|
||||
catenation of two or more compressed files. The result is
|
||||
the concatenation of the corresponding uncompressed files.
|
||||
Integrity testing (-t) of concatenated compressed files is
|
||||
also supported.
|
||||
|
||||
You can also compress or decompress files to the standard
|
||||
output by giving the -c flag. Multiple files may be com-
|
||||
pressed and decompressed like this. The resulting outputs
|
||||
are fed sequentially to stdout. Compression of multiple
|
||||
files in this manner generates a stream containing multi-
|
||||
ple compressed file representations. Such a stream can be
|
||||
decompressed correctly only by bzip2 version 0.9.0 or
|
||||
later. Earlier versions of bzip2 will stop after decom-
|
||||
pressing the first file in the stream.
|
||||
|
||||
bzcat (or bzip2 -dc) decompresses all specified files to
|
||||
the standard output.
|
||||
|
||||
bzip2 will read arguments from the environment variables
|
||||
BZIP2 and BZIP, in that order, and will process them
|
||||
before any arguments read from the command line. This
|
||||
gives a convenient way to supply default arguments.
|
||||
|
||||
Compression is always performed, even if the compressed
|
||||
file is slightly larger than the original. Files of less
|
||||
than about one hundred bytes tend to get larger, since the
|
||||
compression mechanism has a constant overhead in the
|
||||
region of 50 bytes. Random data (including the output of
|
||||
most file compressors) is coded at about 8.05 bits per
|
||||
byte, giving an expansion of around 0.5%.
|
||||
|
||||
As a self-check for your protection, bzip2 uses 32-bit
|
||||
CRCs to make sure that the decompressed version of a file
|
||||
is identical to the original. This guards against corrup-
|
||||
tion of the compressed data, and against undetected bugs
|
||||
in bzip2 (hopefully very unlikely). The chances of data
|
||||
corruption going undetected is microscopic, about one
|
||||
chance in four billion for each file processed. Be aware,
|
||||
though, that the check occurs upon decompression, so it
|
||||
can only tell you that something is wrong. It can't help
|
||||
you recover the original uncompressed data. You can use
|
||||
bzip2recover to try to recover data from damaged files.
|
||||
|
||||
Return values: 0 for a normal exit, 1 for environmental
|
||||
problems (file not found, invalid flags, I/O errors, &c),
|
||||
2 to indicate a corrupt compressed file, 3 for an internal
|
||||
consistency error (eg, bug) which caused bzip2 to panic.
|
||||
|
||||
|
||||
OPTIONS
|
||||
-c --stdout
|
||||
Compress or decompress to standard output.
|
||||
|
||||
-d --decompress
|
||||
Force decompression. bzip2, bunzip2 and bzcat are
|
||||
really the same program, and the decision about
|
||||
what actions to take is done on the basis of which
|
||||
name is used. This flag overrides that mechanism,
|
||||
and forces bzip2 to decompress.
|
||||
|
||||
-z --compress
|
||||
The complement to -d: forces compression,
|
||||
regardless of the invocation name.
|
||||
|
||||
-t --test
|
||||
Check integrity of the specified file(s), but don't
|
||||
decompress them. This really performs a trial
|
||||
decompression and throws away the result.
|
||||
|
||||
-f --force
|
||||
Force overwrite of output files. Normally, bzip2
|
||||
will not overwrite existing output files. Also
|
||||
forces bzip2 to break hard links to files, which it
|
||||
otherwise wouldn't do.
|
||||
|
||||
bzip2 normally declines to decompress files which
|
||||
don't have the correct magic header bytes. If
|
||||
forced (-f), however, it will pass such files
|
||||
through unmodified. This is how GNU gzip behaves.
|
||||
|
||||
-k --keep
|
||||
Keep (don't delete) input files during compression
|
||||
or decompression.
|
||||
|
||||
-s --small
|
||||
Reduce memory usage, for compression, decompression
|
||||
and testing. Files are decompressed and tested
|
||||
using a modified algorithm which only requires 2.5
|
||||
bytes per block byte. This means any file can be
|
||||
decompressed in 2300k of memory, albeit at about
|
||||
half the normal speed.
|
||||
|
||||
During compression, -s selects a block size of
|
||||
200k, which limits memory use to around the same
|
||||
figure, at the expense of your compression ratio.
|
||||
In short, if your machine is low on memory (8
|
||||
megabytes or less), use -s for everything. See
|
||||
MEMORY MANAGEMENT below.
|
||||
|
||||
-q --quiet
|
||||
Suppress non-essential warning messages. Messages
|
||||
pertaining to I/O errors and other critical events
|
||||
will not be suppressed.
|
||||
|
||||
-v --verbose
|
||||
Verbose mode -- show the compression ratio for each
|
||||
file processed. Further -v's increase the ver-
|
||||
bosity level, spewing out lots of information which
|
||||
is primarily of interest for diagnostic purposes.
|
||||
|
||||
-L --license -V --version
|
||||
Display the software version, license terms and
|
||||
conditions.
|
||||
|
||||
-1 (or --fast) to -9 (or --best)
|
||||
Set the block size to 100 k, 200 k .. 900 k when
|
||||
compressing. Has no effect when decompressing.
|
||||
See MEMORY MANAGEMENT below. The --fast and --best
|
||||
aliases are primarily for GNU gzip compatibility.
|
||||
In particular, --fast doesn't make things signifi-
|
||||
cantly faster. And --best merely selects the
|
||||
default behaviour.
|
||||
|
||||
-- Treats all subsequent arguments as file names, even
|
||||
if they start with a dash. This is so you can han-
|
||||
dle files with names beginning with a dash, for
|
||||
example: bzip2 -- -myfilename.
|
||||
|
||||
--repetitive-fast --repetitive-best
|
||||
These flags are redundant in versions 0.9.5 and
|
||||
above. They provided some coarse control over the
|
||||
behaviour of the sorting algorithm in earlier ver-
|
||||
sions, which was sometimes useful. 0.9.5 and above
|
||||
have an improved algorithm which renders these
|
||||
flags irrelevant.
|
||||
|
||||
|
||||
MEMORY MANAGEMENT
|
||||
bzip2 compresses large files in blocks. The block size
|
||||
affects both the compression ratio achieved, and the
|
||||
amount of memory needed for compression and decompression.
|
||||
The flags -1 through -9 specify the block size to be
|
||||
100,000 bytes through 900,000 bytes (the default) respec-
|
||||
tively. At decompression time, the block size used for
|
||||
compression is read from the header of the compressed
|
||||
file, and bunzip2 then allocates itself just enough memory
|
||||
to decompress the file. Since block sizes are stored in
|
||||
compressed files, it follows that the flags -1 to -9 are
|
||||
irrelevant to and so ignored during decompression.
|
||||
|
||||
Compression and decompression requirements, in bytes, can
|
||||
be estimated as:
|
||||
|
||||
Compression: 400k + ( 8 x block size )
|
||||
|
||||
Decompression: 100k + ( 4 x block size ), or
|
||||
100k + ( 2.5 x block size )
|
||||
|
||||
Larger block sizes give rapidly diminishing marginal
|
||||
returns. Most of the compression comes from the first two
|
||||
or three hundred k of block size, a fact worth bearing in
|
||||
mind when using bzip2 on small machines. It is also
|
||||
important to appreciate that the decompression memory
|
||||
requirement is set at compression time by the choice of
|
||||
block size.
|
||||
|
||||
For files compressed with the default 900k block size,
|
||||
bunzip2 will require about 3700 kbytes to decompress. To
|
||||
support decompression of any file on a 4 megabyte machine,
|
||||
bunzip2 has an option to decompress using approximately
|
||||
half this amount of memory, about 2300 kbytes. Decompres-
|
||||
sion speed is also halved, so you should use this option
|
||||
only where necessary. The relevant flag is -s.
|
||||
|
||||
In general, try and use the largest block size memory con-
|
||||
straints allow, since that maximises the compression
|
||||
achieved. Compression and decompression speed are virtu-
|
||||
ally unaffected by block size.
|
||||
|
||||
Another significant point applies to files which fit in a
|
||||
single block -- that means most files you'd encounter
|
||||
using a large block size. The amount of real memory
|
||||
touched is proportional to the size of the file, since the
|
||||
file is smaller than a block. For example, compressing a
|
||||
file 20,000 bytes long with the flag -9 will cause the
|
||||
compressor to allocate around 7600k of memory, but only
|
||||
touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the
|
||||
decompressor will allocate 3700k but only touch 100k +
|
||||
20000 * 4 = 180 kbytes.
|
||||
|
||||
Here is a table which summarises the maximum memory usage
|
||||
for different block sizes. Also recorded is the total
|
||||
compressed size for 14 files of the Calgary Text Compres-
|
||||
sion Corpus totalling 3,141,622 bytes. This column gives
|
||||
some feel for how compression varies with block size.
|
||||
These figures tend to understate the advantage of larger
|
||||
block sizes for larger files, since the Corpus is domi-
|
||||
nated by smaller files.
|
||||
|
||||
Compress Decompress Decompress Corpus
|
||||
Flag usage usage -s usage Size
|
||||
|
||||
-1 1200k 500k 350k 914704
|
||||
-2 2000k 900k 600k 877703
|
||||
-3 2800k 1300k 850k 860338
|
||||
-4 3600k 1700k 1100k 846899
|
||||
-5 4400k 2100k 1350k 845160
|
||||
-6 5200k 2500k 1600k 838626
|
||||
-7 6100k 2900k 1850k 834096
|
||||
-8 6800k 3300k 2100k 828642
|
||||
-9 7600k 3700k 2350k 828642
|
||||
|
||||
|
||||
RECOVERING DATA FROM DAMAGED FILES
|
||||
bzip2 compresses files in blocks, usually 900kbytes long.
|
||||
Each block is handled independently. If a media or trans-
|
||||
mission error causes a multi-block .bz2 file to become
|
||||
damaged, it may be possible to recover data from the
|
||||
undamaged blocks in the file.
|
||||
|
||||
The compressed representation of each block is delimited
|
||||
by a 48-bit pattern, which makes it possible to find the
|
||||
block boundaries with reasonable certainty. Each block
|
||||
also carries its own 32-bit CRC, so damaged blocks can be
|
||||
distinguished from undamaged ones.
|
||||
|
||||
bzip2recover is a simple program whose purpose is to
|
||||
search for blocks in .bz2 files, and write each block out
|
||||
into its own .bz2 file. You can then use bzip2 -t to test
|
||||
the integrity of the resulting files, and decompress those
|
||||
which are undamaged.
|
||||
|
||||
bzip2recover takes a single argument, the name of the dam-
|
||||
aged file, and writes a number of files
|
||||
"rec00001file.bz2", "rec00002file.bz2", etc, containing
|
||||
the extracted blocks. The output filenames are
|
||||
designed so that the use of wildcards in subsequent pro-
|
||||
cessing -- for example, "bzip2 -dc rec*file.bz2 > recov-
|
||||
ered_data" -- processes the files in the correct order.
|
||||
|
||||
bzip2recover should be of most use dealing with large .bz2
|
||||
files, as these will contain many blocks. It is clearly
|
||||
futile to use it on damaged single-block files, since a
|
||||
damaged block cannot be recovered. If you wish to min-
|
||||
imise any potential data loss through media or transmis-
|
||||
sion errors, you might consider compressing with a smaller
|
||||
block size.
|
||||
|
||||
|
||||
PERFORMANCE NOTES
|
||||
The sorting phase of compression gathers together similar
|
||||
strings in the file. Because of this, files containing
|
||||
very long runs of repeated symbols, like "aabaabaabaab
|
||||
..." (repeated several hundred times) may compress more
|
||||
slowly than normal. Versions 0.9.5 and above fare much
|
||||
better than previous versions in this respect. The ratio
|
||||
between worst-case and average-case compression time is in
|
||||
the region of 10:1. For previous versions, this figure
|
||||
was more like 100:1. You can use the -vvvv option to mon-
|
||||
itor progress in great detail, if you want.
|
||||
|
||||
Decompression speed is unaffected by these phenomena.
|
||||
|
||||
bzip2 usually allocates several megabytes of memory to
|
||||
operate in, and then charges all over it in a fairly ran-
|
||||
dom fashion. This means that performance, both for com-
|
||||
pressing and decompressing, is largely determined by the
|
||||
speed at which your machine can service cache misses.
|
||||
Because of this, small changes to the code to reduce the
|
||||
miss rate have been observed to give disproportionately
|
||||
large performance improvements. I imagine bzip2 will per-
|
||||
form best on machines with very large caches.
|
||||
|
||||
|
||||
CAVEATS
|
||||
I/O error messages are not as helpful as they could be.
|
||||
bzip2 tries hard to detect I/O errors and exit cleanly,
|
||||
but the details of what the problem is sometimes seem
|
||||
rather misleading.
|
||||
|
||||
This manual page pertains to version 1.0.6 of bzip2. Com-
|
||||
pressed data created by this version is entirely forwards
|
||||
and backwards compatible with the previous public
|
||||
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
|
||||
1.0.2 and above, but with the following exception: 0.9.0
|
||||
and above can correctly decompress multiple concatenated
|
||||
compressed files. 0.1pl2 cannot do this; it will stop
|
||||
after decompressing just the first file in the stream.
|
||||
|
||||
bzip2recover versions prior to 1.0.2 used 32-bit integers
|
||||
to represent bit positions in compressed files, so they
|
||||
could not handle compressed files more than 512 megabytes
|
||||
long. Versions 1.0.2 and above use 64-bit ints on some
|
||||
platforms which support them (GNU supported targets, and
|
||||
Windows). To establish whether or not bzip2recover was
|
||||
built with such a limitation, run it without arguments.
|
||||
In any event you can build yourself an unlimited version
|
||||
if you can recompile it with MaybeUInt64 set to be an
|
||||
unsigned 64-bit integer.
|
||||
|
||||
|
||||
AUTHOR
|
||||
Julian Seward, jsewardbzip.org.
|
||||
|
||||
http://www.bzip.org
|
||||
|
||||
The ideas embodied in bzip2 are due to (at least) the fol-
|
||||
lowing people: Michael Burrows and David Wheeler (for the
|
||||
block sorting transformation), David Wheeler (again, for
|
||||
the Huffman coder), Peter Fenwick (for the structured cod-
|
||||
ing model in the original bzip, and many refinements), and
|
||||
Alistair Moffat, Radford Neal and Ian Witten (for the
|
||||
arithmetic coder in the original bzip). I am much
|
||||
indebted for their help, support and advice. See the man-
|
||||
ual in the source distribution for pointers to sources of
|
||||
documentation. Christian von Roques encouraged me to look
|
||||
for faster sorting algorithms, so as to speed up compres-
|
||||
sion. Bela Lubkin encouraged me to improve the worst-case
|
||||
compression performance. Donna Robinson XMLised the docu-
|
||||
mentation. The bz* scripts are derived from those of GNU
|
||||
gzip. Many people sent patches, helped with portability
|
||||
problems, lent machines, gave advice and were generally
|
||||
helpful.
|
||||
|
||||
-93
@@ -1,93 +0,0 @@
|
||||
# Microsoft Developer Studio Project File - Name="dlltest" - Package Owner=<4>
|
||||
# Microsoft Developer Studio Generated Build File, Format Version 5.00
|
||||
# ** 編集しないでください **
|
||||
|
||||
# TARGTYPE "Win32 (x86) Console Application" 0x0103
|
||||
|
||||
CFG=dlltest - Win32 Debug
|
||||
!MESSAGE これは有効なﾒｲｸﾌｧｲﾙではありません。 このﾌﾟﾛｼﾞｪｸﾄをﾋﾞﾙﾄﾞするためには NMAKE を使用してください。
|
||||
!MESSAGE [ﾒｲｸﾌｧｲﾙのｴｸｽﾎﾟｰﾄ] ｺﾏﾝﾄﾞを使用して実行してください
|
||||
!MESSAGE
|
||||
!MESSAGE NMAKE /f "dlltest.mak".
|
||||
!MESSAGE
|
||||
!MESSAGE NMAKE の実行時に構成を指定できます
|
||||
!MESSAGE ｺﾏﾝﾄﾞ ﾗｲﾝ上でﾏｸﾛの設定を定義します。例:
|
||||
!MESSAGE
|
||||
!MESSAGE NMAKE /f "dlltest.mak" CFG="dlltest - Win32 Debug"
|
||||
!MESSAGE
|
||||
!MESSAGE 選択可能なﾋﾞﾙﾄﾞ ﾓｰﾄﾞ:
|
||||
!MESSAGE
|
||||
!MESSAGE "dlltest - Win32 Release" ("Win32 (x86) Console Application" 用)
|
||||
!MESSAGE "dlltest - Win32 Debug" ("Win32 (x86) Console Application" 用)
|
||||
!MESSAGE
|
||||
|
||||
# Begin Project
|
||||
# PROP Scc_ProjName ""
|
||||
# PROP Scc_LocalPath ""
|
||||
CPP=cl.exe
|
||||
RSC=rc.exe
|
||||
|
||||
!IF "$(CFG)" == "dlltest - Win32 Release"
|
||||
|
||||
# PROP BASE Use_MFC 0
|
||||
# PROP BASE Use_Debug_Libraries 0
|
||||
# PROP BASE Output_Dir "Release"
|
||||
# PROP BASE Intermediate_Dir "Release"
|
||||
# PROP BASE Target_Dir ""
|
||||
# PROP Use_MFC 0
|
||||
# PROP Use_Debug_Libraries 0
|
||||
# PROP Output_Dir "Release"
|
||||
# PROP Intermediate_Dir "Release"
|
||||
# PROP Ignore_Export_Lib 0
|
||||
# PROP Target_Dir ""
|
||||
# ADD BASE CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
|
||||
# ADD CPP /nologo /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
|
||||
# ADD BASE RSC /l 0x411 /d "NDEBUG"
|
||||
# ADD RSC /l 0x411 /d "NDEBUG"
|
||||
BSC32=bscmake.exe
|
||||
# ADD BASE BSC32 /nologo
|
||||
# ADD BSC32 /nologo
|
||||
LINK32=link.exe
|
||||
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386
|
||||
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /machine:I386 /out:"minibz2.exe"
|
||||
|
||||
!ELSEIF "$(CFG)" == "dlltest - Win32 Debug"
|
||||
|
||||
# PROP BASE Use_MFC 0
|
||||
# PROP BASE Use_Debug_Libraries 1
|
||||
# PROP BASE Output_Dir "dlltest_"
|
||||
# PROP BASE Intermediate_Dir "dlltest_"
|
||||
# PROP BASE Target_Dir ""
|
||||
# PROP Use_MFC 0
|
||||
# PROP Use_Debug_Libraries 1
|
||||
# PROP Output_Dir "dlltest_"
|
||||
# PROP Intermediate_Dir "dlltest_"
|
||||
# PROP Ignore_Export_Lib 0
|
||||
# PROP Target_Dir ""
|
||||
# ADD BASE CPP /nologo /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
|
||||
# ADD CPP /nologo /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_CONSOLE" /D "_MBCS" /YX /FD /c
|
||||
# ADD BASE RSC /l 0x411 /d "_DEBUG"
|
||||
# ADD RSC /l 0x411 /d "_DEBUG"
|
||||
BSC32=bscmake.exe
|
||||
# ADD BASE BSC32 /nologo
|
||||
# ADD BSC32 /nologo
|
||||
LINK32=link.exe
|
||||
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /pdbtype:sept
|
||||
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:console /debug /machine:I386 /out:"minibz2.exe" /pdbtype:sept
|
||||
|
||||
!ENDIF
|
||||
|
||||
# Begin Target
|
||||
|
||||
# Name "dlltest - Win32 Release"
|
||||
# Name "dlltest - Win32 Debug"
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\bzlib.h
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\dlltest.c
|
||||
# End Source File
|
||||
# End Target
|
||||
# End Project
|
||||
-130
@@ -1,130 +0,0 @@
|
||||
# Microsoft Developer Studio Project File - Name="libbz2" - Package Owner=<4>
|
||||
# Microsoft Developer Studio Generated Build File, Format Version 5.00
|
||||
# ** 編集しないでください **
|
||||
|
||||
# TARGTYPE "Win32 (x86) Dynamic-Link Library" 0x0102
|
||||
|
||||
CFG=libbz2 - Win32 Debug
|
||||
!MESSAGE これは有効なﾒｲｸﾌｧｲﾙではありません。 このﾌﾟﾛｼﾞｪｸﾄをﾋﾞﾙﾄﾞするためには NMAKE を使用してください。
|
||||
!MESSAGE [ﾒｲｸﾌｧｲﾙのｴｸｽﾎﾟｰﾄ] ｺﾏﾝﾄﾞを使用して実行してください
|
||||
!MESSAGE
|
||||
!MESSAGE NMAKE /f "libbz2.mak".
|
||||
!MESSAGE
|
||||
!MESSAGE NMAKE の実行時に構成を指定できます
|
||||
!MESSAGE ｺﾏﾝﾄﾞ ﾗｲﾝ上でﾏｸﾛの設定を定義します。例:
|
||||
!MESSAGE
|
||||
!MESSAGE NMAKE /f "libbz2.mak" CFG="libbz2 - Win32 Debug"
|
||||
!MESSAGE
|
||||
!MESSAGE 選択可能なﾋﾞﾙﾄﾞ ﾓｰﾄﾞ:
|
||||
!MESSAGE
|
||||
!MESSAGE "libbz2 - Win32 Release" ("Win32 (x86) Dynamic-Link Library" 用)
|
||||
!MESSAGE "libbz2 - Win32 Debug" ("Win32 (x86) Dynamic-Link Library" 用)
|
||||
!MESSAGE
|
||||
|
||||
# Begin Project
|
||||
# PROP Scc_ProjName ""
|
||||
# PROP Scc_LocalPath ""
|
||||
CPP=cl.exe
|
||||
MTL=midl.exe
|
||||
RSC=rc.exe
|
||||
|
||||
!IF "$(CFG)" == "libbz2 - Win32 Release"
|
||||
|
||||
# PROP BASE Use_MFC 0
|
||||
# PROP BASE Use_Debug_Libraries 0
|
||||
# PROP BASE Output_Dir "Release"
|
||||
# PROP BASE Intermediate_Dir "Release"
|
||||
# PROP BASE Target_Dir ""
|
||||
# PROP Use_MFC 0
|
||||
# PROP Use_Debug_Libraries 0
|
||||
# PROP Output_Dir "Release"
|
||||
# PROP Intermediate_Dir "Release"
|
||||
# PROP Ignore_Export_Lib 0
|
||||
# PROP Target_Dir ""
|
||||
# ADD BASE CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /YX /FD /c
|
||||
# ADD CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D "_WINDOWS" /YX /FD /c
|
||||
# ADD BASE MTL /nologo /D "NDEBUG" /mktyplib203 /o NUL /win32
|
||||
# ADD MTL /nologo /D "NDEBUG" /mktyplib203 /o NUL /win32
|
||||
# ADD BASE RSC /l 0x411 /d "NDEBUG"
|
||||
# ADD RSC /l 0x411 /d "NDEBUG"
|
||||
BSC32=bscmake.exe
|
||||
# ADD BASE BSC32 /nologo
|
||||
# ADD BSC32 /nologo
|
||||
LINK32=link.exe
|
||||
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /machine:I386
|
||||
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /machine:I386 /out:"libbz2.dll"
|
||||
|
||||
!ELSEIF "$(CFG)" == "libbz2 - Win32 Debug"
|
||||
|
||||
# PROP BASE Use_MFC 0
|
||||
# PROP BASE Use_Debug_Libraries 1
|
||||
# PROP BASE Output_Dir "Debug"
|
||||
# PROP BASE Intermediate_Dir "Debug"
|
||||
# PROP BASE Target_Dir ""
|
||||
# PROP Use_MFC 0
|
||||
# PROP Use_Debug_Libraries 1
|
||||
# PROP Output_Dir "Debug"
|
||||
# PROP Intermediate_Dir "Debug"
|
||||
# PROP Ignore_Export_Lib 0
|
||||
# PROP Target_Dir ""
|
||||
# ADD BASE CPP /nologo /MTd /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /YX /FD /c
|
||||
# ADD CPP /nologo /MTd /W3 /Gm /GX /Zi /Od /D "WIN32" /D "_DEBUG" /D "_WINDOWS" /YX /FD /c
|
||||
# ADD BASE MTL /nologo /D "_DEBUG" /mktyplib203 /o NUL /win32
|
||||
# ADD MTL /nologo /D "_DEBUG" /mktyplib203 /o NUL /win32
|
||||
# ADD BASE RSC /l 0x411 /d "_DEBUG"
|
||||
# ADD RSC /l 0x411 /d "_DEBUG"
|
||||
BSC32=bscmake.exe
|
||||
# ADD BASE BSC32 /nologo
|
||||
# ADD BSC32 /nologo
|
||||
LINK32=link.exe
|
||||
# ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /debug /machine:I386 /pdbtype:sept
|
||||
# ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /nologo /subsystem:windows /dll /debug /machine:I386 /out:"libbz2.dll" /pdbtype:sept
|
||||
|
||||
!ENDIF
|
||||
|
||||
# Begin Target
|
||||
|
||||
# Name "libbz2 - Win32 Release"
|
||||
# Name "libbz2 - Win32 Debug"
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\blocksort.c
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\bzlib.c
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\bzlib.h
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\bzlib_private.h
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\compress.c
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\crctable.c
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\decompress.c
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\huffman.c
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\libbz2.def
|
||||
# End Source File
|
||||
# Begin Source File
|
||||
|
||||
SOURCE=.\randtable.c
|
||||
# End Source File
|
||||
# End Target
|
||||
# End Project
|
||||
-63
@@ -1,63 +0,0 @@
|
||||
# Makefile for Microsoft Visual C++ 6.0
|
||||
# usage: nmake -f makefile.msc
|
||||
# K.M. Syring ([email protected])
|
||||
# Fixed up by JRS for bzip2-0.9.5d release.
|
||||
|
||||
CC=cl
|
||||
CFLAGS= -DWIN32 -MD -Ox -D_FILE_OFFSET_BITS=64 -nologo
|
||||
|
||||
OBJS= blocksort.obj \
|
||||
huffman.obj \
|
||||
crctable.obj \
|
||||
randtable.obj \
|
||||
compress.obj \
|
||||
decompress.obj \
|
||||
bzlib.obj
|
||||
|
||||
all: lib bzip2 test
|
||||
|
||||
bzip2: lib
|
||||
$(CC) $(CFLAGS) -o bzip2 bzip2.c libbz2.lib setargv.obj
|
||||
$(CC) $(CFLAGS) -o bzip2recover bzip2recover.c
|
||||
|
||||
lib: $(OBJS)
|
||||
lib /out:libbz2.lib $(OBJS)
|
||||
|
||||
test: bzip2
|
||||
type words1
|
||||
.\\bzip2 -1 < sample1.ref > sample1.rb2
|
||||
.\\bzip2 -2 < sample2.ref > sample2.rb2
|
||||
.\\bzip2 -3 < sample3.ref > sample3.rb2
|
||||
.\\bzip2 -d < sample1.bz2 > sample1.tst
|
||||
.\\bzip2 -d < sample2.bz2 > sample2.tst
|
||||
.\\bzip2 -ds < sample3.bz2 > sample3.tst
|
||||
@echo All six of the fc's should find no differences.
|
||||
@echo If fc finds an error on sample3.bz2, this could be
|
||||
@echo because WinZip's 'TAR file smart CR/LF conversion'
|
||||
@echo is too clever for its own good. Disable this option.
|
||||
@echo The correct size for sample3.ref is 120,244. If it
|
||||
@echo is 150,251, WinZip has messed it up.
|
||||
fc sample1.bz2 sample1.rb2
|
||||
fc sample2.bz2 sample2.rb2
|
||||
fc sample3.bz2 sample3.rb2
|
||||
fc sample1.tst sample1.ref
|
||||
fc sample2.tst sample2.ref
|
||||
fc sample3.tst sample3.ref
|
||||
|
||||
|
||||
|
||||
clean:
|
||||
del *.obj
|
||||
del libbz2.lib
|
||||
del bzip2.exe
|
||||
del bzip2recover.exe
|
||||
del sample1.rb2
|
||||
del sample2.rb2
|
||||
del sample3.rb2
|
||||
del sample1.tst
|
||||
del sample2.tst
|
||||
del sample3.tst
|
||||
|
||||
.c.obj:
|
||||
$(CC) $(CFLAGS) -c $*.c -o $*.obj
|
||||
|
||||
Vendored
-72843
File diff suppressed because it is too large
Load Diff
-2215
File diff suppressed because it is too large
Load Diff
-47
@@ -1,47 +0,0 @@
|
||||
<HTML>
|
||||
<HEAD>
|
||||
<!-- This HTML file has been created by texi2html 1.54
|
||||
from manual.texi on 23 March 2000 -->
|
||||
|
||||
<TITLE>bzip2 and libbzip2 - Introduction</TITLE>
|
||||
<link href="manual_2.html" rel=Next>
|
||||
<link href="manual_toc.html" rel=ToC>
|
||||
|
||||
</HEAD>
|
||||
<BODY>
|
||||
<p>Go to the first, previous, <A HREF="manual_2.html">next</A>, <A HREF="manual_4.html">last</A> section, <A HREF="manual_toc.html">table of contents</A>.
|
||||
<P><HR><P>
|
||||
|
||||
|
||||
<H1><A NAME="SEC1" HREF="manual_toc.html#TOC1">Introduction</A></H1>
|
||||
|
||||
<P>
|
||||
<CODE>bzip2</CODE> compresses files using the Burrows-Wheeler
|
||||
block-sorting text compression algorithm, and Huffman coding.
|
||||
Compression is generally considerably better than that
|
||||
achieved by more conventional LZ77/LZ78-based compressors,
|
||||
and approaches the performance of the PPM family of statistical compressors.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2</CODE> is built on top of <CODE>libbzip2</CODE>, a flexible library
|
||||
for handling compressed data in the <CODE>bzip2</CODE> format. This manual
|
||||
describes both how to use the program and
|
||||
how to work with the library interface. Most of the
|
||||
manual is devoted to this library, not the program,
|
||||
which is good news if your interest is only in the program.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Chapter 2 describes how to use <CODE>bzip2</CODE>; this is the only part
|
||||
you need to read if you just want to know how to operate the program.
|
||||
Chapter 3 describes the programming interfaces in detail, and
|
||||
Chapter 4 records some miscellaneous notes which I thought
|
||||
ought to be recorded somewhere.
|
||||
|
||||
</P>
|
||||
|
||||
<P><HR><P>
|
||||
<p>Go to the first, previous, <A HREF="manual_2.html">next</A>, <A HREF="manual_4.html">last</A> section, <A HREF="manual_toc.html">table of contents</A>.
|
||||
</BODY>
|
||||
</HTML>
|
||||
-484
@@ -1,484 +0,0 @@
|
||||
<HTML>
|
||||
<HEAD>
|
||||
<!-- This HTML file has been created by texi2html 1.54
|
||||
from manual.texi on 23 March 2000 -->
|
||||
|
||||
<TITLE>bzip2 and libbzip2 - How to use bzip2</TITLE>
|
||||
<link href="manual_3.html" rel=Next>
|
||||
<link href="manual_1.html" rel=Previous>
|
||||
<link href="manual_toc.html" rel=ToC>
|
||||
|
||||
</HEAD>
|
||||
<BODY>
|
||||
<p>Go to the <A HREF="manual_1.html">first</A>, <A HREF="manual_1.html">previous</A>, <A HREF="manual_3.html">next</A>, <A HREF="manual_4.html">last</A> section, <A HREF="manual_toc.html">table of contents</A>.
|
||||
<P><HR><P>
|
||||
|
||||
|
||||
<H1><A NAME="SEC2" HREF="manual_toc.html#TOC2">How to use <CODE>bzip2</CODE></A></H1>
|
||||
|
||||
<P>
|
||||
This chapter contains a copy of the <CODE>bzip2</CODE> man page,
|
||||
and nothing else.
|
||||
|
||||
</P>
|
||||
|
||||
<BLOCKQUOTE>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC3" HREF="manual_toc.html#TOC3">NAME</A></H4>
|
||||
|
||||
<UL>
|
||||
<LI><CODE>bzip2</CODE>, <CODE>bunzip2</CODE>
|
||||
|
||||
- a block-sorting file compressor, v1.0
|
||||
<LI><CODE>bzcat</CODE>
|
||||
|
||||
- decompresses files to stdout
|
||||
<LI><CODE>bzip2recover</CODE>
|
||||
|
||||
- recovers data from damaged bzip2 files
|
||||
</UL>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC4" HREF="manual_toc.html#TOC4">SYNOPSIS</A></H4>
|
||||
|
||||
<UL>
|
||||
<LI><CODE>bzip2</CODE> [ -cdfkqstvzVL123456789 ] [ filenames ... ]
|
||||
|
||||
<LI><CODE>bunzip2</CODE> [ -fkvsVL ] [ filenames ... ]
|
||||
|
||||
<LI><CODE>bzcat</CODE> [ -s ] [ filenames ... ]
|
||||
|
||||
<LI><CODE>bzip2recover</CODE> filename
|
||||
|
||||
</UL>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC5" HREF="manual_toc.html#TOC5">DESCRIPTION</A></H4>
|
||||
|
||||
<P>
|
||||
<CODE>bzip2</CODE> compresses files using the Burrows-Wheeler block sorting
|
||||
text compression algorithm, and Huffman coding. Compression is
|
||||
generally considerably better than that achieved by more conventional
|
||||
LZ77/LZ78-based compressors, and approaches the performance of the PPM
|
||||
family of statistical compressors.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
The command-line options are deliberately very similar to those of GNU
|
||||
<CODE>gzip</CODE>, but they are not identical.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2</CODE> expects a list of file names to accompany the command-line
|
||||
flags. Each file is replaced by a compressed version of itself, with
|
||||
the name <CODE>original_name.bz2</CODE>. Each compressed file has the same
|
||||
modification date, permissions, and, when possible, ownership as the
|
||||
corresponding original, so that these properties can be correctly
|
||||
restored at decompression time. File name handling is naive in the
|
||||
sense that there is no mechanism for preserving original file names,
|
||||
permissions, ownerships or dates in filesystems which lack these
|
||||
concepts, or have serious file name length restrictions, such as MS-DOS.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2</CODE> and <CODE>bunzip2</CODE> will by default not overwrite existing
|
||||
files. If you want this to happen, specify the <CODE>-f</CODE> flag.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
If no file names are specified, <CODE>bzip2</CODE> compresses from standard
|
||||
input to standard output. In this case, <CODE>bzip2</CODE> will decline to
|
||||
write compressed output to a terminal, as this would be entirely
|
||||
incomprehensible and therefore pointless.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bunzip2</CODE> (or <CODE>bzip2 -d</CODE>) decompresses all
|
||||
specified files. Files which were not created by <CODE>bzip2</CODE>
|
||||
will be detected and ignored, and a warning issued.
|
||||
<CODE>bzip2</CODE> attempts to guess the filename for the decompressed file
|
||||
from that of the compressed file as follows:
|
||||
|
||||
<UL>
|
||||
<LI><CODE>filename.bz2 </CODE> becomes <CODE>filename</CODE>
|
||||
|
||||
<LI><CODE>filename.bz </CODE> becomes <CODE>filename</CODE>
|
||||
|
||||
<LI><CODE>filename.tbz2</CODE> becomes <CODE>filename.tar</CODE>
|
||||
|
||||
<LI><CODE>filename.tbz </CODE> becomes <CODE>filename.tar</CODE>
|
||||
|
||||
<LI><CODE>anyothername </CODE> becomes <CODE>anyothername.out</CODE>
|
||||
|
||||
</UL>
|
||||
|
||||
<P>
|
||||
If the file does not end in one of the recognised endings,
|
||||
<CODE>.bz2</CODE>, <CODE>.bz</CODE>,
|
||||
<CODE>.tbz2</CODE> or <CODE>.tbz</CODE>, <CODE>bzip2</CODE> complains that it cannot
|
||||
guess the name of the original file, and uses the original name
|
||||
with <CODE>.out</CODE> appended.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
As with compression, supplying no
|
||||
filenames causes decompression from standard input to standard output.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bunzip2</CODE> will correctly decompress a file which is the
|
||||
concatenation of two or more compressed files. The result is the
|
||||
concatenation of the corresponding uncompressed files. Integrity
|
||||
testing (<CODE>-t</CODE>) of concatenated compressed files is also supported.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
You can also compress or decompress files to the standard output by
|
||||
giving the <CODE>-c</CODE> flag. Multiple files may be compressed and
|
||||
decompressed like this. The resulting outputs are fed sequentially to
|
||||
stdout. Compression of multiple files in this manner generates a stream
|
||||
containing multiple compressed file representations. Such a stream
|
||||
can be decompressed correctly only by <CODE>bzip2</CODE> version 0.9.0 or
|
||||
later. Earlier versions of <CODE>bzip2</CODE> will stop after decompressing
|
||||
the first file in the stream.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzcat</CODE> (or <CODE>bzip2 -dc</CODE>) decompresses all specified files to
|
||||
the standard output.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2</CODE> will read arguments from the environment variables
|
||||
<CODE>BZIP2</CODE> and <CODE>BZIP</CODE>, in that order, and will process them
|
||||
before any arguments read from the command line. This gives a
|
||||
convenient way to supply default arguments.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Compression is always performed, even if the compressed file is slightly
|
||||
larger than the original. Files of less than about one hundred bytes
|
||||
tend to get larger, since the compression mechanism has a constant
|
||||
overhead in the region of 50 bytes. Random data (including the output
|
||||
of most file compressors) is coded at about 8.05 bits per byte, giving
|
||||
an expansion of around 0.5%.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
As a self-check for your protection, <CODE>bzip2</CODE> uses 32-bit CRCs to
|
||||
make sure that the decompressed version of a file is identical to the
|
||||
original. This guards against corruption of the compressed data, and
|
||||
against undetected bugs in <CODE>bzip2</CODE> (hopefully very unlikely). The
|
||||
chances of data corruption going undetected is microscopic, about one
|
||||
chance in four billion for each file processed. Be aware, though, that
|
||||
the check occurs upon decompression, so it can only tell you that
|
||||
something is wrong. It can't help you recover the original uncompressed
|
||||
data. You can use <CODE>bzip2recover</CODE> to try to recover data from
|
||||
damaged files.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Return values: 0 for a normal exit, 1 for environmental problems (file
|
||||
not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
|
||||
compressed file, 3 for an internal consistency error (eg, bug) which
|
||||
caused <CODE>bzip2</CODE> to panic.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC6" HREF="manual_toc.html#TOC6">OPTIONS</A></H4>
|
||||
<DL COMPACT>
|
||||
|
||||
<DT><CODE>-c --stdout</CODE>
|
||||
<DD>
|
||||
Compress or decompress to standard output.
|
||||
<DT><CODE>-d --decompress</CODE>
|
||||
<DD>
|
||||
Force decompression. <CODE>bzip2</CODE>, <CODE>bunzip2</CODE> and <CODE>bzcat</CODE> are
|
||||
really the same program, and the decision about what actions to take is
|
||||
done on the basis of which name is used. This flag overrides that
|
||||
mechanism, and forces bzip2 to decompress.
|
||||
<DT><CODE>-z --compress</CODE>
|
||||
<DD>
|
||||
The complement to <CODE>-d</CODE>: forces compression, regardless of the
|
||||
invokation name.
|
||||
<DT><CODE>-t --test</CODE>
|
||||
<DD>
|
||||
Check integrity of the specified file(s), but don't decompress them.
|
||||
This really performs a trial decompression and throws away the result.
|
||||
<DT><CODE>-f --force</CODE>
|
||||
<DD>
|
||||
Force overwrite of output files. Normally, <CODE>bzip2</CODE> will not overwrite
|
||||
existing output files. Also forces <CODE>bzip2</CODE> to break hard links
|
||||
to files, which it otherwise wouldn't do.
|
||||
<DT><CODE>-k --keep</CODE>
|
||||
<DD>
|
||||
Keep (don't delete) input files during compression
|
||||
or decompression.
|
||||
<DT><CODE>-s --small</CODE>
|
||||
<DD>
|
||||
Reduce memory usage, for compression, decompression and testing. Files
|
||||
are decompressed and tested using a modified algorithm which only
|
||||
requires 2.5 bytes per block byte. This means any file can be
|
||||
decompressed in 2300k of memory, albeit at about half the normal speed.
|
||||
|
||||
During compression, <CODE>-s</CODE> selects a block size of 200k, which limits
|
||||
memory use to around the same figure, at the expense of your compression
|
||||
ratio. In short, if your machine is low on memory (8 megabytes or
|
||||
less), use -s for everything. See MEMORY MANAGEMENT below.
|
||||
<DT><CODE>-q --quiet</CODE>
|
||||
<DD>
|
||||
Suppress non-essential warning messages. Messages pertaining to
|
||||
I/O errors and other critical events will not be suppressed.
|
||||
<DT><CODE>-v --verbose</CODE>
|
||||
<DD>
|
||||
Verbose mode -- show the compression ratio for each file processed.
|
||||
Further <CODE>-v</CODE>'s increase the verbosity level, spewing out lots of
|
||||
information which is primarily of interest for diagnostic purposes.
|
||||
<DT><CODE>-L --license -V --version</CODE>
|
||||
<DD>
|
||||
Display the software version, license terms and conditions.
|
||||
<DT><CODE>-1 to -9</CODE>
|
||||
<DD>
|
||||
Set the block size to 100 k, 200 k .. 900 k when compressing. Has no
|
||||
effect when decompressing. See MEMORY MANAGEMENT below.
|
||||
<DT><CODE>--</CODE>
|
||||
<DD>
|
||||
Treats all subsequent arguments as file names, even if they start
|
||||
with a dash. This is so you can handle files with names beginning
|
||||
with a dash, for example: <CODE>bzip2 -- -myfilename</CODE>.
|
||||
<DT><CODE>--repetitive-fast</CODE>
|
||||
<DD>
|
||||
<DT><CODE>--repetitive-best</CODE>
|
||||
<DD>
|
||||
These flags are redundant in versions 0.9.5 and above. They provided
|
||||
some coarse control over the behaviour of the sorting algorithm in
|
||||
earlier versions, which was sometimes useful. 0.9.5 and above have an
|
||||
improved algorithm which renders these flags irrelevant.
|
||||
</DL>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC7" HREF="manual_toc.html#TOC7">MEMORY MANAGEMENT</A></H4>
|
||||
|
||||
<P>
|
||||
<CODE>bzip2</CODE> compresses large files in blocks. The block size affects
|
||||
both the compression ratio achieved, and the amount of memory needed for
|
||||
compression and decompression. The flags <CODE>-1</CODE> through <CODE>-9</CODE>
|
||||
specify the block size to be 100,000 bytes through 900,000 bytes (the
|
||||
default) respectively. At decompression time, the block size used for
|
||||
compression is read from the header of the compressed file, and
|
||||
<CODE>bunzip2</CODE> then allocates itself just enough memory to decompress
|
||||
the file. Since block sizes are stored in compressed files, it follows
|
||||
that the flags <CODE>-1</CODE> to <CODE>-9</CODE> are irrelevant to and so ignored
|
||||
during decompression.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Compression and decompression requirements, in bytes, can be estimated
|
||||
as:
|
||||
|
||||
<PRE>
|
||||
Compression: 400k + ( 8 x block size )
|
||||
|
||||
Decompression: 100k + ( 4 x block size ), or
|
||||
100k + ( 2.5 x block size )
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
Larger block sizes give rapidly diminishing marginal returns. Most of
|
||||
the compression comes from the first two or three hundred k of block
|
||||
size, a fact worth bearing in mind when using <CODE>bzip2</CODE> on small machines.
|
||||
It is also important to appreciate that the decompression memory
|
||||
requirement is set at compression time by the choice of block size.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
For files compressed with the default 900k block size, <CODE>bunzip2</CODE>
|
||||
will require about 3700 kbytes to decompress. To support decompression
|
||||
of any file on a 4 megabyte machine, <CODE>bunzip2</CODE> has an option to
|
||||
decompress using approximately half this amount of memory, about 2300
|
||||
kbytes. Decompression speed is also halved, so you should use this
|
||||
option only where necessary. The relevant flag is <CODE>-s</CODE>.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
In general, try and use the largest block size memory constraints allow,
|
||||
since that maximises the compression achieved. Compression and
|
||||
decompression speed are virtually unaffected by block size.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Another significant point applies to files which fit in a single block
|
||||
-- that means most files you'd encounter using a large block size. The
|
||||
amount of real memory touched is proportional to the size of the file,
|
||||
since the file is smaller than a block. For example, compressing a file
|
||||
20,000 bytes long with the flag <CODE>-9</CODE> will cause the compressor to
|
||||
allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560
|
||||
kbytes of it. Similarly, the decompressor will allocate 3700k but only
|
||||
touch 100k + 20000 * 4 = 180 kbytes.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Here is a table which summarises the maximum memory usage for different
|
||||
block sizes. Also recorded is the total compressed size for 14 files of
|
||||
the Calgary Text Compression Corpus totalling 3,141,622 bytes. This
|
||||
column gives some feel for how compression varies with block size.
|
||||
These figures tend to understate the advantage of larger block sizes for
|
||||
larger files, since the Corpus is dominated by smaller files.
|
||||
|
||||
<PRE>
|
||||
Compress Decompress Decompress Corpus
|
||||
Flag usage usage -s usage Size
|
||||
|
||||
-1 1200k 500k 350k 914704
|
||||
-2 2000k 900k 600k 877703
|
||||
-3 2800k 1300k 850k 860338
|
||||
-4 3600k 1700k 1100k 846899
|
||||
-5 4400k 2100k 1350k 845160
|
||||
-6 5200k 2500k 1600k 838626
|
||||
-7 6100k 2900k 1850k 834096
|
||||
-8 6800k 3300k 2100k 828642
|
||||
-9 7600k 3700k 2350k 828642
|
||||
</PRE>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC8" HREF="manual_toc.html#TOC8">RECOVERING DATA FROM DAMAGED FILES</A></H4>
|
||||
|
||||
<P>
|
||||
<CODE>bzip2</CODE> compresses files in blocks, usually 900kbytes long. Each
|
||||
block is handled independently. If a media or transmission error causes
|
||||
a multi-block <CODE>.bz2</CODE> file to become damaged, it may be possible to
|
||||
recover data from the undamaged blocks in the file.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
The compressed representation of each block is delimited by a 48-bit
|
||||
pattern, which makes it possible to find the block boundaries with
|
||||
reasonable certainty. Each block also carries its own 32-bit CRC, so
|
||||
damaged blocks can be distinguished from undamaged ones.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2recover</CODE> is a simple program whose purpose is to search for
|
||||
blocks in <CODE>.bz2</CODE> files, and write each block out into its own
|
||||
<CODE>.bz2</CODE> file. You can then use <CODE>bzip2 -t</CODE> to test the
|
||||
integrity of the resulting files, and decompress those which are
|
||||
undamaged.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2recover</CODE>
|
||||
takes a single argument, the name of the damaged file,
|
||||
and writes a number of files <CODE>rec0001file.bz2</CODE>,
|
||||
<CODE>rec0002file.bz2</CODE>, etc, containing the extracted blocks.
|
||||
The output filenames are designed so that the use of
|
||||
wildcards in subsequent processing -- for example,
|
||||
<CODE>bzip2 -dc rec*file.bz2 > recovered_data</CODE> -- lists the files in
|
||||
the correct order.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2recover</CODE> should be of most use dealing with large <CODE>.bz2</CODE>
|
||||
files, as these will contain many blocks. It is clearly
|
||||
futile to use it on damaged single-block files, since a
|
||||
damaged block cannot be recovered. If you wish to minimise
|
||||
any potential data loss through media or transmission errors,
|
||||
you might consider compressing with a smaller
|
||||
block size.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC9" HREF="manual_toc.html#TOC9">PERFORMANCE NOTES</A></H4>
|
||||
|
||||
<P>
|
||||
The sorting phase of compression gathers together similar strings in the
|
||||
file. Because of this, files containing very long runs of repeated
|
||||
symbols, like "aabaabaabaab ..." (repeated several hundred times) may
|
||||
compress more slowly than normal. Versions 0.9.5 and above fare much
|
||||
better than previous versions in this respect. The ratio between
|
||||
worst-case and average-case compression time is in the region of 10:1.
|
||||
For previous versions, this figure was more like 100:1. You can use the
|
||||
<CODE>-vvvv</CODE> option to monitor progress in great detail, if you want.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Decompression speed is unaffected by these phenomena.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2</CODE> usually allocates several megabytes of memory to operate
|
||||
in, and then charges all over it in a fairly random fashion. This means
|
||||
that performance, both for compressing and decompressing, is largely
|
||||
determined by the speed at which your machine can service cache misses.
|
||||
Because of this, small changes to the code to reduce the miss rate have
|
||||
been observed to give disproportionately large performance improvements.
|
||||
I imagine <CODE>bzip2</CODE> will perform best on machines with very large
|
||||
caches.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC10" HREF="manual_toc.html#TOC10">CAVEATS</A></H4>
|
||||
|
||||
<P>
|
||||
I/O error messages are not as helpful as they could be. <CODE>bzip2</CODE>
|
||||
tries hard to detect I/O errors and exit cleanly, but the details of
|
||||
what the problem is sometimes seem rather misleading.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
This manual page pertains to version 1.0 of <CODE>bzip2</CODE>. Compressed
|
||||
data created by this version is entirely forwards and backwards
|
||||
compatible with the previous public releases, versions 0.1pl2, 0.9.0 and
|
||||
0.9.5, but with the following exception: 0.9.0 and above can correctly
|
||||
decompress multiple concatenated compressed files. 0.1pl2 cannot do
|
||||
this; it will stop after decompressing just the first file in the
|
||||
stream.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2recover</CODE> uses 32-bit integers to represent bit positions in
|
||||
compressed files, so it cannot handle compressed files more than 512
|
||||
megabytes long. This could easily be fixed.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H4><A NAME="SEC11" HREF="manual_toc.html#TOC11">AUTHOR</A></H4>
|
||||
<P>
|
||||
Julian Seward, <CODE>[email protected]</CODE>.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
The ideas embodied in <CODE>bzip2</CODE> are due to (at least) the following
|
||||
people: Michael Burrows and David Wheeler (for the block sorting
|
||||
transformation), David Wheeler (again, for the Huffman coder), Peter
|
||||
Fenwick (for the structured coding model in the original <CODE>bzip</CODE>,
|
||||
and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
|
||||
(for the arithmetic coder in the original <CODE>bzip</CODE>). I am much
|
||||
indebted for their help, support and advice. See the manual in the
|
||||
source distribution for pointers to sources of documentation. Christian
|
||||
von Roques encouraged me to look for faster sorting algorithms, so as to
|
||||
speed up compression. Bela Lubkin encouraged me to improve the
|
||||
worst-case compression performance. Many people sent patches, helped
|
||||
with portability problems, lent machines, gave advice and were generally
|
||||
helpful.
|
||||
|
||||
</P>
|
||||
</BLOCKQUOTE>
|
||||
|
||||
<P><HR><P>
|
||||
<p>Go to the <A HREF="manual_1.html">first</A>, <A HREF="manual_1.html">previous</A>, <A HREF="manual_3.html">next</A>, <A HREF="manual_4.html">last</A> section, <A HREF="manual_toc.html">table of contents</A>.
|
||||
</BODY>
|
||||
</HTML>
|
||||
-1773
File diff suppressed because it is too large
Load Diff
-528
@@ -1,528 +0,0 @@
|
||||
<HTML>
|
||||
<HEAD>
|
||||
<!-- This HTML file has been created by texi2html 1.54
|
||||
from manual.texi on 23 March 2000 -->
|
||||
|
||||
<TITLE>bzip2 and libbzip2 - Miscellanea</TITLE>
|
||||
<link href="manual_3.html" rel=Previous>
|
||||
<link href="manual_toc.html" rel=ToC>
|
||||
|
||||
</HEAD>
|
||||
<BODY>
|
||||
<p>Go to the <A HREF="manual_1.html">first</A>, <A HREF="manual_3.html">previous</A>, next, last section, <A HREF="manual_toc.html">table of contents</A>.
|
||||
<P><HR><P>
|
||||
|
||||
|
||||
<H1><A NAME="SEC43" HREF="manual_toc.html#TOC43">Miscellanea</A></H1>
|
||||
|
||||
<P>
|
||||
These are just some random thoughts of mine. Your mileage may
|
||||
vary.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
<H2><A NAME="SEC44" HREF="manual_toc.html#TOC44">Limitations of the compressed file format</A></H2>
|
||||
<P>
|
||||
<CODE>bzip2-1.0</CODE>, <CODE>0.9.5</CODE> and <CODE>0.9.0</CODE>
|
||||
use exactly the same file format as the previous
|
||||
version, <CODE>bzip2-0.1</CODE>. This decision was made in the interests of
|
||||
stability. Creating yet another incompatible compressed file format
|
||||
would create further confusion and disruption for users.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Nevertheless, this is not a painless decision. Development
|
||||
work since the release of <CODE>bzip2-0.1</CODE> in August 1997
|
||||
has shown complexities in the file format which slow down
|
||||
decompression and, in retrospect, are unnecessary. These are:
|
||||
|
||||
<UL>
|
||||
<LI>The run-length encoder, which is the first of the
|
||||
|
||||
compression transformations, is entirely irrelevant.
|
||||
The original purpose was to protect the sorting algorithm
|
||||
from the very worst case input: a string of repeated
|
||||
symbols. But algorithm steps Q6a and Q6b in the original
|
||||
Burrows-Wheeler technical report (SRC-124) show how
|
||||
repeats can be handled without difficulty in block
|
||||
sorting.
|
||||
<LI>The randomisation mechanism doesn't really need to be
|
||||
|
||||
there. Udi Manber and Gene Myers published a suffix
|
||||
array construction algorithm a few years back, which
|
||||
can be employed to sort any block, no matter how
|
||||
repetitive, in O(N log N) time. Subsequent work by
|
||||
Kunihiko Sadakane has produced a derivative O(N (log N)^2)
|
||||
algorithm which usually outperforms the Manber-Myers
|
||||
algorithm.
|
||||
|
||||
I could have changed to Sadakane's algorithm, but I find
|
||||
it to be slower than <CODE>bzip2</CODE>'s existing algorithm for
|
||||
most inputs, and the randomisation mechanism protects
|
||||
adequately against bad cases. I didn't think it was
|
||||
a good tradeoff to make. Partly this is due to the fact
|
||||
that I was not flooded with email complaints about
|
||||
<CODE>bzip2-0.1</CODE>'s performance on repetitive data, so
|
||||
perhaps it isn't a problem for real inputs.
|
||||
|
||||
Probably the best long-term solution,
|
||||
and the one I have incorporated into 0.9.5 and above,
|
||||
is to use the existing sorting
|
||||
algorithm initially, and fall back to a O(N (log N)^2)
|
||||
algorithm if the standard algorithm gets into difficulties.
|
||||
<LI>The compressed file format was never designed to be
|
||||
|
||||
handled by a library, and I have had to jump though
|
||||
some hoops to produce an efficient implementation of
|
||||
decompression. It's a bit hairy. Try passing
|
||||
<CODE>decompress.c</CODE> through the C preprocessor
|
||||
and you'll see what I mean. Much of this complexity
|
||||
could have been avoided if the compressed size of
|
||||
each block of data was recorded in the data stream.
|
||||
<LI>An Adler-32 checksum, rather than a CRC32 checksum,
|
||||
|
||||
would be faster to compute.
|
||||
</UL>
|
||||
|
||||
<P>
|
||||
It would be fair to say that the <CODE>bzip2</CODE> format was frozen
|
||||
before I properly and fully understood the performance
|
||||
consequences of doing so.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Improvements which I was able to incorporate into
|
||||
0.9.0, despite using the same file format, are:
|
||||
|
||||
<UL>
|
||||
<LI>Single array implementation of the inverse BWT. This
|
||||
|
||||
significantly speeds up decompression, presumably
|
||||
because it reduces the number of cache misses.
|
||||
<LI>Faster inverse MTF transform for large MTF values. The
|
||||
|
||||
new implementation is based on the notion of sliding blocks
|
||||
of values.
|
||||
<LI><CODE>bzip2-0.9.0</CODE> now reads and writes files with <CODE>fread</CODE>
|
||||
|
||||
and <CODE>fwrite</CODE>; version 0.1 used <CODE>putc</CODE> and <CODE>getc</CODE>.
|
||||
Duh! Well, you live and learn.
|
||||
|
||||
</UL>
|
||||
|
||||
<P>
|
||||
Further ahead, it would be nice
|
||||
to be able to do random access into files. This will
|
||||
require some careful design of compressed file formats.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H2><A NAME="SEC45" HREF="manual_toc.html#TOC45">Portability issues</A></H2>
|
||||
<P>
|
||||
After some consideration, I have decided not to use
|
||||
GNU <CODE>autoconf</CODE> to configure 0.9.5 or 1.0.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>autoconf</CODE>, admirable and wonderful though it is,
|
||||
mainly assists with portability problems between Unix-like
|
||||
platforms. But <CODE>bzip2</CODE> doesn't have much in the way
|
||||
of portability problems on Unix; most of the difficulties appear
|
||||
when porting to the Mac, or to Microsoft's operating systems.
|
||||
<CODE>autoconf</CODE> doesn't help in those cases, and brings in a
|
||||
whole load of new complexity.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Most people should be able to compile the library and program
|
||||
under Unix straight out-of-the-box, so to speak, especially
|
||||
if you have a version of GNU C available.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
There are a couple of <CODE>__inline__</CODE> directives in the code. GNU C
|
||||
(<CODE>gcc</CODE>) should be able to handle them. If you're not using
|
||||
GNU C, your C compiler shouldn't see them at all.
|
||||
If your compiler does, for some reason, see them and doesn't
|
||||
like them, just <CODE>#define</CODE> <CODE>__inline__</CODE> to be <CODE>/* */</CODE>. One
|
||||
easy way to do this is to compile with the flag <CODE>-D__inline__=</CODE>,
|
||||
which should be understood by most Unix compilers.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
If you still have difficulties, try compiling with the macro
|
||||
<CODE>BZ_STRICT_ANSI</CODE> defined. This should enable you to build the
|
||||
library in a strictly ANSI compliant environment. Building the program
|
||||
itself like this is dangerous and not supported, since you remove
|
||||
<CODE>bzip2</CODE>'s checks against compressing directories, symbolic links,
|
||||
devices, and other not-really-a-file entities. This could cause
|
||||
filesystem corruption!
|
||||
|
||||
</P>
|
||||
<P>
|
||||
One other thing: if you create a <CODE>bzip2</CODE> binary for public
|
||||
distribution, please try and link it statically (<CODE>gcc -s</CODE>). This
|
||||
avoids all sorts of library-version issues that others may encounter
|
||||
later on.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
If you build <CODE>bzip2</CODE> on Win32, you must set <CODE>BZ_UNIX</CODE> to 0 and
|
||||
<CODE>BZ_LCCWIN32</CODE> to 1, in the file <CODE>bzip2.c</CODE>, before compiling.
|
||||
Otherwise the resulting binary won't work correctly.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H2><A NAME="SEC46" HREF="manual_toc.html#TOC46">Reporting bugs</A></H2>
|
||||
<P>
|
||||
I tried pretty hard to make sure <CODE>bzip2</CODE> is
|
||||
bug free, both by design and by testing. Hopefully
|
||||
you'll never need to read this section for real.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Nevertheless, if <CODE>bzip2</CODE> dies with a segmentation
|
||||
fault, a bus error or an internal assertion failure, it
|
||||
will ask you to email me a bug report. Experience with
|
||||
version 0.1 shows that almost all these problems can
|
||||
be traced to either compiler bugs or hardware problems.
|
||||
|
||||
<UL>
|
||||
<LI>
|
||||
|
||||
Recompile the program with no optimisation, and see if it
|
||||
works. And/or try a different compiler.
|
||||
I heard all sorts of stories about various flavours
|
||||
of GNU C (and other compilers) generating bad code for
|
||||
<CODE>bzip2</CODE>, and I've run across two such examples myself.
|
||||
|
||||
2.7.X versions of GNU C are known to generate bad code from
|
||||
time to time, at high optimisation levels.
|
||||
If you get problems, try using the flags
|
||||
<CODE>-O2</CODE> <CODE>-fomit-frame-pointer</CODE> <CODE>-fno-strength-reduce</CODE>.
|
||||
You should specifically <EM>not</EM> use <CODE>-funroll-loops</CODE>.
|
||||
|
||||
You may notice that the Makefile runs six tests as part of
|
||||
the build process. If the program passes all of these, it's
|
||||
a pretty good (but not 100%) indication that the compiler has
|
||||
done its job correctly.
|
||||
<LI>
|
||||
|
||||
If <CODE>bzip2</CODE> crashes randomly, and the crashes are not
|
||||
repeatable, you may have a flaky memory subsystem. <CODE>bzip2</CODE>
|
||||
really hammers your memory hierarchy, and if it's a bit marginal,
|
||||
you may get these problems. Ditto if your disk or I/O subsystem
|
||||
is slowly failing. Yup, this really does happen.
|
||||
|
||||
Try using a different machine of the same type, and see if
|
||||
you can repeat the problem.
|
||||
<LI>This isn't really a bug, but ... If <CODE>bzip2</CODE> tells
|
||||
|
||||
you your file is corrupted on decompression, and you
|
||||
obtained the file via FTP, there is a possibility that you
|
||||
forgot to tell FTP to do a binary mode transfer. That absolutely
|
||||
will cause the file to be non-decompressible. You'll have to transfer
|
||||
it again.
|
||||
</UL>
|
||||
|
||||
<P>
|
||||
If you've incorporated <CODE>libbzip2</CODE> into your own program
|
||||
and are getting problems, please, please, please, check that the
|
||||
parameters you are passing in calls to the library, are
|
||||
correct, and in accordance with what the documentation says
|
||||
is allowable. I have tried to make the library robust against
|
||||
such problems, but I'm sure I haven't succeeded.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Finally, if the above comments don't help, you'll have to send
|
||||
me a bug report. Now, it's just amazing how many people will
|
||||
send me a bug report saying something like
|
||||
|
||||
<PRE>
|
||||
bzip2 crashed with segmentation fault on my machine
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
and absolutely nothing else. Needless to say, a such a report
|
||||
is <EM>totally, utterly, completely and comprehensively 100% useless;
|
||||
a waste of your time, my time, and net bandwidth</EM>.
|
||||
With no details at all, there's no way I can possibly begin
|
||||
to figure out what the problem is.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
The rules of the game are: facts, facts, facts. Don't omit
|
||||
them because "oh, they won't be relevant". At the bare
|
||||
minimum:
|
||||
|
||||
<PRE>
|
||||
Machine type. Operating system version.
|
||||
Exact version of <CODE>bzip2</CODE> (do <CODE>bzip2 -V</CODE>).
|
||||
Exact version of the compiler used.
|
||||
Flags passed to the compiler.
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
However, the most important single thing that will help me is
|
||||
the file that you were trying to compress or decompress at the
|
||||
time the problem happened. Without that, my ability to do anything
|
||||
more than speculate about the cause, is limited.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Please remember that I connect to the Internet with a modem, so
|
||||
you should contact me before mailing me huge files.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H2><A NAME="SEC47" HREF="manual_toc.html#TOC47">Did you get the right package?</A></H2>
|
||||
|
||||
<P>
|
||||
<CODE>bzip2</CODE> is a resource hog. It soaks up large amounts of CPU cycles
|
||||
and memory. Also, it gives very large latencies. In the worst case, you
|
||||
can feed many megabytes of uncompressed data into the library before
|
||||
getting any compressed output, so this probably rules out applications
|
||||
requiring interactive behaviour.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
These aren't faults of my implementation, I hope, but more
|
||||
an intrinsic property of the Burrows-Wheeler transform (unfortunately).
|
||||
Maybe this isn't what you want.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
If you want a compressor and/or library which is faster, uses less
|
||||
memory but gets pretty good compression, and has minimal latency,
|
||||
consider Jean-loup
|
||||
Gailly's and Mark Adler's work, <CODE>zlib-1.1.2</CODE> and
|
||||
<CODE>gzip-1.2.4</CODE>. Look for them at
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>http://www.cdrom.com/pub/infozip/zlib</CODE> and
|
||||
<CODE>http://www.gzip.org</CODE> respectively.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
For something faster and lighter still, you might try Markus F X J
|
||||
Oberhumer's <CODE>LZO</CODE> real-time compression/decompression library, at
|
||||
<BR> <CODE>http://wildsau.idv.uni-linz.ac.at/mfx/lzo.html</CODE>.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
If you want to use the <CODE>bzip2</CODE> algorithms to compress small blocks
|
||||
of data, 64k bytes or smaller, for example on an on-the-fly disk
|
||||
compressor, you'd be well advised not to use this library. Instead,
|
||||
I've made a special library tuned for that kind of use. It's part of
|
||||
<CODE>e2compr-0.40</CODE>, an on-the-fly disk compressor for the Linux
|
||||
<CODE>ext2</CODE> filesystem. Look at
|
||||
<CODE>http://www.netspace.net.au/~reiter/e2compr</CODE>.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H2><A NAME="SEC48" HREF="manual_toc.html#TOC48">Testing</A></H2>
|
||||
|
||||
<P>
|
||||
A record of the tests I've done.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
First, some data sets:
|
||||
|
||||
<UL>
|
||||
<LI>B: a directory containing 6001 files, one for every length in the
|
||||
|
||||
range 0 to 6000 bytes. The files contain random lowercase
|
||||
letters. 18.7 megabytes.
|
||||
<LI>H: my home directory tree. Documents, source code, mail files,
|
||||
|
||||
compressed data. H contains B, and also a directory of
|
||||
files designed as boundary cases for the sorting; mostly very
|
||||
repetitive, nasty files. 565 megabytes.
|
||||
<LI>A: directory tree holding various applications built from source:
|
||||
|
||||
<CODE>egcs</CODE>, <CODE>gcc-2.8.1</CODE>, KDE, GTK, Octave, etc.
|
||||
2200 megabytes.
|
||||
</UL>
|
||||
|
||||
<P>
|
||||
The tests conducted are as follows. Each test means compressing
|
||||
(a copy of) each file in the data set, decompressing it and
|
||||
comparing it against the original.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
First, a bunch of tests with block sizes and internal buffer
|
||||
sizes set very small,
|
||||
to detect any problems with the
|
||||
blocking and buffering mechanisms.
|
||||
This required modifying the source code so as to try to
|
||||
break it.
|
||||
|
||||
<OL>
|
||||
<LI>Data set H, with
|
||||
|
||||
buffer size of 1 byte, and block size of 23 bytes.
|
||||
<LI>Data set B, buffer sizes 1 byte, block size 1 byte.
|
||||
|
||||
<LI>As (2) but small-mode decompression.
|
||||
|
||||
<LI>As (2) with block size 2 bytes.
|
||||
|
||||
<LI>As (2) with block size 3 bytes.
|
||||
|
||||
<LI>As (2) with block size 4 bytes.
|
||||
|
||||
<LI>As (2) with block size 5 bytes.
|
||||
|
||||
<LI>As (2) with block size 6 bytes and small-mode decompression.
|
||||
|
||||
<LI>H with buffer size of 1 byte, but normal block
|
||||
|
||||
size (up to 900000 bytes).
|
||||
</OL>
|
||||
|
||||
<P>
|
||||
Then some tests with unmodified source code.
|
||||
|
||||
<OL>
|
||||
<LI>H, all settings normal.
|
||||
|
||||
<LI>As (1), with small-mode decompress.
|
||||
|
||||
<LI>H, compress with flag <CODE>-1</CODE>.
|
||||
|
||||
<LI>H, compress with flag <CODE>-s</CODE>, decompress with flag <CODE>-s</CODE>.
|
||||
|
||||
<LI>Forwards compatibility: H, <CODE>bzip2-0.1pl2</CODE> compressing,
|
||||
|
||||
<CODE>bzip2-0.9.5</CODE> decompressing, all settings normal.
|
||||
<LI>Backwards compatibility: H, <CODE>bzip2-0.9.5</CODE> compressing,
|
||||
|
||||
<CODE>bzip2-0.1pl2</CODE> decompressing, all settings normal.
|
||||
<LI>Bigger tests: A, all settings normal.
|
||||
|
||||
<LI>As (7), using the fallback (Sadakane-like) sorting algorithm.
|
||||
|
||||
<LI>As (8), compress with flag <CODE>-1</CODE>, decompress with flag
|
||||
|
||||
<CODE>-s</CODE>.
|
||||
<LI>H, using the fallback sorting algorithm.
|
||||
|
||||
<LI>Forwards compatibility: A, <CODE>bzip2-0.1pl2</CODE> compressing,
|
||||
|
||||
<CODE>bzip2-0.9.5</CODE> decompressing, all settings normal.
|
||||
<LI>Backwards compatibility: A, <CODE>bzip2-0.9.5</CODE> compressing,
|
||||
|
||||
<CODE>bzip2-0.1pl2</CODE> decompressing, all settings normal.
|
||||
<LI>Misc test: about 400 megabytes of <CODE>.tar</CODE> files with
|
||||
|
||||
<CODE>bzip2</CODE> compiled with Checker (a memory access error
|
||||
detector, like Purify).
|
||||
<LI>Misc tests to make sure it builds and runs ok on non-Linux/x86
|
||||
|
||||
platforms.
|
||||
</OL>
|
||||
|
||||
<P>
|
||||
These tests were conducted on a 225 MHz IDT WinChip machine, running
|
||||
Linux 2.0.36. They represent nearly a week of continuous computation.
|
||||
All tests completed successfully.
|
||||
|
||||
</P>
|
||||
|
||||
|
||||
|
||||
<H2><A NAME="SEC49" HREF="manual_toc.html#TOC49">Further reading</A></H2>
|
||||
<P>
|
||||
<CODE>bzip2</CODE> is not research work, in the sense that it doesn't present
|
||||
any new ideas. Rather, it's an engineering exercise based on existing
|
||||
ideas.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Four documents describe essentially all the ideas behind <CODE>bzip2</CODE>:
|
||||
|
||||
<PRE>
|
||||
Michael Burrows and D. J. Wheeler:
|
||||
"A block-sorting lossless data compression algorithm"
|
||||
10th May 1994.
|
||||
Digital SRC Research Report 124.
|
||||
ftp://ftp.digital.com/pub/DEC/SRC/research-reports/SRC-124.ps.gz
|
||||
If you have trouble finding it, try searching at the
|
||||
New Zealand Digital Library, http://www.nzdl.org.
|
||||
|
||||
Daniel S. Hirschberg and Debra A. LeLewer
|
||||
"Efficient Decoding of Prefix Codes"
|
||||
Communications of the ACM, April 1990, Vol 33, Number 4.
|
||||
You might be able to get an electronic copy of this
|
||||
from the ACM Digital Library.
|
||||
|
||||
David J. Wheeler
|
||||
Program bred3.c and accompanying document bred3.ps.
|
||||
This contains the idea behind the multi-table Huffman
|
||||
coding scheme.
|
||||
ftp://ftp.cl.cam.ac.uk/users/djw3/
|
||||
|
||||
Jon L. Bentley and Robert Sedgewick
|
||||
"Fast Algorithms for Sorting and Searching Strings"
|
||||
Available from Sedgewick's web page,
|
||||
www.cs.princeton.edu/~rs
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
The following paper gives valuable additional insights into the
|
||||
algorithm, but is not immediately the basis of any code
|
||||
used in bzip2.
|
||||
|
||||
<PRE>
|
||||
Peter Fenwick:
|
||||
Block Sorting Text Compression
|
||||
Proceedings of the 19th Australasian Computer Science Conference,
|
||||
Melbourne, Australia. Jan 31 - Feb 2, 1996.
|
||||
ftp://ftp.cs.auckland.ac.nz/pub/peter-f/ACSC96paper.ps
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
Kunihiko Sadakane's sorting algorithm, mentioned above,
|
||||
is available from:
|
||||
|
||||
<PRE>
|
||||
http://naomi.is.s.u-tokyo.ac.jp/~sada/papers/Sada98b.ps.gz
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
The Manber-Myers suffix array construction
|
||||
algorithm is described in a paper
|
||||
available from:
|
||||
|
||||
<PRE>
|
||||
http://www.cs.arizona.edu/people/gene/PAPERS/suffix.ps
|
||||
</PRE>
|
||||
|
||||
<P>
|
||||
Finally, the following paper documents some recent investigations
|
||||
I made into the performance of sorting algorithms:
|
||||
|
||||
<PRE>
|
||||
Julian Seward:
|
||||
On the Performance of BWT Sorting Algorithms
|
||||
Proceedings of the IEEE Data Compression Conference 2000
|
||||
Snowbird, Utah. 28-30 March 2000.
|
||||
</PRE>
|
||||
|
||||
<P><HR><P>
|
||||
<p>Go to the <A HREF="manual_1.html">first</A>, <A HREF="manual_3.html">previous</A>, next, last section, <A HREF="manual_toc.html">table of contents</A>.
|
||||
</BODY>
|
||||
</HTML>
|
||||
-173
@@ -1,173 +0,0 @@
|
||||
<HTML>
|
||||
<HEAD>
|
||||
<!-- This HTML file has been created by texi2html 1.54
|
||||
from manual.texi on 23 March 2000 -->
|
||||
|
||||
<TITLE>bzip2 and libbzip2 - Table of Contents</TITLE>
|
||||
|
||||
</HEAD>
|
||||
<BODY>
|
||||
<H1>bzip2 and libbzip2</H1>
|
||||
<H2>a program and library for data compression</H2>
|
||||
<H2>copyright (C) 1996-2000 Julian Seward</H2>
|
||||
<H2>version 1.0 of 21 March 2000</H2>
|
||||
<ADDRESS>Julian Seward</ADDRESS>
|
||||
<P>
|
||||
<P><HR><P>
|
||||
|
||||
<P>
|
||||
This program, <CODE>bzip2</CODE>,
|
||||
and associated library <CODE>libbzip2</CODE>, are
|
||||
Copyright (C) 1996-2000 Julian R Seward. All rights reserved.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions
|
||||
are met:
|
||||
|
||||
<UL>
|
||||
<LI>
|
||||
|
||||
Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
<LI>
|
||||
|
||||
The origin of this software must not be misrepresented; you must
|
||||
not claim that you wrote the original software. If you use this
|
||||
software in a product, an acknowledgment in the product
|
||||
documentation would be appreciated but is not required.
|
||||
<LI>
|
||||
|
||||
Altered source versions must be plainly marked as such, and must
|
||||
not be misrepresented as being the original software.
|
||||
<LI>
|
||||
|
||||
The name of the author may not be used to endorse or promote
|
||||
products derived from this software without specific prior written
|
||||
permission.
|
||||
</UL>
|
||||
|
||||
<P>
|
||||
THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY EXPRESS
|
||||
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
||||
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
|
||||
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||||
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
|
||||
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
|
||||
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
|
||||
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
||||
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
Julian Seward, Cambridge, UK.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>[email protected]</CODE>
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>http://sourceware.cygnus.com/bzip2</CODE>
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>http://www.cacheprof.org</CODE>
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>http://www.muraroa.demon.co.uk</CODE>
|
||||
|
||||
</P>
|
||||
<P>
|
||||
<CODE>bzip2</CODE>/<CODE>libbzip2</CODE> version 1.0 of 21 March 2000.
|
||||
|
||||
</P>
|
||||
<P>
|
||||
PATENTS: To the best of my knowledge, <CODE>bzip2</CODE> does not use any patented
|
||||
algorithms. However, I do not have the resources available to carry out
|
||||
a full patent search. Therefore I cannot give any guarantee of the
|
||||
above statement.
|
||||
|
||||
</P>
|
||||
|
||||
<UL>
|
||||
<LI><A NAME="TOC1" HREF="manual_1.html#SEC1">Introduction</A>
|
||||
<LI><A NAME="TOC2" HREF="manual_2.html#SEC2">How to use <CODE>bzip2</CODE></A>
|
||||
<UL>
|
||||
<UL>
|
||||
<UL>
|
||||
<LI><A NAME="TOC3" HREF="manual_2.html#SEC3">NAME</A>
|
||||
<LI><A NAME="TOC4" HREF="manual_2.html#SEC4">SYNOPSIS</A>
|
||||
<LI><A NAME="TOC5" HREF="manual_2.html#SEC5">DESCRIPTION</A>
|
||||
<LI><A NAME="TOC6" HREF="manual_2.html#SEC6">OPTIONS</A>
|
||||
<LI><A NAME="TOC7" HREF="manual_2.html#SEC7">MEMORY MANAGEMENT</A>
|
||||
<LI><A NAME="TOC8" HREF="manual_2.html#SEC8">RECOVERING DATA FROM DAMAGED FILES</A>
|
||||
<LI><A NAME="TOC9" HREF="manual_2.html#SEC9">PERFORMANCE NOTES</A>
|
||||
<LI><A NAME="TOC10" HREF="manual_2.html#SEC10">CAVEATS</A>
|
||||
<LI><A NAME="TOC11" HREF="manual_2.html#SEC11">AUTHOR</A>
|
||||
</UL>
|
||||
</UL>
|
||||
</UL>
|
||||
<LI><A NAME="TOC12" HREF="manual_3.html#SEC12">Programming with <CODE>libbzip2</CODE></A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC13" HREF="manual_3.html#SEC13">Top-level structure</A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC14" HREF="manual_3.html#SEC14">Low-level summary</A>
|
||||
<LI><A NAME="TOC15" HREF="manual_3.html#SEC15">High-level summary</A>
|
||||
<LI><A NAME="TOC16" HREF="manual_3.html#SEC16">Utility functions summary</A>
|
||||
</UL>
|
||||
<LI><A NAME="TOC17" HREF="manual_3.html#SEC17">Error handling</A>
|
||||
<LI><A NAME="TOC18" HREF="manual_3.html#SEC18">Low-level interface</A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC19" HREF="manual_3.html#SEC19"><CODE>BZ2_bzCompressInit</CODE></A>
|
||||
<LI><A NAME="TOC20" HREF="manual_3.html#SEC20"><CODE>BZ2_bzCompress</CODE></A>
|
||||
<LI><A NAME="TOC21" HREF="manual_3.html#SEC21"><CODE>BZ2_bzCompressEnd</CODE></A>
|
||||
<LI><A NAME="TOC22" HREF="manual_3.html#SEC22"><CODE>BZ2_bzDecompressInit</CODE></A>
|
||||
<LI><A NAME="TOC23" HREF="manual_3.html#SEC23"><CODE>BZ2_bzDecompress</CODE></A>
|
||||
<LI><A NAME="TOC24" HREF="manual_3.html#SEC24"><CODE>BZ2_bzDecompressEnd</CODE></A>
|
||||
</UL>
|
||||
<LI><A NAME="TOC25" HREF="manual_3.html#SEC25">High-level interface</A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC26" HREF="manual_3.html#SEC26"><CODE>BZ2_bzReadOpen</CODE></A>
|
||||
<LI><A NAME="TOC27" HREF="manual_3.html#SEC27"><CODE>BZ2_bzRead</CODE></A>
|
||||
<LI><A NAME="TOC28" HREF="manual_3.html#SEC28"><CODE>BZ2_bzReadGetUnused</CODE></A>
|
||||
<LI><A NAME="TOC29" HREF="manual_3.html#SEC29"><CODE>BZ2_bzReadClose</CODE></A>
|
||||
<LI><A NAME="TOC30" HREF="manual_3.html#SEC30"><CODE>BZ2_bzWriteOpen</CODE></A>
|
||||
<LI><A NAME="TOC31" HREF="manual_3.html#SEC31"><CODE>BZ2_bzWrite</CODE></A>
|
||||
<LI><A NAME="TOC32" HREF="manual_3.html#SEC32"><CODE>BZ2_bzWriteClose</CODE></A>
|
||||
<LI><A NAME="TOC33" HREF="manual_3.html#SEC33">Handling embedded compressed data streams</A>
|
||||
<LI><A NAME="TOC34" HREF="manual_3.html#SEC34">Standard file-reading/writing code</A>
|
||||
</UL>
|
||||
<LI><A NAME="TOC35" HREF="manual_3.html#SEC35">Utility functions</A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC36" HREF="manual_3.html#SEC36"><CODE>BZ2_bzBuffToBuffCompress</CODE></A>
|
||||
<LI><A NAME="TOC37" HREF="manual_3.html#SEC37"><CODE>BZ2_bzBuffToBuffDecompress</CODE></A>
|
||||
</UL>
|
||||
<LI><A NAME="TOC38" HREF="manual_3.html#SEC38"><CODE>zlib</CODE> compatibility functions</A>
|
||||
<LI><A NAME="TOC39" HREF="manual_3.html#SEC39">Using the library in a <CODE>stdio</CODE>-free environment</A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC40" HREF="manual_3.html#SEC40">Getting rid of <CODE>stdio</CODE></A>
|
||||
<LI><A NAME="TOC41" HREF="manual_3.html#SEC41">Critical error handling</A>
|
||||
</UL>
|
||||
<LI><A NAME="TOC42" HREF="manual_3.html#SEC42">Making a Windows DLL</A>
|
||||
</UL>
|
||||
<LI><A NAME="TOC43" HREF="manual_4.html#SEC43">Miscellanea</A>
|
||||
<UL>
|
||||
<LI><A NAME="TOC44" HREF="manual_4.html#SEC44">Limitations of the compressed file format</A>
|
||||
<LI><A NAME="TOC45" HREF="manual_4.html#SEC45">Portability issues</A>
|
||||
<LI><A NAME="TOC46" HREF="manual_4.html#SEC46">Reporting bugs</A>
|
||||
<LI><A NAME="TOC47" HREF="manual_4.html#SEC47">Did you get the right package?</A>
|
||||
<LI><A NAME="TOC48" HREF="manual_4.html#SEC48">Testing</A>
|
||||
<LI><A NAME="TOC49" HREF="manual_4.html#SEC49">Further reading</A>
|
||||
</UL>
|
||||
</UL>
|
||||
<P><HR><P>
|
||||
This document was generated on 23 March 2000 using the
|
||||
<A HREF="http://wwwcn.cern.ch/dci/texi2html/">texi2html</A>
|
||||
translator version 1.51a.</P>
|
||||
</BODY>
|
||||
</HTML>
|
||||
BIN
Binary file not shown.
BIN
Binary file not shown.
-30007
File diff suppressed because it is too large
Load Diff
Vendored
-9
@@ -1,9 +0,0 @@
|
||||
|
||||
If compilation produces errors, or a large number of warnings,
|
||||
please read README.COMPILATION.PROBLEMS -- you might be able to
|
||||
adjust the flags in this Makefile to improve matters.
|
||||
|
||||
Also in README.COMPILATION.PROBLEMS are some hints that may help
|
||||
if your build produces an executable which is unable to correctly
|
||||
handle so-called 'large files' -- files of size 2GB or more.
|
||||
|
||||
Vendored
-4
@@ -1,4 +0,0 @@
|
||||
|
||||
Doing 6 tests (3 compress, 3 uncompress) ...
|
||||
If there's a problem, things might stop at this point.
|
||||
|
||||
Vendored
-5
@@ -1,5 +0,0 @@
|
||||
|
||||
Checking test results. If any of the four "cmp"s which follow
|
||||
report any differences, something is wrong. If you can't easily
|
||||
figure out what, please let me know ([email protected]).
|
||||
|
||||
Vendored
-30
@@ -1,30 +0,0 @@
|
||||
|
||||
If you got this far and the 'cmp's didn't complain, it looks
|
||||
like you're in business.
|
||||
|
||||
To install in /usr/local/bin, /usr/local/lib, /usr/local/man and
|
||||
/usr/local/include, type
|
||||
|
||||
make install
|
||||
|
||||
To install somewhere else, eg, /xxx/yyy/{bin,lib,man,include}, type
|
||||
|
||||
make install PREFIX=/xxx/yyy
|
||||
|
||||
If you are (justifiably) paranoid and want to see what 'make install'
|
||||
is going to do, you can first do
|
||||
|
||||
make -n install or
|
||||
make -n install PREFIX=/xxx/yyy respectively.
|
||||
|
||||
The -n instructs make to show the commands it would execute, but
|
||||
not actually execute them.
|
||||
|
||||
Instructions for use are in the preformatted manual page, in the file
|
||||
bzip2.txt. For more detailed documentation, read the full manual.
|
||||
It is available in Postscript form (manual.ps), PDF form (manual.pdf),
|
||||
and HTML form (manual.html).
|
||||
|
||||
You can also do "bzip2 --help" to see some helpful information.
|
||||
"bzip2 -L" displays the software license.
|
||||
|
||||
Reference in New Issue
Block a user