|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.io.SequenceFile
public class SequenceFile
Support for flat files of binary key/value pairs.
Nested Class Summary | |
---|---|
static class |
SequenceFile.CompressionType
The type of compression. |
static class |
SequenceFile.Metadata
The class encapsulating with the metadata of a file. |
static class |
SequenceFile.Reader
Reads key/value pairs from a sequence-format file. |
static class |
SequenceFile.Sorter
Sorts key/value pairs in a sequence-format file. |
static interface |
SequenceFile.ValueBytes
The interface to 'raw' values of SequenceFiles. |
static class |
SequenceFile.Writer
Write key/value pairs to a sequence-format file. |
Field Summary | |
---|---|
static org.apache.commons.logging.Log |
LOG
|
static int |
SYNC_INTERVAL
The number of bytes between sync points. |
Method Summary | |
---|---|
static SequenceFile.Writer |
createWriter(Configuration conf,
FSDataOutputStream out,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec)
Construct the preferred type of 'raw' SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(Configuration conf,
FSDataOutputStream out,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
SequenceFile.Metadata metadata)
Construct the preferred type of 'raw' SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass)
Construct the preferred type of SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType)
Construct the preferred type of SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec)
Construct the preferred type of SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress)
Construct the preferred type of SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
CompressionCodec codec,
Progressable progress,
SequenceFile.Metadata metadata)
Construct the preferred type of SequenceFile Writer. |
static SequenceFile.Writer |
createWriter(FileSystem fs,
Configuration conf,
Path name,
Class keyClass,
Class valClass,
SequenceFile.CompressionType compressionType,
Progressable progress)
Construct the preferred type of SequenceFile Writer. |
static SequenceFile.CompressionType |
getCompressionType(Configuration job)
Get the compression type for the reduce outputs |
static void |
setCompressionType(Configuration job,
SequenceFile.CompressionType val)
Set the compression type for sequence files. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final org.apache.commons.logging.Log LOG
public static final int SYNC_INTERVAL
Method Detail |
---|
public static SequenceFile.CompressionType getCompressionType(Configuration job)
job
- the job config to look in
public static void setCompressionType(Configuration job, SequenceFile.CompressionType val)
job
- the configuration to modifyval
- the new compression type (none, block, record)public static SequenceFile.Writer createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass) throws IOException
fs
- The configured filesystem.conf
- The configuration.name
- The name of the file.keyClass
- The 'key' type.valClass
- The 'value' type.
IOException
public static SequenceFile.Writer createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType) throws IOException
fs
- The configured filesystem.conf
- The configuration.name
- The name of the file.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.
IOException
public static SequenceFile.Writer createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, Progressable progress) throws IOException
fs
- The configured filesystem.conf
- The configuration.name
- The name of the file.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.progress
- The Progressable object to track progress.
IOException
public static SequenceFile.Writer createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec) throws IOException
fs
- The configured filesystem.conf
- The configuration.name
- The name of the file.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.codec
- The compression codec.
IOException
public static SequenceFile.Writer createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec, Progressable progress, SequenceFile.Metadata metadata) throws IOException
fs
- The configured filesystem.conf
- The configuration.name
- The name of the file.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.codec
- The compression codec.progress
- The Progressable object to track progress.metadata
- The metadata of the file.
IOException
public static SequenceFile.Writer createWriter(FileSystem fs, Configuration conf, Path name, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec, Progressable progress) throws IOException
fs
- The configured filesystem.conf
- The configuration.name
- The name of the file.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.codec
- The compression codec.progress
- The Progressable object to track progress.
IOException
public static SequenceFile.Writer createWriter(Configuration conf, FSDataOutputStream out, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec, SequenceFile.Metadata metadata) throws IOException
conf
- The configuration.out
- The stream on top which the writer is to be constructed.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.codec
- The compression codec.metadata
- The metadata of the file.
IOException
public static SequenceFile.Writer createWriter(Configuration conf, FSDataOutputStream out, Class keyClass, Class valClass, SequenceFile.CompressionType compressionType, CompressionCodec codec) throws IOException
conf
- The configuration.out
- The stream on top which the writer is to be constructed.keyClass
- The 'key' type.valClass
- The 'value' type.compressionType
- The compression type.codec
- The compression codec.
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |