Struct FileDecompressor 

pub struct FileDecompressor {
    pub lines_consumed: usize,
    /* private fields */
}

State for incremental zstd decompression and line extraction from a single file.

Fields

lines_consumed: usize

Global line index up to which lines have been consumed or skipped. This is the value that should be used for checkpointing. Equals skip_before + number of lines actually returned to caller.

Implementations

impl FileDecompressor

pub fn open(path: &Path) -> Result<Self>

Open a file and prepare for incremental zstd decompression.

pub fn open_with_max_line_size( path: &Path, max_out_buffer: usize, ) -> Result<Self>

Open a file, capping the output buffer at max_out_buffer bytes. A record that fills the cap without a newline is discarded and reading continues with the next record (see NextLine::Skipped).

pub fn next_line(&mut self, skip_before: usize) -> Result<NextLine>

Returns the next line from this file.

skip_before: lines with index < skip_before are discarded.

Returns:

• Ok(NextLine::Line(line)) – a complete line was extracted.
• Ok(NextLine::Skipped { .. }) – a record exceeding the maximum line size was discarded. Reading continues with the next record.
• Ok(NextLine::None) – there isn’t any more data available right now. The caller should check if the file is done or retry later.

pub fn reset_eof(&mut self)

Reset the EOF flag so we can try reading more data (useful when tailing a file that is still being written to).

pub fn has_partial_data(&self) -> bool

Returns true if there is partial (incomplete line) data remaining: either buffered bytes with no terminating newline yet, or a record still being discarded for exceeding the maximum line size. When a done file ends in this state the trailing record is incomplete and is dropped.

pub fn is_discarding_oversized_record(&self) -> bool

Returns true if the trailing partial data is a record being discarded for exceeding the maximum line size, as opposed to an unterminated line left behind by a writer that exited without flushing.