Module BatIO

module BatIO: sig .. end

High-order abstract I/O.

This module deals with BatIO.inputs and BatIO.outputs. Inputs are manners of getting information from the outside world and into your program (for instance, reading from the network, from a file, etc.) Outputs are manners of getting information out from your program and into the outside world (for instance, sending something onto the network, onto a file, etc.) In other words, if you are looking for a way to modify files, read from the network, etc., you're in the right place.

To perform I/O, you first need to open your BatIO.input or your BatIO.output. Chances are that there is an opening operation for this task. Note that most opening operations are defined in their respective module. Operations for opening files are defined in module File, operations for opening communications with the network or with other processes are defined in module Unix. Opening operations related to compression and decompression are defined in module Compress, etc.

Once you have opened an BatIO.input, you may read the data it contains by using functions such as BatIO.read (to read one character), BatIO.nread or BatIO.input (to read one string) or one of the read_* functions. If you need not one information but a complete enumeration, for instance for processing many information before writing them, you may also convert the input into an enumeration, by using one of the *s_of functions.

Once you have opened an BatIO.output, you may write data to this output by using functions scuh as BatIO.write (to write one char), BatIO.nwrite or BatIO.output (to write one string) or one of the write_* functions. If you have not just one piece of data but a complete enumeration, you may write this whole enumeration to the output by using one of the write_*s functions. Note that most operations on output are said to be buffered. This means that small writing operations may be automatically delayed and grouped into large writing operations, as these are generally faster and induce less wear on the hardware. Occasionally, you may wish to force all waiting operations to take place now. For this purpose, you may call function BatIO.flush.

Once you have finished using your BatIO.input or your BatIO.output, chances are that you will want to close it. This is not a strict necessity, as OCaml will eventually close it for you when it detects that you have no more need of that BatIO.input/BatIO.output, but this is generally a good policy, as this will let other programs access the resources which are currently allocated to that BatIO.input/BatIO.output -- typically, under Windows, if you are reading the contents of a file from a program, no other program may read the contents of that file simultaneously and you may also not rename or move the file to another directory. To close an BatIO.input, use function BatIO.close_in and to close an BatIO.output, use function BatIO.close_out.

Note Some BatIO.inputs are built on top of other BatIO.inputs to provide transparent translations (e.g. on-the-fly decompression of a file or network information) and that some BatIO.outputs are built on top of other BatIO.outputs for the same purpose (e.g. on-the-fly compression of a file or network information). In this case, closing the "outer" BatIO.input/BatIO.output (e.g. the decompressor/compressor) will not close the "inner" BatIO.input/BatIO.output (e.g. access to the file or to the network). You will need to close the "inner" BatIO.input/BatIO.output, which will automatically flush the outer BatIO.input/BatIO.output and close it.

Author(s): Nicolas Cannasse, David Teller, Philippe Strauss, Edgar Friendly

type input = BatInnerIO.input

The abstract input type.

type 'a output = 'a BatInnerIO.output

The abstract output type, 'a is the accumulator data, it is returned when the close_out function is called.

type ('a, 'b) printer = 'b output -> 'a -> unit

The type of a printing function to print a 'a to an output that produces 'b as result.

type 'a f_printer = Stdlib.Format.formatter -> 'a -> unit

exception No_more_input

This exception is raised when reading on an input with the read or nread functions while there is no available token to read.

exception Input_closed

This exception is raised when reading on a closed input.

exception Output_closed

This exception is raised when reading on a closed output.

Standard inputs/outputs

val stdin : input

Standard input, as per Unix/Windows conventions (by default, keyboard).

Example: if read_line stdin |> Int.of_string > 10 then failwith "too big a number read";

val stdout : unit output

Standard output, as per Unix/Windows conventions (by default, console).

Use this output to display regular messages. Example: write_string stdout "Enter your name:"; let name = read_line stdin in write_line stdout ("Your name is " ^ name);

val stderr : unit output

Standard error output, as per Unix/Windows conventions.

Use this output to display warnings and error messages.

Example: write_line stderr "Error on Internet - please delete google.com";

val stdnull : unit output

An output which discards everything written to it.

Use this output to ignore messages.

Example: let out_ch = if debug then stderr else stdnull in write_line out_ch "Program running.";

Standard API

val read : input -> char

Read a single char from an input or raise No_more_input if no input is available.

Example: let rec skip_line ch = if read ch = '\n' then skip_line ch else ();

val nread : input -> int -> string

nread i n reads a string of size up to n from an input. The function will raise No_more_input if no input is available. It will raise Invalid_argument if n < 0.

Example: let read_md5 ch = nread ch 32

val really_nread : input -> int -> string

really_nread i n reads a string of exactly n characters from the input.

Raises
- No_more_input if at least n characters are not available.
- Invalid_argument if n < 0. Example: let read_md5 ch = really_nread ch 32

val input : input -> Stdlib.Bytes.t -> int -> int -> int

input i s p len reads up to len characters from the given input, storing them in byte sequence s, starting at character number p. It returns the actual number of characters read (which may be 0) or raise No_more_input if no character can be read. It will raise Invalid_argument if p and len do not designate a valid subsequence of s.

Example: let map_ch f ?(block_size=100) = let b = String.create block_size in try while true do let l = input ch b 0 block_size in f b 0 l; done with No_more_input -> ()

val really_input : input -> Stdlib.Bytes.t -> int -> int -> int

really_input ic s p len reads exactly len characters from the input ic, storing them in the string s, starting at position p. For consistency with BatIO.input it returns len.

Raises
- No_more_input if at len characters are not available.
- Invalid_argument if p and len do not designate a valid substring of s. Example: let _ = really_input stdin b 0 3

val close_in : input -> unit

Close the input. It can no longer be read from.

Example: close_in network_in;

val write : (char, 'a) printer

Write a single char to an output.

Example: write stdout 'x';

val nwrite : (string, 'a) printer

Write a string to an output.

Example: nwrite stdout "Enter your name: ";

val output : 'a output -> Stdlib.Bytes.t -> int -> int -> int

output o s p len writes up to len characters from byte sequence s, starting at offset p. It returns the number of characters written. It will raise Invalid_argument if p and len do not designate a valid subsequence of s.

Example: let written = output stdout (Bytes.to_string "Foo Bar Baz") 2 4

This writes "o Ba" to stdout, and returns 4.

val output_substring : 'a output -> string -> int -> int -> int

like output above, but outputs from a substring instead of a subsequence of bytes

Since 2.8.0

val really_output : 'a output -> Stdlib.Bytes.t -> int -> int -> int

really_output o s p len writes exactly len characters from byte sequence s onto the the output, starting with the character at offset p. For consistency with BatIO.output it returns len.

Raises Invalid_argument if p and len do not designate a valid subsequence of s. This function is useful for networking situations where the output buffer might fill resulting in not the entire substring being readied for transmission. Uses output internally, and will raise Sys_blocked_io in the case that any call returns 0.

val really_output_substring : 'a output -> string -> int -> int -> int

like really_output above, but outputs from a substring instead of a subsequence of bytes

Since 2.8.0

val flush : 'a output -> unit

Flush an output.

If previous write operations have caused errors, this may trigger an exception.

Example: flush stdout;

val flush_all : unit -> unit

Flush all outputs, ignore errors.

Example: flush_all ();

val close_out : 'a output -> 'a

Close the output and return its accumulator data.

The output is flushed before being closed and can no longer be written. Attempting to flush or write after the output has been closed will have no effect.

Example: let strout = output_string () in write strout 'x'; if 2+3>5 then write strout "y"; print_string (close_out strout)

Creation of BatIO Inputs/Outputs

To open a file for reading/writing, see File.open_in and File.open_out

val input_string : string -> input

Create an input that will read from a string.

Example: let inch = input_string "1234554321" in let str1 = nread inch 3 in (* "123" *) let str2 = nread inch 5 in (* "45543" *) let str3 = nread inch 2 in (* "21" *) try string_of_char(read inch) with BatIO.No_more_input -> "End of string";

val output_string : unit -> string output

Create an output that will write into a string in an efficient way. When closed, the output returns all the data written into it.

val input_enum : char BatEnum.t -> input

Create an input that will read from an enum.

val output_enum : unit -> char BatEnum.t output

Create an output that will write into an enum. The final enum is returned when the output is closed.

val combine : 'a output * 'b output -> ('a * 'b) output

combine (a,b) creates a new output c such that writing to c will actually write to both a and b

val tab_out : ?tab:char -> int -> 'a output -> unit output

Create an output shifted to the right by a number of spaces (or other character as specified by tab).

tab_out n out produces a new output for writing into out, in which every new line starts with n spaces.

Raises Invalid_argument if n < 0. Closing tab_out n out does not close out. Rather, closing out closes tab_out n out.

Utilities

val read_all : input -> string

read all the contents of the input until No_more_input is raised.

val pipe : unit -> input * unit output

Create a pipe between an input and an output. Data written from the output can be read from the input.

val copy : ?buffer:int -> input -> 'a output -> unit

Read everything from an input and copy it to an output.

buffer : The size of the buffer to use for copying, in bytes. By default, this is 4,096b.

val pos_in : input -> input * (unit -> int)

Create an input that provide a count function of the number of bytes read from it.

val progress_in : input -> (unit -> unit) -> input

progress_in inp f create an input that calls f () whenever some content is successfully read from it.

val pos_out : 'a output -> unit output * (unit -> int)

Create an output that provide a count function of the number of bytes written through it.

val progress_out : 'a output -> (unit -> unit) -> unit output

progress_out out f create an output that calls f () whenever some content is successfully written to it.

val cast_output : 'a output -> unit output

You can safely transform any output to an unit output in a safe way by using this function.

Binary files API

Here is some API useful for working with binary files, in particular binary files generated by C applications. By default, encoding of multibyte integers is low-endian. The BatIO.BigEndian module provide multibyte operations with other encoding.

exception Overflow of string

Exception raised when a read or write operation cannot be completed.

val read_byte : input -> int

Read an unsigned 8-bit integer.

val read_signed_byte : input -> int

Read an signed 8-bit integer.

val read_ui16 : input -> int

Read an unsigned 16-bit word.

val read_i16 : input -> int

Read a signed 16-bit word.

val read_i32 : input -> int

Read a signed 32-bit integer.

Raises Overflow if the read integer cannot be represented as an OCaml 31-bit integer.

val read_real_i32 : input -> int32

Read a signed 32-bit integer as an OCaml int32.

val read_i64 : input -> int64

Read a signed 64-bit integer as an OCaml int64.

val read_float : input -> float

Read an IEEE single precision floating point value.

val read_double : input -> float

Read an IEEE double precision floating point value.

val read_string : input -> string

Read a null-terminated string.

val read_line : input -> string

Read a LF or CRLF terminated string. If the source runs out of input before a LF is found, returns a string of the remaining input. Will raise No_more_input only if no characters are available.

val write_byte : (int, 'a) printer

Write an unsigned 8-bit byte.

val write_ui16 : (int, 'a) printer

Write an unsigned 16-bit word.

val write_i16 : (int, 'a) printer

Write a signed 16-bit word.

val write_i32 : (int, 'a) printer

Write a signed 32-bit integer.

val write_real_i32 : (int32, 'a) printer

Write an OCaml int32.

val write_i64 : (int64, 'a) printer

Write an OCaml int64.

val write_double : (float, 'a) printer

Write an IEEE double precision floating point value.

val write_float : (float, 'a) printer

Write an IEEE single precision floating point value.

val write_string : (string, 'a) printer

Write a string and append an null character.

val write_line : (string, 'a) printer

Write a line and append a line end.

This adds the correct line end for your operating system. That is, if you are writing to a file and your system imposes that files should end lines with character LF (or '\n'), as Unix, then a LF is inserted at the end of the line. If your system favors CRLF (or '\r\n'), then this is what will be inserted.

module BigEndian: sig .. end

Same operations as module BatIO, but with big-endian encoding

Bits API

This enable you to read and write from an BatIO bit-by-bit or several bits at the same time.

type in_bits

type out_bits

exception Bits_error

val input_bits : input -> in_bits

Read bits from an input

val output_bits : 'a output -> out_bits

Write bits to an output

val read_bits : in_bits -> int -> int

Read up to 31 bits, raise Bits_error if n < 0 or n > 31

val write_bits : out_bits -> nbits:int -> int -> unit

Write up to 31 bits represented as a value, raise Bits_error if nbits < 0 or nbits > 31 or the value representation excess nbits.

val flush_bits : out_bits -> unit

Flush remaining unwritten bits, adding up to 7 bits which values 0.

val drop_bits : in_bits -> unit

Drop up to 7 buffered bits and restart to next input character.

Creating new types of inputs/outputs

val create_in : read:(unit -> char) ->
       input:(Stdlib.Bytes.t -> int -> int -> int) ->
       close:(unit -> unit) -> input

Fully create an input by giving all the needed functions.

Note Do not use this function for creating an input which reads from one or more underlying inputs. Rather, use BatIO.wrap_in.

val wrap_in : read:(unit -> char) ->
       input:(Stdlib.Bytes.t -> int -> int -> int) ->
       close:(unit -> unit) -> underlying:input list -> input

Fully create an input reading from other inputs by giving all the needed functions.

This function is a more general version of BatIO.create_in which also handles dependency management between inputs.

Note When you create an input which reads from another input, function close should not close the inputs of underlying. Doing so is a common error, which could result in inadvertently closing BatIO.stdin or a network socket, etc.

val inherit_in : ?read:(unit -> char) ->
       ?input:(Stdlib.Bytes.t -> int -> int -> int) ->
       ?close:(unit -> unit) -> input -> input

Simplified and optimized version of BatIO.wrap_in which may be used whenever only one input appears as dependency.

inherit_in inp will return an input identical to inp. inherit_in ~read inp will return an input identical to inp except for method read, etc.

You do not need to close inp in close.

val create_out : write:(char -> unit) ->
       output:(Stdlib.Bytes.t -> int -> int -> int) ->
       flush:(unit -> unit) -> close:(unit -> 'a) -> 'a output

Fully create an output by giving all the needed functions.

write : Write one character to the output (see BatIO.write).

output : Write a (sub)string to the output (see BatIO.output).

flush : Flush any buffers of this output (see BatIO.flush).

close : Close this output. The output will be automatically flushed. Note Do not use this function for creating an output which writes to one or more underlying outputs. Rather, use BatIO.wrap_out.

val wrap_out : write:(char -> unit) ->
       output:(Stdlib.Bytes.t -> int -> int -> int) ->
       flush:(unit -> unit) ->
       close:(unit -> 'a) -> underlying:'b output list -> 'a output

Fully create an output that writes to one or more underlying outputs.

This function is a more general version of BatIO.create_out, which also handles dependency management between outputs.

To illustrate the need for dependency management, let us consider the following values:

an output out
a function f : _ output -> _ output, using BatIO.create_out to create a new output for writing some data to an underyling output (for instance, a function comparale to BatIO.tab_out or a function performing transparent compression or transparent traduction between encodings)

With these values, let us consider the following scenario

a new output f out is created
some data is written to f out but not flushed
output out is closed, perhaps manually or as a consequence of garbage-collection, or because the program has ended
data written to f out is flushed.

In this case, data reaches out only after out has been closed. Despite appearances, it is quite easy to reach such situation, especially in short programs.

If, instead, f uses wrap_out, then when output out is closed, f out is first automatically flushed and closed, which avoids the issue.

write : Write one character to the output (see BatIO.write).

output : Write a (sub)string to the output (see BatIO.output).

flush : Flush any buffers of this output (see BatIO.flush).

close : Close this output. The output will be automatically flushed.

underlying : The list of outputs to which the new output will write. Note Function close should not close underlying yourself. This is a common mistake which may cause sockets or standard output to be closed while they are still being used by another part of the program.

val inherit_out : ?write:(char -> unit) ->
       ?output:(Stdlib.Bytes.t -> int -> int -> int) ->
       ?flush:(unit -> unit) ->
       ?close:(unit -> unit) -> 'a output -> unit output

Simplified and optimized version of BatIO.wrap_out whenever only one output appears as dependency.

inherit_out out will return an output identical to out. inherit_out ~write out will return an output identical to out except for its write method, etc.

You do not need to close out in close.

For compatibility purposes

val input_channel : ?autoclose:bool -> ?cleanup:bool -> Stdlib.in_channel -> input

Create an input that will read from a channel.

autoclose : If true or unspecified, the BatIO.input will be automatically closed when the underlying in_channel has reached its end.

cleanup : If true, the channel will be automatically closed when the BatIO.input is closed. Otherwise, you will need to close the channel manually. Default is true.

val output_channel : ?cleanup:bool -> Stdlib.out_channel -> unit output

Create an output that will write into a channel.

cleanup : If true, the channel will be automatically closed when the BatIO.output is closed. Otherwise, you will need to close the channel manually.

val to_input_channel : input -> Stdlib.in_channel

Create a channel that will read from an input.

Note This function is extremely costly and is provided essentially for debugging purposes or for reusing legacy libraries which can't be adapted. As a general rule, if you can avoid using this function, don't use it.

Generic BatIO Object Wrappers

These OO Wrappers have been written to provide easy support of BatIO by external libraries. If you want your library to support BatIO without actually requiring Batteries to compile, you can implement the classes in_channel, out_channel, poly_in_channel and/or poly_out_channel which are the common BatIO specifications established for ExtLib, OCamlNet and Camomile.

(see http://www.ocaml-programming.de/tmp/BatIO-Classes.html for more details).

Note In this version of Batteries Included, the object wrappers are not closed automatically by garbage-collection.

class in_channel : input -> object .. end

class out_channel : 'a output -> object .. end

class in_chars : input -> object .. end

class out_chars : 'a output -> object .. end

val from_in_channel : #in_channel -> input

val from_out_channel : #out_channel -> unit output

val from_in_chars : #in_chars -> input

val from_out_chars : #out_chars -> unit output

Enumeration API

val bytes_of : input -> int BatEnum.t

Read an enumeration of unsigned 8-bit integers.

val signed_bytes_of : input -> int BatEnum.t

Read an enumeration of signed 8-bit integers.

val ui16s_of : input -> int BatEnum.t

Read an enumeration of unsigned 16-bit words.

val i16s_of : input -> int BatEnum.t

Read an enumartion of signed 16-bit words.

val i32s_of : input -> int BatEnum.t

Read an enumeration of signed 32-bit integers.

Raises Overflow if the read integer cannot be represented as an OCaml 31-bit integer.

val real_i32s_of : input -> int32 BatEnum.t

Read an enumeration of signed 32-bit integers as OCaml int32s.

val i64s_of : input -> int64 BatEnum.t

Read an enumeration of signed 64-bit integers as OCaml int64s.

val doubles_of : input -> float BatEnum.t

Read an enumeration of IEEE double precision floating point values.

val floats_of : input -> float BatEnum.t

Read an enumeration of IEEE single precision floating point values.

val strings_of : input -> string BatEnum.t

Read an enumeration of null-terminated strings.

val lines_of : input -> string BatEnum.t

Read an enumeration of LF or CRLF terminated strings.

val lines_of2 : input -> string BatEnum.t

Buffered version of BatIO.lines_of, for performance.

val chunks_of : int -> input -> string BatEnum.t

Read an input as an enumeration of strings of given length. If the input isn't a multiple of that length, the final string will be smaller than the rest.

val chars_of : input -> char BatEnum.t

Read an enumeration of Latin-1 characters.

Note Usually faster than calling read several times.

val bits_of : in_bits -> int BatEnum.t

Read an enumeration of bits

val write_bitss : nbits:int -> out_bits -> int BatEnum.t -> unit

Write an enumeration of bits

val default_buffer_size : int

The default size for internal buffers.

Thread-safety

val synchronize_in : ?lock:BatConcurrent.lock -> input -> input

synchronize_in inp produces a new BatIO.input which reads from input in a thread-safe way. In other words, a lock prevents two distinct threads from reading from that input simultaneously, something which would potentially wreak havoc otherwise

lock : An optional lock. If none is provided, the lock will be specific to this input. Specifying a custom lock may be useful to associate one common lock for several inputs and/or outputs, for instance in the case of pipes.

val synchronize_out : ?lock:BatConcurrent.lock -> 'a output -> unit output

synchronize_out out produces a new BatIO.output which writes to output in a thread-safe way. In other words, a lock prevents two distinct threads from writing to that output simultaneously, something which would potentially wreak havoc otherwise

lock : An optional lock. If none is provided, the lock will be specific to this output. Specifying a custom lock may be useful to associate one common lock for several inputs and/or outputs, for instance in the case of pipes.

Thread-safety internals

Unless you are attempting to adapt Batteries Included to a new model of concurrency, you probably won't need this.

val lock : BatConcurrent.lock Stdlib.ref

A lock used to synchronize internal operations.

By default, this is BatConcurrent.nolock. However, if you're using a version of Batteries compiled in threaded mode, this uses BatMutex. If you're attempting to use Batteries with another concurrency model, set the lock appropriately.

val lock_factory : (unit -> BatConcurrent.lock) Stdlib.ref

A factory used to create locks. This is used transparently by BatIO.synchronize_in and BatIO.synchronize_out.

By default, this always returns BatConcurrent.nolock. However, if you're using a version of Batteries compiled in threaded mode, this uses BatMutex.

val to_string : ('a, string) printer -> 'a -> string

val to_f_printer : ('a, 'b) printer -> 'a f_printer