Text

# Module Text

    module Text: sig .. end

UTF-8 encoded strings

This modules is intended for ``text'' manipulation. By text we mean sequence of unicode characters.

For compatibility and simplicity reasons, text is represented by UTF-8 encoded strings, and there is no special types for unicode characters, whose are just represented by 1-length text.

All functions of this module expect to by applied on valid UTF-8 encoded strings, and may raise `Invalid` if this is not the case.

    type t = string

Type of text

    exception Invalid of string * string

`Invalid(error, text)` Exception raised when an invalid UTF-8 encoded string is encountered. `text` is the faulty text and `error` is a description of the first error in `text`.

    val check : string -> string option

`check str` checks that `str` is a valid UTF-8 encoded string. Returns `None` if it is the case, or `Some error` otherwise.

    val validate : string -> unit

Same as check but raises an exception in case the argument is not a valid text.

###### Encoding/decoding

    val encode : ?encoding:Encoding.t -> t -> string

`encode ?encoding txt` encode the given text with `encoding`, which defaults to `Encoding.system` plus transliteration.

    val decode : ?encoding:Encoding.t -> string -> t

`decode ?encoding str` decode the given string encoded in `encoding`, which defaults to `Encoding.system`

    val to_ascii : t -> t

`to_ascii txt` returns an approximative ascii version of `txt`. This is the same as `encode ~encoding:"ASCII//TRANSLIT" txt`

###### Informations

    val length : t -> int

Return the number of unicode character contained in the given text

###### Construction/access

    val code : t -> int

`code text` returns the unicode code-point of first character of `text`.

For example:

- `code "A" = 65`
- `code "é" = 0xe9`

    val char : int -> t

`char code` returns the character corresponding to the given unicode code-point.

For example:

- `char 65 = "A"`
- `char 0xe9 = "é"`

Raises `Invalid_argument` if `code` is not a valid unicode code-point. Valid code-point are all integers in the range `0..0x10ffff`.

    val get : t -> int -> t

`get text n` returns the `n`-th character of `text`. `n` is a number of unicode character, not bytes. A negative value is interpreted as a position from the end of the text.

For example:

- `get "abc" 0 = "a"`
- `get "abc" 2 = "c"`
- `get "aéb" 1 = "é"`
- `get "aéb" 2 = "b"`
- `nth "abc" (-1) = "c"`

    val sub : t -> int -> int -> t

`sub text pos len` Returns the sub-text of `text` starting at position `pos` and of length `len`. `pos` and/or `len` may be negative.

For example:

- `sub "ocaml" 1 2 = "ca"`
- `sub "ocaml" 3 (-2) = "ca"`
- `sub "ocaml" (-2) 1 = "m"`

    val slice : t -> int -> int -> t

`slice text a b` returns the text contained in `txt` between `a` and `b` (exlusive). `a` and/or `b` may be negative.

For example:

- `slice "abc" 0 1 = "a"`
- `slice "abcdef" 1 (-1) = "bcde"`

    val splice : t -> int -> int -> t -> t

`splice text a b repl` replace the text between `a` and `b` (exclusive) by `repl`.

For example:

- `splice "abcd" 1 2 "plop" = "aplopcd"`
- `splice "abcd" 1 2 "" = "acd"`

    val repeat : int -> t -> t

`repeat n text` returns `text` concatened `n`-times with itself.

    val init : int -> (int -> t) -> t

`init n f` returns `f 0 ^ f 1 ^ ... ^ f (n - 1)`

    val rev_init : int -> (int -> t) -> t

`rev_init n f` returns `f (n - 1) ^ f 1 ^ ... ^ f 0`

###### Locale specific functions

    val upper : t -> t

`upper t` returns the upper-cased version of `t`.

    val lower : t -> t

`lower t` returns the upper-cased version of `t`.

    val capitalize : t -> t

`capitalize t` returns `t` with its first letter upper-cased

    val uncapitalize : t -> t

`capitalize t` returns `t` with its first letter lower-cased

    val compare : t -> t -> int

Compares two texts according to the current locale

    val icompare : t -> t -> int

Compares two texts, case-insensitive

    val transform : t -> t

`transform str` transforms `str` in a way such that comparing two string `str1` and `str2` transformed with `Pervasives.compare` give the same result as comparing them with `Text.compare`.

###### Transformations

    val rev : t -> t

`rev t` returns the sequence of characters of `t` in reverse order.

For example:

- `rev "ocaml" = "lmaco"`
- `rev "héhé" = "éhéh"`

    val concat : t -> t list -> t

`concat sep l` returns the concatenation of all texts contained in `l`, separated by `sep`.

For example:

- `concat "/" ["a"; "b"; "c"] = "a/b/c"`

    val rev_concat : t -> t list -> t

`rev_concat sep l` returns the concatenation of all texts contained in `l`, separated by `sep`.

For example:

- `concat "/" ["a"; "b"; "c"] = "c/b/a"`

    val explode : t -> t list

`explode txt` returns the list of all characters of `txt`.

For example:

- `explode "" = []`
- `explode "abé" = ["a"; "b"; "é"]`

    val rev_explode : t -> t list

`rev_explode txt` returns the list of all characters of `txt`, in reverse order.

For example:

- `rev_explode "ocaml" = ["l"; "m"; "a"; "c"; "o"]`

    val implode : t list -> t

`implode l` returns the concatenation of all texts contained in `l`. This is the same as `concat "" l`, but a bit more efficient.

For example:

- `implode ["o"; "c"; "a"; "m"; "l"] = "ocaml"`
- `implode ["abc"; "def"] = "abcdef"`

    val rev_implode : t list -> t

`rev_implode l` returns the concatenation of all texts contained in `l`, in reverse order.

For example:

- `implode ["o"; "c"; "a"; "m"; "l"] = "lmaco"`
- `implode ["abc"; "def"] = "defabc"`

###### Tests

The following functions tests whether all characters of the given text verify a property:

    val is_ascii : t -> bool

    val is_alnum : t -> bool

    val is_alpha : t -> bool

    val is_blank : t -> bool

    val is_cntrl : t -> bool

    val is_digit : t -> bool

    val is_graph : t -> bool

    val is_lower : t -> bool

    val is_print : t -> bool

    val is_punct : t -> bool

    val is_space : t -> bool

    val is_upper : t -> bool

    val is_xdigit : t -> bool

###### Text traversals

For all the following functions we give a equivalent implementation, and examples. They have the same semantic as the equivalent implementation but are more efficient.

    val map : (t -> t) -> t -> t

`map f text` ~ `implode (List.map f (explode text))`

`map (function "a" -> "x" | t -> t) "abc" = "xbc"`

    val rev_map : (t -> t) -> t -> t

`rev_map f text` ~ `implode (List.rev_map f (explode text))`

`rev_map (function "a" -> "x" | t -> t) "abc" = "cbx"`

    val fold : (t -> 'a -> 'a) -> t -> 'a -> 'a

`fold f x text` ~ `List.fold_left f x (explode text)`

`fold (fun acc t -> acc + code t) 0 "ABC" = 198`

    val rev_fold : (t -> 'a -> 'a) -> t -> 'a -> 'a

`fold f text x` ~ `List.fold_left f x (rev_explode text)`

`rev_fold (fun t acc -> acc + code t) "ABC" 0 = 198`

    val filter : (t -> bool) -> t -> t

`filter text` ~ `implode (List.filter f (explode text))`

`filter is_alpha "1a2E" = "aE"`

    val rev_filter : (t -> bool) -> t -> t

`rev_filter text` ~ `implode (List.filter f (rev_explode text))`

`rev_filter is_alpha "1a2E" = "Ea"`

    val iter : (t -> unit) -> t -> unit

`iter f text` ~ `List.iter f (explode text)`

    val rev_iter : (t -> unit) -> t -> unit

`iter f text` ~ `List.iter f (rev_explode text)`

###### Scanning

    val for_all : (t -> bool) -> t -> bool

`for_all f text` returns whether all characters of `text` verify the predicate `f`

    val exists : (t -> bool) -> t -> bool

`exists f text` returns whether at least one character of `text` verify `f`

    val count : (t -> bool) -> t -> int

`count f text` returhs the number of characters of `text` verifying `f`

###### Splitting

    val words : t -> t list

Returns all words of the given text. Words are sequence of non-space and non-punct characters.

    val lines : t -> t list

Returns all lines of the given text, without end of line characters. Both `"\r\n"` and `"\n"` are recognized as end of line delimiters.

    val split : ?max:int -> ?sep:t -> t -> t list

`split ?max ?sep text` split `text` according to `sep`. If `max` is specified, returns at most `max` splits. `sep` defaults to `" "`.

For example:

- `split ~sep:"/" "a/b/c" = ["a"; "b"; "c"]`
- `split ~sep:".." "a..b..c" = ["a"; "b"; "c"]`
- `split ~max:1 "a b c" = ["a b c"]`
- `split ~max:2 "a b c" = ["a"; "b c"]`

    val rev_split : ?max:int -> ?sep:t -> t -> t list

`rev_split ?max text sep` split `text` according to `sep` in reverse order.

For example:

- `split ~sep:"/" "a/b/c" = ["c"; "b"; "a"]`
- `split ~max:1 "a b c" = ["a b c"]`
- `split ~max:2 "a b c" = ["a b"; "c"]`
- `rev_split ~max:2 ~sep:"." "toto.mli" = ["toto"; "mli"]`

    val replace : t -> patt:t -> repl:t -> t

`replace text ~patt ~repl` replace all occurences of `patt` in `text` by `repl`.

For example:

- `replace "abcd" ~patt:"b" ~repl:"x" = "axcd"`
- `replace "Hello world!" ~patt:"world" ~repl:"you" = "Hello you!"`

###### Tests

    val contains : t -> t -> bool

`contains text sub` returns whether `sub` appears in `text`

    val starts_with : t -> t -> bool

`starts_with text prefix` returns `true` iff `s` starts with `prefix`.

For example:

- `starts_with "abcd" "ab" = true`
- `starts_with "abcd" "af" = false`
- `starts_with "ab" "abcd" = false`

    val ends_with : t -> t -> bool

`ends_with s suffix` returns `true` iff `s` ends with `suffix`.

For example:

- `ends_with "abcd" "cd" = true`
- `ends_with "abcd" "hd" = false`
- `ends_with "ab" "abc" = false`

###### Stripping

    val strip : ?chars:t list -> t -> t

`strip ?chars text` removes all characters of `text` which are part of `chars` at the right and left. `chars` defaults to whitespaces.

    val rstrip : ?chars:t list -> t -> t

`rstrip ?chars text` removes all characters of `text` which are part of `chars` at the right.

    val lstrip : ?chars:t list -> t -> t

`lstrip ?chars text` removes all characters of `text` which are part of `chars` at the left.

    val rchop : t -> t

`rchop t` returns `t` without is last character. Returns `""` if `t = ""`.

    val lchop : t -> t

`lchop t` returns `t` without is first character. Returns `""` if `t = ""`

###### Pointers

Since characters are not encoded by a fixed number of bytes, accessing them by character position is not efficient. The following functions allow you to iterates in a string in an efficient way.

    type pointer

A pointer to a unicode character in a text.

    val pointer_l : t -> pointer

Returns a pointer to the left of the given text

    val pointer_r : t -> pointer

Returns a pointer to the right of the given text

    val pointer_at : t -> int -> pointer

`pointer_at txt n` returns a pointer to the character at position `n` in `txt`.

    val next : pointer -> (t * pointer) option

`next ptr` if `ptr` is at the end of text, returns `None`, otherwise, returns `Some(ch, ptr')` where `ch` is the character at current position and `ptr'` is the pointer to the next character of the text.

    val prev : pointer -> (t * pointer) option

`prev ptr` if `ptr` is at the beginning of text, returns `None`, otherwise, returns `Some(ch, ptr')` where `ptr'` points to the previous character and `ch` is the character at `ptr'`.

    val move : int -> pointer -> pointer

`move n ptr` moves `ptr` by `n` unicode characters. If `n < 0` then `ptr` is moved to the left. Raises `Invalid_argument` if the result is outside the text.

    val chunk : pointer -> pointer -> t

`chunk a b` returns the chunk of text between `a` and `b`. Raises `Invalid_arugment` if `a` or `b`.

    val offset : pointer -> int

`offset ptr` returns the position in bytes of `ptr`

    val position : pointer -> int

`position ptr` returns the position in unicode character of `ptr`

    val equal_at : pointer -> t -> bool

`equal_at ptr str` returns wether `ptr` points to a substring equal to `str`

    val find : ?from:pointer -> t -> t -> pointer option

`find ?from text patt` returns a pointer to the first occurrence of `patt` in `text`.

    val rev_find : ?from:pointer -> t -> t -> pointer option

`find ?from text patt` returns a pointer to the last occurrence of `patt` in `text`.