Wrappers

Alex User Guide
Prev | Chapter 5. The Interface to an Alex-generated lexer | Next

# 5.2. Wrappers

To use one of the provided wrappers, include the following declaration in your file:

        %wrapper "name"

where `name` is the name of the wrapper, eg. `basic`. The following sections describe each of the wrappers that come with Alex.

## 5.2.1. The "basic" wrapper

The basic wrapper is a good way to obtain a function of type `String -> [token]` from a lexer specification, with little fuss.

It provides definitions for `AlexInput`, `alexGetChar` and `alexInputPrevChar` that are suitable for lexing a `String` input. It also provides a function `alexScanTokens` which takes a `String` input and returns a list of the tokens it contains.

The `basic` wrapper provides no support for using startcodes; the initial startcode is always set to zero.

Here is the actual code included in the lexer when the basic wrapper is selected:

        type AlexInput = (Char,	-- previous char
    		      String)	-- current input string

        alexGetChar :: AlexInput -> Maybe (Char,AlexInput)
        alexGetChar (_, [])   = Nothing
        alexGetChar (_, c:cs) = Just (c, (c,cs))

        alexInputPrevChar :: AlexInput -> Char
        alexInputPrevChar (c,_) = c

        -- alexScanTokens :: String -> [token]
        alexScanTokens str = go ('\n',str)
          where go inp@(_,str) =
        	      case alexScan inp 0 of
        		    AlexEOF -> []
        		    AlexError _ -> error "lexical error"
        		    AlexSkip  inp' len     -> go inp'
        		    AlexToken inp' len act -> act (take len str) : go inp'

The type signature for `alexScanTokens` is commented out, because the `token` type is unkonwn. All of the actions in your lexical specification should have type:

        { ... } :: String -> token

for some type `token`.

For an example of the use of the basic wrapper, see the file `examples/Tokens_basic.x` in the Alex distribution.

## 5.2.2. The "posn" wrapper

The posn wrapper provides slightly more functionality than the basic wrapper: it keeps track of line and column numbers of tokens in the input text.

The posn wrapper provides the following, in addition to the straightforward definitions of `alexGetChar` and `alexInputPrevChar`:

        data AlexPosn = AlexPn !Int  -- absolute character offset
                               !Int  -- line number
                               !Int  -- column number

        type AlexInput = (AlexPosn, -- current position,
    		      Char,	-- previous char
    		      String)	-- current input string

        --alexScanTokens :: String -> [token]
        alexScanTokens str = go (alexStartPos,'\n',str)
          where go inp@(pos,_,str) =
        	      case alexScan inp 0 of
        		    AlexEOF -> []
        		    AlexError _ -> error "lexical error"
        		    AlexSkip  inp' len     -> go inp'
        		    AlexToken inp' len act -> act pos (take len str) : go inp'

The types of the token actions should be:

        { ... } :: AlexPosn -> String -> token

For an example using the `posn` wrapper, see the file `examples/Tokens_posn.x` in the Alex distribution.

## 5.2.3. The "monad" wrapper

The `monad` wrapper is the most flexible of the wrappers provided with Alex. It includes a state monad which keeps track of the current input and text position, and the startcode. It is intended to be a template for building your own monads - feel free to copy the code and modify it to build a monad with the facilities you need.

    data AlexState = AlexState {
            alex_pos :: !AlexPosn,	-- position at current input location
            alex_inp :: String,	-- the current input
            alex_chr :: !Char,	-- the character before the input
            alex_scd :: !Int 	-- the current startcode
        }

    newtype Alex a = Alex { unAlex :: AlexState
                                   -> Either String (AlexState, a) }

    runAlex          :: String -> Alex a -> Either String a

    alexGetInput     :: Alex AlexInput
    alexSetInput     :: AlexInput -> Alex ()

    alexError        :: String -> Alex a

    alexGetStartCode :: Alex Int
    alexSetStartCode :: Int -> Alex ()

To invoke a scanner under the `monad` wrapper, use `alexMonadScan`:

    alexMonadScan    :: Alex result

The token actions should have the following type:

    type AlexAction result = AlexInput -> Int -> Alex result
    { ... }  :: AlexAction result

The `monad` wrapper also provides some useful combinators for constructing token actions:

    -- skip :: AlexAction result
    skip input len = alexMonadScan

    -- andBegin :: AlexAction result -> Int -> AlexAction result
    (action `andBegin` code) input len = do alexSetStartCode code; action input len

    -- begin :: Int -> AlexAction result
    begin code = skip `andBegin` code

    -- token :: (String -> Int -> token) -> AlexAction token
    token t input len = return (t input len)

## 5.2.4. The "gscan" wrapper

The `gscan` wrapper is provided mainly for historical reasons: it exposes an interface which is very similar to that provided by Alex version 1.x. The interface is intended to be very general, allowing actions to modify the startcode, and pass around an arbitrary state value.

        alexGScan :: StopAction state result -> state -> String -> result

        type StopAction state result
                 = AlexPosn -> Char -> String -> (Int,state) -> result

The token actions should all have this type:

        { ... }      :: AlexPosn                -- token position
                     -> Char                    -- previous character
                     -> String                  -- input string at token
                     -> Int                     -- length of token
                     -> ((Int,state) -> result) -- continuation
                     -> (Int,state)             -- current (startcode,state)
                     -> result

Prev | Home | Next
The Interface to an Alex-generated lexer | Up | Invoking Alex