PYRO - Usage

## 4. Pyro Usage

- Introduction
- Pyro script tools
- Writing the remote class
- Writing the server
- Writing the client
- Runtime setup
- Runtime control and Logging
- Threads, sessions and objects
- Last Notes

### Introduction

This chapter will show the Pyro development process: how to build a Pyro application. Let's repeat the scenario from the Introduction chapter here, but with some more detail:

1. You write a Python class that you want to access remotely. Do this as if it were a normal Python class (but see the Features and Guidelines chapter).
2. Write a server process that performs the following tasks:
   - Create one or more instances of your remote object.
   - Create a Pyro Daemon instance.
   - Find the Name Server.
   - Connect the remote object instances to the Daemon that will register them with the Name Server.
   - Sit in a loop telling the Daemon to handle incoming requests.
3. Write a client program that does the following:
   - Find the Name Server.
   - Query the NS for the requires objects. You'll get URIs back.
   - Create a Dynamic Proxy for the URI.
   - Call methods on the proxy object as if it were the real thing.
4. Make sure the Pyro Name Server is running.
5. Start your server process. If it complains that the names it wants to register already exist, use the `pyro-nsc` tool to unregister them, or restart the NS.
6. Run the client!

In the following sections each step is explained in more detail.

### Pyro script tools

Before using them let us first study the usage of the script tools. Pyro comes with two flavors, Un*x-style shellscripts and Windows/DOS command files. The Windows-style command files have the '.cmd' extension.

`pyro-genguid` (GUID generator)
  No arguments.
  This is a very simple GUID generator. It uses the internal Pyro GUID generator to print a new GUID.
`pyro-ns, pyro-rns` (Name Server)
  These scripts are explained in the Name Server chapter.
`pyro-es` (Event Server)
  This script is explained in the Event Server (Pyro Services) chapter.
`pyro-nsc` (Name Server Control tool)
  - Arguments: [-h host] [-p port] [-c bcaddr] [-i identification] command [args...]
  - Controls the Pyro Name Server. '-h host' specifies the host where the Name Server should be contacted. '-p port' specifies a non-standard NS broadcast port to contact. '-c bcaddr' allows you to override the broadcast address. With '-i identification' you can supply the authentication passphrase that is used to connect to the Name Server. When it contains spaces, use quotes around it. 'command' is one of the following:
  - ping: just check if the NS is up and running.
  - list: prints the contents of a name group. Argument is the group name to list.
  - listall: prints a list of all registered names (expanded).
  - register: register a new name. Arguments are the name and the URI.
  - resolve: search for names. Arguments are the names to search for.
  - remove: remove registered names from the NS. Arguments are the names to remove.
  - creategroup: create a name group. Argument is the group to create.
  - deletegroup: delete a name group and all contents. Argument is the group to delete.
  - showmeta: Shows system and user meta info. Argument is the group or object to show meta info from.
  - setmeta: Set user meta info. Arguments are the object name and the meta data string.
  - shutdown: send the NS a shutdown request, so that a clean shutdown is performed. No arguments.
`pyro-xnsc` (Graphical NS control tool)
  - No arguments
  - This is a graphical version of the `nsc` command-line tool. Currently it needs Tk for the GUI, so you have to have a Tk-enabled Python on your system. The GUI is simple and should explain itself. You can enter the hostname in the textbox at the top and press <enter> to contact the NS at that host, or just press the 'Auto Discover' button at the top right. If the NS has been found, the rest of the buttons are enabled. If your Name Server requires an authorization passphrase, you must enter that first in the ID entry box. After that, you can connect to the NS. Once connected, the passphrase is erased in the display for security reasons. You have to type it again if you need to reconnect.
`pyro-wxnsc` (Alternative Graphical NS control tool)
  - No arguments
  - This is similar to the `xnsc` tool, but based on the WxPython GUI toolkit.
`pyro-nssvc, pyro-essvc` (Windows-only Name Server and Event Server 'NT-service' control scripts)
  These scripts are explained in the Name Server chapter and the Event Server chapter.
Using `python -m` to start various tools
  `python -m Pyro.naming` - start the name server
  `python -m Pyro.EventService.Server` - start the event server
  `python -m Pyro.nsc` - start the nsc tool. Also works with xnsc and wxnsc.
  `python -m Pyro.configuration` - print a dump of Pyro's active configuration settings.

### Steps 1, 2 and 3: Writing the remote class

Just create a Python module containing the classes you want to access remotely. There are some restrictions induced by Pyro:

- The remote class can't have a remote `__init__` method. You should use a regular initialization method that you must call explicitly after binding to the remote object. The `__init__` method will only be called on the server side when the object is created.
- The remote class can't have direct attribute access unless you conciously choose to use a special proxy that supports attribute access. See below. You don't have to use getters and setters for each member variable any more.

If you keep those in mind, you should be safe. You can use all Python types and parameter lists and exceptions in your code. Pyro will deal with those nicely.

### Step 4: Writing the server

#### Initialization

You should initialize Pyro before using it in your server program. This is done by calling

       Pyro.core.initServer()

If you provide the optional argument `banner=1`, a short version message is printed on the standard output. There is also a second optional argument `storageCheck`. By default it is 1 and Pyro will check the availability of the `PYRO_STORAGE` directory. If you set it to 0, Pyro will not perform this check.

If the tracelevel is not zero, a startup message is written to the log. This message shows the active configuration options.

It is not strictly required to call `Pyro.core.initServer()`, if you are creating a Pyro Daemon first. If you're doing that (see next paragraph-- it's a very common thing to do first), Pyro will initialise itself automatically. If you're not doing this, and are using other Pyro things first, it won't work because Pyro will then think you are a client, and call the wrong initialization function. So it's best to call `Pyro.core.initServer()` yourself. All Pyro code you see in this manual and the Pyro examples do this too.

#### Create a Pyro Daemon

Your server program must create a Pyro Daemon object, which contains all logic necessary for accepting incoming requests and dispatching them to your objects by invoking their methods. You also have to tell the daemon which Name Server to use. When connecting objects to the daemon (see below) it uses this NS to register those objects for you. This is convenient as you don't have to do it yourself.

       daemon = Pyro.core.Daemon()
       daemon.useNameServer(ns)

You can provide several arguments when creating the Daemon:

`protocol` | the protocol to use (defaults to "PYRO")
`host` | the hostname to bind the server on (defaults to '' - the default host). This may be necessary in the case where your system has more than one hostname/IP address, for instance, when it has multiple network adapters. With this argument you can select the specific hostname to bind the server on.
`port` | the socket number to use (defaults to the `PYRO_PORT` configuration item). Keep in mind that Pyro will pay attention to the `PYRO_PORT_RANGE` config item: if it cannot claim the socket on the given port, it will try the next higher port, and so on, as long as `PYRO_PORT_RANGE` allows. Setting this to 0 lets the operating system choose a random port for you (you need to set `norange` to 1 or True as well, if you want this).
`norange` | whether or not to try a range of sockets, i.e. don't pay attention to the `PYRO_PORT_RANGE` setting. (It's usually best leave this at the default value, 0) You need to set this to 1 or True if you want to use the random port selection (when setting `port=0`).
`publishhost` | the hostname that the daemon will use when publishing URIs, in case of a firewall/NAT setup. See the Features chapter. Defaults to the value given to the `host` parameter.

The second line tells the daemon to use a certain Name Server (`ns` is a proxy for the NS, see the next paragraph how to get this proxy). It's possible to omit this call but the Daemon will no longer be able to register your objects with the NS. If you didn't register them yourself, it is impossible to find them. The daemon will log a warning if it doesn't know your NS.

If your daemon is no longer referenced, it might be garbage collected (destroyed) by Python. Even if you connected Pyro objects to the daemon. So you have to make sure that you keep a reference to your daemon object at all time. This is recommended anyway because you can then cleanly terminate your Pyro application by calling `daemon.shutdown()` when it exits. Usually this is not a problem because your program creates a deamon and calls its `requestLoop`. But a situation might arise where you don't keep a reference to the daemon object, and then things break.

#### Find the Name Server

You have to get a reference to the Pyro Name Server, which itself is a Pyro object. The easiest way is by using the NS Locator:

       locator = Pyro.naming.NameServerLocator()
       ns = locator.getNS()

`ns` now contains a reference. There are more advanced ways to get a reference to the NS, please read the chapter about the Name Server to find out about them.

#### Create object instances

The objects you create in the server that have to be remotely accessible can't be created bare-bones. They have to be decorated with some logic to fool them into thinking it is a regular python program that invokes their methods. This logic is incorporated in a special generic object base class that is part of the Pyro core: `Pyro.core.ObjBase`. There are three ways to achieve this:

- Derive a new class from both `Pyro.core.ObjBase` and your original class. The class body can be a simple '`pass`'. If you want to add a custom `__init__` method, make sure you call the `__init__` method of `Pyro.core.ObjBase` and of your own class, if it has one.

         class ObjectImpl(Pyro.core.ObjBase, test.MyClass):
            def __init__(self):
               Pyro.core.ObjBase.__init__(self)
               test.MyClass.__init__(self)
            ...
         obj = ObjectImpl()
          ... use obj ...

- Delegate Pattern. In this pattern you create two objects, and you tell one of them to delegate all calls to the other. Instead of deriving from `Pyro.core.ObjBase` you just create that object and tell it to use your own object as a delegate, by calling the `delegateTo` method.

         obj = Pyro.core.ObjBase()
         myobj = MyClass()
         obj.delegateTo(myobj)
          ... use obj ...

- Direct inheritance: subclass your class directly from `Pyro.core.ObjBase`. This is the least hassle but you have to change existing code if you want to make classes suitable for Pyro.

         class MyPyroObj(Pyro.core.ObjBase):
            def __init__(self):
               Pyro.core.ObjBase.__init__(self)
               ...obj init here...
             ...
         obj = MyPyroObj()
          ... use obj ...

For advanced purposes, there are two other base classes that you can use instead of `ObjBase`:

`Pyro.core.SynchronizedObjBase`
  Use this to make your Pyro object thread-safe; all (remote) method calls are automatically synchronized for this object.
`Pyro.core.CallbackObjBase`
  Use this for special callback objects that need to report errors also on the client, not only on the server. For more information, please read about Callbacks in the Features and Guidelines chapter.

#### Connect object instances

Ok, we're going nicely up to this point. We have some objects that even already have gotten a unique ID (that's part of the logic `Pyro.core.ObjBase` gives us). But Pyro still knows nothing about them. We have to let Pyro know we've created some objects and how they are called. Only then can they be accessed by remote client programs. So let's connect our objects with the Pyro Daemon we've created before:

       daemon.connect(obj,'our_object')

That done, the daemon has registered our object with the NS too (if you told it where to find the NS, as we explained earlier: `daemon.useNameServer(ns)`). The NS will now have an entry in its table that connects the name "our_object" to our specific object.
Note 1: if you don't provide a name, your object is a so-called transient object. The daemon will not register it with the Name Server. This is useful when you create new Pyro objects on the server that are not full-blown objects but rather objects that are only accessible by the code that created them. Have a look at the factory and Bank2 examples if this is not clear.
Note 2: the `connect` method actually returns the URI that will identify this object. You can ignore this if you don't want to use it immediately without having to consult the name service.
Note 3: there is also a `connectPersistent` method that is used for a special purpose. Look under the "Automatic Rebinding" topic in the "Features and guidelines" chapter for more info.

In contrast to the simple (flat) name shown above ("our_object"), Pyro's Name Server supports a hierarchical object naming scheme.

#### Disconnecting object instances

Usually you don't have to worry about cleaning up, the daemon will cleanly remove any registered objects from the Name Server if it exits. (Note that 'persistently' connected objects are not removed automatically.) But sometimes it can be better to manually remove any objects that you don't need any longer. Use the following method to do that:

       daemon.disconnect(obj)

Just pass the Pyro object you want to remove from the Daemon (and the Name Server). It is also possible to pass the object's UID (string) instead of the object itself. This allows you to remove objects from the daemon that you only know by their UID (for instance, you only have a Pyro URI or you got the object ID directly from the daemon's `getRegistered()` object list).

#### The Daemon handleRequest loop

We're near the end of our server coding effort. The only thing left is the code that sits in a loop and processes incoming requests. Fortunately most of that is handled by a single method in the daemon. For many applications calling `daemon.requestLoop()` is enough. For finer control, you can give a few arguments to the function:

    requestLoop(condition, timeout, others, callback)

All arguments are optional. The default is that `requestLoop` enters an endless loop waiting and handling Pyro requests. You can specify a `condition` callable object (for instance, a lambda function) that is evaluated each cycle of the loop to see if the loop should continue (the condition must evaluate to 1). The `timeout` can be used to adjust the timeout between loop cycles (default=3 seconds). The `requestLoop` doesn't use the timeout (it only returns when the optional loop condition is no longer true), the timeout is simply passed to the underlying `handleRequests` call. This is required on some platforms (windows) to cleanly handle break signals like ^C. The `others` and `callbacks` can be used to add your own socket or file objects to the request handling loop, and act on them if they trigger. For more details, see the paragraph below.

For those that like to have more control over the request handling loop, there is also `handleRequests`. Usually your loop will look something like this:

       while continueLoop:
          daemon.handleRequests(3.0)
          ... do something when a timeout occured ...

The timeout value in this example is three seconds. The call to `handleRequests` returns when the timeout period has passed, or when a new proxy object got a connection to the daemon. You could use '`0`' for timeout, but this means the call returns directly if no requests are pending. If you want infinite timeout, use '`None`'. You can also provide additional objects the daemon should wait on (multiplexing), to avoid having to split your program into multiple threads. You pass those objects, including a special callback function, as follows:

       daemon.handleRequests(timeout, [obj1,obj2,obj3], callback_func)

The second argument is a list of objects suitable for passing as ins list to the `select` system call. The last argument is a callback function. This function will be called when one of the objects in your list triggers. The function is called with one argument: the list of ready objects. For more information about this multiplexing issue, see the manual page about the Un*x `select` system call.

#### Including the Pyro Daemon in another (external) event loop

Some applications already have their own event loop. If it is `select`-based, or can process additional sockets to wait on, you can also use your application's event loop instead of the Daemon's `requestLoop`. Do this by querying the Daemon for a list of active socket objects that it is currently listening on, and pass every socket in that list to your external event loop. The Daemon has a method `getServerSockets()` that returns this list of `socket` objects. This list changes so you have to call it every time you enter the 'foreign' event loop. When your code returns from the 'foreign' event loop, check if one of Pyro's sockets has an event, and if so, call the regular `handleRequests()`. Pyro will then process every event that's pending for it. An example:

    while some_condition :
            socks=daemon.getServerSockets()
            ins,outs,exs=select.select(socks,[],[],2)   # 'foreign' event loop
            for s in socks:
                    if s in ins:
                            daemon.handleRequests()
                            break    # no need to continue with the for loop

Have a look at the "AllInOne" example. It shows two approaches of starting various Pyro servers from within a single program and then using a custom event loop to wait for incoming requests. That code is easily adapted to integrate Pyro in a GUI toolkit's event loop, for instance.

#### Stopping the server, cleaning up

To signal the Daemon that it should stop its requestloop, you can call `daemon.shutdown()` or send the process a break signal (ctrl-C). This issues an asynchronous request to the Daemon to terminate the request loop once any processing that is currently going on, is finished (it can still take a while before the requestloop is actually stopped). Once the loop stops, and all references to the daemon object are gone, it is garbage collected and Python tries to run the finalizer code that nicely unregisters any connected objects (so their names are removed from the Name Server unless you're using persistent mode).

However this may not work in all cases, or perhaps you want to control it explicitly. If you want to explicitly tell the daemon to unregister its objects and shut down, you should use `daemon.shutdown(True)`. So your code might look like this:

     …
    daemon.connect( … )
    try:
        daemon.requestLoop()
    finally:
        daemon.shutdown(True)
        # at this moment, the objects have been unregistered
     …

If you're not doing any more processing in your server after the requestloop, it is usually not necessary to add this explicit cleanup logic. However, if the server is aborted in a 'hard' way (terminated, crash) instead of a normal shutdown or ctrl-C signal, Python may not execute the finalizer code and your objects are still registered in the NS. There is not much you can do about this; even the explicit shutdown code above doesn't help (because it is not executed as well!). A solution is to change the registration of the objects: if you encounter errors because the name already exists in the NS, just unregister the old name and re-register.

This concludes our server. Full listings can be found in the Example chapter.

### Step 5: Writing the client

#### Initialization

You should initialize Pyro before using it in your client program. This is done by calling

       Pyro.core.initClient()

If you provide the argument 'banner=1', a short version message is printed on the standard output. In contrast to the server initialization (see above), this method does not check the availability of the `PYRO_STORAGE` directory. This means that you can run Pyro clients on a read-only system, as long as they don't have to write something (log!) to `PYRO_STORAGE`!

If the tracelevel is not zero, a startup message is written to the log. This message shows the active configuration options.

It is not strictly required to call `Pyro.core.initClient()`. If you don't call it, Pyro will initialise itself automatically.

#### Find the Name Server

This part is identical to the way this is done in the server. See above. Let's assume that the variable `ns` now contains the proxy for the NS.

#### Find object URIs

There are essentially three ways to find an object URI by name:

- Query the NS. This is the best way to go. You ask the NS to give you the URI for the object with the right name ("my_object" in this example):

         uri = ns.resolve('my_object')

- Read it from a special file that was written by the object. This is like what is done in the previous paragraph, about finding the NameServer. Be sure to convert the string you read from the file to a real `PyroURI` object before you use it. Just pass it to the constructur of `PyroURI` and you'll be fine.
- Use the special `PYRONAME://` or `PYROLOC://` URI strings. The first is a shortcut to the Name Server, the second bypasses the Name Server completely. More info is in the Name Server chapter.

#### Create a proxy

You now have a URI in your posession. But you need an object to call methods on. So you create a proxy object for the URI.

       obj = Pyro.core.getProxyForURI(uri)     # get a dynamic proxy
       obj = Pyro.core.getAttrProxyForURI(uri) # get a dyn proxy with attribute support

       # if you're sure that the URI is a real PyroURI object, you can do this:
       obj = uri.getProxy()                    # get a dynamic proxy directly from the URI
       obj = uri.getAttrProxy()                # same, but with attribute support

If you're using attribute proxies, be aware of their properties and limitations.

#### Remote method invocations

And now what we've all been waiting for: calling remote methods. This is what's Pyro's all about: there is no difference in calling a remote method or calling a method on a regular (local) Python object. Just go on and write:

       obj.method(arg1, arg2)
       print obj.getName()
       a = obj.answerQuestion('What is the meaning of life?')
       # the following statements only work with a attribute-capable proxy:
       attrib = obj.attrib
       obj.sum = obj.sum+1

or whatever methods your objects provide. The only thing to keep in mind is that you need a proxy object whose methods you call.

This concludes our client. Full listings can be found in the Example chapter. For information on using Pyro's logging/tracing facility, see Runtime control and Logging, below.

### Steps 6, 7 and 8: Runtime setup

This part is a no-brainer, really. There may be some extra configuration necessary when you're running Pyro behind a firewall, and want to access it from outside the firewall, or have machines with dynamic IP addresses. You can find more information about this in the Features and Guidelines chapter. Otherwise it's simple:

#### Starting the Name Server

A Pyro system needs at least one running Name Server. So, if it's not already running, start one using the `ns` utility. See Pyro script tools. After starting it will print some information and then the Name Server sits in a loop waiting for requests:

    irmen@atlantis:~ > projects/Pyro/bin/ns
    *** Pyro Name Server ***
    Pyro Server Initialized. Using Pyro V2.4
    Will accept shutdown requests.
    URI written to: /home/irmen/Pyro_NS_URI
    URI is: PYRO://10.0.0.150:9090/0a000096-08620ada-6697d564-62110a9f
    Name Server started.

The NS writes its URI to a file, as it says. This file can be read by other programs, and this is another -very portable- way to discover the NS. Usually you'll want to use the default mechanism from the `NameServerLocator` (automatic discovery using broadcasting). This is easier. But if your network doesn't support broadcasting, or the NS can't be reached by a broadcast (because it sits on another subnet, for instance), you have to use another method to reach the NS.

#### Running the server

Just start the python module as you do normally. Before starting, you may want to set certain environment variables to change some of Pyro's configuration items. After starting, your server will usually sit in a loop waiting for incoming requests (method calls, actually).

#### Running the client

Just start the python module as you do normally. Before starting, you may want to set certain environment variables to change some of Pyro's configuration items.

### Runtime control and Logging

#### Controlling the Name Server

You might want to control the NS while it's running. For instance, to inspect the current registered names or to remove an old name, or to register a new one by hand. You use the `pyro-nsc` command-line utility or the `pyro-xnsc` graphical tool for this purpose, see Pyro script tools.

#### Controlling Pyro

Pyro has many configuration items that can be changed also during runtime. You might want to set the tracelevel to 3 during a special function, for instance. See the Installation and Configuration chapter for more information.

#### Tracing (logging)

Pyro has two distinct logs: the system log and the user log. The system log is used by Pyro itself. You can use it in your own code too, but generally it's better to use the user log.

- System Log
  The system log is implemented by the `Pyro.util.Log` object, which is an instance of `Pyro.util.SystemLogger`. System log tracelevel is configured using the `PYRO_TRACELEVEL` config item, the logfile location is configured using the `PYRO_LOGFILE` config item.
- User log
  You should create your own user log object by creating a `Pyro.util.UserLogger` instance. User log tracelevel is configured using the `PYRO_USER_TRACELEVEL` config item, the user logfile location is configured using the `PYRO_USER_LOGFILE` config item.
- Using the Logger object: logging entries
  The logger class provides four methods:
  - `msg(source, *args)` - log a simple message (note). `source` is a string that identifies the source of the log entry, after that, any argument may follow to be printed in the logfile.
  - `error(source, *args)` - log an error. `source` is a string that identifies the source of the log entry, after that, any argument may follow to be printed in the logfile.
  - `warn(source, *args)` - log a warning. `source` is a string that identifies the source of the log entry, after that, any argument may follow to be printed in the logfile.
  - `raw(string)` - log a string (unformatted). `string` is the string to write to the logfile. This logging is done unconditionally, the tracelevel setting has no influence here.
  Logfile entries have the following format:
  "`2002-01-16 16:45:02 [5884:MainThread] ** ERR! ** NameServerLocator ** Name Server not responding`"
  (a date and timestamp, process ID:thread name, then "NOTE", "WARN" or "ERR!", indicating if this was a simple message, a warning or an error. After that, the source of the log entry - this can be any string but should be meaningful for the developer. After that, the actual log message. All elements are separated by two asterisks). Each log entry is one line in the logfile. Entries written by the `raw` method can have any format, including multiple lines.

### Threads, sessions and objects

For more complex uses of Pyro, it is important to understand how Pyro uses threads and how the objects interact. Below are, in condensed form, the rules Pyro follows. For detailed information about these subjects, please refer to the relevant chapters elsewhere in the manual.

- A Pyro server object is only created once. The same instance is reused over and over by the Daemon. It is never deleted, unless the Daemon itself shuts down or the object is explicitly disconnected from it.
- Nameless ('transient') objects created on the server are not automatically deleted, the follow the same rules. You can use Pyro's transient object timeout feature to get rid of them.
- Pyro server objects may be accessed from multiple threads at the same time. So they need to be reentrant (threadsafe).
- A thread is created for every distinct connection that is created to the Daemon.
- A connection is a link between a Pyro proxy object and a Pyro Daemon. A proxy object has exactly one connection to a Daemon, a Daemon can have lots of connected proxy objects.
- Every distinct proxy object has its own connection. That means that when you create a new proxy object, a new connection will be made to the Daemon, and a new thread to handle the method calls will be spawned.
- Multiple method calls can occur over a single connection, they are handled serially by the thread associated to the connection. If you share a proxy object among several threads in your client application, all method calls will be serialized. If you make a copy of the proxy object and use those in different threads in your client application, the method calls will not be serialized and the server object will process them concurrently.
- 'oneway' calls can be an exception and can execute in their own short-lived thread.
- TLS can be used to store objects that must be unique to a thread or to a client session. For example, open files or database connection objects.
- You can use the `caller` property of the TLS object if required: that object always equals the currently active client connection. Multithreaded or not.
- If you disable multithreading using the config item, there will be only a single thread that will be shared by all method calls from all proxies! There will only be one shared TLS as well! If your code needs to be able to deal with this possibility, you always have to use the `caller` property of the TLS in some way instead of just storing stuff on the TLS directly.

### Last Notes

Please be sure to read the chapter on Configuration, the Name Server and the chapter about Pyro's Features and Guidelines.

These chapters contain invaluable information about the more detailed aspects and possibilities of Pyro.

Also have a look at the extensions package `Pyro.ext`, it contains two modules that provide extremely easy remoting for your programs. Take a look at the "quickstart" and "quickstart-noNS" examples for more details.