RPyC
Encyclopedia
RPyC or Remote Python Call, is a python library for remote procedure call
s (RPC), as well as distributed computing
. Unlike regular RPC mechanisms, such as ONC RPC, CORBA
or Java RMI, RPyC is transparent, symmetrical, and requires no special decoration or definition languages. Moreover, it provides programmatic access to any pythonic element, be it functions, classes, instances or modules.
Once a client is connected to the server, it has one of two ways to perform remote operations:
Remote operations return something called a NetProxy, which is an intermediate object that reflects any operation performed locally on it to the remote object. For example, conn.modules.sys.path is a NetProxy for the sys.path object of the server. Any local changes done to conn.modules.sys.path are reflected immediately on the remote object.
Note: NetProxies are not used for simple objects, such as numbers and strings, which are immutable.
Async is a proxy wrapper, meaning, it takes a NetProxy and returns another that wraps it with asynchronous functionallity. Operations done to an AsyncNetProxy return something called AsyncResult. These objects have a '.is_ready' predicate, '.result' property that holds the result (or blocks until it arrives), and '.on_ready' callback, which will be called when the result arrives.
RPyC 3 is a complete rewrite of the library, adding a capability
-based security model, explicit services, and various other improvements.
Remote procedure call
In computer science, a remote procedure call is an inter-process communication that allows a computer program to cause a subroutine or procedure to execute in another address space without the programmer explicitly coding the details for this remote interaction...
s (RPC), as well as distributed computing
Distributed computing
Distributed computing is a field of computer science that studies distributed systems. A distributed system consists of multiple autonomous computers that communicate through a computer network. The computers interact with each other in order to achieve a common goal...
. Unlike regular RPC mechanisms, such as ONC RPC, CORBA
Çorba
Chorba , ciorbă , shurpa , shorpo , or sorpa is one of various kinds of soup or stew found in national cuisines across Middle East...
or Java RMI, RPyC is transparent, symmetrical, and requires no special decoration or definition languages. Moreover, it provides programmatic access to any pythonic element, be it functions, classes, instances or modules.
Features
- Symmetrical—there is no difference between the client and the server—both can serve. The only different aspect is that the client is usually the side that initiates the action. Being symmetrical, for example, allows the client to pass callback functions to the server.
- Transparent—remote objects look and behave the same as local objects would.
- Exceptions propagate like local ones
- Allows for synchronous and asynchronous operation:
- Synchronous operations return a NetProxy (see below)
- Asynchronous operations return an AsyncResult, which is like promise objects
- AsyncResults can be used as events
- ThreadsThread (computer science)In computer science, a thread of execution is the smallest unit of processing that can be scheduled by an operating system. The implementation of threads and processes differs from one operating system to another, but in most cases, a thread is contained inside a process...
are supported (though not recommended) - UNIX specific: server integration with inetdInetdinetd is a super-server daemon on many Unix systems that manages Internet services. First appearing in 4.3BSD , it is generally located at /usr/sbin/inetd.-Function:...
Architecture
RPyC gives the programmer a slave python interpreter at his or her control. In this essence, RPyC is different than other RPCs, that require registration of resources prior to accessing them. As a result, using RPyC is much more straight-forward, but this comes at the expense of security (you cannot limit access). RPyC is intended to be used within a trusted network, there are various schemes including VPN for achieving this.Once a client is connected to the server, it has one of two ways to perform remote operations:
- The modules property, that exposes the server's modules namespace:
doc = conn.modules.sys.pathorconn.modules["xml.dom.minidom"].parseString("<some>xml</some>"). - The execute function, that executes the given code on the server:
conn.execute("print 'hello world'")
Remote operations return something called a NetProxy, which is an intermediate object that reflects any operation performed locally on it to the remote object. For example, conn.modules.sys.path is a NetProxy for the sys.path object of the server. Any local changes done to conn.modules.sys.path are reflected immediately on the remote object.
Note: NetProxies are not used for simple objects, such as numbers and strings, which are immutable.
Async is a proxy wrapper, meaning, it takes a NetProxy and returns another that wraps it with asynchronous functionallity. Operations done to an AsyncNetProxy return something called AsyncResult. These objects have a '.is_ready' predicate, '.result' property that holds the result (or blocks until it arrives), and '.on_ready' callback, which will be called when the result arrives.
Usage
Originally, RPyC was developed for managing distributed testing of products over a range of different platforms (all capable of running python). However, RPyC has evolved since then, and now its use cases include:- Distributed computing (splitting workload between machines)
- Distributed testing (running tests that connect multiple platforms and abstracting hardware resources)
- Remote administration (tweaking config files from one central place, etc.)
- Tunneling or chaining (crossing over routable network boundaries)
Demo
import rpyc
conn = rpyc.classic.connect("hostname") # assuming a classic server is running on 'hostname'
print conn.modules.sys.path
conn.modules.sys.path.append("lucy")
print conn.modules.sys.path[-1]
- a version of 'ls' that runs remotely
def remote_ls(path):
ros = conn.modules.os
for filename in ros.listdir(path):
stats = ros.stat(ros.path.join(path, filename))
print "%d\t%d\t%s" % (stats.st_size, stats.st_uid, filename)
remote_ls("/usr/bin")
- and exceptions...
try:
f = conn.builtin.open("/non/existent/file/name")
except IOError:
pass
History
RPyC is based on the work of Eyal Lotem (aka Lotex) on PyInvoke, which is no longer maintained. The first public release was 1.20, which allowed for symmetrical and transparent RPC, but not for asynchronous operation. Version 1.6, while never publicly released, added the concept of 'events', as a means for the server to inform the client. Version 2.X, the first release of which was 2.2, added thread synchronization and the Async concept, which can be used as a superset of events. Version 2.40 adds the execute method, that can be used to execute code on the other side of the connection directly.RPyC 3 is a complete rewrite of the library, adding a capability
Capability-based security
Capability-based security is a concept in the design of secure computing systems, one of the existing security models. A capability is a communicable, unforgeable token of authority. It refers to a value that references an object along with an associated set of access rights...
-based security model, explicit services, and various other improvements.