Erlang An Open Source Language for Robust Distributed Applications
Dr Lawrie Brown
School of Computer Science,
Australian Defence Force Academy,
Canberra, Australia
Email: Lawrie.Brown@adfa.edu.au
Abstract
This paper introduces Erlang, a functional language designed to
support robust, reliable, distributed, near real-time applications.
Its was developed by the Ericsson Computer Science Laboratories, and
was released as open source in late 1998. Erlang includes not just
the base language, but the Mnesia DBMS, a web server, an ORB, and an
SNMP agent. Collectively these provide some very good tools for
applications development.
Introduction
Erlang is a declarative language for programming
concurrent and distributed systems. It was developed at the Ericsson
Computer Science Laboratories
to satisfy a requirement for a language suitable for building large
soft real-time control systems, particularly telecommunications systems
[AVWW96] [Arms97]
[Arms96] [Wiks94].
It is a dynamically typed, single assignment language which uses pattern
matching for variable binding and function selection, has inherent
support for lightweight concurrent and distributed processes, and has
error detection and recovery mechanisms. Most of the Erlang system is
written in Erlang (including compilers, debuggers, standard libraries),
with just the core run-time system and a number of low-level system
calls (known as BIFs for BuiltIn Functions) written in C. Distributed
use is almost transparent, with processes being spawned on other nodes,
instead of locally. An Erlang node is one instance of the run-time
environment, often implemented as a single process (with many threads)
on Unix systems. Erlang is currently being used in the development of
a number of very large telecommunications applications, and this usage
is expected to increase.
A binary distribution of the Erlang system has been available for several
years. In late 1998, Ericsson announced the release of Erlang as
open source, under the Erlang Public Licence (a derivative
of the Mozilla licence). The release comprises full source for the
run-time system, compilers (both the original JAM and the new BEAM),
the Mnesia DBMS, libraries and utilities. It is available from the
erlang.org
site [Erlang].
The features of open-source Erlang are described in a
White Paper [ADLM98], and in the references
listed above.
Why a Functional Language?
Erlang is a functional language. It inherits benefits such
as pattern matching for selecting between alternate function clauses,
minimal side-effects which greatly simplify analysis and testing of code,
and dynamically typed symbolic data representation. Application are
composed of many functions, and the lack of side-effects in most means
its easy to test and validate components of the application. Generally
the only side-effects occur when interacting with other processes or
external resources. This interaction can be easily isolated to assist
in the testing process.
In additional, because the primary design focus was on distributed
soft real-time systems, the language includes an extremely effective
distribution and concurency mechanisms. Spawning a new process (thread) is
extremely cheap and efficient, and spawning a process on another node is a
trivial extension. Given a process identifier, all other communication and
interaction with it is independent of whether it is local or remote. This
greatly eases the implementation of robust distributed applications.
Experience with a number of large application developments seems to show
that using Erlang leads to a faster implementation cycle and time to
market, compared to using other more traditional development languages
(see [AVWW96]).
There are of course, some disadvantages. Many numerical algorithms
don't translate well into Erlang since they usually assume mutable
data elements. The use of dynamic typing and garbage collection does
involve some runtime overhead, though given the speed of current systems,
in many applications this has not proved to be a problem. Also, because
the original design focus was not on user interface issues, graphical
interfaces may not be as efficient as in some other languages, though
they certainly are available.
There is however, always the possibility of either calling out to
an external program or using a linked in driver, which can be derived
from code in any language. It comes down to a case of selecting the best
language for any particular task. And in real, large scale, applications,
this may often involve using several langauges for different components
of the system (cf the AXD301 ATM switch later).
Features
Because of its design goal to support soft real-time control systems,
Erlang includes a number of features more often considered
as part of the operating system, including:
- Concurrency
- using extremely light-weight processes communicating by message passing
- Distribution
- with easy creation of and interaction with processes on remote nodes
- Robustness
- using error and exception handling mechanisms to support fault-tolerant
systems, including monitoring of processes on remote nodes
- Soft real-time response
- capable of responding to events within a few milliseconds, and
with garbage collection handler capable of supporting this feature
- Hot code upgrade
- of code modules on a running system with handover from previous to
current code versions
- Incremental code loading
- where code can be loaded either at boot time (embedded systems) or as
needed (general purpose systems)
- External interfaces to devices, files, network, other processes
- using the same message passing mechanism as used with other Erlang
processes (Erlang has the view everything is a process, much like
the Unix view that everything is a file)
More details are provided in the White Paper [ADLM98].
Components
As well as the basic language development and run-time system, the
open-source Erlang distribution includes a number of standalone
components which can be used as building blocks when developing
applications including:
- Mnesia
- a fast, distributed, real-time database for Erlang, running in
the same address space as the applications that use it, which supports
arbitrarily complex data structures and dynamic scheme changes. It was
designed specifically to meet the robust, fault tolerant, requirements
of telecommunications applications (eg. billing), but is suitable
for a number of other applications. It includes automatic consistency
maintenance of replicated data distributed over a number of systems.
A SQL interface will be available shortly.
See [MNW99] for additional details.
- Inets
- a package of Internet tools including a fully featured HTTP server
with Apache style configuration. The server can be used as a standalone
web server, or the HTTP protocol modules can be built into applications
to provide a web user interface to them.
- Orber
- a CORBA v2.0 object request broker (ORB). Erlang programs can be
clients or servers. It includes support for common object services
such as persistence, naming & events.
- SNMP
- an extensible SNMP v1/v2 agent and MIB (ASN.1) compiler.
As well, the standard library includes a number of modules for applications
monitoring and debugging, GUI interface support (using Tk widgets),
parse tools, etc.
A Trivial Example
The following is a very simple illustration of concurrent distributed
Erlang use, programming a very simple bank account server
(mickey mouse student example :-)
-module(bankserver).
-export([start/1]).
start(Sum) -> % start account server
register(bank,self()), % register as 'bank'
account(Sum). % process transactions
account(Sum) -> % transaction processing loop
receive % await transaction
{Pid, Ref, Amount} -> % account update msg received
NewSum = Sum+Amount, % update sum
Pid ! {Ref,NewSum}, % send response back
account(NewSum); % loop (recurse)
stop -> nil % end server msg received
end.
This could be started on some remote node specified by BankNode
,
with a balance of 1000, and updated, as follows:
...
% spawn a bank account process with initial Sum 1000
Bank_pid = spawn(BankNode,bankserver,start,[1000]),
...
Ref = make_ref(), % make a unique ref value
Bank_pid ! {self(),Ref,17}, % send msg to server
receive % await reply from server
{Ref,New_balance} -> % reply says updated ok
...
end,
...
Alternatively, any other process in that node could communicate with
it using the registered name (which maps to the appropriate pid):
bank ! {self(),Ref,-12}, % send msg to server
...
Note that this example doesn't use any of the standard mechanisms
for programming and monitoring client-server applications, just
the core language features. It could be made much smaller and
more reliable using them (but then you wouldn't see what was
going on behind the scenes!)
A Slightly More Serious Example
In the Appendices at the end of this paper is code for a very
simple HTTPD/0.9 web server. It has just the minimum functionality,
extracting the name from the request, appending it to the supplied
document root, and returning either that file or an error response.
However it does illustrate how Erlang can be used to implement a
TCP/IP server, and shows some more extensive error handling.
Whilst probably too lean for use as is, this code could form the
outline of code permitting configuration of some application
using a web interface. Of course if serious web server is required,
then a fully featured HTTPD/1.0 server is supplied as part of
the Open Source Erlang system.
In the second Appendix is the code for a simple application monitor,
which starts the requested application, catches and logs an errors
that killed it, and then restarts the application. Again this is
minimal code, and a fully featured monitor is provided as part of
the system, but it illustrates how the process works. It could be
used to monitor the above httpd server by being run as:
monitor:start(httpd,start,[]).
It could easily be extended to monitor an application on a remote node,
simply by providing an additional argument Node
, and
then changing the spawning of the application to be remote using:
spawn_link(Node,Module,Function,Args)
Production Applications
Erlang has been used as the major language in the development of
a number of Ericsson products in the last few years. The two
best known examples are the Mobility Server, and the AXD301 ATM switch.
The Mobility Server is an intelligent call control system incorporated
within the Consomo PBX to provide a personal number service for mobile
phone users. It comprises 486 Erlang modules with 230k loc (lines of
code) and 20k loc in C for the device interfaces, and was written by a
team of 35 people on time and under budget. It has now been sold to a
number of customers.
The AXD301 ATM switch is a new asynchronous transfer mode (ATM) switching
system which combines features associated with data communication, such as
compactness and high functionality, with features from telecommunications,
such as robustness and scaleability. The AXD 301 system is designed
for nonstop operation, with duplicated hardware, and modular software,
which can be upgraded to facilitate the introduction of new functionality
without disturbing traffic.
There are a number of components in its application architecture, and
different languages were used for each. All of the device interfaces
and interaction with the switching fabric was written in C. The web
based management interface was written in Java. The remainder of the
application, including the overall control and management was written in
Erlang. Each of these components comprise about 150k loc.
There are consistent reports that programming in a functional language
such as Erlang is faster, results in smaller source code size,
and is more maintainable than when traditional procedural languages
are used. This is discussed in [Arms96].
And Where do I come In?
For those of you who know me, and are wondering how I came to be playing with
Erlang when my research areas are cryptography and security: well during
my sabbatical in 1997 I started a project looking at what extensions would be
needed to a functional language like Erlang to support safe mobile code
execution. The answer is surprisingly little, and you can read about it in
[BrSa97], [Bro97d].
It primarily involves replacing forgeable identifiers for pids and ports
etc with capabilities which explicitly state what operations are
permitted on them, the provision of a hierarchy of sub-nodes within
each physical node, which can support a range of security restrictions,
and support for remote code loading in context (so references to
modules in code loaded from a remote site are also resolved from
that remote site). Currently these modifications are being evaluated
using a prototype which rewrites the Erlang source in a "safe" manner.
For production use though, these changes would need to be incorporated
into the actual Erlang run-time system.
Conclusions
In this paper I have introduced Erlang, an open-source language from
Ericsson, which I believe provide some very useful tools for building
large, scaleable, robust, distributed applications. Take a look for
yourselves.
Online Resources
Information and sources are available for:
- Erlang
- at http://www.erlang.org
or
http://www.serc.rmit.edu.au/mirrors/ose_mirror/ (oz shadow)
- Eddie
- at http://www.eddieware.org
References
- ADLM98
-
Joe Armstrong, Bjarne Däcker, Thomas Lindgren, Håkan Millroth,
"Open-source Erlang - White Paper",
Ericsson Computer Science Laboratory, Stockholm, Sweden, white paper, 1998.
http://www.erlang.org/white_paper.html.
- AVWW96
-
J. Armstrong, R. Virding, C. Wikstrom, M. Williams,
"Concurrent Programming in Erlang",
2nd edn, Prentice Hall, 1996.
http://www.ericsson.se/erlang/sure/main/news/book.shtml.
- Arms96
-
J. Armstrong,
"Erlang - A Survey of the Language and its Industrial Applications",
in INAP'96 - The 9th Exhibitions and Symposium on Industrial Applications of Prolog,
Hino, Tokyo, Japan, Oct 1996.
http://www.ericsson.se/cslab/erlang/publications/inap96.ps.
- Arms97
-
Joe Armstrong,
"The Development of Erlang",
in Proceedings of the ACM SIGPLAN International Conference on Functional Programming,
ACM, pp 196-203, 1997.
- BrSa97
-
L. Brown, D. Sahlin,
"Extending Erlang for Safe Mobile Code Execution",
School of Computer Science, Australian Defence Force Academy, Canberra, Australia, Technical Report, No CS03/97, Nov 1997.
http://lpb.canb.auug.org.au/adfa/papers/tr9703.ps.gz.
- Bro97d
-
L. Brown,
"SSErl - Prototype of a Safer Erlang",
School of Computer Science, Australian Defence Force Academy, Canberra, Australia, Technical Report, No CS04/97, Nov 1997.
http://lpb.canb.auug.org.au/adfa/papers/tr9704.html.
- Erlang
-
Erlang Systems,
"Open Source Erlang Distribution",
Ericsson Software Technology AB, Erlang Systems, 1999.
http://www.erlang.org/.
- MNW99
-
H Mattsson, H Nilsson, C Wikstrom,
"Mnesia: A Distributed Robust DBMS for Telecommunications Applications",
in First Intl. Workshop on Practical Aspects of Declarative Languages (PADL'99),
1999.
http://www.ericsson.se/cslab/~klacke/padl99.ps.
- Wiks94
-
C. Wikstrom,
"Distributed Programming in Erlang",
in PASCO'94 - First International Symposium on Parallel Symbolic Computation,
Sep 1994.
http://www.ericsson.se/cslab/erlang/publications/dist-erlang.ps.
Appendix - httpd.erl
A very simple HTTPD/0.9 web server, capable of parsing a request and
returning the named file below the configured document root.
%% httpd.erl - a simple HTTPD/0.9 web server in Erlang
-module(httpd).
-export([start/0,start/1,start/2,process/2]).
-import(regexp,[split/2]).
-define(defPort,8888). %% port to use if not given
-define(docRoot,"./htdocs"). %% HTML document root
%% start mini HTTPD/0.9 server, can specify port/docroot if wanted
start() -> start(?defPort,?docRoot).
start(Port) -> start(Port,?docRoot).
start(Port,DocRoot) ->
case gen_tcp:listen(Port, [binary,{packet, 0},{active, false}]) of
{ok, LSock} -> server_loop(LSock,DocRoot);
{error, Reason} -> exit({Port,Reason})
end.
%% main server loop - wait for next connection, spawn child to process it
server_loop(LSock,DocRoot) ->
case gen_tcp:accept(LSock) of
{ok, Sock} ->
spawn(?MODULE,process,[Sock,DocRoot]),
server_loop(LSock,DocRoot);
{error, Reason} ->
exit({accept,Reason})
end.
%% process current connection
process(Sock,DocRoot) ->
Req = do_recv(Sock),
{ok,[Cmd|[Name|[Vers|_]]]} = split(Req,"[ \r\n]"),
FileName = DocRoot ++ Name,
LogReq = Cmd ++ " " ++ Name ++ " " ++ Vers,
Resp = case file:read_file(FileName) of
{ok, Data} ->
io:format("~p ~p ok~n",[LogReq,FileName]),
Data;
{error, Reason} ->
io:format("~p ~p failed ~p~n",[LogReq,FileName,Reason]),
error_response(LogReq,file:format_error(Reason))
end,
do_send(Sock,Resp),
gen_tcp:close(Sock).
%% construct HTML for failure message
error_response(LogReq,Reason) ->
"<html><head><title>Request Failed</title></head><body>\n" ++
"<h1>Request Failed</h1>\n" ++ "Your request to " ++ LogReq ++
" failed due to: " ++ Reason ++ "\n</body></html>\n".
%% send a line of text to the socket
do_send(Sock,Msg) ->
case gen_tcp:send(Sock, Msg) of
ok -> ok;
{error, Reason} -> exit(Reason)
end.
%% receive data from the socket
do_recv(Sock) ->
case gen_tcp:recv(Sock, 0) of
{ok, Bin} -> binary_to_list(Bin);
{error, closed} -> exit(closed);
{error, Reason} -> exit(Reason)
end.
Appendix - monitor.erl
A very simple application monitor which illustrates how errors can be
caught and an application restarted.
%% monitor - simple application monitor
-module(monitor).
-export([start/3,run/3]).
-define(max,5).
%% spawn of monitor process
start(Module, Function, Args) ->
spawn(?MODULE,run,[Module, Function, Args]).
%% start application being monitored with traps being caught
run(Module, Function, Args) ->
process_flag(trap_exit, true),
Child = spawn_link(Module,Function,Args),
monitor(Child, ?max, Module, Function, Args).
%% wait for something to go wrong, log, restart application
monitor(Child, N, Module, Function, Args) ->
receive
{'EXIT', Child, Why} when N > 0 ->
io:format("~p:~p~p died ~p~n", [Module,Function,Args,Why]),
NewChild = spawn_link(Module,Function,Args),
monitor(NewChild,N-1,Module,Function,Args);
{'EXIT', Child, _} ->
io:format('too many restarts on ~p:~p~p!~n', [Module,Function,Args])
end.
The latest version of this paper may be found at:
http://lpb.canb.auug.org.au/adfa/papers/auug99-erl.html.
This paper was last revised: 30 July 1999.