/ Perl

Serverless Tutorial: Open FaaS Meets Perl

Serverless (or FaaS: Function-as-a-Service) is a relatively new processing paradigm. Amazon Lambda or Google Cloud Functions are great examples of commercial offerings where functions can be deployed and run. What is so special about FaaS?

Serverless vs. Microservice

The main difference between a microservice and a function is the way they run and behave when called. A microservice must run in order to accept requests. If the container that encapsulates a service is stopped, the service cannot accept requests.
In contrast to this, a function runs only when it is called and stops when the processing in the function stops. Requests to a FaaS function are monitored by a watchdog that stars the function process and stops it when processing is done. The watchdog process must run all the time, but fortunately it is quite tiny and does not consume much resources. The function is started and stopped multiple times, so it is important that there is no heavy weight lifting in initialization and cleanup. It is recommended that FaaS functions are stateless and lightweight.

A Function in Perl

We need a function to get some hands-on experience with FaaS in Perl context. Let's start with something very simple. The function will read from STDIN and write to STDOUT.

# file: function/Handler.pm

package Handler;
use strict;
use warnings;

sub handle {
    my $st = shift;
    return "Hello $st !";
}

1;

The function handle takes the input parameter $st and returns a scalar (string) that prepends Hello and appends exclamation mark to the input. Let's write a script that allows to run function handle.

# file: main.pl

use lib qw(function local/lib/perl5 function/local/lib/perl5);
use strict;
use warnings;

require Handler;

sub get_stdin {
    my $buf = "";
    while (<STDIN>) {
        $buf .= $_;
    }
    return $buf;
}

my $st = get_stdin();
print Handler::handle($st);

Script main.pl is responsible for parsing the input from STDIN (get_stdin) and returning it as a scalar (return $buf). Next, the function handle in the Handler module is called and the result is printed to STDOUT. Let's see how the function should be called from a console.

$ echo -n "James Bond" | perl main.pl
Hello James Bond !

Everything is simple here. In the context of FaaS it is important to remember that a function should:

  • read from STDIN,
  • write to STDOUT,
  • be stateless.

Running Perl Function with OpenFaaS

I will use the OpenFaaS framework and Docker Swarm on a local machine to demonstrate how to run our example function.

Installing OpenFaaS and Preparing Machine

Instead of running our function on Amazon Lambda, we will use Docker Swarm as container runtime and OpenFaaS as a wrapper for functions. To setup your machine to run Docker Swarm just run docker swarm init.

$ docker swarm init
Swarm initialized: current node (wicllwro5khbx93ilo822zy3d) is now a manager.
...

To stop Docker Swarm, use

$ docker swarm leave --force
Node left the swarm.

Now install OpenFaaS tools

$ git clone https://github.com/openfaas/faas && \
  cd faas && \
  git checkout master && \
  ./deploy_stack.sh && \
  docker service ls
Deploying stack
Creating network func_functions
Creating service func_webhookstash
Creating service func_base64
Creating service func_echoit
Creating service func_hubstats
Creating service func_decodebase64
Creating service func_prometheus
Creating service func_alertmanager
Creating service func_wordcount
Creating service func_nodeinfo
Creating service func_gateway
Creating service func_markdown  

That's it! You can verify if everything works properly by navigating to http://localhost:8080/. With the default installation, several functions have been deployed. Try them out using the UI in the browser.

Template for Perl Functions

Unfortunately, quick boostrapping of Perl functions is not officially supported in OpenFaaS and it does not have a specialized template for a quick start. In order to start a new Perl project, we need to use generic Dockerfile template.

$ mkdir my-perl-fass && cd my-perl-fass
$ faas-cli new --lang Dockerfile perl-echo
2017/12/03 15:52:11 No templates found in current directory.
2017/12/03 15:52:11 HTTP GET https://github.com/openfaas/faas-cli/archive/master.zip
2017/12/03 15:52:13 Writing 287Kb to master.zip

2017/12/03 15:52:13 Attempting to expand templates from master.zip
2017/12/03 15:52:13 Fetched 10 template(s) : [csharp go-armhf go node-arm64 node-armhf node python-armhf python python3 ruby] from https://github.com/openfaas/faas-cli
2017/12/03 15:52:13 Cleaning up zip file...
Folder: perl-echo created.
  ___                   _____           ____
 / _ \ _ __   ___ _ __ |  ___|_ _  __ _/ ___|
| | | | '_ \ / _ \ '_ \| |_ / _` |/ _` \___ \
| |_| | |_) |  __/ | | |  _| (_| | (_| |___) |
 \___/| .__/ \___|_| |_|_|  \__,_|\__,_|____/
      |_|


Function created in folder: perl-echo
Stack file written: perl-echo.yml

We created a project folder named my-perl-fass and initialized a function based on the generic Dockerfile template. The directory structure looks as follows:

$ pwd 
~/Workspace/perl/my-perl-fass
$ tree -L 2
.
├── perl-echo
│   └── Dockerfile
├── perl-echo.yml
└── template
    ├── csharp
    ├── go
    ├── go-armhf
    ├── node
    ├── node-arm64
    ├── node-armhf
    ├── python
    ├── python-armhf
    ├── python3
    └── ruby

As we see, there is no Perl template, we just got the perl-echo.yml file and Dockerfile inside perl-echo directory. The Dockerfile is our starting point to run Perl scripts. It looks as follows:

FROM alpine:3.6
# Use any image as your base image, or "scratch"
# Add fwatchdog binary via https://github.com/openfaas/faas/releases/
# Then set fprocess to the process you want to invoke per request - i.e. "cat" or "my_binary"

ADD https://github.com/openfaas/faas/releases/download/0.6.9/fwatchdog /usr/bin
# COPY ./fwatchdog /usr/bin/
RUN chmod +x /usr/bin/fwatchdog

# Populate example here - i.e. "cat", "sha512sum" or "node index.js"
ENV fprocess="wc -l"

HEALTHCHECK --interval=5s CMD [ -e /tmp/.lock ] || exit 1
CMD ["fwatchdog"]

Lets take a deeper look on the Dockerfile. In line 6, it installs the fwatchdog from OpenFaaS GitHub repo and makes it executable (line 8). Next, in line 11, it defines an environmental variable fprocess that defines a function that should be run by the watchdog. By default it is set to wc -l, so the function being run serverless is wc -l.
Next, in line 13 a healthcheck is defined. It returns true (exit status = 0) when file /tmp/.lock exists; otherwise it returns error (exit status = 1). This is used to check if the watchdog is still runinng. In case it stops, the lock file will dissapear and Docker swarm will spawn a new container. This is great, because we do not need to care if the container is running all the time - it will be restarted if it happens.
Finally, line 14 defines a process to be started when we issue the docker run command. It is set to fwatchdog, because the watchdog manages the function. It also means that the only process running in the container is watchdog, and not our function. The function will be triggered by the watchdog when requests arrive.
Everything is ready, lets build and run it!

$ faas-cli build -f perl-echo.yml
[0] > Building: perl-echo.
Building: perl-echo with Dockerfile. Please wait..
...
Image: perl-echo built.
[0] < Builder done.
$ faas-cli deploy -f perl-echo.yml
Deploying: perl-echo.
No existing function to remove
Deployed.
URL: http://localhost:8080/function/perl-echo

200 OK

Now the function can be run using faas-cli invoke perl-echo and piping inputs into the function.

$ echo -n "James" | faas-cli invoke perl-echo
0
$ echo -n "James\nBond" | faas-cli invoke perl-echo
1
$ echo -n "James\nBond\n007" | faas-cli invoke perl-echo
2

As demonstrated, inputs with various number of lines produced expected results on the output. Note, that echo -n does not include end-of-line, thus result was 0.

Calling Perl from Dockerfile

Binding Perl function to the generated template consist of changing the line defining the fprocess variable. This gives (after cleaning up the comments) the following Dockerfile.

FROM perl:5.26

ADD https://github.com/openfaas/faas/releases/download/0.6.9/fwatchdog /usr/bin
RUN chmod +x /usr/bin/fwatchdog

RUN cpanm Carton

WORKDIR /root/
COPY main.pl .
COPY cpanfile .
RUN carton install

COPY function function

ENV fprocess="carton exec perl main.pl"

HEALTHCHECK --interval=5s CMD [ -e /tmp/.lock ] || exit 1
CMD ["fwatchdog"]

The changes to the Dockerfile contain (referred are numbered comments):

  • Line 1: Change of the base image from alpine:3.6 to perl:5.26
  • Line 6: Installing carton in the container
  • Line 8: Changing the working directory
  • Line 9: Copying of the main.pl file
  • Line 10: Copying of the cpanfile file
  • Line 11: Run of dependencies installation using carton
  • Line 13: Copying of the function directory (including Handler.pm)
  • Line 15: Change of the call to the function, i.e., running the perl code

Let's build the new container using cd my-perl-faas && faas-cli build -f perl-echo.yml.
Now, the function can be triggered:

$ echo -n "James Bond" | faas-cli invoke perl-echo
Hello James Bond !

Great! Everything works as expected!

In case of execution errors, for example:

$ echo -n "James\nBond\n007" | faas-cli invoke perl-echo
Server returned unexpected status code: 500 - Can't reach service: perl-echo

you may wait a moment or check the status using docker ps. If component is in status starting waiting is the solution. Status healthy confirms that function is ready to use. Any other status denotes an error with starting the fwatchdog or the function itself. The errors will be returned by faas-cli or can be investigated using docker ps -a and docker logs <container-sha>.

Tutorial Summary

A function that can be deployed as serverless should read from standard input and return values to standard output. The code that handles the function can be written in any programming language, including Perl. Using generic Dockerfile template provided by OpenFaaS allows to bind the function call using the fprocess environemntal variable. Required installation steps must be added to the Dockerfile manually.

The project structure after the tutorial looks as follows:

$ cd ~/Workspace/perl/my-perl-fass
$ tree -L 3
.
├── perl-echo
│   ├── Dockerfile
│   ├── cpanfile
│   ├── function
│   │   └── Handler.pm
│   └── main.pl
├── perl-echo.yml
└── template
    ├── ...

Why FaaS?

You may ask yourself a question, what is the benefit of using FaaS when such a simple function may be run as usual.

  1. Functions can be deployed to external services (e.g., similar to AWS Lambda) and reduce the complexity of locally installed software.
  2. The watchdog process running inside Docker container can spawn multiple processes with our functions which allows to parallelize execution.
  3. Using Docker Swarm (or Kubernetes) guarantees that the Docker container will be restarted in case of a failure.
  4. OpenFaaS provides Prometheus monitoring out-of-the-box, so that invocations of the service can be monitored.

Missing Bits

Unfortunately, I was unable to find any free hosting offering for deploying OpenFaaS functions. The most of the available tutorials and manuals refer either to paid offerings, or suggest to run a server on our own (e.g., using Digital Ocean). This gives me the feeling that the idea of being serverless is somehow not complete.

Unfortunately, according to my current knowledge, OpenFaaS functions cannot be deployed to AWS Lambda.

Further Steps

This tutorial presents very simple Perl function that reads and returns unformatted text. Further improvements could include, for example, support for JSON-formatted input and output.

More complex example using OpenFaaS and Perl is available in my GitHub repo faas-perl-bibtex-2-html

Piotr

Piotr

IT specialist with 7 years of experience in research at four European universities and in private sector. I believe that good quality is always worth to invest few € and some free time.

Read More