Skip to content

M.E.L.I.G.: From VM to Unikernel and SOA to Serverless

During the latest MeetUp we talked about the rise of virtualization techniques and how software changed from clunky big services to state-less functions.

Slides & Videos

M.E.L.I.G. Unikernel and Serverless

Virtualization

While VMs and containers are all over the place I wanted to give an introduction to Unikernels. Which merges application code, libraries and the bits&pieces of the kernel needed into an 'application kernel'. One project to make use of it is Unik.

Setup UniK

To install it on macOS, the following steps have to be done, as described here: github.com/emc-advanced-dev/unik/blob/master/docs/install.md

git clone git@github.com:emc-advanced-dev/unik.git ~/src/github.com/emc-advanced-dev/unik
cd ~/src/github.com/emc-advanced-dev/unik
make
cp _build/unik /usr/local/bin

Afterwards open up VirtualBox and check on the vboxnet0 adapter (under Preferences).

The unikernel will get IP addresses from the range of the DHCP server, the address of the adapter has to be in the same network as the range. Next up, configuration of the unik daemon:

$ mkdir $HOME/.unik
$ cat << \EOF > $HOME/.unik/daemon-config.yaml
providers:
  virtualbox:
    - name: my-vbox
      adapter_type: host_only
      adapter_name: vboxnet0
EOF
$

And start the daemon in one terminal window...

$ unik daemon
INFO[0000] daemon started                                config={Providers:{Aws:[] Gcloud:[] Vsphere:[] Virtualbox:[{Name:my-vbox AdapterName:vboxnet0 VirtualboxAdapterType:host_only}] Qemu:[] Photon:[] Xen:[] Openstack:[] Ukvm:[]} Version:}
*snip*

This compiles a little unikernel itself and starts it. After a little while we can open another tab and connect to the daemon.

$ unik target --host localhost
$ unik instances
INFO[0000] listing instances                             host=localhost:3000
NAME            ID                   INFRASTRUCTURE CREATED                        IMAGE                IPADDRESS       STATE
VboxUnikInstanc 265604ea-40a6-4183-9 VIRTUALBOX     2017-03-23 15:45:04.317971783  VboxUnikInstanceList 192.168.100.101 running

Create First Unikernel

So far, so good, but let's create our own little unikernel following the little example: emc-advanced-dev/unik/blob/master/docs/getting_started.md

$ git clone git@github.com:qnib/unik-http.git ~/src/github.com/qnib/unik-http/
$ cd ~/src/github.com/qnib/unik-http/
$ unik build --name httpImage --path ./ --base rump --language go --provider virtualbox
INFO[0000] running unik build                            args= base=rump force=false host=localhost:3000 language=go mountPoints=[] name=httpImage path=./ provider=virtualbox
INFO[0000] App packaged as tarball: /var/folders/4x/s7z45gq93y5cq1cyccjbzw3h0000gn/T/sources.tar.gz.892223842

NAME                 ID                   INFRASTRUCTURE  CREATED                        SIZE(MB) MOUNTPOINTS
httpImage            httpImage            VIRTUALBOX      2017-03-23 15:50:12.887106166  39
$

Now we start the unikernel...

$ unik run --instanceName httpInstance --imageName httpImage
INFO[0000] running unik run                              env=map[] host=localhost:3000 imageName=httpImage instanceName=httpInstance mounts=map[]
NAME            ID                   INFRASTRUCTURE CREATED                        IMAGE                IPADDRESS       STATE
httpInstance    919ffb07-3788-4159-9 VIRTUALBOX     2017-03-23 15:59:57.960772954  httpImage                            pending
$ sleep 30 ; curl -s http://192.168.100.102:8080
my first unikernel!

Et voila...

Function-as-a-Service (FaaS)

The second outcome is somehow related, but not necessarily. Lately serverless, function-as-a-service caught my interest...

This new paradigm breaks down microservices into single functions, without the need of an actual service anymore - it's just the function which is scheduled and load-balanced by some framework. In the case of FaaS (github.com/alexellis/faas) Docker Services are used to scale up and a golang watchdog just schedules a binary over and over again.

Since it runs on SWARM, it's pretty hassle-free to set up:

$ git clone git@github.com:alexellis/faas.git ~/src/github.com/alexellis/faas/
Cloning into '/Users/kniepbert/src/github.com/alexellis/faas'...
remote: Counting objects: 997, done.
remote: Compressing objects: 100% (139/139), done.
remote: Total 997 (delta 66), reused 0 (delta 0), pack-reused 853
Receiving objects: 100% (997/997), 1.55 MiB | 1.24 MiB/s, done.
Resolving deltas: 100% (515/515), done.
Checking connectivity... done.
$ ./deploy_stack.sh
Deploying stack
Creating service func_hubstats
Creating service func_alertmanager
Creating service func_wordcount
Creating service func_gateway
Creating service func_alexacolorchange
Creating service func_nodeinfo
Creating service func_echoit
Creating service func_base64
Creating service func_markdown
Creating service func_prometheus
Creating service func_decodebase64
Creating service func_webhookstash
$

This creates a bunch of exemplary functions (as docker services). Once every docker image is downloaded and started...

$ docker service ls
ID            NAME                   MODE        REPLICAS  IMAGE
25m7f8sz72fl  func_prometheus        replicated  1/1       quay.io/prometheus/prometheus:latest
2wev4d40mfxp  func_nodeinfo          replicated  1/1       alexellis2/faas-nodeinfo:latest
749jiszgx2t2  func_gateway           replicated  1/1       alexellis2/faas-gateway:latest
7subbjju3oc2  func_alexacolorchange  replicated  1/1       alexellis2/faas-alexachangecolorintent:latest
atqkvvfm6ek5  func_hubstats          replicated  1/1       alexellis2/faas-dockerhubstats:latest
lwp6xe4s8z76  func_echoit            replicated  1/1       alexellis2/faas-alpinefunction:latest
m9dxiljg51mf  func_decodebase64      replicated  1/1       alexellis2/faas-alpinefunction:latest
n5bnq69ffxdf  func_markdown          replicated  1/1       alexellis2/faas-markdownrender:latest
ptm6wfn2roov  func_base64            replicated  1/1       alexellis2/faas-alpinefunction:latest
qtsom1nqqpl8  func_webhookstash      replicated  1/1       alexellis2/faas-webhookstash:latest
tyeeksxdyxpq  func_alertmanager      replicated  1/1       quay.io/prometheus/alertmanager:latest
xgv1gpso6fot  func_wordcount         replicated  1/1       alexellis2/faas-alpinefunction:latest
$

... it is time to check it out. Open the dashboard (localhost:8080).

faas_dash

A function can be invoked via the WebUI.

faas_hubstats

Or using the bash...

$ curl --data qnib http://localhost:8080/function/func_hubstats
The organisation or user qnib has 228 repositories on the Docker hub.
$

What I dislike here is that the containers are not trashed and rescheduled after use.

Fission (FaaS for Kubernetes)

I am not that much of a Kubernetes guy, but I have to admit that the fission project might convince me otherwise.

Install Minikube

Not unlike OpenStack, when I first encountered kubernetes it was hard to set it up locally. And on a normal host there were a lot of moving parts to consider. This times are gone, welcome minikube...

curl -sLO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl
chmod +x kubectl
mv kubectl /usr/local/bin
curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.17.1/minikube-darwin-amd64 
chmod +x minikube
mv minikube /usr/local/bin/

Deploy Fission

$ minikube start
Starting local Kubernetes cluster...
Starting VM...
Downloading Minikube ISO
 89.24 MB / 89.24 MB [==============================================] 100.00% 0s
SSH-ing files into VM...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.
$ kubectl create -f http://fission.io/fission.yaml
namespace "fission" created
namespace "fission-function" created
deployment "controller" created
deployment "router" created
service "poolmgr" created
deployment "poolmgr" created
deployment "kubewatcher" created
service "etcd" created
deployment "etcd" created
$ kubectl create -f http://fission.io/fission-nodeport.yaml
service "router" created
service "controller" created
$ export FISSION_URL=http://$(minikube ip):31313
$ export FISSION_ROUTER=$(minikube ip):31314
$ curl -s http://fission.io/mac/fission > fission
$ chmod +x fission
$ mv fission /usr/local/bin/

Create example

Let us go through the little example...

$ fission env create --name nodejs --image fission/node-env
$ cat << \EOF > hello.js
module.exports = async function(context) {
    return {
        status: 200,
        body: 'Your body here\n'
    };
}
EOF
$ fission function create --name hellojs --env nodejs --code hello.js
$ fission route create --method GET --url /hellojs --function hellojs
$ curl http://$FISSION_ROUTER/hellojs
Your body here
$

Spin up Kubernetes

$ git clone https://github.com/marselester/prometheus-on-kubernetes.git ~/src/github.com/marselester/prometheus-on-kubernetes
Cloning into '/Users/kniepbert/src/github.com/marselester/prometheus-on-kubernetes'...
remote: Counting objects: 56, done.
remote: Total 56 (delta 0), reused 0 (delta 0), pack-reused 56
Unpacking objects: 100% (56/56), done.
Checking connectivity... done.
$ cd ~/src/github.com/marselester/prometheus-on-kubernetes
$ kubectl create -f kube/prometheus/deployment-v1.yml
deployment "prometheus-deployment" created
$ kubectl expose deployment prometheus-deployment --type=NodePort --name=prometheus-service
service "prometheus-service" exposed
$ kubectl describe service prometheus-service
Name: prometheus-service
Namespace: default
Labels: app=prometheus-server
Selector: app=prometheus-server
Type:    NodePort
IP:      10.0.0.102
Port:     <unset> 9090/TCP
NodePort: <unset> 32241/TCP
Endpoints: 172.17.0.12:9090
Session Affinity: None
No events.
$ open http://$(minikube ip):32241

Conclusion

This new approaches starting to boggle my mind.

We had a nice discussion about what one might gain and my argument was:

Optimize Mixed Workloads

When a heterogenous workload is what your users are putting upon your shoulders (like AWS experiences), functions can help fill the unused gaps and provide more utilization.

Iteration Speed and Reproducibility

As the function is stateless, it can be updated on the spot, without the need for an actual update-path. And each execution is completely the same - without any side-effects.

So far... thanks everyone for attending - always a pleasure...

Comments