Extend Falco outputs with falcosidekick

(2021-04-13) edit: update to integrate Falcosidekick-UI use last versions of Falco helm chart which embeds Falcosidekick as dependency

By default, Falco has 5 outputs for its events: stdout, file, gRPC, shell and http. As you can see in the following diagram:

Even if they're convenient, we can quickly be limited to integrating Falco with other components. Here comes Falcosidekick, a little daemon that extends that number of possible outputs.

The current list of available Falcosidekick outputs (version v2.22.0) is:

• Slack
• Rocketchat
• Mattermost
• Teams
• Datadog
• Discord
• AlertManager
• Elasticsearch
• Loki
• NATS
• STAN (NATS Streaming)
• Influxdb
• AWS Lambda
• AWS SQS
• AWS SNS
• AWS CloudWatchLogs
• AWS S3
• SMTP (email)
• Opsgenie
• StatsD (for monitoring of falcosidekick)
• DogStatsD (for monitoring of falcosidekick)
• Webhook
• Azure Event Hubs
• Prometheus (for both events and monitoring of falcosidekick)
• GCP PubSub
• GCP Storage
• Google Chat
• Apache Kafka
• PagerDuty
• Kubeless
• OpenFaaS
• WebUI (a Web UI for displaying latest events in real time)

Beyond that, it provides metrics about the number of events and let you add custom fields in events, for example environment, region, etc

In this article, we'll see how to deploy together in a Kubernetes cluster Falco, Falcosidekick and Falcosidekick-UI.

We'll use Helm (version 3) for installing all components and for a better user experience, the official Falco chart is able to install and set all configurations for us:

For this tutorial, we'll send the events in a Slack channel, so get your webhook URL first.

Run the following Helm

kubectl create namespace falco
helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco \
--set falcosidekick.enabled=true \
--set falcosidekick.webui.enabled=true \
--set falcosidekick.config.slack.webhookurl="https://hooks.slack.com/services/XXXX" \
-n falco 

• --set falcosidekick.enabled=true enables deployment of Falcosidekick aside Falco and configures Falco for sending its events to Falcosidekick
• --set falcosidekick.webui.enabled=true enables deployment of Falcosidekick-UI and configure Falcosidekick for using it as output
• --set falcosidekick.config.slack.webhookurl="https://hooks.slack.com/services/XXXX" enables Slack as output for Falcosidekick

All possible values can be seen in the according Helm charts, see the repository

After few seconds you should get:

kubectl -n falco get pods

NAME                                      READY   STATUS    RESTARTS   AGE
falco-falcosidekick-ui-7bdc54fb4c-h99b6   1/1     Running   0          26s
falco-falcosidekick-7779579477-pfspz      1/1     Running   0          26s
falco-falcosidekick-7779579477-bn8pv      1/1     Running   0          26s
falco-6ksbx                               1/1     Running   0          26s
falco-pxvbk                               1/1     Running   0          26s
falco-5cg5b                               1/1     Running   0          26s

kubectl -n falco get svc

NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
falco-falcosidekick      ClusterIP   10.43.212.119   <none>        2801/TCP   61s
falco-falcosidekick-ui   ClusterIP   10.43.35.87     <none>        2802/TCP   60s

You can test the deployment of Falcosidekick with a typical port forward:

kubectl -n falco port-forward svc/falco-falcosidekick 2801

curl -s http://localhost:2801/ping

pong

It's alive !

We can send a test event to Slack to test whether it works or not. Falcosidekick provides a useful endpoint for that:

curl -sI -XPOST http://localhost:2801/test

HTTP/1.1 200 OK
Date: Tue, 13 Apr 2021 20:42:32 GMT
Content-Length: 0

In logs you'll get:

kubectl -n falco logs deployment/falcosidekick

kubectl logs deployment/falco-falcosidekick -n falco
Found 2 pods, using pod/falco-falcosidekick-7779579477-pfspz
2021/04/13 20:40:11 [INFO]  : Enabled Outputs : [Slack WebUI]
2021/04/13 20:40:11 [INFO]  : Falco Sidekick is up and listening on :2801
2021/04/13 20:41:35 [INFO]  : WebUI - Post OK (200)
2021/04/13 20:41:35 [INFO]  : WebUI - Publish OK
2021/04/13 20:41:35 [INFO]  : Slack - Post OK (200)
2021/04/13 20:41:35 [INFO]  : Slack - Publish OK

We can notice the fist line of logs [INFO] : Enabled Outputs : [Slack WebUI], we do have 2 enabled outputs, Slack and WebUI (Falcosidekick-UI)

And in your Slack channel:

Tip: For Slack and some other ouputs, the message format can be customized, more informations in README

We'll now add some custom fields and test a more realistic event.

Upgrade your deployment:

helm upgrade falco falcosecurity/falco \
--set falcosidekick.enabled=true \
--set falcosidekick.webui.enabled=true \
--set falcosidekick.config.slack.webhookurl="https://hooks.slack.com/services/XXXX" \
--set falcosidekick.config.customfields="environment:production\,datacenter:paris"
-n falco 

Send a more advanced test event to Falcosidekick (still with the port forward aside):

curl "http://localhost:2801/" -d'{"output":"A more realistic test event","priority":"Error","rule":"Fake rule","time":"2021-04-13T20:58:00.746609046Z+2", "output_fields": {"evt.time":1618347519000000,"fd.name":"/bin/hack","proc.cmdline":"touch /bin/hack","user.name":"root"}}'

Falco community also provides a Web UI for following live events an get statistics about last. (Tip: you can add filters by clicking on any label)

By default, you can access to it through a port forward too:

kubectl port-forward svc/falco-falcosidekick-ui -n falco 2802

You now have access in your browser with the URL: http://localhost:2802/ui

Get involved

If you would like to find out more about Falco:

• Get started in Falco.org.
• Check out the Falco project on GitHub.
• Get involved Falco community.
• Meet the maintainers on the Falco Slack.
• Follow @falco_org on Twitter.

And that's it!

Enjoy

• ←Previous
Next→