This project creates an end-to-end event pipeline that reads event from a webhook, generates a formatted string, and publishes the result to Slack. While this is a simeple example, it has many event notification use cases, such as:
- submission frpm website forms (via Cloudflare workers or your custom backend).
- activity from ecommerce platforms on purchases and shopping carts.
- notifications from github an your projects' activities.
- alerts from financial products on your transactions.
- notifications form any product that can invoke a webhook.
This pipeline uses the following features:
- webhook: that creates a public API to receive external events, transform and publish them to a topic.
- http-sink: that listens the same topic, and publish them on Slack.
Show an example on how you build an event streaming pipeline that receies webhook evnts, transforms the input into a readable form, and generates an alert. We assume the events are generated by a user submitting a form, and we'll format accordingly.
- Fluvio CLI running locally
- Account on InfinyOn Cloud
- Create webhook configuration file
- Create http-sink configuration file
- Download SmartModules
- Start Webhook and Connector
- Test Data Pipeline
Create a webhook configuration file called form-webhook.yaml
:
meta:
name: form-webhook
topic: form-events
webhook:
outputParts: body
outputType: json
transforms:
- uses: infinyon-labs/[email protected]
with:
spec:
match:
- key: "/type"
value: "subscribe"
format:
with: ":loudspeaker: {} ({}) subscribed on {}"
using:
- "/name"
- "/email"
- "/source"
output: "/formatted"
- key: "/type"
value: "use-case"
format:
with: ":confetti_ball: {} ({}) wants to solve the following '{}' use-case:\n>{}"
using:
- "/name"
- "/email"
- "/source"
- "/description"
output: "/formatted"
default:
format:
with: "{} ({}) submitted a request"
using:
- "/name"
- "/email"
output: "/formatted"
The webhook reads the JSON body, applies the json-formatter
smartmodule to generate readable text, and writes the new record to a topic called form-events
. Checkout labs-json-formatter-sm in github for additional information.
Create an HTTP source connector configuration file called slack-form-alerts.yaml
:
apiVersion: 0.1.0
meta:
version: 0.2.8
name: slack-form-alerts
type: http-sink
topic: form-events
secrets:
- name: SLACK_USER_ALERTS
http:
endpoint: "https://hooks.slack.com/services/${{ secrets.SLACK_USER_ALERTS }}"
headers:
- "Content-Type: application/json"
transforms:
- uses: infinyon/[email protected]
with:
spec:
- operation: shift
spec:
"formatted": "text"
The sink connector reads from the form-events
topic and uses the jolt
smartmodule to shift the formatted string into a field called text
per the Slack instructions. Checkout fluvio-jolt in github for additional information.
The Slack webhook link is sensitive information, let's add the access token part to secret
in InfinyOn Cloud :
fluvio cloud secret set SLACK_USER_ALERTS <webhook-token>
Check out Slack Webhooks on how to create the webhook token.
Download the smartmodules used by the webhook ad the connector:
fluvio hub sm download infinyon/[email protected]
fluvio hub sm download infinyon-labs/[email protected]
Check fluvio smartmodule list
to ensure they've been downloaded.
Start webhook listener:
fluvio cloud webhook create --config form-webhook.yaml
Check fluvio cloud webhook list
to ensure it has been successfully provisioned. In checkout the webhook link that we'll use to test the pipeline: https://infinyon.cloud/webhooks/v1/[token]
Start sink connector:
fluvio cloud connector create -config slack-form-alerts.yaml
Check fluvio cloud connector list
to ensure it has been successfully provisioned.
Use curl
to send a POST request with a fictious user request to our webhook link. In production environments, this iw what a website would send:
curl -X POST https://infinyon.cloud/webhooks/v1/<token> \
-H "Content-Type: application/json" \
-d '{ "email": "[email protected]", "name": "Alice Liddell", "type": "subscribe", "source": "front-page" }'
The following alert is displayed in Slack:
`:loudspeaker: Alice Liddell ("[email protected]) subscribed on front-page` will show-up in your slack channel.
That's all folks!