Nakadi @ Zalando
team RAY
Why
async microservices communication
event-driven
REST oriented
reliability
responsibility
Centralised event bus for Zalando
all events are transported via Nakadi
tooling and analysis on top
What
secured RESTful API
underlying infrastructure is proven and well-known
event type registry
compatibility with STUPS
Everything is REST
Underlying infrastructure
Underlying infrastructure
Underlying infrastructure
Terminology
event
event type - similar to topic in MQ, channel for communication
producer - one who is publishing an event to an event type
consumer - one who is listening on an event type and gets notified when an event occurs
offset - point from which to start consuming events
partition - not needed at the moment!
Challenge
As consumers we want to be notified when an event occurs (in real-time).
Challenge
Two solutions
Low-level API
High-level API
Low-level
A consumer sends an HTTP request on a specific event type
Nakadi does not respond...
...but instead sends an event as it occurs
HTTP connection "never" closes
Subscription API (High-Level)
A consumer subscribes to a specific event type...
...providing a callback API
An event occurs and is POSTed on the callback API
Current status
Low-Level API - implemented (and later demoed)
Subscription API - under construction
Event filtering - yet to come
How to use nakadi
Prerequisites
Create a new event type
Connect to consume an event on newly created type
Produce an event on newly created type
Offset in practice
Nakadi inspector
Backend libraries
Prerequisites
Your favorite HTTP crawler (curl, http, postman, etc.)
Create a new event type
curl -v -X POST https://nakadi-staging.aruha-test.zalan.do/event-types \
--header "Authorization: Bearer your-zign-token-here" \
--header "Content-Type:application/json" \
-d '{
"name": "brand-solutions.test",
"owning_application": "bs.test",
"category": "undefined",
"partition_strategy": "random",
"schema": {
"type": "json_schema",
"schema": "{ \"additionalProperties\": true }"
}
}'
Check if the event type was created
curl -v -X GET https://nakadi-staging.aruha-test.zalan.do/event-types/brand-solutions.test \
--header "Authorization: Bearer your-zign-token-here" \
--header "Content-Type:application/json"
Connect to consume an event on newly created type
curl -v -X GET https://nakadi-staging.aruha-test.zalan.do/event-types/brand-solutions.test/events \
--header "Authorization: Bearer your-zign-token-here" \
--header "Content-Type:application/json" \
--header 'X-Nakadi-Cursors:[{"partition": "0", "offset":"BEGIN"}]'
Produce an event on newly created type
curl -v -X POST https://nakadi-staging.aruha-test.zalan.do/event-types/brand-solutions.test/events \
--header "Authorization: Bearer your-zign-token-here" \
--header "Content-Type:application/json" \
-d '[{"foo": "bar"}]'
Offset in practice
curl -v -X GET https://nakadi-staging.aruha-test.zalan.do/event-types/brand-solutions.test/events \
--header "Authorization: Bearer your-zign-token-here" \
--header "Content-Type:application/json" \
--header 'X-Nakadi-Cursors:[{"partition": "0", "offset":"10"}]'
Nakadi inspector
Nikita did 5-minute-prototype-jQuery project
Inner-source: Nakadi inspector
DEMO
Backend libraries