Introduction to Apache Pulsar
Jowanza Joseph
@jowanza
Agenda
- Pulsar Internals
- Pulsar Usages
- The Future
- Questions
Problems
- Data Storage
- Data Transmission
- Multi-casting
- Delivery Guarantees
- Ecosystem
Immutable Log
Tiered Storage
{"namespace": "com.bigdatums.avro",
"type": "record",
"name": "BdPerson",
"fields": [
{"name": "id", "type": "int"},
{"name": "username", "type": "string"},
]
}Schema Evolution
{"namespace": "com.bigdatums.avro",
"type": "record",
"name": "BdPerson",
"fields": [
{"name": "id", "type": "int"},
{"name": "username", "type": ["string", "null"], "default": "null"}
}Multi-Casting
Delivery Guarantees
No Guarantees
At Most Once
At Least Once
Exactly Once
Stream Processing
Pulsar SQL
Pulsar Functions
package main
import (
"context"
"fmt"
"log"
"github.com/apache/pulsar/pulsar-client-go/pulsar"
)
func main() {
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
if err != nil {
log.Fatal(err)
return
}
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "my-topic",
})
defer producer.Close()
ctx := context.Background()
for i := 0; i < 10; i++ {
if err := producer.Send(ctx, pulsar.ProducerMessage{
Payload: []byte(fmt.Sprintf("hello-%d", i)),
}); err != nil {
log.Fatal(err)
}
}
}Pulsar IO
Support for More Connectors
Function Mesh
Try Pulsar
- Read the Pulsar Docs
- Try Pulsar Locally
- Free account from StreamNative
- Free account from DataStax
Questions?
@jowanza
me@jowanza.com
Intro to Apache Pulsar
By Jowanza Joseph
