Introduction to Apache Pulsar

Jowanza Joseph

@jowanza

Agenda

Pulsar Internals
Pulsar Usages
The Future
Questions

Problems

Data Storage
Data Transmission
Multi-casting
Delivery Guarantees
Ecosystem

Immutable Log

Tiered Storage

{"namespace": "com.bigdatums.avro",
 "type": "record",
 "name": "BdPerson",
 "fields": [
     {"name": "id", "type": "int"},
     {"name": "username",  "type": "string"},
 ]
}

Schema Evolution

{"namespace": "com.bigdatums.avro",
 "type": "record",
 "name": "BdPerson",
 "fields": [
     {"name": "id", "type": "int"},
     {"name": "username",  "type": ["string", "null"], "default": "null"}
}

Multi-Casting

Delivery Guarantees

No Guarantees

At Most Once

At Least Once

Exactly Once

Stream Processing

Pulsar SQL

Pulsar Functions

package main
	
	import (
		"context"
		"fmt"
		"log"
	
		"github.com/apache/pulsar/pulsar-client-go/pulsar"
	)
	
	func main() {
		client, err := pulsar.NewClient(pulsar.ClientOptions{
			URL: "pulsar://localhost:6650",
		})
	
		if err != nil {
			log.Fatal(err)
			return
		}
	
		defer client.Close()
	
		producer, err := client.CreateProducer(pulsar.ProducerOptions{
			Topic: "my-topic",
		})
	
		defer producer.Close()
	
		ctx := context.Background()
	
		for i := 0; i < 10; i++ {
			if err := producer.Send(ctx, pulsar.ProducerMessage{
				Payload: []byte(fmt.Sprintf("hello-%d", i)),
			}); err != nil {
				log.Fatal(err)
			}
		}
	}

Pulsar IO

Support for More Connectors

Function Mesh

Try Pulsar

Read the Pulsar Docs
Try Pulsar Locally
Free account from StreamNative
Free account from DataStax

Questions?

@jowanza

me@jowanza.com