Sponsorslides

Integrating systems

in the age of

Quarkus, Knative and Kafka Connect

Peter Palaga

@ppalaga

Peter Palaga

Agenda

• Apache Camel intro
• Camel on Quarkus
• Camel on Knative/Kubernetes
• Camel on Kafka Connect

What is Apache Camel?

Since 2007

Huge community

• ~500 contributors
• ~3k stars on Github
• ~3.9k forks on Github
• ~1k subscribers on the Users mailing list
• ~9.3k questions on StackOverFlow

Follow @ApacheCamel (new) on Twitter!

An integration library

Enterprise Integration Patterns (EIP)

integrate anything

through its

~300 components

Domain Specific Languages (DSL)

class MyCamelRouter extends RouteBuilder {
  @Override
  public void configure() throws Exception {
    from("twitter-search:"
        + "word1,word2")
      .log("Got a tweet: ${body}")
      .to("telegram:bots");
  }
}

<routes
  xmlns="http://camel.apache.org/schema/spring">
  <route id="my-route">
    <from uri="twitter-search:word1,word2"/>
    <log message="Got a tweet: ${body}"/>
    <to uri="telegram:bots"/>
  </route>
</routes>

Groovy, Scala, Kotlin, ...

Runnable on

Standalone
Spring Boot
	Knative/Kubernetes

Why Camel on Quarkus?

Java slow to boot and memory intensive for
containers and serverless

What is

?

Supersonic, subatomic Java!

Camel Quarkus

• Incubated inside Quarkus
• Now https://github.com/apache/camel-quarkus
• ~50 components supported as of v1.0.0-M2
• Available via code.quarkus.io

Camel Quarkus Demo

Demo code:
github.com/ppalaga/camel-quarkus-cnorris-demo

Camel Quarkus links

• Project starter code.quarkus.io
• User guide
• Examples

Camel on Kubernetes and Knative

How fast can you (re-)deploy

on Kubernetes?

Camel K demo

Camel K internals

• kamel client side binary
  • github.com/apache/camel-k/releases
• Camel K Operator

What is Kubernetes Operator?

• An Application Management concept
• Does what a human operator would have to do
  • Based on declarative input
    
    
• A pattern
• (Ex-)CoreOS Operator Framework

How it works

A running container

• Monitors the cluster via Kubernetes API
• Reacts:
  • Add/remove/modify resources (pods/services/custom CRDs)
  • Scale up/down
  • Call endpoints of the running applications
• Implements custom logic

Custom Resource Definitions (CRD)

Camel K Operator

Image resolution (1/2)

Collect Maven dependencies based on

• DSL used (Groovy, XML, Java, Kotlin, JavaScript, YAML)
• Camel components used in the routes

Image resolution (2/2)

• Based on the found dependency set
  • Either take an available Image (buildless deploy)
  • Or build a new image


• Route definition not a part of the image
  • Mounted to the container as a ConfigMap

Camel on Knative

What is Knative?

• Building blocks for Serverless applications
• CRDs and operators

https://knative.dev

Knative Serving (1/2)

Knative Serving (2/2)

Knative Serving (2/2)

Knative Serving (2/2)

Knative Eventing

Knative Event Consumer

Knative Event Consumer

Knative Event Consumer

Knative Event Consumer

Knative Event Consumer

Knative Event Source

apiVersion: sources.eventing.knative.dev/v1alpha1
kind: CamelSource
metadata:
  name: bot-source
spec:
  source:
    flow:
      from:
        uri: telegram:bots
        parameters:
          authorizationToken: xxx
  sink:
    apiVersion: messaging.knative.dev/v1alpha1
    kind: InMemoryChannel
    name: mychannel

Roles in Knative Eventing

Camel Kafka Connector

What is

Apache Kafka?

• A stream-processing platform
• Distributed, scalable, fault-tolerant, fast

Several APIs

• Producer API - publish records to topics
• Consumer API - subscribe to topics to consume records
• Streams API - act as a stream processor/transformer
• Connector API - for streaming data between Apache Kafka and other systems

Connector API

a.k.a. Kafka Connect

• A framework to write Connectors
• A runtime to run them

Camel Kafka Connector

• Sink
• Source

to exchange data with 300+ Camel components

with minimal config

Camel Kafka Connector

Demo

Wrap up

• Apache Camel - an integration lib with 300+ connectors
• Supersonic subatomic on Quarkus
• Producer/Consumer/Router, scale to zero on Knative
• Streaming data from/to Kafka using Camel Kafka Connect

• https://camel.apache.org
• Chat: https://gitter.im/apache/camel
• Mailing list: dev@camel.apache.org
• Follow @ApacheCamel