Sponsorslides

Apache Camel K

Integrating systems

in the age of

Quarkus, Knative and Kafka Connect

 

Peter Palaga

@ppalaga

Peter Palaga

Agenda

  • Apache Camel intro
  • Camel on Quarkus
  • Camel on Knative/Kubernetes
  • Camel on Kafka Connect

What is Apache Camel?

http://camel.apache.org/

Since 2007

Huge community


Follow @ApacheCamel (new) on Twitter!

Apache Camel is...

An integration library



Systems integration
Apache Camel is based on

Enterprise Integration Patterns (EIP)

EIPs
https://www.enterpriseintegrationpatterns.com/
Apache Camel can...

integrate anything

through its

~300 components

Filetypes:

  • Plaintext, XML, HTML, CSV, JSON, ZIP, ...
  • Apache Tika (MS Office, OpenDocument, ...)

Protocols:

  • AMQP, DNS, FTP, HTTP, IRC, SSH, TCP/UDP, ...
  • REST, GRPC, git, ...
  • IoT: CoAP, MQTT, PubNub

Public Clouds:   

  • AWS: S3, SQS, Kinesis, ...
  • Azure: Blob, Queue, ...
  • Google: BigQuery, PubSub

Enterprise:

  • Caches: Hazelcast, Infinispan, Redis, ...
  • Cassandra, MongoDB, Kafka
  • JDBC, SQL, JPA, CDI, EJB, JMS, JMX, OpenTracing, OptaPlanner, Syslog,

APIs:

  • Social: Facebook, Twitter, LinkedIn
  • Box, Dropbox, Google Calendar/Drive/Mail/Docs
  • Salesforce, SAP, ServiceNow, FHIR
  • GitHub, Jira, Telegram

Misc:

  • Barcode
  • Base64, JsonPath, Freemarker, Groovy, Mustache, Ruby
  • Docker, etcd, Kubernetes, OpenShift,
Routes defined via

Domain Specific Languages (DSL)

Java XML
class MyCamelRouter extends RouteBuilder {
  @Override
  public void configure() throws Exception {
    from("twitter-search:"
        + "word1,word2")
      .log("Got a tweet: ${body}")
      .to("telegram:bots");
  }
}
<routes
  xmlns="http://camel.apache.org/schema/spring">
  <route id="my-route">
    <from uri="twitter-search:word1,word2"/>
    <log message="Got a tweet: ${body}"/>
    <to uri="telegram:bots"/>
  </route>
</routes>

Groovy, Scala, Kotlin, ...

Apache Camel is...

Runnable on


Standalone
Spring Boot Spring Boot Apache Karaf WildFly
Quarkus Apache Karaf Knative/Kubernetes WildFly

Why Camel on Quarkus?

 

Java slow to boot and memory intensive for
containers and serverless

What is

?

Supersonic, subatomic Java!

An intro

Camel Quarkus

Camel Quarkus Demo



Demo code:
github.com/ppalaga/camel-quarkus-cnorris-demo

Camel Quarkus links


Camel on Kubernetes and Knative

How fast can you (re-)deploy

on Kubernetes?

Web UI for "Citizen integrators"

Fast feedback required

Camel K demo

Camel K internals

What is Kubernetes Operator?

  • An Application Management concept
  • Does what a human operator would have to do
    • Based on declarative input

  • A pattern
  • (Ex-)CoreOS Operator Framework
A Kubernetes Operator

How it works

A running container

  • Monitors the cluster via Kubernetes API
  • Reacts:
    • Add/remove/modify resources (pods/services/custom CRDs)
    • Scale up/down
    • Call endpoints of the running applications
  • Implements custom logic
Kubernetes

Custom Resource Definitions (CRD)

Built-in resources
  • pods, services, ...
  • oc create, oc get, oc delete
CRDs
  • Extend the cluster model
  • oc create, oc get, oc delete
  • Declarative input for operators

Camel K Operator

Camel K internals
Apache Camel K

Image resolution (1/2)

Collect Maven dependencies based on

  • DSL used (Groovy, XML, Java, Kotlin, JavaScript, YAML)
  • Camel components used in the routes
Camel K

Image resolution (2/2)

  • Based on the found dependency set
    • Either take an available Image (buildless deploy)
    • Or build a new image

  • Route definition not a part of the image
    • Mounted to the container as a ConfigMap

Camel on Knative

What is Knative?

  • Building blocks for Serverless applications
  • CRDs and operators

https://knative.dev

Camel

Knative Serving (1/2)

Camel

Knative Serving (2/2)

Camel

Knative Serving (2/2)

Camel

Knative Serving (2/2)

Knative Eventing

https://cloudevents.io
Camel as

Knative Event Consumer

Camel as

Knative Event Consumer

Camel as

Knative Event Consumer

Camel as

Knative Event Consumer

Camel as

Knative Event Consumer

Camel as

Knative Event Source

New: a pure kubectl / oc variant

apiVersion: sources.eventing.knative.dev/v1alpha1
kind: CamelSource
metadata:
  name: bot-source
spec:
  source:
    flow:
      from:
        uri: telegram:bots
        parameters:
          authorizationToken: xxx
  sink:
    apiVersion: messaging.knative.dev/v1alpha1
    kind: InMemoryChannel
    name: mychannel
Camel

Roles in Knative Eventing

Camel Kafka Connector

What is

Apache Kafka?

  • A stream-processing platform
  • Distributed, scalable, fault-tolerant, fast
Apache Kafka offers

Several APIs

  • Producer API - publish records to topics
  • Consumer API - subscribe to topics to consume records
  • Streams API - act as a stream processor/transformer
  • Connector API - for streaming data between Apache Kafka and other systems
Apache Kafka

Connector API

a.k.a. Kafka Connect

  • A framework to write Connectors
  • A runtime to run them

Camel Kafka Connector

 

  • Sink
  • Source

to exchange data with 300+ Camel components

with minimal config

Camel Kafka Connector

Demo

Wrap up

  • Apache Camel - an integration lib with 300+ connectors
  • Supersonic subatomic on Quarkus
  • Producer/Consumer/Router, scale to zero on Knative
  • Streaming data from/to Kafka using Camel Kafka Connect


Feedback and contributions are welcome!