Contact Info
London Office :
Denizli Office :
Pamukkale Üniversitesi Teknoloji Geliştirme Merkezi B Blok Oda:Z17, 20070 Denizli
info@cloudnesil.com
+88 (0) 101 0000 000
Certified Kubernetes Administrator Certified Developer on Apache Cassandra

CloudNesil

kafka   

Kafka Encryption with SSL

Apache Kafka is a community distributed event streaming platform capable of handling events. This Article is about how you can secure Kafka communication both inter-broker and external. You can find plenty of information in official Confluent web site but through this article you will find useful commands how can you deploy Kafka with SSL encryption. Last but not least I will tell you how to deploy Kafka on top Kubernetes with helm chart of Kafka from canonical incubator helm chart repository.

There are plenty of information and explanation on Confluent web site I will just skip them and get going:

 1. Create Root CA

PASSWORD=cloudnesil
VALIDITY=365CA_CONFIG=$(echo -n '[dn]\n
CN=CloudNesil\n
[req]\n
distinguished_name=dn\n
[EXT]\n
keyUsage=digitalSignature,keyCertSign\n
basicConstraints=CA:TRUE\n
extendedKeyUsage=serverAuth'| tr -d '\n')openssl req -new -x509 -keyout ca-key -out root-ca -days $VALIDITY -subj '/CN=CloudNesil' -extensions EXT -config <(printf $CA_CONFIG) -passout pass:$PASSWORD
With the command above, you should have two files created: “root-ca” and “ca-key.

2. Create Kafka Server Keystore

SERVER_CONFIG=$(echo -n '[dn]\n
CN=Kafka-brokers\n
[req]\n
distinguished_name=dn\n
[EXT]\n
keyUsage=digitalSignature,keyCertSign\n
extendedKeyUsage=serverAuth\n
subjectAltName=@alt_names\n
[alt_names]\n
DNS.0=kafka.cloudnesil.com\n
DNS.1=kafka-0\n
DNS.2=kafka-1\n
DNS.3=kafka-2\n
DNS.4=kafka-0.kafka-headless\n
DNS.5=kafka-1.kafka-headless\n
DNS.6=kafka-2.kafka-headless'| tr -d '\n')keytool -keystore server.keystore.jks -alias kubernetes -validity $VALIDITY -genkey -dname "CN=CloudNesil" -storepass $PASSWORDkeytool -keystore server.keystore.jks -alias kubernetes -certreq -file server-cert-request -storepass $PASSWORD
You should have two files created: “server.keystore.jks” and “server-cert-request”.

3. Create Server Certifate and Sign

openssl x509 -req -CA root-ca -CAkey ca-key -in server-cert-request -out server-cert-request-signed -days $VALIDITY -CAcreateserial -passin pass:$PASSWORD -extensions EXT -extfile <(printf $SERVER_CONFIG)
You should have two files created: “server-cert-request-signed” and “root-ca.srl”.

4. Import RootCA and Server Certificate to keystore

keytool -keystore server.keystore.jks -alias RootCA -import -noprompt -file root-ca -storepass $PASSWORD
keytool -keystore server.keystore.jks -alias kubernetes -import -noprompt -file server-cert-request-signed -storepass $PASSWORD
You can verify your server.keystore.jks with: 
keytool -list -v -keystore server.keystore.jks -storepass $PASSWORD
You should see something like this: 
Keystore type: jks
Keystore provider: SUNYour keystore contains 2 entriesAlias name: rootca
Creation date: May 10, 2020
Entry type: trustedCertEntryOwner: CN=CloudNesil
Issuer: CN=CloudNesil
Serial number: b023dd2e7023518d
Valid from: Sun May 10 00:49:31 EET 2020 until: Mon May 10 00:49:31 EET 2021
Certificate fingerprints:
  MD5:  7B:B0:62:20:42:73:5D:8A:8B:27:35:77:5A:0E:FE:E1
  SHA1: 49:74:55:0D:F8:22:84:A9:84:03:53:A0:C6:7F:9E:19:35:59:07:5E
  SHA256: 8F:11:BB:4F:D6:C9:F6:FC:2E:47:77:CB:AE:07:96:65:0F:2D:1C:45:EB:DA:ED:C6:B2:8E:C5:FE:92:C6:7A:19
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3Extensions:#1: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]#2: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
]#3: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
  DigitalSignature
  Key_CertSign
]*******************************************
*******************************************Alias name: kubernetes
Creation date: May 10, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=CloudNesil
Issuer: CN=CloudNesil
Serial number: ed6c79362f53647b
Valid from: Sun May 10 01:16:33 EET 2020 until: Mon May 10 01:16:33 EET 2021
Certificate fingerprints:
  MD5:  3C:F8:6A:11:71:A6:FF:BD:67:1B:21:6A:20:23:E2:6A
  SHA1: 59:3F:42:BA:68:61:DB:F9:EB:A5:57:44:F5:96:92:5D:AF:6F:65:51
  SHA256: 5C:A8:D5:1E:3A:04:9D:80:4F:0A:88:F3:41:50:6C:12:24:2A:50:17:7A:D7:0B:29:86:9A:0F:A7:A5:B9:5F:DB
Signature algorithm name: SHA1withRSA
Subject Public Key Algorithm: 2048-bit DSA key
Version: 3Extensions:#1: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
]#2: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
  DigitalSignature
  Key_CertSign
]#3: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: kafka.cloudnesil.com
  DNSName: kafka-0
  DNSName: kafka-1
  DNSName: kafka-2
  DNSName: kafka-0.kafka-headless
  DNSName: kafka-1.kafka-headless
  DNSName: kafka-2.kafka-headless
]Certificate[2]:
Owner: CN=CloudNesil
Issuer: CN=CloudNesil
Serial number: b023dd2e7023518d
Valid from: Sun May 10 00:49:31 EET 2020 until: Mon May 10 00:49:31 EET 2021
Certificate fingerprints:
  MD5:  7B:B0:62:20:42:73:5D:8A:8B:27:35:77:5A:0E:FE:E1
  SHA1: 49:74:55:0D:F8:22:84:A9:84:03:53:A0:C6:7F:9E:19:35:59:07:5E
  SHA256: 8F:11:BB:4F:D6:C9:F6:FC:2E:47:77:CB:AE:07:96:65:0F:2D:1C:45:EB:DA:ED:C6:B2:8E:C5:FE:92:C6:7A:19
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3Extensions:#1: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]#2: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
]#3: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
  DigitalSignature
  Key_CertSign
]

5. Create Server Trust Store and Client Trust Store

keytool -keystore server.truststore.jks -alias RootCA -import -noprompt -file root-ca -storepass $PASSWORD
keytool -keystore client.truststore.jks -alias RootCA -import -noprompt -file root-ca -storepass $PASSWORD
You shoould have a file created: “server.truststore.jks” and “client.truststore.jks”. You can verify your client.truststore.jks with: 
keytool -list -v -keystore client.truststore.jks -storepass $PASSWORD
You should see something like this: 
Keystore type: jks
Keystore provider: SUNYour keystore contains 1 entryAlias name: rootca
Creation date: May 10, 2020
Entry type: trustedCertEntryOwner: CN=CloudNesil
Issuer: CN=CloudNesil
Serial number: b023dd2e7023518d
Valid from: Sun May 10 00:49:31 EET 2020 until: Mon May 10 00:49:31 EET 2021
Certificate fingerprints:
  MD5:  7B:B0:62:20:42:73:5D:8A:8B:27:35:77:5A:0E:FE:E1
  SHA1: 49:74:55:0D:F8:22:84:A9:84:03:53:A0:C6:7F:9E:19:35:59:07:5E
  SHA256: 8F:11:BB:4F:D6:C9:F6:FC:2E:47:77:CB:AE:07:96:65:0F:2D:1C:45:EB:DA:ED:C6:B2:8E:C5:FE:92:C6:7A:19
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3Extensions:#1: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]#2: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
]#3: ObjectId: 2.5.29.15 Criticality=false
KeyUsage [
  DigitalSignature
  Key_CertSign
]

6. Create kubernetes secret

kubectl create secret generic -n kafka kafka-ssl-store --from-file=server.keystore.jks --from-file=server.truststore.jks

7. Deploy Kafka via helm

helm install  my-kafka --namespace kafka  incubator/kafka  --version 0.21.0 \
--set fullnameOverride=kafka \
--set configurationOverrides."advertised\.listeners"=SSL_EXTERNAL_CONNECTION://kafka.cloudnesil.com:"\$((31090 + \${KAFKA_BROKER_ID}))"\\,SSL_INTERNAL://kafka-"\$((\${KAFKA_BROKER_ID}))".kafka-headless:9093 \
--set configurationOverrides."inter\.broker\.listener\.name"=SSL_INTERNAL \
--set configurationOverrides."listener\.security\.protocol\.map"=SSL_EXTERNAL_CONNECTION:SSL\\,SSL_INTERNAL:SSL\\,PLAINTEXT:PLAINTEXT \
--set configurationOverrides."ssl\.keystore\.location"=/var/private/ssl/server.keystore.jks \
--set configurationOverrides."ssl\.keystore\.password"=cloudnesil \
--set configurationOverrides."ssl\.truststore\.location"=/var/private/ssl/server.truststore.jks \
--set configurationOverrides."ssl\.truststore\.password"=cloudnesil \
--set configurationOverrides."ssl\.key\.password"=cloudnesil \
--set external.enabled=true \
--set external.distinct=true \
--set zookeeper.fullnameOverride=kafka-zookeeper \
--set zookeeper.service.type=NodePort \
--set zookeeper.service.ports.client.nodePort=32022 \
--set secrets[0].name=kafka-ssl-store \
--set secrets[0].keys[0]=server.keystore.jks \
--set secrets[0].keys[1]=server.truststore.jks \
--set secrets[0].mountPath=/var/private/ssl
Now you can connect zookeeper from: kafka.cloudnesil.com:32022 and kafka from kafka.cloudnesil.com:31090 using “client.trust.store”, enjoy..!
Like
Share

Related Posts

Software
Software
Software

Post a Comment

Retype the CAPTCHA code from the image
Change the CAPTCHA codeSpeak the CAPTCHA code