Kafka Developer - #1711379

Job Role: Kafka Developer with Confluent Migration

Location: Sheffield, UK (Hybrid – 2 days onsite every week)

Employment Type: Contract B2B/Free Lance (Inside IR35)

Duration: Initially 6 Months (Extendable)

Roles & Responsibilities:

Summary:

We are seeking a highly experienced and skilled Senior Kafka Engineer to join our team. The ideal candidate will have a proven track record of designing, implementing, and maintaining large-scale Kafka deployments, with specific experience upgrading Kafka clusters. This role requires a deep understanding of Kafka's architecture, internals, and best practices. The candidate must be comfortable working independently and collaboratively within a fast-paced environment.

Responsibilities:

• Design, implement, and maintain high-throughput, low-latency Kafka clusters.


• Lead the upgrade of existing Kafka clusters from version 3.3.1 to the latest supported version, ensuring minimal downtime and data integrity.


• Experience in confluent migration .


• Develop and implement monitoring and alerting systems for Kafka.


• Troubleshoot and resolve Kafka-related issues.


• Participate in on-call rotation to ensure 24/7 system availability.


• Collaborate with other engineers to integrate Kafka into existing systems.


• Design and implement Kafka security best practices.


• Contribute to the improvement of our Kafka infrastructure.


• Mentor junior engineers and share expertise.


• Stay current with the latest Kafka technologies and best practices.

Qualifications:

• 8+ years of experience working with Apache Kafka.


• Proven experience upgrading Kafka clusters, specifically from version 3.3.1 to a current supported version. Detail of the specific versions involved is highly desirable.


• Strong understanding of Kafka architecture, including topics, partitions, consumers, and producers.


• Experience with Kafka Connect.
  


• Experience with Kafka Zookeeper, Kraft.
  


• Experience working in Confluent Kafka.
  


• Experience in deploying the Kafka image in Kubernetes.
  


• Experience in setting up Stretched Kafka Cluster.
  


• Experience with Kafka Streams or ksqlDB a plus.
  


• Proficiency in at least one scripting language (e.g., Python, Shell).


• Experience with cloud-based Kafka deployments (AWS, Azure, GCP) is highly preferred.


• Experience with Kafka monitoring and alerting tools.
  


• Excellent problem-solving and troubleshooting skills.


• Strong communication and collaboration skills.