Data Engineer (ClickHouse)

Project Overview: 
High-load in-house platform handling multiple third-party integrations, data collection, and comprehensive reporting systems. The system processes large volumes of real-time data and requires high availability with performance optimization.

Stack:

AWS, S3, Lambda, Kafka/MSK, Debezium/CilckPipes, MySQL (RDS), ClickHouse, Tablue, Terraform/CDK, Git, NodeJs/Python, SQL

📌 Key Responsibilities: 

• Design and maintain scalable ETL/ELT pipelines

• Handle CDC ingestion from Amazon RDS (MySQL) via Debezium + Kafka

• Load and optimize high-throughput data flows into ClickHouse

• Tune ingestion in ClickHouse: partitioning, TTLs, ORDER BY keys

• Use Parquet/ORC and partitioning for efficient storage in S3

• Integrate datasets into BI

• Implement monitoring and alerting for data pipelines

• Use Git + CI/CD and manage infrastructure with Terraform or AWS CDK

📌 Requirements:

• Proven experience building high-throughput data ingestion pipelines

• Proficiency with Kafka/MSK and Debezium or CilckPipes

• Experience with ClickHouse and Warehouse concepts and architecture

• SQL, Git, CI/CD

• Infrastructure-as-Code: Terraform

📌 Nice to have:

• Design and maintain scalable ETL/ELT pipelines using AWS Glue, Airflow, and S3

• Experience with dbt

• Familiarity with MySQL binlog-based CDC

• BI reporting knowledge

• Experience monitoring and tuning ClickHouse, quries, NodeJS

📌 Benefits:

• 24 working days of paid annual leave

• 6 days of paid sick leave 

• Official employment

• Medical insurance

• Coffee zone with fruit & snacks available in the office

• Corporate Lunch provided by the company

• Gym and sports classes

• Healthy and friendly work atmosphere