Senior Streaming Data Engineer

CyprusRelocationHybrid

Cyprus, Ypsonas

As a Senior Streaming Data Engineer, you will be responsible for building, operating, and scaling our on-premise data platform running on Linux virtual machines. You will also enable streaming data ingestion from multiple sources and ensure reliable, production-grade data availability for analytics and machine learning use cases.

Responsibilities

Build and operate streaming ingestion pipelines from multiple sources into on-premise systems and databases hosted on Linux VMs
Design, implement, and optimize end-to-end ETL/ELT pipelines
Implement and maintain Kafka-based ingestion patterns, including topics, partitions, consumer groups, and reliability strategies
Use Redis where appropriate for caching, buffering, and real-time processing patterns
Maintain and optimize analytical storage solutions (e.g., ClickHouse or similar) used for BI and analytics workloads
Ensure reliable data ingestion from relational databases (PostgreSQL, MS SQL)
Implement monitoring, logging, alerting, and operational runbooks for pipelines and data systems
Ensure secure access and auditability across services and databases
Collaborate with BI, Analytics, and Data Science teams to deliver reliable datasets

Requirements

Strong Linux administration skills, including troubleshooting and production operations
Strong Python skills for building production-grade data services, automation, and testing
Strong SQL skills (PostgreSQL and/or MS SQL), including performance tuning
Hands-on experience building and operating ETL/ELT pipelines in production environments
Hands-on experience with Kafka and streaming data systems in production environments
Good understanding of core data engineering concepts such as data quality, schema evolution, partitioning, and idempotency
Experience working with analytical databases such as ClickHouse or similar technologies

Nice to have

Experience with CDC tooling or patterns (e.g., Debezium or equivalent approaches)
Familiarity with observability tools such as Prometheus, Grafana, ELK, or OpenSearch
Experience working in security-constrained production environments

Interview steps