Job Description

We are looking for a highly skilled Senior Data Engineer with 10+ years of experience in designing, developing, and maintaining scalable data platforms and enterprise-grade data pipelines. The ideal candidate should possess strong expertise in big data technologies, cloud platforms, distributed processing systems, and modern data engineering practices.

As a Senior Data Engineer, you will be responsible for building robust ETL/ELT frameworks, optimizing large-scale data workflows, enabling real-time and batch processing, and supporting analytics and AI-driven initiatives across the organization.

You will collaborate with Data Architects, Data Scientists, Analysts, Product Teams, and DevOps teams to deliver scalable, reliable, and secure data solutions.

Roles & Responsibilities

• Design and develop scalable ETL/ELT pipelines for structured and unstructured data
• Build and maintain enterprise data lakes, data warehouses, and lakehouse architectures
• Develop high-performance batch and streaming data processing systems
• Optimize Spark jobs, SQL queries, and cloud infrastructure performance
• Implement data quality, monitoring, observability, and governance solutions
• Integrate data from multiple internal and external sources
• Collaborate with cross-functional teams to understand business and analytics requirements
• Ensure platform scalability, reliability, security, and fault tolerance
• Lead technical discussions, code reviews, and mentoring of junior engineers
• Support CI/CD automation and DevOps best practices
• Troubleshoot production issues and perform root cause analysis
• Contribute to architectural decisions and technology evaluations

Required Skills

• Strong expertise in Python and SQL
• Hands-on experience with Apache Spark / PySpark
• Experience with Apache Kafka or streaming technologies
• Strong understanding of data modeling and database design
• Experience with ETL/ELT frameworks and orchestration tools
• Expertise in GCP, Snowflake, Airflow, Hadoop ecosystem
• Knowledge of distributed systems and scalable architectures

Preferred Skills

• Experience with Delta Lake, Apache Iceberg, or Hudi
• Exposure to real-time analytics and event-driven architecture
• Docker and Kubernetes experience
• Infrastructure as Code (Terraform)
• Experience supporting AI/ML or LLM data pipelines
• Knowledge of data governance and observability tool

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or related field