Lead Data Engineer

We are looking for a highly skilled Senior Data Engineer with strong expertise in Google Cloud Platform (GCP), Apache Beam/Dataflow, Java, and BigQuery. The ideal candidate will be responsible for designing, building, and optimizing scalable data pipelines and cloud-based data processing solutions for enterprise applications.

Key Responsibilities

• Design and develop scalable ETL/ELT pipelines using GCP services.
  
• Build and maintain real-time and batch data processing pipelines using Dataflow and Apache Beam.
  
• Develop robust backend processing solutions using Java.
  
• Work extensively with BigQuery for data warehousing, analytics, and performance optimization.
  
• Integrate data from multiple sources including APIs, databases, and streaming platforms.
  
• Optimize pipeline performance, cost, scalability, and reliability on GCP.
  
• Collaborate with cross-functional teams including Architects, Analysts, DevOps, and Business stakeholders.
  
• Ensure data quality, governance, monitoring, and security best practices.
  
• Troubleshoot production issues and provide long-term scalable solutions.
  

Requirements

• Strong hands-on experience with GCP services.
  
• Expertise in Google Dataflow / Apache Beam.
  
• Strong programming skills in Java.
  
• Hands-on experience with BigQuery.
  
• Experience in building batch and streaming data pipelines.
  
• Good knowledge of SQL and data modeling concepts.
  
• Experience with CI/CD, Git, and Agile methodologies.
  
• Strong problem-solving and debugging skills.
  

Good to Have

• Experience with Pub/Sub, Cloud Composer, Cloud Storage, Dataproc, or Kafka.
  
• Exposure to Python or Spark.
  
• Experience in handling large-scale enterprise data platforms.
  

Preferred Qualifications

• Bachelor’s/Master’s degree in Computer Science, Engineering, or related field.
  
• GCP certifications are an added advantage.