Data Engineer with Pyspark

As a Data Engineer with PySpark, you will:

• Design, develop, and maintain scalable and reliable data processing solutions using PySpark.
  
• Build and manage robust batch and streaming data pipelines.
  
• Develop efficient data transformation and processing solutions using Python, PySpark, and SQL.
  
• Design, develop, and optimize data models to support scalable and high-performance data processing.
  
• Work with large datasets and complex data processing workloads.
  
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.
  
• Implement data quality, validation, and error-handling mechanisms across data pipelines.
  
• Collaborate with engineering, architecture, and business teams to deliver reliable data solutions.
  
• Contribute to Agile development practices and continuously improve data engineering standards and processes.
  

What You Bring to the Table:

• 5+ years of professional experience in Data Engineering or a related field.
  
• Strong hands-on experience with PySpark as a core data processing technology.
  
• Strong proficiency in Python, PySpark, and SQL.
  
• Hands-on experience developing and maintaining batch and streaming data pipelines.
  
• Experience working with large-scale data processing and transformation.
  
• Good understanding of data modeling, data integration, and data optimization.
  
• Experience with cloud-based data platforms and modern data engineering architectures.
  
• Experience with data quality, validation, troubleshooting, and performance optimization.
  
• Familiarity with CI/CD and modern software engineering practices.
  
• Experience working in Agile development and delivery environments.
  
• Strong communication, collaboration, analytical, and problem-solving skills.
  

You Should Possess the Ability to:

• Develop scalable and high-performance data pipelines using PySpark.
  
• Build efficient data transformations using Python, PySpark, and SQL.
  
• Process and manage large volumes of structured and unstructured data.
  
• Optimize Spark jobs, data processing performance, and resource utilization.
  
• Troubleshoot complex data pipeline and processing issues.
  
• Design reliable and maintainable data engineering solutions.
  
• Implement data quality and validation processes.
  
• Work effectively with technical and business stakeholders.
  
• Apply best practices for code quality, scalability, maintainability, and reliability.
  
• Continuously improve data engineering processes and solutions.
  

What We Bring to the Table:

• Opportunity to work on enterprise-scale data engineering initiatives in Spain.
  
• Exposure to modern data platforms, PySpark, cloud technologies, and large-scale data processing.
  
• A collaborative Agile environment focused on technical excellence and innovation.
  
• Opportunities to work on complex data pipelines and data transformation solutions.
  
• Continuous learning and opportunities for technical and professional growth.
  
• A culture focused on quality, ownership, scalability, and sustainable data engineering solutions.
  

Let’s Connect:

Want to discuss this opportunity in more detail? Feel free to reach out.

Recruiter: Hema Murali
Phone: +31 20 369 0609 ; Extn :148
Email: hema.m@stafide.nl
LinkedIn: https://www.linkedin.com/in/hema-murali-315999329/