AWS Data Engineer

Experience: 3-8 Years

Location: Pune (Kharadi)

About the Role

We are looking for a highly skilled Spark SQL Developer with a solid background in Data Engineering. The candidate should have hands-on experience in building and optimizing large-scale data pipelines using Apache Spark, along with strong expertise in SQL and ETL processes.

Key Responsibilities

• Design, develop, and maintain scalable ETL data pipelines using Apache Spark and SQL.
• Write and optimize Spark SQL queries for large and complex datasets.
• Collaborate with data engineers, analysts, and business stakeholders to deliver efficient data solutions.
• Perform data validation, transformation, and performance tuning on big data platforms.
• Implement data ingestion workflows from multiple structured and unstructured sources.
• Ensure data quality, reliability, and consistency across environments.

Required Skills & Experience

• 38 years of hands-on experience in Apache Spark (PySpark or Scala Spark).
• Strong proficiency in SQL, including complex joins, window functions, and performance optimization.
• Experience in ETL development and data pipeline orchestration.
• Solid understanding of data warehousing and data modelling concepts.
• Familiarity with distributed computing and Spark performance tuning.
• Working knowledge of Git and CI/CD processes.

Preferred Skills (Good to Have)

• Exposure to cloud data platforms such as AWS Glue, Databricks, GCP BigQuery, or Azure Data Factory.
• Experience with workflow orchestration tools like Airflow or Oozie.
• Knowledge of Python or Scala scripting.

Soft Skills

• Strong analytical and problem-solving abilities.
• Excellent communication and collaboration skills.
• Self-motivated with a focus on quality and performance.

Educational Qualification

Bachelors or Masters degree in Computer Science, Information Technology, or a related field.

If interested, please share your resume on ms00883967@techmahindra.com