Role & responsibilities

• Minimum of 5 years of hands-on experience in designing, building, and optimizing data pipelines, data models and PySpark for large-scale data processing and analysis
• Extensive experience and deep expertise in data modeling and data model concepts, particularly with large datasets, ensuring the design and implementation of efficient, scalable, and high-performing data models.
• Strong software engineer, you take pride in what youre developing with a strong testing ethos.
• String proficiency in Python programming, with a focus on data processing and analysis
• Extensive experience in designing, building, and optimizing ADB/PySpark pipelines and architectures, with a strong focus on supporting both batch and real-time data workflows.
• Hand-on Experience with cloud technologies and big data concepts, especially in Azure Databricks, Azure Data Lake and Azure Data Factory
• In-Depth Knowledge of PySpark, including experience with PySpark performance tuning techniques to optimal processing efficiency
• Strong SQL skills for querying and manipulating large datasets, with experience in optimizing complex queries for performance