Lead Data Engineer (AWS Data Platform)

Position: Lead Data Engineer (AWS Data Platform) Experience: 5 to 8+ Years Location: Hybrid (Pakistan) Note: (Candidates must be available during UAE business hours and follow UAE public holidays) Job Summary We are seeking a highly skilled Lead Data Engineer with 5 to 8+ years of experience in designing, developing, and optimizing large-scale data platforms and ETL/ELT pipelines. The ideal candidate will have strong hands-on expertise in PySpark, AWS Glue, Amazon EMR, Amazon Redshift, and SQL-based data warehousing, along with proven experience in performance tuning and data optimization. Key Responsibilities - Design, develop, and maintain scalable data pipelines using PySpark, AWS Glue, and Amazon EMR. - Build and optimize data ingestion, transformation, and processing frameworks for structured and semi-structured data. - Develop and maintain enterprise data warehouse solutions using Amazon Redshift. - Write complex SQL queries, stored procedures, and data transformations to support analytics and reporting requirements. - Implement ETL/ELT processes to move data efficiently across multiple systems and platforms. - Perform performance tuning and optimization of Spark jobs, ETL pipelines, SQL queries, and Redshift workloads. - Ensure data quality, integrity, security, and governance across data platforms. - Troubleshoot production issues and perform root cause analysis for data-related incidents. - Collaborate with business stakeholders, analysts, architects, and engineering teams to understand data requirements. - Participate in code reviews, technical design discussions, and best-practice implementation. - Monitor data pipelines and proactively identify opportunities for performance improvements and automation. - Create and maintain technical documentation, data models, and operational procedures. Required Skills & Experience Must-Have Skills: - 5 to 8+ years of experience in Data Engineering and Data Warehousing. - Strong hands-on experience with PySpark. - Extensive experience with AWS Glue. - Experience building and managing workloads on Amazon EMR. - Strong expertise in Amazon Redshift. - Excellent SQL development and query optimization skills. - Strong understanding of Data Warehousing concepts, dimensional modeling, and ETL/ELT processes. - Experience in performance tuning of Spark jobs, SQL queries, ETL pipelines, and data warehouse workloads. - Experience handling large-scale datasets and distributed data processing. - Strong debugging, troubleshooting, and analytical skills. Preferred Skills: - Experience with additional AWS services such as S3, IAM, CloudWatch, Lambda, and Step Functions. - Knowledge of CI/CD pipelines and DevOps practices for data platforms. - Experience with workflow orchestration tools. - Familiarity with data governance, security, and compliance practices. - Exposure to Agile/Scrum development methodologies. Qualifications: - Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related field. - Relevant AWS certifications will be considered an advantage.

(تازہ ترین نوکریاں، سلیبس اور ٹیسٹ کی تیاری کی مکمل معلومات)