.Job Description:Euronext is seeking a talented Senior Data Engineer with expertise in AWS and a strong background in solution architecture, database management, and data engineering. The ideal candidate will have extensive experience with AWS services such as Lambda, Glue, Step Functions, and Cloud Formation, along with proficiency in Python, SQL, and database technologies. Experience with Iceberg tables for managing large datasets is highly desirable.As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data solutions on the AWS cloud platform. You will collaborate with functional teams to develop robust data pipelines, ETL processes, and orchestration workflows, leveraging tools like Apache Airflow and AWS Step Functions. Strong production awareness and troubleshooting skills are essential, along with the ability to provide technical leadership, mentorship, and guidance to junior team members.Key AccountabilitiesThe Senior Data Engineer will be assigned to one or more projects and might change projects when higher priorities are identified. When the Senior Data Engineer has acquired sufficient experience, it can be exposed to support activity of production systems, namely being on call for support.Design and Implement AWS Solutions:Utilize expert knowledge of AWS services such as Lambda, Glue, Step Functions, and others to design, implement, and maintain scalable and efficient data solutions on the cloud platform.Solution Architecture and Cloud Infrastructure:Develop robust solution architectures and cloud infrastructure designs, considering factors such as scalability, performance, security, and cost optimization. Demonstrate proficiency in cloud networking, including VPCs, subnets, security groups, and routing tables, to ensure secure and reliable data transmission.Data Engineering and Database Management:Data Modeling: Designing efficient data models for optimal query performance.SQL Proficiency: Writing and optimizing SQL queries.Performance Tuning: Identifying and optimizing performance bottlenecks.ETL and Data Integration: Extracting, transforming, and loading data into Redshift, MySQL, PostgreSQL.Cluster Management: Provisioning, scaling, and monitoring Redshift clusters.Security and Compliance: Implementing security measures and ensuring compliance.AWS Integration: Integrating Redshift with other AWS services.Monitoring and Troubleshooting: Monitoring cluster performance and resolving issues.Documentation and Training: Creating documentation and providing training to team members.Logging and Tracing: Proficiency in setting up and managing logging and tracing mechanisms in AWS, including leveraging services like AWS Cloud Trail for auditing API calls and AWS X-Ray for distributed tracing and performance analysis.Orchestration and Workflow Management:Implement orchestration solutions using tools like Apache Airflow and AWS Step Functions to automate and manage data workflows effectively
