Senior Big Data Python Engineer Let'S Make Things Fly! Lisbon

Detalhes da Vaga

Let's make things Fly! Your Role & Responsibility:
Data Transformation and Processing: Extract, transform, and load (ETL) Flight Data Monitoring data using Spark and PySpark, applying relevant algorithms based on engineering requirements to ensure accuracy and consistency.
Algorithm Implementation: Develop, code, and test PySpark algorithms that meet specific definitions and requirements provided by aircraft engineers, addressing use cases such as anomaly detection, event recognition, and trend analysis in flight data.
Timeseries Data Management: Structure and optimize large time-series datasets in PostgreSQL Timeseries, with a focus on efficient storage, retrieval, and processing for real-time insights.
Platform Management: Maintain data processing pipelines within Azure, ensuring scalability, performance, and data security for big data workloads.
API and Data Delivery: Publish processed data via API or other methods to ensure seamless integration with frontend applications, enabling real-time data access.
Monitoring and Optimization: Continuously monitor data pipelines and workflows for performance and reliability, optimizing code and infrastructure to meet SLAs and operational goals.
Collaboration with Aircraft Engineers: Work directly with aircraft engineers to translate their data analysis needs and specifications into PySpark algorithms, ensuring that data processing outputs align with technical and operational requirements.
Documentation and Best Practices: Document data transformation processes, maintain code quality standards, and contribute to best practices in big data engineering.

#J-18808-Ljbffr


Salário Nominal: A acordar

Fonte: Allthetopbananas_Ppc

Função de trabalho:

Requisitos

Senior Backend Software Engineer | Base

At Wellhub (formerly Gympass) (Permanent), in Lisbon, Portugal Expires at: 2025-06-09 Remote policy: Full remote Apply now for a career that puts wellbeing f...


Landing.Jobs - Lisboa

Publicado 13 days ago

Juniot Isv Partnerships Executive

ISV Partnerships Manager, are you passionate about payments and fintech? Do you want to join one of Europe's technology leading digital payments companies th...


Viva.Com - Lisboa

Publicado 13 days ago

.Net Developer (Portugal) - Hybrid Remote

Boost IT is a Portuguese technology consultancy company, we are integrated into one of the most entrepreneurial groups in Portugal, with investment in more t...


Boost-It - Lisboa

Publicado 13 days ago

Software Engineer

About UsCompensation is more than a pay-check, and traditional approaches don't adequately reflect it or the way people work today. We're on a mission to cha...


Coverflex - Lisboa

Publicado 13 days ago

Built at: 2024-12-23T14:42:58.363Z