Data Engineering Lead

Detalhes da Vaga

.What we offerInteract with senior stakeholders at our clients on a regular basis to drive their business towards impactful change. Become the go-to person for end-to-end data handling, management, and analytics processes. Lead your team in creating the pipeline for Data management, data visualization, and analytics products, including automated services and APIs. Work with Data Scientists to take data throughout its lifecycle - acquisition, exploration, data cleaning, integration, analysis, interpretation, and visualization. Become part of a fast-growing international and diverse team.What you will doLead, influence & implement the technical roadmap of our clients, in light of overall technology and architecture roadmaps.Be responsible for the Architecture, ensuring that technical and non-technical components work together in the product to deliver customer needs.Lead and develop digital development and operations using Python, Spark, RESTful API, and Microsoft Azure Cloud, ideally with some Data Insights experience.Steer the technical excellence of the application and consistency with relevant digital frameworks, best practices, and standards.Coach and enable other product team members to deliver best-in-class products.Deliver data platforms, data insights, and in collaboration with the data science team – data products, such as recommendation systems, data lakes/hubs, data insights, and dashboards.Lead project activities and be responsible for the evaluation of business needs and objectives in collaboration with the project team and other stakeholders.What you'll bring5-8 years of experience in a similar role with experience in developing and deploying ETL solutions on Azure, GCP or AWS.Solid knowledge of data warehousing principles, concepts, and best practices (e.G. ODS, data marts, data lakes, data vault, 3NF).Good understanding of common platforms and practices regarding digital development; Cloud-based setups, Advanced Analytics and computing environments, and GitHub (or similar) and CI/CD workflows.Experience in setting up automated testing frameworks including unit tests and integration tests.Understanding of modern cloud-based architecture (Lambda & Kappa architectures).Advanced SQL, data transformation, and data profiling skills.Experience in building production ETL/ELT pipelines at scale.Data governance experience.3-5 years of hands-on experience with Azure: Data Factory, Databricks, Synapse (DWH), Azure Functions, App logic, and other data analytics services, including streaming.Experience with Airflow, Kubernetes, Great expectations is a plus.Programming languages: Python (PySpark), scripting languages like Bash.Basic knowledge of PowerBI is a plus.Customer-centric, passionate about delivering great digital products and services.Demonstrating a true engineering craftsmanship mindset.Passionate about continuous improvement and working collaboratively


Salário Nominal: A acordar

Fonte: Jobtome_Ppc

Função de trabalho:

Requisitos

Work From Home Portuguese (Portugal) Freelance Writer

Work from Home Portuguese (Portugal) Freelance Writer - Vila Nova de Gaia Outlier helps the world's most innovative companies improve their AI models by prov...


Outlier - Porto

Publicado a month ago

Salesforce Developer

We're looking for the special, unique, and amazing YOU! At KCS IT, we look for those that stand out, for those that always want to be better and fight for it...


Kcs It - Porto

Publicado a month ago

Functional Tester

n What the Job is About n Do you get excited when you find a bug and can trace it back to its source? Are you looking for a role where your efforts directly ...


Sofico Nv - Porto

Publicado a month ago

Qae

We're looking for the special, unique, and amazing YOU! At KCS IT, we look for those that stand out, for those that always want to be better and fight for it...


Kcs It - Porto

Publicado a month ago

Built at: 2024-12-19T01:48:13.598Z