DTx is looking for a Senior Data Engineer to integrate its Data & Application Engineer (DAE) Group within the project "R2UTechnologies – modular systems" (nr. C644876810-00000019), financed by Business Innovation – Innovation Pact, within the call n. º 02/C05-i01/2022 of PRR – Plano de Recuperação e Resiliência. The "R2UTechnologies – Modular system" Innovation Pact aims to execute the development and industrialization on Portuguese soil of the disruptive industrial concept of modular construction. This innovative concept will respond to the growing needs of the global market and the new challenges facing the sector today, particularly in the fields of design, sustainability, customization, intelligent building, and environmental protection.
Candidate Profile:
We are seeking a skilled and experienced data engineer to join our team. In this role, you will be an essential part of all data engineering pipeline, conducting data migration, data wrangling, and data integration. You will design and develop the ETL/ELT pipeline architecture.
The ideal candidate will have a strong background working with data engineering methodologies and techniques, have vast experience with Spark, SQL, Data warehouses, and knowledge in machine learning.
You should be able to work independently and as part of a team to identify and resolve issues in a timely manner.
Requirements:
Master's or Ph.D. degree in Computer Science, or a related field with a strong emphasis on software development;
Experience (5+ years) in data engineering processes;
Preferably with understanding of technologies such as Databricks, Data Factory, and Azure Functions;
Proficiency in SQL, Scala/Python;
Vast experience with Spark;
Familiarity with Database technologies such as Cassandra, MongoDB, SQL, PostgreSQL;
Knowledge of MLOps, ETL processes;
Familiarity with REST API;
Experience in Machine Learning, including the ability to implement machine learning models in data pipelines;
Knowledge of machine learning libraries and frameworks, such as TensorFlow, PyTorch, or Scikit-learn;
Ability to integrate and optimize machine learning models in production environments;
Analytical and problem-solving skills;
Motivation to learn and apply technologies in which you have no experience;
Capacity for innovation and constant progression;
Autonomy, accuracy, responsibility, and awareness of the importance of quality in software development;
Good verbal and written communication and presentation skills in Portuguese and/or English;
Ability to analyze and structure information to be shared and communicated to other stakeholders and development teams.
Responsibilities:
Connect to raw data: Perform distribution analysis, data quality checks, and validation checks;
Design architectures and technologies: Explore new technologies and solutions;
Design Feature Engineering Pipelines: Ingest data and perform data transformation, cleaning, and feature engineering;
Continuous Improvement: Your insights will not only fix current issues but also inform our future development processes, helping prevent potential problems before they arise;
Collaborate and Communicate: Work closely with cross-functional teams, identify defects, suggest improvements, and advocate for quality at every stage of development.
About DTx: Digital Transformation CoLab DTx carries out its activity of applied research in the areas related to digital transformation. DTx aims at being a reference player in the scope of digital transformation and focuses its research in the intersection of physical, digital, and cybernetic domains, with the purpose of creating the next generation of advanced cyber-physical systems.
Application: Submission of the CV and Degree Certificates to the address ******, indicating in subject: DTx/12/2024.
#J-18808-Ljbffr