.DATA ENGINEER (REMOTE) Portuguese company hires for remote work FLUENT PORTUGUESE AND ENGLISH (C1) SEND CV IN ENGLISH Description Data Engineer PURPOSE OF THE JOB You will work on one of the world's largest platforms which deals with a few petabytes of data coming to the system daily. You will contribute as part of R&D self-organized team working in a challenging, innovative environment for our client. Investigate, create, and implement the solutions for many technical challenges using cutting edge technologies, including building/enhancing data processing platform enabling work of software used by hundreds of millions of users. Responsibilities Obtains tasks from the project lead or Team Lead (TL), prepares functional and design specifications, approves them with all stakeholders. Ensures that assigned area/areas are delivered within set deadlines and required quality objectives. Provides estimations, agrees task duration with the manager and contributes to project plan of assigned area. Evaluating existing data systems. Developing best practices for data coding to ensure consistency within the system. Updating and optimizing local and metadata models. Evaluating implemented data systems for variances, discrepancies, and efficiency. Troubleshooting and optimizing data systems. Understand the business drivers and analytical use-cases and translate these to data products. Design, implement and maintain pipelines that produce business critical data reliably and efficiently using cloud technology. Addresses area-level risks, provides, and implements mitigation plan. Reports about area readiness/quality, and raise red flags in crisis situations which are beyond his/her AOR. Responsible for resolving crisis situations within his/her AOR. Initiates and conducts code reviews, creates code standards, conventions, and guidelines. Suggests technical and functional improvements to add value to the product. Constantly improves his/her professional level. Collaborates with other teams. MUST HAVE Having a solid understanding of data engineering principles and proficiency in designing, building, and optimizing data pipelines. Having knowledge of ETL tools and frameworks and an understanding of native capabilities for data ingestion, transformation, and orchestration, such AIRFLOW. Strong SQL Skills Proficiency in Python Minimum 4 Years Experience In The Technologies Above PLUS In-depth knowledge of both Snowflake and Databricks platforms is required. Proficiency in data modeling and designing efficient data structures/ETL processes and optimizing data storage. AWS cloud experience (EC2/S3). Awareness of data governance principles and best practices for ensuring data security, and compliance. Tableau, Python/Scala, PySpark, SparkSQL, Airflow, Kafka, Flink, Trino, Snowflake, Deltalake, Databricks, AWS (EC2/S3). Office hours Eastern Daylight Time - 14:30 PM - 22:30 PM (Lisbon Time) 09:30 AM- 17
