Data Engineer

As a Data Engineer at our company, you will be responsible for leveraging Data Lake architecture and the Azure cloud platform to develop, deploy, and optimize data-driven solutions. Your role will involve transforming raw data into actionable insights to support strategic decision-making across the organization. Key Responsibilities: - Develop and optimize scalable data pipelines using Python and PySpark - Build and orchestrate data workflows with Azure Data Factory - Design and implement solutions using Azure Databricks and Synapse Analytics - Manage and maintain data storage solutions in Azure Data Lake Storage Gen2, focusing on cost-efficient architecture - Implement and manage CI/CD pipelines using Azure Logic Apps, integrating tools like Azure DevOps, GitHub Actions, or Jenkins - Model and maintain Medallion Architecture across Bronze, Silver, and Gold layers based on business requirements - Collaborate with data scientists, analysts, and business stakeholders to ensure reliable data availability - Monitor and optimize performance and cost efficiency of data solutions across the ecosystem Required Skills & Experience: - Strong proficiency in Python and PySpark for data manipulation and transformation - Hands-on experience with Azure Data Factory, Azure Databricks, and Synapse Analytics - Familiarity with Azure Logic Apps for CI/CD pipeline management - Knowledge of CI/CD tools such as: - Azure DevOps Pipelines - GitHub Actions - Jenkins - Expertise in managing Azure Data Lake Storage Gen2 environments with a focus on security and cost optimization - Deep understanding of Medallion Architecture principles: - Bronze Layer: Raw data ingestion - Silver Layer: Cleansed, enriched, and business-ready data - Gold Layer: Aggregated and analytics-ready data models - Strong problem-solving and communication skills - Experience in cost optimization (Note: No additional details of the company were mentioned in the job description) As a Data Engineer at our company, you will be responsible for leveraging Data Lake architecture and the Azure cloud platform to develop, deploy, and optimize data-driven solutions. Your role will involve transforming raw data into actionable insights to support strategic decision-making across the organization. Key Responsibilities: - Develop and optimize scalable data pipelines using Python and PySpark - Build and orchestrate data workflows with Azure Data Factory - Design and implement solutions using Azure Databricks and Synapse Analytics - Manage and maintain data storage solutions in Azure Data Lake Storage Gen2, focusing on cost-efficient architecture - Implement and manage CI/CD pipelines using Azure Logic Apps, integrating tools like Azure DevOps, GitHub Actions, or Jenkins - Model and maintain Medallion Architecture across Bronze, Silver, and Gold layers based on business requirements - Collaborate with data scientists, analysts, and business stakeholders to ensure reliable data availability - Monitor and optimize performance and cost efficiency of data solutions across the ecosystem Required Skills & Experience: - Strong proficiency in Python and PySpark for data manipulation and transformation - Hands-on experience with Azure Data Factory, Azure Databricks, and Synapse Analytics - Familiarity with Azure Logic Apps for CI/CD pipeline management - Knowledge of CI/CD tools such as: - Azure DevOps Pipelines - GitHub Actions - Jenkins - Expertise in managing Azure Data Lake Storage Gen2 environments with a focus on security and cost optimization - Deep understanding of Medallion Architecture principles: - Bronze Layer: Raw data ingestion - Silver Layer: Cleansed, enriched, and business-ready data - Gold Layer: Aggregated and analytics-ready data models - Strong problem-solving and communication skills - Experience in cost optimization (Note: No additional details of the company were mentioned in the job description)