Data Engineering Consultant

As a Data Engineering Consultant at Codvo in Doha, Qatar, you will be responsible for designing and maintaining enterprise data architecture, building data pipelines, integrating diverse data sources, developing cloud data platforms, data modeling, warehousing, and enabling advanced AI/ML models. Your key responsibilities include: - Define and maintain the enterprise data architecture blueprint, including data lakes, warehouses, streaming platforms, and analytics layers. - Design logical, physical, and conceptual data models to support operational, analytical, and AI use cases. - Ensure architecture supports scalability, performance, and high availability. - Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines. - Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians, DCS, PLC, and IoT sensors. - Implement and manage data solutions on the Microsoft Azure cloud platform, leveraging services like Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics. - Design and implement data models optimized for time-series data from industrial assets, supporting operational dashboards and real-time analytics. - Build the data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data. Qualifications required for this role include: - 10+ years of experience in data architecture, data engineering, or enterprise analytics roles. - Strong experience designing enterprise-scale data platforms in cloud and hybrid environments. - Expertise in data modeling and data integration patterns. - Deep understanding of data lakes, data warehouses, Lakehouse architectures, and real-time streaming platforms. - Hands-on experience with cloud data services, especially Azure. - Strong knowledge of SQL, data transformation frameworks, and metadata management. - Familiarity with DevOps and DataOps practices. Technical proficiencies expected: - Expert-level proficiency in SQL and Python for data manipulation and pipeline development. - Hands-on experience with distributed computing frameworks like Apache Spark and streaming technologies like Kafka. - Deep experience with Microsoft Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Databricks, Azure Synapse. - Proven experience with modern data platforms like Databricks Delta Lake. - Understanding of machine learning lifecycles and the data requirements for AI/ML models. - Proficiency with Git and CI/CD best practices. Preferred qualifications include certifications in Azure, hands-on experience with OSDU Data Platform, experience in Oil & Gas or industrial data environments, understanding of cybersecurity considerations, familiarity with ERP systems like SAP, advanced SQL skills, and experience in constructing and maintaining enterprise knowledge graphs. As a Data Engineering Consultant at Codvo in Doha, Qatar, you will be responsible for designing and maintaining enterprise data architecture, building data pipelines, integrating diverse data sources, developing cloud data platforms, data modeling, warehousing, and enabling advanced AI/ML models. Your key responsibilities include: - Define and maintain the enterprise data architecture blueprint, including data lakes, warehouses, streaming platforms, and analytics layers. - Design logical, physical, and conceptual data models to support operational, analytical, and AI use cases. - Ensure architecture supports scalability, performance, and high availability. - Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines. - Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians, DCS, PLC, and IoT sensors. - Implement and manage data solutions on the Microsoft Azure cloud platform, leveraging services like Azure IoT Hub, Azure Event Hubs, and Azure Stream Analytics. - Design and implement data models optimized for time-series data from industrial assets, supporting operational dashboards and real-time analytics. - Build the data infrastructure to support AI/ML models for predictive maintenance, operational anomaly detection, and process optimization using real-time OT data. Qualifications required for this role include: - 10+ years of experience in data architecture, data engineering, or enterprise analytics roles. - Strong experience designing enterprise-scale data platforms in cloud and hybrid environments. - Expertise in data modeling and data integration patterns. - Deep understanding of data lakes, data warehouses, Lakehouse architectures, and real-time streaming platforms. - Hands-on experience with cloud data services, especially Azure. - Strong knowledge of SQL, data transformation frameworks, and metadata management. - Familiarity with DevOps and DataOps practices. Technical proficiencies expected: - Expert-level pr