Data Engineer

In this role as a Data Engineer at Elgebra, you will be responsible for designing, building, and maintaining scalable and reliable data pipelines using Airflow, Python, and other cutting-edge technologies. You will collaborate closely with data scientists, analysts, and other engineers to ensure the smooth flow of data from various sources to our data warehouse and analytical platforms. Your work will directly impact the quality and availability of data used for critical business decisions, enabling Elgebra and its clients to gain a competitive edge. **Key Responsibilities:** - Design and implement robust and scalable ETL pipelines using Airflow and Python to ingest, transform, and load data from diverse sources into our data warehouse. - Develop and maintain data models and schemas that support efficient querying and analysis by data scientists and analysts. - Monitor and troubleshoot data pipeline performance, ensuring data quality and availability for downstream applications. - Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications. - Optimize data storage and processing infrastructure for performance and cost-effectiveness. - Implement data governance and security policies to ensure data integrity and compliance. - Contribute to the development of best practices for data engineering and pipeline automation. **Required Skillset:** - Demonstrated ability to design, develop, and maintain complex data pipelines using Airflow. - Proficient in Python programming, with experience in data manipulation and ETL processes. - Strong understanding of SQL and relational database concepts, with experience in writing complex queries. - Experience with cloud platforms such as AWS, including services like S3, EC2, and Redshift. - Solid understanding of big data technologies and concepts, such as Hadoop, Spark, and Hive. - Excellent problem-solving and analytical skills, with the ability to identify and resolve data-related issues. - Strong communication and collaboration skills, with the ability to work effectively in a team environment. - Bachelor's or Master's degree in Computer Science, Engineering, or a related field. Elgebra is a leading data analytics and solutions provider, empowering businesses across the financial services, healthcare, and retail sectors to unlock the full potential of their data. The company specializes in building robust data pipelines and delivering actionable insights that drive strategic decision-making and improve operational efficiency. With expertise in big data technologies, Elgebra handles massive datasets and provides customized solutions tailored to clients' specific needs. In this role as a Data Engineer at Elgebra, you will be responsible for designing, building, and maintaining scalable and reliable data pipelines using Airflow, Python, and other cutting-edge technologies. You will collaborate closely with data scientists, analysts, and other engineers to ensure the smooth flow of data from various sources to our data warehouse and analytical platforms. Your work will directly impact the quality and availability of data used for critical business decisions, enabling Elgebra and its clients to gain a competitive edge. **Key Responsibilities:** - Design and implement robust and scalable ETL pipelines using Airflow and Python to ingest, transform, and load data from diverse sources into our data warehouse. - Develop and maintain data models and schemas that support efficient querying and analysis by data scientists and analysts. - Monitor and troubleshoot data pipeline performance, ensuring data quality and availability for downstream applications. - Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications. - Optimize data storage and processing infrastructure for performance and cost-effectiveness. - Implement data governance and security policies to ensure data integrity and compliance. - Contribute to the development of best practices for data engineering and pipeline automation. **Required Skillset:** - Demonstrated ability to design, develop, and maintain complex data pipelines using Airflow. - Proficient in Python programming, with experience in data manipulation and ETL processes. - Strong understanding of SQL and relational database concepts, with experience in writing complex queries. - Experience with cloud platforms such as AWS, including services like S3, EC2, and Redshift. - Solid understanding of big data technologies and concepts, such as Hadoop, Spark, and Hive. - Excellent problem-solving and analytical skills, with the ability to identify and resolve data-related issues. - Strong communication and collaboration skills, with the ability to work effectively in a team environment. - Bachelor's or Master's degree in Computer Science, Engineering, or a related field. Elgebra is a leading data analytics and solutions provider, empowering