Senior Big Data Engineer

As a Sr Big Data Engineer at dunnhumby, you will be responsible for designing end-to-end data solutions, architecting scalable data pipelines, developing automation frameworks, and ensuring data integrity and availability. Your role will involve leading architectural reviews, implementing data governance standards, and mentoring technical teams. Your technical expertise should include proficiency in data pipeline tools, experience with cloud platforms, understanding of API design, and familiarity with modern data stack tools. Strong problem-solving skills, effective communication, collaboration with cross-functional teams, and adaptability to new technologies are essential soft skills for this role. **Key Responsibilities:** - Design end-to-end data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms. - Architect scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming. - Design and orchestrate end-to-end automation using frameworks like Apache Airflow. - Develop intelligent systems for anomaly detection, alerts, and process maintenance. - Define data architecture strategies supporting advanced analytics, machine learning, and real-time processing. - Implement data governance, metadata management, and data quality standards. - Lead architectural reviews and technical design sessions. - Translate business needs into data architecture requirements. - Explore tools, platforms, and technologies aligned with organizational standards. - Ensure security, compliance, and regulatory requirements in data solutions. - Evaluate and recommend improvements to existing data architecture and processes. - Provide mentorship and guidance to data engineers and technical teams. **Qualification Required:** - Bachelor's or master's degree in computer science, Information Systems, Data Science, or related field. - 5+ years of experience in data architecture, data engineering, or related field. - Proficiency in data pipeline tools such as Apache Spark, Kafka, Airflow, or similar. - Experience with data governance frameworks and cloud platforms. - Strong understanding of API design, data security, and modern data stack tools. - Knowledge of high-level programming languages like Python, Java & Scala. - Experience with Hadoop/Spark Toolsets, relational database management systems, and data flow development. In your role at dunnhumby, you can expect not only to meet but exceed your expectations. You will enjoy a comprehensive rewards package, personal flexibility, and thoughtful perks like flexible working hours and your birthday off. Additionally, you will benefit from an investment in cutting-edge technology that reflects the company's global ambition, with a small-business feel that fosters innovation and growth. As a Sr Big Data Engineer at dunnhumby, you will be responsible for designing end-to-end data solutions, architecting scalable data pipelines, developing automation frameworks, and ensuring data integrity and availability. Your role will involve leading architectural reviews, implementing data governance standards, and mentoring technical teams. Your technical expertise should include proficiency in data pipeline tools, experience with cloud platforms, understanding of API design, and familiarity with modern data stack tools. Strong problem-solving skills, effective communication, collaboration with cross-functional teams, and adaptability to new technologies are essential soft skills for this role. **Key Responsibilities:** - Design end-to-end data solutions, including data lakes, data warehouses, ETL/ELT pipelines, APIs, and analytics platforms. - Architect scalable and low-latency data pipelines using tools like Apache Kafka, Flink, or Spark Streaming. - Design and orchestrate end-to-end automation using frameworks like Apache Airflow. - Develop intelligent systems for anomaly detection, alerts, and process maintenance. - Define data architecture strategies supporting advanced analytics, machine learning, and real-time processing. - Implement data governance, metadata management, and data quality standards. - Lead architectural reviews and technical design sessions. - Translate business needs into data architecture requirements. - Explore tools, platforms, and technologies aligned with organizational standards. - Ensure security, compliance, and regulatory requirements in data solutions. - Evaluate and recommend improvements to existing data architecture and processes. - Provide mentorship and guidance to data engineers and technical teams. **Qualification Required:** - Bachelor's or master's degree in computer science, Information Systems, Data Science, or related field. - 5+ years of experience in data architecture, data engineering, or related field. - Proficiency in data pipeline tools such as Apache Spark, Kafka, Airflow, or similar. - Experience with data governance frameworks and cloud platforms. - Strong understanding of