Senior Data Engineer

As a Senior Data Engineer, you will be responsible for designing, building, and optimizing large-scale data pipelines and data warehouse solutions. Your expertise in big data technologies, cloud platforms, and modern data engineering frameworks will support advanced analytics and business intelligence initiatives. **Key Responsibilities:** - Design, develop, and maintain scalable data pipelines (ETL/ELT) for processing large-scale datasets (multi-terabyte scale). - Build and optimize batch and real-time data processing systems. - Develop robust data architectures to support analytics, reporting, and business intelligence. - Implement data solutions using Spark, PySpark, and SQL for high-performance data transformation. - Work with cloud-based data platforms (AWS/GCP) for scalable and secure data solutions. - Develop and manage orchestration workflows using Airflow. - Integrate streaming solutions using Kafka and Spark Structured Streaming. - Collaborate with cross-functional teams including analytics, product, and engineering stakeholders. - Ensure data quality, governance, performance optimization, and reliability. **Qualifications Required:** - Programming Languages: Python, SQL, Scala, Java - Big Data Technologies: Apache Spark, PySpark, Spark SQL - Streaming Technologies: Kafka, Spark Structured Streaming - Cloud Platforms: AWS or GCP, Databricks, Snowflake - Data Engineering & Orchestration: ETL/ELT Pipeline Development, Apache Airflow The company is looking for someone with 7 to 9 years of experience in this field. It is good to have experience with Java-based frameworks like Spring Boot, familiarity with FastAPI or Django, and exposure to data warehouse modernization initiatives. As a Senior Data Engineer, you will be responsible for designing, building, and optimizing large-scale data pipelines and data warehouse solutions. Your expertise in big data technologies, cloud platforms, and modern data engineering frameworks will support advanced analytics and business intelligence initiatives. **Key Responsibilities:** - Design, develop, and maintain scalable data pipelines (ETL/ELT) for processing large-scale datasets (multi-terabyte scale). - Build and optimize batch and real-time data processing systems. - Develop robust data architectures to support analytics, reporting, and business intelligence. - Implement data solutions using Spark, PySpark, and SQL for high-performance data transformation. - Work with cloud-based data platforms (AWS/GCP) for scalable and secure data solutions. - Develop and manage orchestration workflows using Airflow. - Integrate streaming solutions using Kafka and Spark Structured Streaming. - Collaborate with cross-functional teams including analytics, product, and engineering stakeholders. - Ensure data quality, governance, performance optimization, and reliability. **Qualifications Required:** - Programming Languages: Python, SQL, Scala, Java - Big Data Technologies: Apache Spark, PySpark, Spark SQL - Streaming Technologies: Kafka, Spark Structured Streaming - Cloud Platforms: AWS or GCP, Databricks, Snowflake - Data Engineering & Orchestration: ETL/ELT Pipeline Development, Apache Airflow The company is looking for someone with 7 to 9 years of experience in this field. It is good to have experience with Java-based frameworks like Spring Boot, familiarity with FastAPI or Django, and exposure to data warehouse modernization initiatives.