Senior Data Engineer - GCP

Role Overview: As a Senior Data Engineer at Equinix, you will play a crucial role in driving the end-to-end delivery of enterprise-scale data solutions on Google Cloud Platform (GCP). Your responsibilities will include designing sophisticated system architectures, leading cross-functional teams, and mentoring staff and other data engineers to ensure the successful delivery of mission-critical data products. Your technical expertise in GCP data technologies, leadership skills in guiding engineering teams through complex challenges, and ability to translate strategic business objectives into scalable technical solutions will be key to your success in this role. Key Responsibilities: - System Design & Architecture Leadership: - Architect complex, distributed data systems capable of handling petabyte-scale workloads with fault tolerance and high availability - Design data platform architectures considering scalability, performance, cost optimization, and operational excellence - Evaluate and recommend emerging technologies, tools, and architectural patterns to drive innovation - Create comprehensive architectural documentation, design patterns, and best practices guidelines - Technical Solutioning & Delivery: - Own end-to-end delivery of complex data engineering initiatives from conception to production - Design and implement sophisticated solutions leveraging BigQuery, Cloud Dataflow, Composer, Dataform/dbt, Pub/Sub, and Vertex AI - Architect and build real-time streaming data pipelines using Cloud Pub/Sub, Kafka, and Apache Beam - Develop advanced data transformation frameworks and custom processing engines - Design microservices-based data platforms with containerization (Docker/Kubernetes) - Establish comprehensive observability, monitoring, and alerting frameworks - Drive performance optimization, cost management, and operational efficiency initiatives - Team Leadership & Mentorship: - Lead and mentor Data Engineers and other team members, fostering technical excellence and professional growth - Guide teams through complex technical challenges and architectural decision-making - Conduct technical design reviews and ensure adherence to architectural standards - Establish coding standards, quality assurance processes, and engineering best practices - Lead technical training programs and knowledge-sharing initiatives - Drive continuous improvement in team processes, tools, and methodologies - Participate in technical hiring processes and team capability building Qualifications: - Technical Expertise: - GCP Platform Mastery: 8+ years of hands-on experience with Google Cloud Platform, including deep expertise in BigQuery, Cloud Dataflow, Cloud Composer, Cloud Storage, Pub/Sub, Dataproc, and Vertex AI - Programming Excellence: Expert-level Python/Java programming, proficiency in Python/Scala for Spark development - Advanced Data Technologies: Deep expertise in Apache Beam, Apache Spark, Airflow, Kafka, and distributed computing architectures - Infrastructure: Extensive experience with Terraform, CI/CD pipelines, and cloud infrastructure management - Database & Data Modeling: Advanced knowledge of relational databases (PostgreSQL, MySQL) and NoSQL systems (BigTable, Firestore, MongoDB); expertise in data modeling, normalization, and performance optimization - System Design: Proven ability to architect complex, distributed systems with considerations for scalability, reliability, security, and cost-effectiveness Additional Company Details: Equinix is the worlds digital infrastructure company, dedicated to shortening the path to connectivity to enable innovations that enrich work, life, and the planet. At Equinix, bold ideas are welcomed, human connection is valued, and everyone has the opportunity to shape their future. The company is committed to ensuring an open employment process for all individuals, including those with disabilities, and is an Equal Employment Opportunity and, in the U.S., an Affirmative Action employer. Role Overview: As a Senior Data Engineer at Equinix, you will play a crucial role in driving the end-to-end delivery of enterprise-scale data solutions on Google Cloud Platform (GCP). Your responsibilities will include designing sophisticated system architectures, leading cross-functional teams, and mentoring staff and other data engineers to ensure the successful delivery of mission-critical data products. Your technical expertise in GCP data technologies, leadership skills in guiding engineering teams through complex challenges, and ability to translate strategic business objectives into scalable technical solutions will be key to your success in this role. Key Responsibilities: - System Design & Architecture Leadership: - Architect complex, distributed data systems capable of handling petabyte-scale workloads with fault tolerance and high availability - Design data platform architectures considering scalability, performance, cost o