Data Engineer

As a skilled Data Engineer, your role will involve designing, building, and maintaining data pipelines and data models to support analytical and business intelligence needs. Your expertise in Python or SQL, Google Cloud Platform (GCP), and a solid understanding of data management, quality, and security best practices will be crucial for this position. Key Responsibilities: - Build and maintain moderately complex data pipelines to ensure smooth data flow, transformation, and usability for analytical projects. - Design and implement data models, focusing on performance and scalability optimization. - Utilize knowledge of data characteristics and supply patterns to create rules and tracking processes that enhance data quality models. - Prepare data for analytical purposes by collecting, integrating, cleansing, and structuring data from various sources and systems. - Design, create, and interpret large and highly complex datasets. - Troubleshoot pipeline and data issues to ensure accuracy and reliability. - Stay updated with GCP advancements and suggest innovative solutions. - Implement security best practices within data pipelines and cloud infrastructure. - Collaborate with global teams to exchange best practices in data management, maintenance, reporting, and security. - Develop and execute data quality checks to maintain consistency and integrity. - Utilize credit data products and conduct analysis using tools like Google BigQuery, BigTable, DataFlow, and Spark/PySpark. Mandatory Skills: - Python or SQL Proficiency: Ability to use Python or SQL for data manipulation and processing. - GCP & Cloud Fundamentals: Intermediate understanding and experience with Google Cloud Platform (GCP) and cloud computing concepts. - Data Pipeline Construction: Demonstrated capability in building, maintaining, and troubleshooting moderately complex pipelines. - Data Modeling & Optimization: Experience in designing and optimizing data models for improved performance. - Data Quality Governance: Skill in developing rules, tracking processes, and checks to uphold a data quality model. - Data Preparation & Structuring: Proficiency in integrating, consolidating, cleansing, and structuring data for analytical purposes. - Security Implementation: Knowledge of security best practices in pipelines and cloud infrastructure. - Big Data Analysis Tools: Hands-on experience with Google BigQuery, BigTable, DataFlow, Scala + Spark or PySpark. - Advanced Data Formats: Familiarity with working on JSON, AVRO, and PARQUET formats. - Communication & Best Practices: Strong communication skills to advocate global best practices and facilitate adoption. Preferred Qualifications: - Cloud certification (e.g., GCP Data Engineer, AWS, Azure). - Experience with credit data products. - Familiarity with data governance frameworks and metadata management tools. (Note: The company details were not provided in the job description.) As a skilled Data Engineer, your role will involve designing, building, and maintaining data pipelines and data models to support analytical and business intelligence needs. Your expertise in Python or SQL, Google Cloud Platform (GCP), and a solid understanding of data management, quality, and security best practices will be crucial for this position. Key Responsibilities: - Build and maintain moderately complex data pipelines to ensure smooth data flow, transformation, and usability for analytical projects. - Design and implement data models, focusing on performance and scalability optimization. - Utilize knowledge of data characteristics and supply patterns to create rules and tracking processes that enhance data quality models. - Prepare data for analytical purposes by collecting, integrating, cleansing, and structuring data from various sources and systems. - Design, create, and interpret large and highly complex datasets. - Troubleshoot pipeline and data issues to ensure accuracy and reliability. - Stay updated with GCP advancements and suggest innovative solutions. - Implement security best practices within data pipelines and cloud infrastructure. - Collaborate with global teams to exchange best practices in data management, maintenance, reporting, and security. - Develop and execute data quality checks to maintain consistency and integrity. - Utilize credit data products and conduct analysis using tools like Google BigQuery, BigTable, DataFlow, and Spark/PySpark. Mandatory Skills: - Python or SQL Proficiency: Ability to use Python or SQL for data manipulation and processing. - GCP & Cloud Fundamentals: Intermediate understanding and experience with Google Cloud Platform (GCP) and cloud computing concepts. - Data Pipeline Construction: Demonstrated capability in building, maintaining, and troubleshooting moderately complex pipelines. - Data Modeling & Optimization: Experience in designing and optimizing data models for improved performance. - Data Quality Governance: Skill in developing rules, tracking processes, and chec