Sr. Database Engineer

As a Data Engineer, your role will involve creating and maintaining optimal data pipeline architecture. You will be responsible for assembling large, complex data sets that meet functional/non-functional business requirements. Furthermore, you will identify, design, and implement internal process improvements such as automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability using Python/open source technologies. Key Responsibilities: - Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and cloud database technologies. - Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs. - Keep our data separated and secure across national boundaries through multiple data centers and regions. - Test databases and perform bug fixes. - Develop best practices for database design and development activities. - Take on technical leadership responsibilities of database projects across various scrum teams. - Manage exploratory data analysis to support database and dashboard development. Qualification Required: - Expert knowledge in databases like PostgreSQL and any cloud-based Data Warehouse with strong programming experience in SQL. - Competence in data preparation and/or ETL tools like snapLogic, MATILLION, Azure Data Factory, AWS Glue, and SSIS. - Understanding of data modeling techniques and working knowledge with OLTP and OLAP systems. - Deep knowledge of databases, stored procedures, optimizations of huge data. - In-depth knowledge of ingestion techniques, data cleaning, de-dupe, and partitioning. - Experience with building the infrastructure required for data ingestion and analytics. - Ability to fine-tune report-generating queries. - Solid understanding of normalization and denormalization of data, database exception handling, transactions, profiling queries, performance counters, debugging, database & query optimization techniques. - Understanding of index design and performance-tuning techniques. - Familiarity with SQL security techniques such as data encryption at the column level, Transparent Data Encryption (TDE), signed stored procedures, and assignment of user permissions. - Experience in understanding the source data from various platforms and mapping them into Entity Relationship Models (ER) for data integration and reporting. - Adhere to standards for all databases e.g., Data Models, Data Architecture, and Naming Conventions. - Exposure to Source control like GIT, Azure DevOps. - Understanding of Agile methodologies (Scrum, Kanban). - Preferably experience with NoSQL database to migrate data into other types of databases with real-time replication. - Experience with automated testing and coverage tools. - Experience with CI/CD automation tools (desirable). As a Data Engineer, your role will involve creating and maintaining optimal data pipeline architecture. You will be responsible for assembling large, complex data sets that meet functional/non-functional business requirements. Furthermore, you will identify, design, and implement internal process improvements such as automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability using Python/open source technologies. Key Responsibilities: - Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and cloud database technologies. - Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs. - Keep our data separated and secure across national boundaries through multiple data centers and regions. - Test databases and perform bug fixes. - Develop best practices for database design and development activities. - Take on technical leadership responsibilities of database projects across various scrum teams. - Manage exploratory data analysis to support database and dashboard development. Qualification Required: - Expert knowledge in databases like PostgreSQL and any cloud-based Data Warehouse with strong programming experience in SQL. - Competence in data preparation and/or ETL tools like snapLogic, MATILLION, Azure Data Factory, AWS Glue, and SSIS. - Understanding of data modeling techniques and working knowledge with OLTP and OLAP systems. - Deep knowledge of databases, stored procedures, optimizations of huge data. - In-depth knowledge of ingestion techniques, data cleaning, de-dupe, and partitioning. - Experience with building the infrastructure required for data ingestion and analytics. - Ability to fine-tune report-generating queries. - Solid understanding of normalization and denormalization of data, database exception handling, transactions, profiling queries, perfo