Senior Data Engineer
Mumbai, 6th Floor, WeWork, Chincholi, Bandar Road, Mumbai 400064
We are looking for a Senior data engineer to take on the responsibility of designing data pipelines and enabling Azure cloud services, perform hands-on development, and maintaining data pipelines and build/deploy Azure services. This is also a role that provides leadership opportunities for the candidate to head a new function within GPI, called the ‘Data Engineering team’.
The Senior Data Engineer will oversee GPI’s data integration work, including developing robust data models, developing a data warehouse & data lake and analytics environment and writing code scripts for data integration and analysis. The candidate needs to ensure that automated streams of daily data updates and data management can be successfully implemented, alongside our designated Azure services enablement.
This role will work closely and collaboratively with members of our established Data teams, Product team, and Software Engineering teams to define requirements, mine and analyse data, integrate data from a variety of sources, and deploy high-quality data pipelines in support of the analytics needs of GPI.
We offer a flat working structure and are looking for engineers at all levels to join the team. We value people being engaged and caring about customers, caring about the code they write and the contribution they make to GPI. People with a broad ability to apply themselves to a multitude of problems and challenges, who can work across teams do great things here at GPI. If you care enough to find elegant solutions to difficult technical problems, we’d love to hear from you.
Responsibilities and Duties:
- Design, implement, test, deploy, and maintain stable, secure, and scalable data engineering solutions and pipelines in support of data and analytics projects, including integrating new sources of data into our central data warehouse/data lake, and moving data out to applications and affiliates.
- Responsible in delivering the design and implementation of data lakes and data warehouses for our Azure architecture, supporting GPI’s products.
- Responsible in designing and delivering data models required to migrate GPI’s current manual data collection in Excel (from different data teams), and overall data management to an online automated workflow on our new Azure architecture.
- Responsible in ensuring all requirements are meticulously captured from different Data teams, and Software engineers during the process of designing the Azure stack, data models and overall architecture.
- Responsible in maintaining and providing continuous support to the data teams, and other technical teams in ensuring the overall Azure architecture implementation is continuously optimised for the needs of the business.
- Responsible in developing and maintaining all ETL/ELT pipelines on our Azure platform for the different technical teams.
- Able to begin this role as a single contributor first, and ensure that the overall architecture transformation is realised within the first year.
- Build reports and data visualizations, using data from the data warehouse and other sources.
- Be able to build, lead and manage a data engineering team in GPI.
Key Subject Matter / Technical Skills:
- Degree in Computer science or related discipline
- At least 6+ years of experience in designing Microsoft Azure architecture as well as Data Warehousing (Databricks, Snowflake) models, Data Lakes and database systems.
- Previous experiences working with SaaS products as Sr Data Engineer/Data Engineer
- Extensive experience in designing Azure Synapse Analytics solutions and providing strong integration with other Azure cloud architecture services.
- Build and deliver data pipelines that ingests data (structured and unstructured) from multiple sources into target data lake and data warehouse.
- Experienced in integrating a wide variety of web-scraped data into Azure platform.
- Extensive experience in Python development and scripting
- Able to develop and optimise queries in SQL and NoSQL
- Strong experience in the concepts of Extract, Transform, and Load (ETL) / (ELT)
- Experience in creating large scale data pipelines using Azure Data Factory for data teams and enabling automation for daily data manipulation and data updates.
- Strong experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, Cosmos DB,
- Cassandra, Azure Functions, Serverless Architecture, ARM Templates
- Experience working with varied forms of data infrastructure inclusive of Hadoop, Spark and column-oriented databases.
- Extremely comfortable with Python scripting, and in another programming language (C++ or Java)
- Familiarity with APIs (REST, SOAP, and GraphQL)
- Experience with CI/CD workflows (e.g. Azure DevOps) and Git best practices
- Experienced in Microsoft Dot Net framework development
- Familiar with working with Agile projects
- Experience with C#
- Experience with batch and streaming processing, real-time applications, event-based architectures and pub/sub patterns (e.g., Kafka)
- Experience with Docker, Kubernetes and Terraform
- Working knowledge and basics in NLP, Machine learning
- A passion for developing code “the right way” according to sound development principles and an advocate of unit testing and continuous integration
- Excellent verbal and written communication and reporting skills (English)
- Ability to work independently and inspire a small team
- Strong attention to detail and ability to deliver high quality work within defined timelines
- A keenness to learn and commitment to a rapid development curve
- A strong motivation to thrive in a client-focused, international team environment
- Excellent analytical & problem-solving skills, willingness to take ownership and resolve.
Share this job:
GPI is a price and market access insights company, providing innovative approaches to the biopharma industry. Headquartered London, GPI utilise analytical approaches to provide evidence-based decision support, helping clients to make better investment decisions.
As a people-centric organisation, we strive to create opportunities for the best people in the industry. Our cultural values are centred on our corporate values: authenticity, sustainability, quality matters and collaboration. Through our training and career programme LEAP we promote the ambitions of inspiring and talented individuals so that together we can make a difference to the life sciences industry.
As a business, we support our biopharma clients through:
- GPI pulse™: a powerful data analytics platform
- GPI consulting: bespoke engagements with a focus on strategic pricing, market access excellence, and commercial strategy
- GPI horizon: dynamic value assessment and price forecasting solution
Our mission is to lead the way forward to a future where data-driven strategies are the foundation for confident decision-making. We begin with our clients’ greatest challenges and harness the power of technology and our team of experts to offer holistic solutions.
If you are an ambitious individual, looking for the next step in your career and want to work in a collaborative and open environment, please send across your CV to email@example.com