Principal Data Engineer
Job Title – Principle Data Engineer
Overview of Providence:
At Providence, we use our voice to advocate for vulnerable populations and needed reforms in health care. We pursue innovative ways to transform health care by keeping people healthy, and making our services more convenient, accessible and
affordable for all. In an increasingly uncertain world, we are committed to high-quality, compassionate health care for everyone—regardless of coverage or ability to pay. We help people and communities’ benefit from the best health care model for the future—today.
Together, our 119,000-plus caregivers/employees serve in 51 hospitals, more than 1000 clinics and a comprehensive range of health and social services across Alaska, California, Montana, New Mexico, Oregon, Texas and Washington in United States.
Providence Information Service (commonly known as Information technology) is aimed to digitally enable our vision of Health for a better world. We make use of multiple technologies spread across Microsoft, Oracle, JAVA, Python and many new open sources.
Providence Global Center recently launched in Hyderabad, India as Global Capability Center for Providence looking to leverage the India talent to help meet our global vision and scale our Information Services and products to the world of Cloud.
About Healthcare Intelligence
Healthcare Intelligence is the pillar that focuses on creating intelligent data products within Providence and provides unique opportunities in Data Engineering, Operations, Data Science, BI Reporting and Data Analytics on cloud stack. We are a group of data professionals who work towards enabling decisions that improve patient and caregiver experience.
About the role
A Data Engineer is responsible for designing and modelling Data lake solutions/data ingestion pipelines by collaborating with our customers and understanding requirements on digital transformation. Able to navigate through ambiguous situations, to deal with aggressively changing environment. The drive to collaborate, gather feedback, solve problems, and tackle challenges through test and learn is highly valuable in this position. Should work closely with Software Engineers, Data Scientists, Azure & Network Administrator teams to build a scalable & compliant data system. Work with different stakeholders as SME for Data Engineering work.
Responsibilities
- Be the owner for data engineering solutions within Healthcare Intelligence. Ensure the continuity of data processes and the associated batch jobs.
- Partner with leadership, engineers, program managers, data analysts and data scientists to understand data needs
- Identify, design and implement/coordinate implementation of scalable processes and infrastructure to have good governance of automated data processes.
- Manage the end-to-end data solutions of our customers: from raw data analysis to data flow and predictive framework configurations.
- Manage performance, capacity, availability, security and compliance of data platform and data solutions.
- Should be able to work on a problem independently and prepare client ready deliverable with minimal supervision. Elicit, analyze, and validate customer data from ingestion to production
- Monitor all data update processes and outputs to ensure predictive quality
- Communicate with customers to discuss any issues with received data and help them identify and fix data issues
- Solve day-to-day Data problems and customer challenges
- Own the automation, deployment and operation of data pipelines on MS Azure
- Build tools and mechanism to monitor and optimize different parts of the systems
- Build custom integrations between cloud-based systems using APIs
Skills and qualifications
- Required: Expertise in ETL/ELT tools like Informatica/DataStage/Ab initio. And exposure to reporting tools like PowerBI/ Tableau etc.
- Familiarity with Linux/Unix scripting, Python, SQL Queries and Database concepts required. Exposure to cloud onboard from legacy data sets.
- Fluent and fast with T-SQL, query analysis and optimization. Deep skills in modeling transactional databases and data warehouses.
- Required: Working experience on Azure technologies like Datalake ADLSGen2, DataBricks, Azure Data factory, Azure SQL, Azure Synapse etc.
- Required: Experience in creating data orchestration using ADF and optimizing them through regular monitoring.
- Good to have: Experience in batch scheduling & rationalization through Control M
- Good to have: Familiarity in administering Azure-based data infrastructures / Windows systems administration (VMs or physical machines), storage, networking, security access, certificate, and compliance management.