Data Engineering Senior Advisor 

Job Description & Responsibilities: 

 

Work with business and technical leadership to understand requirements 

Design to the requirements and document the designs 

Ability to write product-grade performant code for data extraction, transformations and loading using Spark, Py-Spark 

Do data modeling as needed for the requirements 

Write performant queries using Teradata SQL, Hive SQL and Spark SQL against Teradata and Hive 

Implementing dev-ops pipelines to deploy code artifacts on to the designated platform/servers like AWS or Hadoop Edge Nodes 

Implement Hadoop job orchestration using Shell scripting, Apache Oozie, CA7 Enterprise Scheduler and Airflow  

Troubleshooting the issues, providing effective solutions and jobs monitoring in the production environment 

Participate in sprint planning sessions, refinement/story-grooming sessions, daily scrums, demos and retrospectives 

 

 

 

Experience Required: 

 

Overall 13 - 16 Years 

 

 

Experience Desired: 

 

Experience in Jira and Confluence 

Health care domain knowledge is a plus 

Excellent work experience on Hadoop as data warehouse 

Experience in Agile and working knowledge on DevOps tools 

 

 

 

Education and Training Required: 

 

Primary Skills: 

 

Spark, Py-Spark, Shell scripting, Teradata, Hive and Hadoop 

SQLs (using Teradata SQL, Hive SQL and Spark SQL) and Stored Procedures 

Git, Jenkins, Artifactory 

Unix/Linux Shell scripting (KSH) and basic administration of Unix servers 

CA7 Enterprise Scheduler 

AWS (S3, EC2, SNS, SQS, Lambda, ECS, Glue, IAM, and CloudWatch) 

Databricks (Delta lake, Notebooks, Pipelines, cluster management, Azure/AWS integration) 

 

 

 

Additional Skills: 

 

Exercises considerable creativity, foresight, and judgment in conceiving, planning, and delivering initiatives. 

 

About Evernorth Health Services

Evernorth Health Services, a division of The Cigna Group, creates pharmacy, care and benefit solutions to improve health and increase vitality. We relentlessly innovate to make the prediction, prevention and treatment of illness and disease more accessible to millions of people. Join us in driving growth and improving lives.