PySpark Data Engineer

Key Responsibilities:

· Design, develop, test, deploy, maintain, and optimize scalable data pipelines and integration solutions using PySpark.

· Build and refine data pipelines and analysis tools to support business needs and enhance data-driven insights.

· Collaborate closely with cross-functional teams to understand data requirements and integrate data solutions into our Enterprise Data Platform.

· Conduct requirements gathering, author technical design documents, and create testing plans and scripts to ensure robust data solutions.

· Provide analytical expertise by writing complex SQL queries, optimizing performance, and implementing user-defined functions, views, indexes, and more.

· Facilitate process mapping workshops and support standard operating procedure implementation.