S&P Global ETL/Data Engineer - Tech Lead in New York, New York
The ETL Tech Lead will design, develop and deliver strategic data-centric applications leveraging the firm s next generation ETL technologies with a strong focus on tools like Informatica etc. These solutions will be architected in alignment with the underlying S&P Global technology infrastructure, as well as the foundational Data Services framework.
This role is a hands-on technical role.
Provide leadership for all stages for the software development - maintenance for the Business Intelligence application project portfolio. This includes oversight for the design, specification, ongoing maintenance and roadmap of this project portfolio:*
Development of Mappings, data flows
Data structure optimization and integration
Automation, deployment, scheduling & distribution of feeds
Auditing & value realization (usage metrics)
Collaborating with on - off-shore resources
Knowledge about Data Analytics and ability to provide end to end solutions
Ability to take Initiative and Proactively address problems
Partner with enterprise architects to define and ensure proposed Business Intelligence solutions adhere to enterprise reference architecture.
Design robust data centric Business Intelligence solutions that consider technology from a development, operations, business, and vendor management perspective.
Actively participates and represents Business Intelligence Solutions team in meetings and facilitates cross-functional team collaboration.
Create and deliver project communications and presentations to relevant stakeholders
Aligns application systems design with the business voice of the customer and articulates trade-offs and alternative options.
Deliver the optimal mix of approach, process, and technology that best accomplish the goals of an application - project while adhering to enterprise reference architecture.
5 years "hands-on" experience designing Data Integration solutions with the latest ETL and Analytic Tools.
- 5 years "hands-on" experience using Oracle PL - SQL, stored procedures, SQL optimization skills
Proven experience with data integration & Data Analytic tool set(s). Data warehousing skill set is a plus.
Very strong data modeling skills
Experience with Taxonomy (XSD) and generating XML files is a plus.
Must be "hands-on" as well as be able to manage other resources in completing a project;
Experience with software development lifecycle methodologies (Agile preferred)
Strong communication, presentation and interpersonal skills are essential
Technical leadership and mentoring skills
Bachelor's degree and - or Masters degree in computer science or related fields
Ability to collaborate with business and technology teams to create practical, robust and scalable architectures and solutions meeting the business and technology goals - strategy ofof the organization
Hands on with Python ,with prior experience in handling ETL - ELT workload using Python Scripts.
Hands on with loading and manipulating large data sets using Spark - PySpark and SparkSQL.
Prior Experience with consuming - Persisting data from and to Relational databases, S3 and Redshift using Python - PySpark.
Experience with various AWS EMR components ( Hive - HDFS - Spark - EMFRS - Scoop) handling very large data sets in a large Data Lake setup.
Good Understanding of other AWS services like S3 ,EC2 , IAM , RDS Experience with Orchestration and Data Pipeline like AWS Step functions - Data Pipeline - Glue