Lead Cloud Engineer, Big Data 1096 views

Job Overview

Macys is looking to hire experienced Lead (Big Data) Cloud Engineer to play a key role in designing and developing batch and real time data acquisition pipelines and analytics platform using latest open source big data technologies. In this role candidate will report into DaaS Director in our west coast (San Francisco) office. Perform other duties as assigned.

Data as a Service (DaaS) is an enterprise platform team that organizes Macy’s data and makes it available for applications and business units. It collects, correlates, enriches, and manages Macy’s core data to produce single view of truth for use at speed and at scale. We are passionate technologists who thrive on simple and elegant architecture and agility. We own the architecture and development of comprehensive data management, bringing real-time operational use, near real-time visibility use, and off-line deep analytical use together in a cohesive and holistic way. Innovation and engineering prowess is our core to stay ahead of our user’s demand and the industry changing landscape. Come get challenged in a fast, agile environment with other A players.

Essential Functions

• Design and develop scalable data pipelines for optimal ingestion, transformation, storage and computation using latest big data technologies.
• Work with engineering, product, and business team(s) to understand requirements, evaluate new features and architecture to help drive decisions.
• Design, develop and maintain automated ETL processes for efficient batch records-matching of multiple large-scale datasets.
• Translate complex business requirements into a scalable, efficient and high availability data platform. Strong ownership of the full backend stack – from design through to deploy and beyond.
• Experience working with continuous integration (CI/CD) framework, building regression-able code within data world using GitHub, Jenkins, Cloud Build, Spinnaker and related applications.
• Actively participate in code review and test solutions to ensure it meets best practice specifications. Prototype ideas quickly using cutting edge and new generation technologies.
• Build collaborative partnerships with architects, technical leads and key individuals within other cross functional groups. Work with business teams/engineers to define instrumentation and data requirements.
• Lead projects independently and also mentor team members and other engineers in the team.
• Regular, dependable attendance & punctuality.

Education/Experience

Qualifications:

• Bachelor or Master’s degree or equivalent in computer science or related field with minimum of 5+ years of directly related work experience.
• Good understanding of data structures and algorithms.
• Programming experience in Python, Java or Scala.
• Extensive experience with Elastic stack, Kafka and Spark.
• Experience with big data technologies like Hadoop, Map Reduce, Hive, Spark/Storm, Big Query.
• Experience with time series and NoSQL databases is a plus.
• Experience working with containers and kubernetes is good to have.
• Experience designing, implementing and scaling big data solutions on public cloud.
• Experience with Unix/ Linux and shell scripting.
• Experience working with various data sources like unstructured data files, flat files, message queues, JSON based events, databases.
• Experience driving architectural discussions and proposing solutions for system and product changes.
• Experience writing high-performance, reliable and maintainable code.
• Experience with large distributed services is a plus as is building/operating highly available systems.

Communication Skills

• Able to effectively communicate across teams and roles.
• Excellent verbal and written communication skills.

Mathematical Skills

• Basic math functions such as addition, subtraction, multiplication, division, and analytical skills.

Reasoning Ability

• Proven ability as a problem-solver.

Physical Demands

• This position involves regular walking, standing, sitting for extended periods of time, hearing, and talking.
• May occasionally involve stooping, kneeling, or crouching.
• May involve close vision, color vision, depth perception, focus adjustment, and viewing computer monitor for extended periods of time.
• Involves manual dexterity for using keyboard, mouse, and other office equipment.
• May involve moving or lifting items under 10 pounds.

Other Skills

• Strong estimating and planning skills
• Ability and desire to thrive in a proactive, high-pressure, client-services environment
• Proven ability to effectively work across multiple teams on solutions that have complex dependencies and requirements in a fast-paced environment
• Ability to produce clear, well-documented designs for product features.
• Proven track record of strong analytical and problem-solving skills.
• Ability to adapt to new development environments, changing business requirements.

Work Hours

• Ability to work a flexible schedule based on department and company needs.

Company Profile

As the fastest growing part of Macy’s Inc. business, macys.com is achieving record sales and broadening our workforce. Macys.com offers the entrepreneurial culture of a web business with the stability and support of the best brand in retailing. Creativity and ingenuity partner with business acumen and tech savvy to build a unique business poised for substantial growth. If you’re interested in being a part of that growth and want to know what it’s really like to work at macys.com, get an inside look at http://ecommerce.macysjobs.com/

Our employees have long-term opportunities and are encouraged to utilize their Supervisors and Human Resources for cross-functional movement to further their careers. At macys.com we are committed to giving back to the community by partnering with local charitable organizations. By skillfully combining the power of digital technology and omnichannel integration with the best in retailing, macys.com is reaching new heights.

This job overview is not all inclusive. In addition, Macy’s, Inc. reserves the right to amend this job overview at any time. Macy’s is an Equal Opportunity Employer, committed to a diverse and inclusive work environment. Macy’s, Inc. – including Macy’s and Bloomingdale’s – will consider for employment qualified applicants with criminal convictions in a manner consistent with SFPC Art. 49 and LA MC ch.XVIII Art. 9.