Cloud DevOps Engineer 1080 views

Nielsen Connect is making a large cloud-based data platform of petabyte scale. The platform is heavily based on distributed systems and horizontally scalable big-data technologies including Apache Spark, Apache Nifi, Kubernetes, and many more. It primarily exists on the Microsoft Azure cloud and its underlying technologies such as Azure Data Lake Store (ADLS) , Azure Kubernetes Service (AKS), etc.

Primary Responsibilities Of The Group Include

The candidate for this role will work in our Cloud DevOps group. This group manages the cloud as a whole for the many application teams.

• • Creation of all cloud resources (including user permissions management).
  • Automation of infrastructure provisioning via Packer, Terraform, Ansible, etc.
  • Development of sites/tools to support infrastructure (Java/Angular/Python/SQL/etc).
  • Design of data lakes and related components (security, scalability, fault tolerance, etc).
  • Design and enforcement of application and data security rules.
  • Training of application teams to do their own devops work in lower-environments.

The candidate will need to be a fast learner and open to trying new things. The devops team designs, deploys, and manages virtually every technology on the cloud, and it is impossible to know them all in advance. Strong fundamentals with the Linux OS and distributed systems will be required to enable the candidate to adopt technologies at the required pace.

Key Requirements

• • Experience working with Linux, preferably CentOS/RHEL
  • Experience with CI/CD – e.g. Jenkins Pipelines or GitLab CI
  • Some knowledge of databases/SQL
  • Some understanding of distributed systems
  • Understanding of high availability concepts

Differentiating Skills

• • Java, Angular, Python, or Spark development
  • Cloud experience with Azure (or AWS/GCP)
  • Infrastructure automation (Terraform, Packer, Ansible)
  • Application automation (Ansible, Puppet, Chef, or similar)
  • Networking knowledge
  • General system administration skills
  • Prior experience with Data Lake design