Open Data Science job portal

Cloud DevOps Engineer 302 views

Job description

Nielsen Connect is making a large cloud-based data platform of petabyte scale. The platform is heavily based on distributed systems and horizontally scalable big-data technologies including Apache Spark, Apache Nifi, Kubernetes, and many more. It primarily exists on the Microsoft Azure cloud and its underlying technologies such as Azure Data Lake Store (ADLS) , Azure Kubernetes Service (AKS), etc.

Primary Responsibilities Of The Group Include

The candidate for this role will work in our Cloud DevOps group. This group manages the cloud as a whole for the many application teams.

    • Creation of all cloud resources (including user permissions management).
    • Automation of infrastructure provisioning via Packer, Terraform, Ansible, etc.
    • Development of sites/tools to support infrastructure (Java/Angular/Python/SQL/etc).
    • Design of data lakes and related components (security, scalability, fault tolerance, etc).
    • Design and enforcement of application and data security rules.
    • Training of application teams to do their own devops work in lower-environments.

The candidate will need to be a fast learner and open to trying new things. The devops team designs, deploys, and manages virtually every technology on the cloud, and it is impossible to know them all in advance. Strong fundamentals with the Linux OS and distributed systems will be required to enable the candidate to adopt technologies at the required pace.

Key Requirements

    • Experience working with Linux, preferably CentOS/RHEL
    • Experience with CI/CD – e.g. Jenkins Pipelines or GitLab CI
    • Some knowledge of databases/SQL
    • Some understanding of distributed systems
    • Understanding of high availability concepts

Differentiating Skills

    • Java, Angular, Python, or Spark development
    • Cloud experience with Azure (or AWS/GCP)
    • Infrastructure automation (Terraform, Packer, Ansible)
    • Application automation (Ansible, Puppet, Chef, or similar)
    • Networking knowledge
    • General system administration skills
    • Prior experience with Data Lake design

Industry

  • Market Research
  • Management Consulting

Employment Type

Full-time

Job Functions

  • Other

More Information

Share this job
Company Information
Connect with us
Contact Us
https://jobs.opendatascience.com/wp-content/themes/noo-jobmonster/framework/functions/noo-captcha.php?code=14c07

Here at the Open Data Science Conference we gather the attendees, presenters, and companies that are working on shaping the present and future of AI and data science. ODSC hosts one of the largest gatherings of professional data scientists with major conferences in the USA, Europe, and Asia.