Open Data Science job portal

Software Engineer, Big Data Workflow Platform 156 views

They’re looking for Big data workflow/orchestration platform engineers (across levels) to help them build and lead the next generation of ETL orchestration platform at Pinterest. You’ll be working on some of the most exciting big data open source technologies (Airflow, Hadoop, Hive, Spark, Presto, etc.), at the scale of hundreds of petabytes of data to help Pinners discover and do what they love.

What You’ll Do

    • Improve and customize the internals of open source big data technologies to meet their challenges at scale
    • Build and scale workflow orchestration platforms using the latest open-source technologies to process petabytes-scale datasets
    • Build and scale systems that orchestrate and execute complex workflows in big-data pipelines
    • Provide reliable, performant workflow engines as services to enable ETL, big data analytics and actionable insights
    • Contribute to the team’s technical vision and long-term roadmap

What They’re Looking For

  • Experience with large scale distributed framework
  • Experience with big data workflow orchestration engines for ETL jobs, such as Airflow, Oozie, Azkaban, etc.
  • Knowledge of container and orchestration frameworks such as Docker and Kubernetes
  • Experiences in one or more open-source Big Data technologies ( Airflow, Hadoop, YARN, Spark, Hive, Presto, SQL, Kafka, Impala, Parquet, HDFS, etc.)
  • Proficiency in one or more programming languages (Python, Java, Scala)

More Information

Share this job
Company Information
Connect with us
Contact Us
https://jobs.opendatascience.com/wp-content/themes/noo-jobmonster/framework/functions/noo-captcha.php?code=34881

Here at the Open Data Science Conference we gather the attendees, presenters, and companies that are working on shaping the present and future of AI and data science. ODSC hosts one of the largest gatherings of professional data scientists with major conferences in the USA, Europe, and Asia.