Staff Big Data Engineer - Jobs Opendatascience

This is a post on behalf of the AIPlus hiring partner.

Apply with Job ID # 100060

Our hiring partner is a global technology and data company that builds verification, optimization, and analytics solutions for the advertising industry and they’re looking for a Staff Big Data Engineer to join their Data Engineering team. If you are excited by technology that has the power to handle hundreds of thousands of transactions per second; collect tens of billions of events each day, and evaluate thousands of data-points in real-time all while responding in just a few milliseconds, then this is the place for you!

As a Staff Big Data Engineer you will lead, design, implement and maintain big data pipelines responsible for aggregating tens of billions of daily transactions. You will lead the entire software development lifecycle including hands-on development, code reviews, testing, and deployment for streaming, batch ETL, and RESTful APIs. You will be using cutting-edge Big Data technologies on cloud platforms. As a senior contributor, you will help guide, mentor, and provide technical leadership to junior team members.

What you’ll get to do:

Planning delivery of data pipelines and API endpoints for the company product offerings
Architect and build data pipelines and data stores specialized for data science needs
Design architecture to streamline data science SDLC
Design and implement systems for large-scale data analysis and machine learning
Build tools and frameworks for use across the data science and data engineering teams
Define and implement best practices and development processes
Partner with engineering teams on deployments of data science-driven solutions
Participate in training and mentoring of more junior data engineers
The ideal candidate is naturally curious, dedicated, detail-oriented with a strong desire to work with awesome people in a highly collaborative environment.

You should apply if you have most of this experience:

Bachelors or Masters in Computer Engineering, Computer Science, Electronics Engineering, or related fields
- 10+ years of experience designing and building data-intensive applications
- Hands-on coding experience in an object-oriented language, preferably Java/Scala/Python
- In-depth understanding of algorithms, scalability, and various tradeoffs in a Big Data setting
- Expertise in using Big Data frameworks (e.g., Hadoop, Spark) and MPP databases (e.g., Redshift, Snowflake) for complex data assembly and transformation
- Experience building production level systems in a cloud environment (AWS, Azure, or GCP)
- Orchestrating data pipelines using tools such as Airflow
- Knowledge of database technology, SQL, schema design, and query optimization techniques
- Understanding of full software development life cycle, agile development, and continuous integration
- Excellent interpersonal and communication skills
What puts you over the top:
- Digital advertising or web technology experience
- Experience with Spark streaming or Flink
- Experience implementing machine learning algorithms
- Experience with MLOps tools
About the company

Our partner is the global market leader in digital ad verification, offering technologies that drive high-quality advertising media. The company equips advertisers and publishers with both the insight and technology to protect their advertising investments from fraud and unsafe environments as well as to capture consumer attention, and drive business outcomes. Founded in 2009, they are headquartered in New York with global operations in 18 offices across 13 countries. The company is part of the Vista Equity Partners portfolio of software companies.

Equal Opportunity Employer:

Our hiring partner is an equal opportunity employer, committed to their diversity and inclusiveness. They will consider all qualified applicants without regard to race, color, nationality, gender, gender identity or expression, sexual orientation, religion, disability, or age. They strongly encourage women, people of color, members of the LGBTQIA community, people with disabilities, and veterans to apply.

More Information

Salary Offer 0 ~ $3000
Experience Level Junior
Total Years Experience 0-5
Dropdown field Option 1