Senior Data Engineer - Jobs Opendatascience

This is a post on behalf of the AIPlus hiring partner.

Apply with Job ID # 100061

Our hiring partner is a global technology and data company that builds verification, optimization, and analytics solutions for the advertising industry and they’re looking for a Senior Data Engineer to join their Data Engineering team. If you are excited by technology that has the power to handle hundreds of thousands of transactions per second; collect tens of billions of events each day, and evaluate thousands of data-points in real-time all while responding in just a few milliseconds, then this is the place for you!

As a Senior Data Engineer, you will design, implement, and maintain big data pipelines responsible for aggregating tens of billions of daily transactions. You will be part of a growing team to develop and mature their capability in video ad verification, analytics, and anti-ad fraud software products. The ideal candidate is naturally curious, dedicated, detailed-oriented with a strong desire to work with awesome people in a highly collaborative environment. You should be able to not take yourself too seriously as well.

What you’ll do:

Architect, design, code and maintain components for aggregating tens of billions of daily transactions
Work on Big Data technologies such as Hadoop, MapReduce, Kafka, and/or Spark in columnar databases on AWS
Contribute to the entire software lifecycle including hands-on development, code reviews, testing, deployment, and documentation for streaming & batch ETL and RESTful APIs
Provide leadership, work collaboratively, and be a mentor in an awesome team

You should apply if you have most of this:

Bachelors or Masters in Computer Engineering, Computer Science, Electrical Engineering or related field
5+ years of recent hands-on in an object-oriented language (Java, Scala, Python)
5+ years of experience designing and building data pipelines and data-intensive applications
Experience using Big Data frameworks (e.g., Hadoop, Spark, MapReduce) and MPP databases (e.g., Redshift, Snowflake) for complex data assembly and transformation
Strong knowledge of collections, multi-threading, JVM memory model, etc.
In-depth understanding of algorithms, performance, scalability, and reliability in a Big Data setting
Good knowledge of SQL
Experience in full software development, Agile, and continuous integration/deployment
Excellent interpersonal and communication skills

What puts you over the top:

Experience building production level systems in a cloud environment (AWS, Azure, or GCP)
Building event-driven microservices applications using spring boot or gRPC
Orchestrating data pipelines using tools such as Airflow
Familiarity with messaging frameworks like Kafka or RabbitMQ
Experience with Spark streaming or Flink
Solid understanding of OLTP and OLAP systems, database fundamentals

About the company

Our partner is the global market leader in digital ad verification, offering technologies that drive high-quality advertising media. The company equips advertisers and publishers with both the insight and technology to protect their advertising investments from fraud and unsafe environments as well as to capture consumer attention, and drive business outcomes. Founded in 2009, they are headquartered in New York with global operations in 18 offices across 13 countries. The company is part of the Vista Equity Partners portfolio of software companies.

Equal Opportunity Employer:

Our hiring partner is an equal opportunity employer, committed to their diversity and inclusiveness. They will consider all qualified applicants without regard to race, color, nationality, gender, gender identity or expression, sexual orientation, religion, disability, or age. They strongly encourage women, people of color, members of the LGBTQIA community, people with disabilities, and veterans to apply.

More Information

Salary Offer 0 ~ $3000
Experience Level Junior
Total Years Experience 0-5
Dropdown field Option 1