Data Scientist, Machine Learning Evaluation 1253 views

At Adobe, you will be immersed in an exceptional work environment that is recognized around the world. You will also be surrounded by colleagues who are committed to helping each other grow through their unique Check-In approach where ongoing feedback flows freely. If you’re looking to make an impact, Adobe’s the place for you. Discover what their employees are saying about their career experiences on the Adobe Life blog and explore the meaningful benefits we offer.

The Opportunity

Machine Learning is a critical part of Adobe’s Cloud offering. Adobe Clouds enable customers to build and manage digital content, such as assets, composites, 3D, documents, etc., and digital experience and transformations.

Creative Cloud is focused on visual imagination and creation and includes such well-known products as Photoshop, Illustrator, Lightroom, and Adobe Stock. Document Cloud is centered around the creation and management of textual documents, including Acrobat and Adobe Sign. Experience Cloud is the set of offerings for the large-scale enterprise, covering marketing, advertising, and analytics.

Adobe Sensei powers machine learning and artificial intelligence across all of these Adobe Cloud product lines. This platform enables thousands of applied researchers, millions of users, and billions of content pieces. The objective is to make machine learning offerings a world-class, leading-edge, differentiating product in Adobe Cloud ecosystem. They match the pace, innovation, and excitement of a startup, backed by the resources and infrastructure of Adobe!

Data is a critical piece of developing machine learning solutions. ML data can include the datasets that train algorithms, but also large-scale evaluation datasets which give insights into algorithm behavior that helps to refine and evolve ML products.

Training data – Adobe’s training datasets often contain digital art, photos, videos, text, or audio files, along with some type of additional knowledge data (user behaviors, annotations, manipulations) to enable meaningful machine learning. How do they build these datasets to maximize performance and accuracy for the customer? How do they ensure that their models are robust and unbiased?

Evaluation – How do they assess the quality of their algorithms and models? What is the ground truth they use to judge them? Where do they succeed and how do they fail? These problems are even more subtle in the complex, creative domains of Adobe products – for example, how should they judge the quality of similarity search, where a user finds new images with an image as the query?

They’re looking for data scientists to define, drive, and coordinate training data construction and model evaluation for the Applied Machine Learning team at Adobe. This will involve thinking hard about product quality, the role of machine learning in those products, and the capabilities of large-scale human-judgment tasks to support scalable solutions that generate the fuel for the success of their AI/ML algorithms.

Responsibilities

• Work closely with product, research, and engineering leaders to understand ML applications and define success.
• Interact with APIs to effectively sample raw data to optimize for ML training and evaluation.
• Design tasks and guidelines for human judgment jobs to annotate images, videos, text, audio; and also to measure the performance of machine-learned and other advanced algorithms.
• Help design A/B tests for our cloud offerings and use human judgment vs. end-user A/B tests as appropriate
• Create a winning culture built on collaboration and shared accomplishments- have fun along the way!

Qualifications

• BS or MS in a technical field (Computer Science, Physics, Math, Statistics) OR in a human-centered field such as Psychology, Linguistics, Human-Computer Interaction.
• Exposure to applied machine learning in an enterprise setting.
• Proven experience invalidation and evaluation of machine-learned models.
• Strong quantitative and analytical orientation. Experience in evaluation metrics, Python or R, JSON parsing, custom HTML for visualization.
• Strong qualitative skillset. Experience in taxonomies, survey design.
• Understanding of issues around ground truth and training/evaluation data for ML, including human judgment and A/B testing
• Hands-on experience using third-party judgment services such as Figure 8, Appen, or Amazon Mechanical Turk.

At Adobe, you will be immersed in an exceptional work environment that is recognized throughout the world on Best Companies lists. You will also be surrounded by colleagues who are committed to helping each other grow through their unique Check-In approach where ongoing feedback flows freely.

If you’re looking to make an impact, Adobe’s the place for you. Discover what their employees are saying about their career experiences on the Adobe Life blog and explore the meaningful benefits they offer.

Adobe is an equal opportunity employer. They welcome and encourage diversity in the workplace regardless of gender, race or color, ethnicity or national origin, age, disability, religion, sexual orientation, gender identity or expression, or veteran status.

At Adobe, you will be immersed in an exceptional work environment that is recognized throughout the world on Best Companies lists. You will also be surrounded by colleagues who are committed to helping each other grow through their unique Check-In approach where ongoing feedback flows freely.

If you’re looking to make an impact, Adobe’s the place for you. Discover what their employees are saying about their career experiences on the Adobe Life blog and explore the meaningful benefits they offer.

Adobe is an equal opportunity and affirmative action employer. They welcome and encourage diversity in the workplace regardless of gender, race or color, ethnicity or national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, or any other characteristics protected by law.