Pinterest on AWS
Pinterest is a visual discovery engine that hosts billions of images for over 450 million users to explore, save, and share as “Pins” to personalized digital inspiration boards. Born in the Amazon Web Services (AWS) Cloud, Pinterest can scale processing, storage, and analysis of its rapidly increasing data, all while reducing infrastructure management and focusing on innovation. Using compute solutions from AWS, Pinterest migrated its iOS continuous integration and continuous delivery (CI/CD) pipeline from on-premises to reduce build failures by over 80 percent. Pinterest also uses machine learning (ML) to power its visual search tool Pinterest Lens that can recognize over 2.5 billion objects and match them to products. Today, Pinterest’s exabyte data platform runs entirely on AWS, scaling log search and analytics to over 1.7TB while reducing operations costs by 30%.
-
Pinterest re:Invent 2023 Recap
-
Storage
-
Data Solutions
-
Machine Learning
-
Migration
-
Cost Optimization
-
Security
-
Sustainability
-
Pinterest re:Invent 2023 Recap
-
2023
AWS Graviton: The best price performance for your AWS workloads
AWS Graviton-based Amazon EC2 instances provide the best price performance for workloads in Amazon EC2. In this session, discover AWS Graviton processor and review workload performance, AWS customer success stories, energy efficiency, and software offerings. Learn about best practices to optimize workloads on AWS Graviton across various applications and how you can get started with your AWS Graviton journey.
2023Large model training on AWS Deep Learning AMIs & PyTorch, ft. Pinterest
Join this session for a deep dive into the infrastructure used to train large models at Pinterest. It covers training hardware, compute orchestration, and ML application development SDKs as well as how infrastructure choices speed up development and improve model training efficiency. See how Pinterest integrates AWS services like Amazon EC2 UltraClusters and AWS Deep Learning AMIs to reduce the total cost of ownership for ML infrastructure.
2023Pinterest extends existing data lake with generative AI
Pinterest is the visual inspiration platform where people come to search, save and shop the best ideas in the world. Hear from Dave Burgess, Vice President of Data Engineering at Pinterest, about their exabyte-scale data lake and how they are putting that data to work with generative AI.
-
Storage
-
2024
Spark on Amazon EKS networking – Part 1
In this post, we show you how Pinterest and AWS went about gathering requirements and forming tenets for a networking design project to support large scale, real-world deployments. The process was key to defining the scope of the problem, establishing the boundaries to operate within, aligning with stakeholders on requirements, and establishing a process for collaborating across teams to achieve our goals.
2024Spark on Amazon EKS networking – Part 2
In this post, we show you an iterative and collaborative approach as a strong foundation for Pinterest’s new data platform, but also for the future of Pinterest on the cloud. This is a point in time, and this design serves us well, but we can and will iterate with new technologies, different workloads, and new scale.
2023Addressing tech debt and critical data back up at Pinterest
Learn how Ambud Sharma, tech lead and engineering manager at Pinterest, discovered his passion for tech and how his research in distributor systems eventually led him toward building them from scratch. Ambud also explains how Amazon Simple Storage Service (Amazon S3) has been foundational for some of Pinterest's work, describing it as a game changer for its on-going work.
2022Build With Me – Pinterest and AWS
Learn how Ambud Sharma, tech lead and engineering manager at Pinterest, discovered his passion for tech and how his research in distributor systems eventually led him toward building them from scratch. Ambud also explains how Amazon Simple Storage Service (Amazon S3) has been foundational for some of Pinterest's work, describing it as a game changer for its on-going work.
2022How Pinterest Uses AWS to Create a System to Speed Up Development Times
Looking to enable its users while deploying analytic tools, Pinterest identified that those users should have permission in order to access the data. Pinterest used Amazon S3 and AWS Identity and Access Management (IAM) to create a fine grain access control system to ensure the correct user permissions were in place which, in return, improved development time for the internal users.
2021How Pinterest Worked with AWS to Create a New Way to Manage Data Access
With thousands of engineers and developers working on its platform, Pinterest needed to restrict data access to specific users and processes, turning to AWS for help building a solution. Learn about the collaboration between Pinterest and AWS to develop the scalable and secure Fine Grain Access Control (FGAC) system for Pinterest’s data on Amazon S3 and how FGAC helps Pinterest amplify underrepresented creators.
2021How Pinterest Uses Amazon S3 Glacier Deep Archive to Manage Storage for its Visual Discovery Engine
As a large-scale user of Amazon S3, Pinterest stores billions of objects and nearly an exabyte of data across multiple AWS Regions. Learn how Pinterest uses Amazon S3 Lifecycle to assign data to optimal Amazon S3 storage class assignments, helping meet large-scale cost goals and maximize storage efficiency.
2021Tips for Effective Innovation Collaboration from Pinterest and AWS
Keith Regier, Pinterest engineering manager, and Doug Youd, AWS senior solutions architect, share six collaboration tips gleaned from their experience working together to build Pinterest’s FGAC system. Discover actionable insights, including how to clearly define a problem, why even “bad” ideas can be useful, and what AWS can do to help customers like Pinterest develop solutions to tough technical challenges.
As a visually driven platform, Pinterest relies heavily on the speed and quality of images. But, the text behind those images is just as important as it provides context and makes Pins actionable for our 200 million active 'Pinners.' By working with the Amazon Rekognition Text in Image feature, we can better extract the rich text captured in images at scale and with low latency for the millions of Pins stored in Amazon S3."
Vanja Josifovski
Chief Technology Officer, Pinterest
-
Data Solutions
-
2022
How Pinterest Engaged with AWS Proserve to Ensure Pinners have Always-On Access to Its Platform
Growing rapidly with large amounts of data, Pinterest identified the need to establish a disaster recovery plan. With existing solutions already on AWS, Pinterest engaged with AWS Professional Services to build on top of those solutions and take its nearly exabyte of data, version it, and distribute it regionally.
-
Machine Learning
-
2022
Pinterest Lens Helps Users Find and Buy the Perfect Item
Pinterest builds on AWS storage and compute solutions to power the ML engines behind the Lens camera feature on its app, which is used to conduct hundreds of millions of visual searches each month. Learn how Lens can be used as a digital shopping concierge to help users source and purchase items in minutes.
Pinterest is continuously developing machine learning systems to detect objects for visual search and moderation use cases. To accomplish this, we need to label millions of images to generate the required training datasets. Pinterest has an existing labeling platform that has integrated Amazon services like Amazon Mechanical Turk. We were excited to explore using SageMaker Ground Truth to extend this platform to support bounding box labeling tasks. We found SageMaker Ground Truth provides a simple, streamlined interface to kick off labeling jobs."
Veronica Mapes
Technical Program Manager, Pinterest
2018Deep Dive on Amazon Rekognition, featuring Pinterest
Learn how Pinterest can easily add intelligent image and video analysis to applications using Amazon Rekognition in order to automate manual workflows, enhance creativity, and provide more personalized customer experiences.
-
Migration
-
2021
Pinterest Improves iOS Build Pipeline Reliability by 80.5% Using Amazon EC2 Mac Instances
In this case study, learn how Pinterest migrated its continuous integration and continuous delivery (CI/CD) pipeline for iOS applications from on-premises computers to Amazon Elastic Compute Cloud (Amazon EC2) Mac Instances. On AWS, Pinterest developers can now seamlessly provision access to macOS compute environments in the cloud, reducing machine-related build failures by 80.5 percent and improving development speeds by 18.4 percent.
-
Cost Optimization
-
2023
Pinterest implements a cost-effective strategy by leveraging AWS
Understand how Pinterest is addressing challenges around their compute footprint relating to availability and cost effectiveness. Molly Junck, Infrastructure Governance Technical Program Manager at Pinterest shares how working closely with the Amazon Web Services (AWS) account team has contributed towards how Pinterest learning to address their evolving technology needs as they've grown and scale.
2020Pinterest Scales Daily Log Search and Analytics from 500 GB to 1.7 TB, Reduces Costs by 30% on Amazon OpenSearch Service
Read how Pinterest migrated its log and search analytics workloads from self-managed and third-party Elasticsearch tools to Amazon OpenSearch Service. Following the migration, Pinterest scaled its daily data-ingestion capabilities from 500 GB to 1.7 TB in only 1 year while reducing operational costs by 30 percent, improving data security, and increasing engineer productivity.
-
Security
-
2024
Achieving Zero Trust Security on Amazon EKS with Istio
In this post, we cover Istio’s security mechanisms, which allow us to implement a true zero trust security architecture on Amazon EKS. In addition to the built-in security features for Request Authentication, Peer Authentication, and Ingress security, we learned to leverage add-on tools such as Keycloak and OPA.
2022How Pinterest Engaged with AWS Proserve to Ensure Pinners have Always-On Access to their Platform
How Pinterest engaged with AWS Proserve to ensure Pinners have always-on access to their platform.
2022Scalable Access Control with STS Token Vending
In this video, Keith Regier, engineering manager at Pinterest, talks about the scalable access control solution for clusters of Amazon EC2 instances that need access to data in Amazon S3. Learn how Pinterest built a token vending service using AWS Secure Token Service (STS) that helps to bridge its internal concept of users and authentication to managed policies and AWS Identity and Access Management (IAM).
At Pinterest we use Amazon Elastic Container Registry (Amazon ECR) for managing our Docker container images. We use Amazon ECR’s image scanning feature to help us improve security of our container images. Amazon ECR scans images for a broad range of operating system vulnerabilities and lets us build tools to act on the results.”
Cedric Staub
Engineering Manager, Pinterest
-
Sustainability
-
2023
Pinterest and AWS Graviton: more performance and lower carbon footprint
Pinterest provides inspiration for millions of Pinners to create a life they love. To do so, Pinterest wants to provide the latest innovations to Pinners while becoming more responsible on sustainability. Through its adoption of AWS Graviton, Pinterest is able to do more with less. Building on AWS gave Pinterest the flexibility to have the latest technology, the support of a wonderful best-in-class team, and a fearlessness of approaching any challenge together.
2024Improving Sustainability and Price Performance Using AWS Graviton–Based Instances with Pinterest
Learn how Pinterest improved infrastructure cost efficiency and environmental sustainability using AWS Graviton–based instances.
2022Sustainability and AWS Silicon
Learn how sustainability is integrated into Pinterest’s AWS architecture decisions and how AWS continues to innovate on chip design as the organization works toward Amazon’s goal of achieving net-zero carbon by 2040.
About Pinterest
Pinterest is a visual-discovery platform and social commerce network with a mission to inspire. Building on AWS storage and compute solutions, Pinterest uses sophisticated machine learning engines to deliver personalized content to its users.