We all know that change is inevitable. Yet, for cloud service providers like AWS, introducing even the smallest change can ripple across thousands of businesses, disrupting operations and workflows. In a recent Breaking Changes episode, Becky Weiss, Vice President / Distinguished Engineer at AWS, shared her insights with Postman’s Jean Yang on how AWS managed to introduce a potentially breaking change in S3—not only successfully, but in a way that greatly enhanced security and usability for customers. Her story offers valuable lessons for tech leaders looking to implement significant changes in their own organizations while minimizing disruption.
The challenge: implementing change in a long-standing system
AWS’s Simple Storage Service (S3) has been around since 2006, and the scale and scope of its operations are staggering. According to Weiss, “If it is not literally true that 100% of AWS customers are storing things in S3, it’s pretty darn close to true.” This ubiquity means that any changes introduced to S3 affect nearly every AWS customer. Let’s take a look at some of the key takeaways for tech leadership based on Weiss’s experience navigating this challenge.
Understand the evolution of your product and how customers use it
One of the critical realizations was that the way customers use S3 has changed dramatically over the years. Initially conceived as “storage for the Internet,” S3 now serves as secure, scalable cloud storage for enterprise data analytics, among other use cases. Recognizing this shift, AWS knew that the default security settings needed to evolve to meet current customer needs.
“Would most of our customers today think of S3 as storage for the Internet? No, S3 for them is storage in the cloud for their own data where they store their data, and they do analytics on it. That’s the same S3… but it’s a very, very different S3.”
—Becky Weiss
Advice for leadership: Regularly evaluate how your product is being used by your customers, and be ready to adapt your offering to align with their evolving needs. According to Salesforce’s State of Service Report, 73% of customers expect that companies will understand and cater to their unique needs. Staying static in a changing landscape risks becoming obsolete.
Prioritize security but avoid disruption
For AWS, security was always a top concern. Weiss emphasized that “by default, S3 buckets have always been private,” but the introduction of IAM (Identity and Access Management) in 2011 presented a more scalable and efficient way to handle permissions compared to the older ACL (Access Control List) system. However, AWS couldn’t just enforce a switch overnight.
In 2021, AWS introduced the option for customers to disable ACLs, a move that Weiss described as a significant but gradual transition toward a more secure S3 environment. This decision wasn’t made lightly; it involved years of preparation, incremental steps, and feedback from customers to ensure the change didn’t cause disruption.
Advice for leadership: When planning a significant change, prioritize customer security, but avoid imposing abrupt shifts that could disrupt workflows. Gradually introduce the change, and provide options to accommodate different user needs.
Make the path to success as easy as possible for customers
One of the more striking parts of Weiss’s story is how AWS focused on making security improvements effortless for customers. She used the phrase “fall into the pit of success,” meaning that AWS aimed to make it nearly impossible for customers to get things wrong, even if they weren’t paying attention.
“That’s what winning looks like where I work,” Weiss said. This approach was evident in features like the Block Public Access tool introduced in 2018, which allowed customers to have absolute confidence that their data wasn’t being inadvertently shared publicly. By creating features that automatically guided customers toward best practices, AWS ensured that even those who were less familiar with AWS’s complexity could achieve optimal security. This approach not only enables customers, it also helps prevent churn—a critical factor in long-term SaaS growth—leading to higher satisfaction and potential upsell opportunities (Matrix Partners).
Advice for leadership: Design your solutions in a way that makes it difficult for users to make mistakes. The easier you make it for them to adopt best practices, the more likely they are to succeed.
Use data and visibility to build confidence in change
Introducing changes, especially security-related ones, requires visibility and proof that the changes are beneficial and won’t introduce new risks. AWS tackled this by incorporating logging and instrumentation features, enabling customers to monitor how changes affected their setups. As Weiss explained, they provided tools that allowed customers to check if their workloads depended on ACLs before disabling them.
This transparency empowered customers to make informed decisions. AWS also added ACL data to S3 bucket inventories, offering clear insights into whether any non-standard configurations existed. This gave customers the confidence to proceed with changes.
Advice for leadership: Provide your customers with clear visibility into how changes will impact them, and offer tools to monitor and validate the effects of these changes. This approach builds trust and minimizes resistance to adopting new practices.
Communicate, communicate, communicate
One of the most understated yet critical components of AWS’s success in implementing this change was their communication strategy. Weiss highlighted how they sent out emails informing customers of the changes and explaining what actions, if any, were required on their end. But she also revealed that “a ton of work goes on behind the scenes to make sure that the right people are getting emails that are going to be actionable to minimize the amount of action that’s actually required.”
Advice for leadership: Effective communication is key when introducing changes, especially those that impact a wide user base. Tailor your messaging, ensure it’s sent to the right stakeholders, and provide clear, actionable steps to avoid confusion.
Embrace iterative and customer-centric development
Finally, AWS’s journey with this change wasn’t preordained. They didn’t have a “crystal ball,” as Weiss put it. Instead, they iteratively introduced features, gathered feedback, and adapted. For example, AWS released the option to disable ACLs in 2021, but only changed the default settings after observing customer adoption patterns and ensuring the transition was smooth.
Advice for leadership: Accept that not all aspects of change can be foreseen. For example, according to research by Boston Consulting Group, 80% of enterprise resource projects take longer than expected. Leaders can mitigate this risk by approaching change iteratively, collecting feedback, and remaining flexible to adjust your path based on real-world usage and customer needs.
Key takeaways
AWS’s experience in implementing a seemingly small but impactful change to S3’s security model offers a powerful example of how to handle breaking changes at scale. It emphasizes the importance of understanding your product’s evolution, prioritizing security without disrupting customers, and making the path to success as seamless as possible.
For tech and engineering leaders looking to introduce change in their own organizations, these lessons are invaluable. By planning meticulously, iterating based on feedback, and building features that guide customers to the “pit of success,” you can implement changes that not only improve your product but also enhance trust and loyalty among your user base.
In the words of Weiss, “It’s one of the wonderful things about being a customer-obsessed company… We are very good at kind of listening and watching where our customers are going and anticipating what their next need is going to be.” Follow this approach, and your organization can navigate change with the same level of success.
For more of Becky Weiss’s insights, be sure to check out the full episode, “Modernizing Storage for the Internet.” Learn more wisdom from industry experts by subscribing to Breaking Changes on Apple, Spotify, and YouTube.