In this season’s finale of Breaking Changes, Jean Yang, Head of Product - Observability at Postman, had the opportunity to speak with Nick Schrock, widely recognized for co-creating GraphQL at Facebook. Schrock is now the driving force behind Dagster, an open-source data orchestrator that aims to revolutionize data infrastructure. Schrock’s journey is rooted in identifying gaps where engineers face unnecessary pain points due to inadequate abstractions and tooling. Through his work, he has been consistently driven by a desire to create tools that unlock new capabilities for software developers.
“Engineers and people building stuff are in pain, and they’re in pain only because of abstraction and tooling problems. And that makes me, like, fundamentally angry. In a productive way, like the Sith Lord of dev tools.”
—Nick Schrock
The shift from GraphQL to data infrastructure
When Schrock left Facebook in 2017, he observed something that felt both familiar and troubling: a gap between the critical importance of data infrastructure and the tools available to manage it. This was similar to the pre-2010 state of web development, where engineers lacked the robust tools necessary for the job. Schrock noted, “It was the biggest mismatch between the criticality and complexity of a problem domain and the tools and processes to address that domain that I’ve ever seen.”
Data infrastructure is the foundation of decision-making in organizations, influencing everything from loan approvals to healthcare pricing. Schrock saw an opportunity to create a tool that could dramatically improve the reliability and transparency of data pipelines.
Advice for leadership: To develop more impactful technologies, identify industry pain points and envision how they can be addressed with better abstractions.
The birth of Dagster: orchestrating data the right way
Schrock’s latest project, Dagster, aims to be a critical part in the data ecosystem. Dagster serves as an orchestration layer that invokes computational runtimes and connects to every storage layer, giving engineers a unified platform for managing data workflows. Think of it this way: If GraphQL acted as a translator between clients and backends, Dagster functions more like the conductor of a data orchestra, ensuring all components play in harmony.
From internal tool to game-changer
Inception: solving Facebook’s mobile data problem
The story of GraphQL began as an internal Facebook project, created to solve the challenges of rebuilding Facebook’s mobile apps. Back in 2012, the company faced immense pressure to transition from HTML5 to native apps for iOS and Android. The existing data-fetching mechanisms were not sufficient to support the complexity and performance needs of the mobile experience.
Prototype phase: a rapid solution under intense pressure
GraphQL was developed in a matter of weeks. The prototype provided a thin but powerful wrapper over Facebook’s internal data-fetching stack, allowing engineers to access complex, hierarchical data structures far more efficiently than before.
What made it work
GraphQL’s early success came from building a tool that directly addressed the needs of product engineers working under intense time pressure. This alignment between technical innovation and business urgency ensured that GraphQL was rapidly adopted by the internal teams at Facebook.
Sometimes, the most impactful solutions arise from the most pressing problems. By focusing on a specific challenge and user group, Schrock and his team were able to build something that not only worked but transformed how data was handled at scale.
From internal adoption to industry standard
How did GraphQL spread beyond Facebook? It wasn’t the usual “convince everyone” approach. Instead, Schrock focused on the teams that mattered most at the time—the ones building Facebook’s News Feed.
“We didn’t convince everyone. We convinced the people who mattered at that time.”
—Nick Schrock
Instead of open-sourcing a codebase, as many open-source projects do, the team released a detailed specification. This allowed other developers to implement GraphQL in their preferred languages, accelerating its adoption. GitHub’s use of GraphQL for its public API was a defining moment, solidifying its relevance beyond Facebook.
Advice for leadership: Are there opportunities in your organization to create a “first customer” that will drive the adoption of new technologies across teams?
Cultivating a community
One of the more unconventional aspects of GraphQL’s journey was Schrock’s initial skepticism about open-sourcing it. This move turned out to be a strength. By focusing on the specification, they enabled a vibrant community to build their own versions, driving widespread adoption.
When Facebook eventually released GraphQL as a specification, it relied heavily on the community to build various implementations. Apollo and Graphcool were two of the early startups that took the mantle of driving GraphQL’s growth and adoption outside of Facebook.
Open-sourcing a technology doesn’t always mean handing over code—it can be just as effective, if not more, to provide a comprehensive spec that allows others to build their own versions.
Leadership as a “quarterback”
Schrock’s transition from being the CEO to the CTO of Dagster Labs offers valuable insights into how technical leaders can maximize their impact. He compares his role to that of a “quarterback,” as opposed to the head coach.
“My job is to score the touchdown. I’m not dealing with administrative aspects, but I’m still making strategic decisions and have to understand what’s going on.”
—Nick Schrock
This shift allowed Schrock to stay close to the technology and execution, focusing on where he could make the most impact.
Advice for leadership: As a technical leader, your role doesn’t have to fit the traditional management mold. Focus on where you can add the most value by leveraging your technical expertise and strategic vision. Identify whether you thrive as a coach (mentoring and guiding from the sidelines) or a quarterback (leading from the front and directly contributing to the project). There’s room for both types of leadership in tech organizations.
Driving change in large organizations
Schrock’s experience with GraphQL offers valuable insights into driving change and innovation:
- Focus on key stakeholders: Schrock didn’t try to convince the entire organization. Instead, he concentrated on the teams that would benefit most from GraphQL. This allowed for organic adoption and growth.
- Product-driven development: The success of GraphQL was rooted in its ability to provide immediate, tangible benefits. It wasn’t built as a technology for technology’s sake but as a solution to a pressing problem.
Advice for leadership: Start with a highly motivated, early adopter team, and let the success of your solution create momentum for broader adoption. Don’t waste time trying to convince everyone upfront.
How AI might reshape the developer tools landscape
Reflecting on how both GraphQL and Dagster addressed major pain points in software development, Schrock believes that the next frontier is AI-driven developer tools. He suggests that just as GraphQL and Dagster provided new layers of abstraction, AI could introduce entirely new paradigms for how developers interact with complex systems.
Schrock suggested that AI might soon be the “quarterback” in software development, hinting at a future where AI could take over routine orchestration tasks, allowing developers to focus on higher-level problem-solving.
Embracing the journey from GraphQL to Dagster
Schrock’s journey from building GraphQL to launching Dagster illustrates the power of identifying pain points and crafting elegant, scalable abstractions to address them. His story offers a roadmap for how to create impactful technologies that not only solve immediate problems but also set the stage for broader industry transformation.
Closing thoughts for tech leadership:
- Identify where engineers are struggling and build tools that address those pain points.
- Start with one highly motivated team or customer and let their success drive broader adoption.
- Think strategically about where your solution can serve as a leverage point across an organization or ecosystem.
- Find a leadership style that allows you to have the most impact, whether that’s as a coach, quarterback, or somewhere in between.
By embracing these principles, you can follow in Schrock’s footsteps and build technologies that change the landscape of software engineering for years to come.
For more of Nick Schrock’s insights, be sure to check out the full episode, “From GraphQL to Dagster Labs: How Nick Schrock Is Reinventing Data Infrastructure.” Learn more wisdom from industry experts by subscribing to Breaking Changes on Apple, Spotify, and YouTube.