The Rise of Self-Service Platforms and Backstage Framework

A Paradigm Shift in DevOps

The domain of DevOps has undergone a remarkable evolution, transitioning from basic automation of infrastructure provisioning and software delivery to the adoption of Continuous Integration and Continuous Delivery (CI/CD) pipelines. Today, the latest innovation reshaping this field is the emergence of self-service platforms. This article delves into the rationale behind this shift and explores best practices for implementing such platforms, with a particular focus on the widely-used framework, Backstage.

The Rationale Behind Adopting Self-Service Platforms

The decision to adopt self-service platforms in software development arises from a fundamental organizational change: the segregation of development (dev) and operations (ops) teams. Traditionally, ops teams, often centralized, serve as a support mechanism for numerous application development teams. This arrangement allows developers to concentrate on feature creation and domain expertise, while ops teams manage the underlying infrastructure and operational tasks.

However, this division of labor frequently leads to inefficiencies. The typical interaction between app teams and ops teams—through a ticket-based system where ops prioritize and address requests—often creates bottlenecks. These bottlenecks occur because the ops team, limited in size, can only address tickets based on their priority, leaving many development needs unmet.

To combat this inefficiency, some organizations have attempted to scale their ops teams in response to the growing demand for operational support. However, this approach has proven ineffective at scale, prompting a shift towards self-service platforms. These platforms empower developers to independently access services and manage infrastructure needs through a centralized portal, circumventing the need for traditional ticket-based requests and thereby alleviating bottlenecks.

Balancing Automation and Enhancing DevEx

Self-service platforms significantly enhance the Developer Experience (DevEx) by eliminating delays in fulfilling requests, thereby streamlining workflows.

However, the goal of achieving fully automated or entirely self-serving platforms often remains more aspirational than practical. Organizations typically adopt a pragmatic approach, starting with smaller, semi-automated systems. This strategy allows for a manageable integration of automated processes, ensuring each phase aligns with the organization's operational capacity and the developers' needs. By gradually refining and expanding automation capabilities, this incremental approach facilitates a smoother transition for development teams, allows for necessary system adjustments, and ensures that the platforms evolve in response to real-world feedback, ultimately enhancing DevEx while setting a realistic course towards more sophisticated, fully automated solutions.

Spotify's Backstage

Spotify's Backstage framework marks a significant advancement in the development of self-service developer platforms. Open-sourced by Spotify, Backstage embodies the ethos of "you build it, you run it," allowing application developers to directly access and consume services via a dedicated portal. This setup frees the platform team to focus on enhancing and adding new features to the self-service platform, rather than being bogged down by routine support tasks.

A notable advantage of Backstage is its role in streamlining the onboarding process for new employees. In environments characterized by complex microservice architectures, Backstage functions as a comprehensive catalogue. It aids newcomers in navigating through various services, endpoints, documentation, and identifying key personnel responsible for different aspects of the system. This feature is especially beneficial in setting up development environments for new hires, thereby minimizing disruptions in the existing development workflow and enabling organizations to integrate new employees more efficiently.

However, it's crucial to dispel a common misconception about Backstage: it is a framework, not a plug-and-play product. This distinction means that organizations can't simply install Backstage and expect it to fit their needs right out of the box. Instead, Backstage requires customization to align with specific organizational use cases. Its modular and plugin-rich architecture invites the integration of third-party plugins or the development of custom plugins to seamlessly blend into an organization's technical ecosystem. This flexibility and adaptability make Backstage a powerful tool for organizations looking to empower their development teams with a tailored, self-service platform.

Strategic Implementation of Backstage for Long-Term Organizational Benefit

When considering the implementation of Backstage as a framework for self-service platforms, organizations must develop a strategy that reflects a deep understanding of their unique needs and environment. The integration of Backstage should align seamlessly with the organization's technological ecosystem and culture, as well as the specific ways in which developers approach their projects. Recognizing that adopting Backstage is a long-term investment is crucial. This perspective ensures that the platform is not just set up but also continuously reviewed and improved to meet the evolving needs of the organization.

The strategy should involve treating the platform team like any other application team within the organization. This approach allows for the platform to be developed, managed, and iterated upon as an internal product, with the development team as its primary users. Such an alignment ensures that the platform is always tuned to the real-world requirements of its users, enabling developers to work more efficiently and effectively.

Moreover, organizations should view the implementation of Backstage as an ongoing process rather than a one-time project. Regular feedback loops with the developer users, combined with iterative development of the platform, will ensure that it remains relevant and continues to cater to the critical needs of the organization. This strategy fosters a culture of continuous improvement and adaptation, crucial in today's rapidly changing technological landscape.

Overcoming Initial Hurdles

While Backstage presents numerous advantages in streamlining developer workflows, it's important to acknowledge the challenges and friction points that come with its implementation. One of the primary concerns is the learning curve associated with setting up and customizing Backstage. Unlike some off-the-shelf solutions, Backstage requires a certain level of technical proficiency and time investment to adapt fully to an organization’s specific needs. This aspect can initially pose a challenge, especially for teams new to such extensive customization.

From the user interface (UI) perspective, although Backstage offers extensive customization options, the default material interface is often sufficient for most needs. A significant advantage of this UI is its ability to standardize the interfaces provided by various plugins. The DevOps tools ecosystem is notably diverse and sometimes inconsistent, but integrating these tools into Backstage forces them to adhere to a unified design principle. This standardization results in cleaner, less distracting dashboards, enabling users to concentrate on the essentials of their work.

Another notable challenge is the ongoing maintenance of the platform. Once implemented, Backstage essentially becomes a product managed by the DevOps team. Like any other product, it requires dedicated resources for continuous support and maintenance. This aspect means that organizations must allocate a budget for these ongoing needs, just as they would for any other critical software tool. Understanding and planning for these maintenance requirements is essential for the long-term success and sustainability of a Backstage implementation.

Key Strategies for Self-Service Platform Adoption

Adopting a self-service development platform is a strategic decision that requires careful planning and execution. A practical approach begins with starting small, especially in organizations with numerous microservices. Onboarding teams one at a time allows for focused attention on each group's specific needs and a better understanding of how the platform can be optimized. Early phases should include extensive A/B testing to identify the most effective use cases and adapt the platform accordingly. Rapid prototyping and iterative improvement are key to refining the platform in real-time based on actual user feedback and requirements.

Stakeholder involvement from the outset is crucial. Different users interact with the platform in diverse ways, and understanding these interactions is vital for a successful rollout. Developers might focus on the system's health and the outcomes of code deployment across various environments, while project managers may require a broader view of system health. For security teams, the emphasis might be on compliance with DevSecOps practices. Recognizing and accommodating these varied needs ensures that the platform serves its purpose across the organization.

Providing comprehensive training and support to end-users is another critical element. Initially, the platform might function in a semi-automated manner, gradually moving towards full autonomy in development workflows. During this transition, the platform team should be prepared to offer support, easing users into the new system and ensuring a smooth adaptation process.

A vital consideration in rolling out an Internal Developer Platform is understanding how engineers prefer to work, the cognitive load they can manage, and the cultural shifts necessary for adoption. Tailoring the rollout to these factors can significantly enhance the chances of a successful implementation and long-term utilization of the platform.

Insights from Wise: On Conducting the User Surveys

Wise's journey in implementing Backstage for their internal developer platform offers valuable insights into the power of user surveys in shaping successful self-service portals. The process began with identifying three core objectives: recognizing the need, scoping an alpha version, and validating the results. Biannual surveys conducted within the organization revealed key challenges such as difficulty in locating relevant documentation, issues with service ownership discoverability, and high cognitive load due to complex code navigation. These findings highlighted areas of friction in the engineering process that needed addressing.

To delve deeper, the platform team conducted user interviews, targeting both new joiners and experienced personnel. These interviews focused on gathering comprehensive information on user personas, jobs to be done, user journeys, and value stream mapping – similar to UX surveys in many organizations. A three-person team conducted these interviews, with roles including a main interviewer, a journey mapper for specific questions, and a note-taker to capture nuances.

These interviews proved highly beneficial. They facilitated direct involvement of more stakeholders, provided Platform engineers with opportunities to engage with users firsthand, and fostered empathy for user challenges. The outcome was the creation of multiple user personas and distinct user journeys, enriching the team’s understanding of diverse user needs.

Based on these insights, the alpha version of the platform was developed, from which several key lessons emerged. The team learned to prevent solution bias, used wireframes for better alignment, and grappled with the decision of when to discard versus when to incrementally improve features.

Subsequent A/B testing, both with and without the developer portal, yielded revealing results. For the public beta release, significant improvements were made. Documentation quality was enhanced through a quality scoring system, frictionless editing capabilities, and the integration of Backstage's documentation module. Additionally, the search experience was upgraded with federated search functionality, enabling users to efficiently parse information from various sources.

Maximizing Value with Backstage: Key Considerations and Implementations

Implementing Backstage in an organization requires a thoughtful approach centered on delivering genuine value to users. The foremost question to address is the selection of plugins and design strategies that resonate most effectively with user needs. A crucial part of this process involves engaging directly with developers, understanding their requirements, and identifying what they need to optimize their productivity with minimal effort.

Iterative Development Cycle: Feedback, Implement, Repeat

A continuous, iterative cycle of collecting feedback, implementing changes, and repeating the process is vital for the successful adoption of Backstage. This approach ensures that the platform evolves based on real user experiences and needs, minimizing assumptions and maximizing relevance.

Conducting User Research

As exemplified in the Wise case study, thorough user research is instrumental in understanding the nuances of existing services and how they can be enhanced using Backstage. This research provides insights into what users truly need from the platform.

Improving Documentation Quality

A common challenge for developers is the overwhelming volume of information, often leading to a high signal-to-noise ratio in documentation search. Backstage can address this by indexing documentation from various sources, enhancing searchability. For instance, using Backstage’s collector mechanism to index Confluence pages can significantly improve the search experience. Adding tags about the source of information further aids in easy navigation.

Scaffolding New Projects

The Backstage scaffolder plugin is a powerful tool for streamlining project initiation. By providing premade templates that adhere to best practices, such as generating 12-factor application code with preconfigured CI/CD pipelines, developers can start new projects with just a few clicks.

Accessing Service Deployment Information

Backstage plugins can be used to access logs and other pertinent information directly from the development portal. Some organizations have even integrated it with Kubernetes Lens to display pod-level information, enhancing the visibility and management of services.

Annotations for Enhanced Navigation

Annotations in Backstage are a key feature, determining what users see on each Entity page in the portal. They are especially useful in building catalogs for complex software systems, such as those with a large number of microservices. Annotations simplify navigation and information retrieval, enabling users to link directly to relevant documentation, Kubernetes lens, logs, GitHub project repositories, teams, and more.

Harnessing Data Insights with Platform Analytics

Incorporating analytics into your developer platforms is a pivotal strategy for measuring and enhancing effectiveness, as well as improving the Developer Experience (DevEx). This approach revolves around identifying key performance metrics that provide insights into the health and efficiency of the development process.

Strategic Application of Platform Analytics

• Measuring Delivery Pipeline Velocity: One of the primary metrics to track is the velocity of the delivery pipeline. This measurement helps in understanding how quickly and efficiently new features and updates are moving from development to production.

• Evaluating Throughput and Release Cadence: Throughput, or the number of tasks completed over a given period, along with the frequency of releases (release cadence), provides a clear picture of the team's productivity. Regular monitoring of these metrics can help in identifying trends, forecasting future performance, and making informed decisions to optimize workflows.

• Identifying Potential Risk Areas for Burnout and Bottlenecks: Platform analytics are invaluable in pinpointing areas where developers might be facing excessive workload or where processes are less efficient, leading to bottlenecks. Identifying these areas early allows for timely interventions, such as process adjustments or resource reallocation, to prevent burnout and maintain a sustainable pace of work.

How Organizations Leverage Backstage for Developer Platforms

Backstage, originally developed by Spotify for their complex microservices architecture, has been widely adopted and customized by various organizations for their unique development needs.

“We identify the various components of our system and locate them vertically by how much they will impact the end-user experience and horizontally by how much they're commoditized in the industry. Components are broken down into their constituent parts, pulling out the pieces that are commoditized.” — Brian Leathem, Netflix

Netflix expanded on Backstage, creating 'The Paved Road,' a federated console tailored by their Platform Experience and Design team to meet their extensive development requirements. Zalando implemented Backstage for 'Sunrise,' a comprehensive developer portal with features like a user-friendly dashboard, API catalog, and CI/CD tools, showcasing its adaptability for Kubernetes-based systems.

WeWork's use of Backstage to manage Kubernetes-based infrastructure underlines its efficacy in cloud-native technology and infrastructure management. Tidepool, an open-source project, leveraged Backstage to centralize and streamline its development pipeline, highlighting the framework's suitability for open-source environments.

Additionally, SADA, a cloud solution provider, utilized Backstage to manage their development pipeline and to provide developers with a centralized view of their cloud services and tools.

Case Study of Veterans United Home Loans

“As a developer, you’re not waiting so much and you can be autonomous. You can help the business as much as possible because you’re not waiting to add value”.

- Alan Barr, Product owner - IDP, Veteran united home loans

Veterans United Home Loans, a full-service mortgage lender based out of CA. Initially, their legacy environment was bogged down with extensive time spent in various stages of development, predominantly due to prolonged periods waiting for ticket resolutions. The process involved multiple stages, including centralized project planning, check-in and dependency management, build configuration, release configuration, server configuration, testing, and monitoring and alerting. These stages cumulatively accounted for a significant amount of time, often leading to inefficiencies.

The introduction of their Internal Developer Platform, dev.lab, marked a pivotal shift in this workflow. Post-implementation, the majority of these stages were automated, significantly reducing the time spent on each. This automation led to a streamlined process where only centralized project planning and monitoring and alerting required the same amount of time as before, with the rest handled efficiently by dev.lab.

One of the most notable impacts of this shift was the reduced cognitive load on engineers. They were no longer required to navigate through the intricacies of infrastructure changes and security, allowing them to focus more on core development tasks. The adoption of this platform, however, was not instantaneous. It required an adjustment period of 18-24 months, a testament to the time and effort needed for organizations and their personnel to adapt to new work processes and technologies.

Veterans United Home Loans' approach to building their developer platform involved carefully selecting tools that complemented their rapidly expanding cloud-native toolchain. By standardizing crucial aspects like security practices and commit formatting, the platform team was able to minimize the cognitive load on developers significantly.

A key understanding that emerged from this transformation was recognizing that a developer platform cannot cater to every single use case. The platform teams learned to manage their workload by not overstretching themselves across too many feature requests. Instead, they directed teams towards other solutions like cloud services, virtual machines, or specific tools when the platform did not cover particular needs.

---