Sourcegraph Managed Services Platform (MSP)

The Sourcegraph Managed Services Platform (MSP) is the standardized tooling and infrastructure for deploying and operating managed Sourcegraph services. MSP takes a service specification and generates Terraform manifests and adjacent resources required to operate a service, aiming to provide a simple, Heroku-like experience to spin up infrastructure for standalone managed services.

By adopting MSP for your managed service, it will benefit from an expanding set of features and integrations, alignment with infrastructure and security best practices at Sourcegraph, and support from the Core Services team.

All assets are managed in sourcegraph/managed-services, and the tooling is being developed in sourcegraph/sourcegraph/dev/sg/msp.

Features

MSP supports single-container:

  • stateless, horizontally scaling services
  • scheduled cron jobs

From a simple service configuration YAML (examples) and the sg msp toolchain for managing configuration, we currently support:

  • Generating infrastructure-as-code, deployed via Terraform Cloud
  • Service initialization and runtime boilerplate via sourcegraph/lib/managedservicesplatform, which includes:
    • initialization of OpenTelemetry tracing and metrics, logging, and error reporting
    • integration guidance for provisioned data backends like Redis and PostgreSQL
  • Provisioning of data backends, configured with secure, highly available defaults and regular backups out of the box where applicable:
    • Redis for ephemereal data and synchronization between instances of a service.
    • PostgreSQL for persistent, relational data.
    • BigQuery dataset and tables for high-volume analytics and usage data specific to your feature.
  • Service-specific features
    • Configuring a domain and TLS through Cloudflare and GCP load balancing
    • Scaling capabilities backed by Cloud Run
  • Job-specific features
  • Commands for easy access to infrastructure
    • Shortcuts to relevant UIs in sg msp tfc view, sg msp logs, etc.
    • Securely connect to your PostgreSQL instance using sg msp pg connect
  • Generated infrastructure guidance, rendered in the Managed Services infrastructure pages.

See our GitHub roadmap and 2023 Managed Services Platform (MSP) proof-of-concept update for more details on things we will be adding to MSP.

Building a new service

Before deploying a service, you will need to build it! The Core Services team recommends building your service in Go to leverage the service initialization and runtime boilerplate provided by the standalone github.com/sourcegraph/sourcegraph/lib/managedservicesplatform module.

The runtime.Start function outlines the expected “contract” the MSP runtime expects services to fulfill:

import (
  "github.com/sourcegraph/sourcegraph/lib/managedservicesplatform/runtime"

  // Your implementation!
  "github.com/sourcegraph/my-service/service"
)

func main() {
  runtime.Start[service.Config](service.Service{})
}

In your implementation of runtime.Service, the primary entrypoint Initialize provides a runtime.Contract that is pre-configured with MSP defaults and offers helpers to connecting to MSP-provisioned resources. For example, to serve your service, you must use (runtime.Contract).Port, and to get a securely authenticated PostgreSQL connection, you can use (runtime.Contract).PostgreSQL.OpenDatabase(...).

A full example service is available in cmd/msp-example that makes use of most MSP functionality.

Creating and configuring infrastructure for services

Refer to the sourcegraph/managed-services README for all documentation for creating configuring MSP deployments and using sg msp.

Operating services