A blog by the Guardian's internal engineering team. We build and run the Guardian website, mobile apps, editorial tools, revenue products, advertising, and data and identity platforms. This blog is where we share our experiences and approaches, including software development tips, code examples, open source software and code stories behind product development.
The Guardian replaced password-based authentication with email-delivered one-time passcodes across account creation, sign-in and password reset to reduce friction and improve security, addressed account enumeration and mobile deep-link issues, rolled out UX improvements via their Gateway frontend, measured strong adoption of passwordless, and are planning further steps such as MFA and FIDO passkeys.
The Guardian built a structured recipe dataset to power its Feast cooking app by collaboratively defining a recipe data model, training an in-house machine-learning model to pre-structure legacy recipes, creating a temporary curation tool (Hatch) for journalists to refine output, integrating the workflow into Composer and the Source design system, and storing curated records in a database — producing 4,300+ structured recipes and enabling web recipe schema/SEO benefits.
Employee profile: Alex Guild describes their path into product management, a typical hybrid workday (morning routine, standups, meetings, focus time), what they enjoy about The Guardian’s collaborative culture, and why they applied (alignment with journalism and digital product quality). The article mentions prior experience owning a mobile app but contains no technical implementation details.
The article discusses the pitfalls of relying solely on TypeScript static types for JSON from REST APIs (JSON.parse returning any) and recommends treating parsed data as unknown and validating it at runtime using a small parsing/schema library (the Guardian chose valibot, noting similarity to zod). It shows how schema parsing improves safety and developer confidence, and suggests handling failures with safeParse/Result patterns to fail gracefully.
Recap of The Guardian's 2024 hack day highlighting 19 prototypes (favicon rendering, a browser extension that turns paragraphs into puzzles, tennis-score parsing, a guest seating puzzle, etc.), with commentary on experimentation, structuring content, gaming experiences and using technology (python, ai, tor, private cloud) as a competitive edge.
The Guardian describes their experience setting up SecureDrop Workstation on Qubes OS. They walk through using Salt state files to create TemplateVMs/AppVMs/DispVMs (amnesic VMs), installing packages (from default and custom apt repos such as dangerzone), adding a Nautilus extension in Python, templating secrets with jinja2 and JSON, packaging Salt state files into an RPM with FPM, and deploying/applying the configuration using qubesctl — with security notes about network access, disposable VMs and key handling.
Part three of a three-part retrospective on Pinboard, an internal Guardian tool for newsroom communication. The post discusses MVP trade-offs (critical-mass concerns), trialing with liveblogs and desks, an interactive onboarding tour, the org-wide rollout (inferring permissions from Composer/Grid), adoption and metrics, and planned improvements (Workflow integration, ‘important messages’, imaging requests).
Part two of a three-part engineering post explaining how Pinboard integrates into the Guardian's newsroom tools. The team chose a one-line bootstrapping script (served by an AWS Lambda/API Gateway endpoint) that generates AppSync auth tokens and then loads a cached hashed JS bundle. On the frontend Pinboard mounts a React app into host pages (works alongside Angular hosts) using MutationObserver, React portals and Shadow DOM to isolate styles. The backend uses GraphQL via AWS AppSync with Lambda resolvers and subscriptions (real-time collaboration). The article discusses trade-offs (extension vs library vs standalone loader), release/deployment considerations, and auth options.
Part one of a three-part Guardian devblog describing the origin, discovery work and technical architecture for Pinboard — a newsroom communication tool. It covers UX/design challenges and then details the technology stack: full‑stack TypeScript, infrastructure-as-code with CDK/GuCDK, esbuild for builds, an all-serverless server-side using AWS Lambda, AppSync (GraphQL) with Apollo on the client, AWS RDS (Postgres) after migrating from DynamoDB, and a client built with React/Preact + Emotion bundled with Webpack (previously Parcel). The write-up emphasizes integration with Composer and Grid, fast CI/builds, and rapid lambda deployments.
First-person account of the Guardian’s Digital Fellowship describing how the program integrates non-traditional entrants into the editorial tools team, offers hands-on production experience (AWS Lambda, Scala, Guardian API, d3, React), encourages learning and rotations across teams, and emphasizes inclusion and diversity. The post concludes with a call to apply to the 2019 Fellowship (application closing date).
Recap of The Guardian’s Product & Engineering hack day where many teams explored large language models and generative AI. Notable hacks: CrosswordsPlus (multiplayer via SharePlay on iOS), 5/15s (OpenAI GPT-3.5 for summarisation + text-to-speech to create a short podcast), and Linguini (using LLMs as annotators to speed NLP dataset labelling). The event also featured experiments with ChatGPT, Google Bard and integrations like Trello.
The Guardian improved Typerighter by replacing a Google Sheets rule source with a Postgres-backed Rule Manager (a second Scala Play app), importing the Collins English Dictionary to add dictionary-based spellchecking via LanguageTool, integrating a named-entity-recognition service to avoid false positives, and adding a React/ElasticUI front end while migrating editors (Scribe -> ProseMirror).
The Guardian and UCL researchers trained and fine-tuned language models to perform coreference resolution for accurate quote attribution. They manually annotated ~100 articles, fine-tuned off-the-shelf models (Coreferee, spaCy coreference model and FastCoref from ExplosionAI), used word embeddings and language-model techniques, and plan large-scale tests across the Guardian archive to match quotes to sources automatically.
The article from The Guardian advocates for standardisation to reduce cognitive load and speed development. It describes how a developer experience team creates shared abstractions and defaults (testing suites, package managers, build scripts, linting/formatting, deployment strategies). It cites concrete tools and patterns — CDK and AWS for infra-as-code and microservice deployment, Riff‑raff for deployments and rollbacks, an isomorphic JavaScript front-end using TypeScript and ESLint, defaults published on npm and a client-side monorepo — and discusses balancing familiarity with innovation.
A seven-year engineering retrospective from The Guardian’s Supporter Revenue team describing the evolution of supporter products and organisation: early proof-of-concepts (Zebra/Giraffe), payment and billing integrations, a Snowplow fork (“Bolt”) for analytics, A/B testing of the after-article "epic" messaging, contribution reminders and automation, frontend architecture and banner tooling upgrades (including a rotating 3D banner), and repeated reorganisations into cross-functional huddles aligned to the customer lifecycle.
