Magrep

Python

FastAPI

Oracle

Docker

Overview

MagRep generates witness reports for Nb₃Sn superconducting cable at CERN, documenting critical current (Ic), residual resistivity ratio (RRR) and heat treatment characterisation for accelerator magnet programmes. The original was a ~72,000-line C++/Qt desktop application.

Stabilising the legacy application

When I inherited it, the application couldn't be distributed at all: there was no installer, and running it required every dependency to already be in exactly the right place on the machine. The RRR section of the report was also largely unimplemented. I built Windows and macOS installers so it could reach end users, and completed the RRR reporting to parity with the rest of the report.

Why reimplement

Even once packaged, the structural problems remained. Business logic, data access, physics and LaTeX generation were all compiled into one binary, so any change meant a full rebuild and redistribution. More seriously, users had no visibility into the data feeding a report, and the class-based selection logic occasionally picked up the wrong sample or measurement.

Those are fundamentally problems of data selection and joining, so rather than porting the same class hierarchy to another language, the reimplementation moves that logic into SQL, where every join and filter is visible and can be checked independently.

Proof of concept

I first built a Python proof of concept to validate the approach: an ETL pipeline extracting Oracle data into SQLite with Pydantic validation at every stage, a reproduction of the scaling-law analysis, and PDF output matching the C++ reports. It also added a key feature the original lacked: letting users inspect and edit the underlying data before a report is generated.


Production Architecture

The production system has three independent layers:

  1. Oracle PL/SQL backend (~9,500 lines): owns all data aggregation, conditional logic and business rules, returning render-ready JSON for each report block.
  2. FastAPI rendering microservice: stateless, deployed on CERN OKD (OpenShift). It walks a report spec, fetches each block's data from Oracle, and renders PDF and HTML.
  3. Oracle APEX frontend: built collaboratively with the group, for selecting heat treatment orders, configuring and triggering reports, and managing documents.

Unusually, Oracle drives generation rather than the frontend calling the service:

  1. APEX calls a PL/SQL procedure.
  2. Oracle assembles the request and POSTs it to the microservice.
  3. The microservice walks the spec's block tree, calling back into Oracle for each block's data.
  4. It renders and returns the document.
  5. Oracle stores it for APEX to preview or download.

Python performs only rendering and placeholder substitution, so a data or logic change never requires touching the service.


Spec-Driven Reporting Engine

Rather than hardcoded report classes, each report is a JSON specification: a tree of typed blocks (title_page, section, text, table, figure, repeater), each declaring the Oracle function that supplies its data. {{token}} placeholders are substituted from request metadata at runtime, and repeater blocks render their children once per item in a list, for example one table per magnet aperture type.

Every renderer is implemented once for LaTeX and once for HTML, so a single spec produces both a PDF and a web report. Adding a new report type needs only a new spec and its Oracle functions, with no Python changes.

An /analyse endpoint statically validates a spec without generating anything, listing every Oracle function referenced and every token, and whether each is satisfied. This lets a database engineer check a spec's wiring before a full generation.


Superconductor Physics

A core part of the analysis is fitting the Summers scaling law to critical current measurements:

Ic(B,T)=C(T)⋅bp(1−b)qBpI_c(B, T) = C(T) \cdot \frac{b^p (1-b)^q}{B_p}Ic​(B,T)=C(T)⋅Bp​bp(1−b)q​

Given measured (Bp,Ic,T)(B_p, I_c, T)(Bp​,Ic​,T) triplets, a dedicated fitting service uses Levenberg-Marquardt least squares (scipy.optimize.least_squares) to extract the material parameters Bc20B_{c20}Bc20​ and C0C_0C0​, and corrects IcI_cIc​ between 1.9 K and 4.3 K using the fitted law. It also finds where a magnet's load line intersects the fitted critical surface, using Brent root-finding, which is a key acceptance criterion for accelerator magnets. PL/SQL calls the fitting service over HTTP, so numerical results arrive in the same call as the rest of the business logic.

Material constants are looked up by fabrication type and strand diameter, with a designated default as a fallback.


Validation

Every computed value (band fit parameters, degradation percentages, RRR pairings) was cross-checked against the C++ application's output for known heat treatment orders. Agreement is typically within a few percent, and each residual difference was traced to a specific, documented methodological difference rather than an implementation bug.


Deployment

The microservice runs as a container (python:3.12-slim, TeX Live, Oracle Instant Client) on CERN OKD, behind a TLS-terminated route with readiness and liveness probes. The image also builds a MkDocs user guide, served from the running service.


Key Features

  • Logic moved from a 72k-line C++ binary into inspectable SQL, addressing wrong-data incidents at the root
  • JSON spec-driven reports, so new report types need no Python changes
  • PDF and HTML output from a single spec
  • Scaling-law fits and load-line intersections validated against the legacy application
  • Users no longer need Oracle credentials or a desktop toolchain to generate reports

Reflections

The biggest lesson was that the hardest bugs weren't in the physics or the rendering; they were in which data got selected. Moving that logic somewhere it could be inspected did more for correctness than any amount of careful porting would have. Next on my list would be an automated regression suite comparing Oracle output and rendered reports against recorded references, plus a CI/CD pipeline for the service.

On this page

OverviewStabilising the legacy applicationWhy reimplementProof of conceptProduction ArchitectureSpec-Driven Reporting EngineSuperconductor PhysicsValidationDeploymentKey FeaturesReflections

Links

Fit Service Gitlab (Restricted Access to CERN Members)Report Generator Gitlab (Restricted Access to CERN Members)