Collaborative Mantid development project delivers more than 10x faster processing for single-crystal INS data

Single crystal INS is a technique that measures the motion of atoms in materials, allowing researchers to precisely determine interaction and forces between atoms or spins in solid matter. This can be used to explain the atomic scale mechanisms behind properties such as heat conduction and to probe quantum phenomena across a wide field from energy materials to superconductors.

At ISIS, we acquire INS spectra using time-of-flight spectrometers with large 2D detectors that measure the neutron speed and scattering direction after they interact with a material. This gives valuable yet incomplete information about the 3D structure of the material. Thus, rotating the crystalline sample and acquiring data at different positions is often necessary to assemble all the 2D “slices” at different times into a complete representation of the sample structure.

Previously, each such 2D “slice” was measured at a set of fixed crystal orientation angles (e.g. every one degree) and processed individually into separate data collections that were binned into histogram representations. But data acquisition systems can now record each individual neutron event and motor position, enabling an alternative “continuous” rotation mode of operation in which the sample is slowly rotated at 0.1 degree per second while all neutron count events are monitored in a single acquisition experiment.

This advantageous measurement mode did not have a compatible software processing method, as the continuous dataset had to be split into separate data chunks to be able to reconstruct the full volumetric data, which was computationally very demanding and time and resource expensive.

To address this, we formed a novel collaboration where an instrument scientist from the ISIS excitations group joined the Mantid software team working directly with the research software scientist to design and develop the new feature. Previously, instrument scientists had a more hands-off role of defining the problem and testing solutions that the research software team implemented. The active participation of both sides in the software development process in this project meant that we could deliver the new capability to process continuous rotation data in Mantid on a much shorter time scale. Its success has prompted the Mantid team to explore further close collaboration with other instrument scientists to temporarily join the Mantid team for projects.

The new continuous rotation workflow in Mantid has reduced the time to process a representative dataset from around 20 minutes to less than 2 minutes. In addition, as the new workflow does not need to rebin the data it also saves a significant amount of memory and disk storage, thus reducing the cost and environmental impact of our data analysis computations. This new approach also grants more flexibility to instrument scientists and users. Measurements can be stopped once sufficient data have been collected, rather than waiting for all crystal angles to be completed in the fixed-angle mode. This can create time for additional measurements, for example at different temperatures or applied magnetic fields, within the same allocation of beam time. Finally, the faster processing time will also enable new approaches such as a “live data” viewer or autonomous data collection strategies for these instruments.

The new processing method will be available from Mantid 7.0, which has a planned release by the end of October 2026, integrated as part of the ConvertToMD and related set of algorithms.

Article by Duc Le, Instrument Scientist and Adrian Diaz, Scientific Software Engineer.