STFC-ISIS-8 Data Streaming and Data Reduction High-throughput for the European Spallation Source
04 Feb 2019
Yes
-  

 

 

 

No

​​

 

Background

The European Spallation Source, currently under construction in Sweden, will be the world's most powerful pulsed neutron source upon completion in 2023. The experiments will produce very high data volumes of critical scientific data. This requires lossless transport, and high frequency IO for essential operations of the facility. Each experiment, of which there may be 10+ running simultaneously may, generate 10 GBs-1 of neutron data. This represents a major and fundamental software engineering challenge for the science programme.

Description

This is a 3-month placement based at RAL/ISIS which will involve collaboration with the data-streaming and data-reduction development teams as well as our international partners.

The project goals are defined as:

  1. To improve the functionality of file IO at several critical points in the data lifecycle. We have several new formats providing richer information and more complete set of information for our users, which we must code against. What is implemented must work cross-platform and must be proven to work in a range of scenarios.
  2. Improve performance from the above, and ensure that pre-existing solutions are suitably optimised to provide the high-throughput required.
  3. Roll all of the above into a large-scale production codebase used daily across the globe

You will be experienced with c++ and have a keen interest in optimising large-scale codebases. Python experience is also desirable. In return, you will work directly with a very experienced, professional, software development team. You will be treated as a full team member for the project and will work with our distributed partners to achieve the best outcomes.

You will gain experience writing modern high-performance c++ and python. You will gain a greater appreciation for practice software engineering including test driven development, continuous integration, performance profiling. You will gain experience working on a distributed agile software development team. You will have opportunities to present your work at key points.

Supervisor: Owen Arnold, owen.arnold@stfc.ac.uk

Contact: