REMS helps researchers find, acquire and share scientific materials

In brief

Customer

CSC

Challenge CSC needed a system for finding and analysing research data that would adapt to the needs of various organisations.

Results An open-source system that serves many users and that will be further developed and improved in the future.

Benefits The REMS system can be used to flexibly meet the needs of each organisation.

csc.fi

Together with the IT Center for Science (CSC), Nitor is developing the REMS system for managing scientific data. The system allows users in different organisations to apply for and authorise access to different materials. The agile solution is implemented in Clojure and published as open-source.

REMS helps researchers find scientific materials
The IT Center for Science, or CSC, is a Finnish center of expertise in information technology owned by the Finnish state and higher education institutions. Together with CSC, Nitor is renewing the center’s REMS system utilised by Findata and THL Biobank, among others.

REMS or Resource Entitlement Management System has been implemented as part of the international ELIXIR project, which brings together data and tools from the life and health sciences across Europe. The aim of the project is to create a research infrastructure that will make it easier for researchers to find and analyse data and share their knowledge, implementing best practices in the field.

REMS is a secure way of sharing research data that cannot be shared publicly, such as sensitive information. REMS provides researchers from different organisations access to data and materials, and the application process can be handled entirely within the service.

A public system published as open-source
Built by using the Clojure programming language, REMS is an open-source public system with CSC and Nitor as the main implementers.

3

public actors controlling access to data via REMS

1

petabyte of scientific data

127

data sets available

A robust solution built on 42,000 lines of pure Clojure

REMS acts as an integration point for other licensing and processing systems
REMS includes an interface through which the license application circulates the material from the applicant to the owner or their representative for approval. In addition, it provides an extensive interface that shares information on applications and licenses granted and acts as an integration point with other licensing or processing systems. REMS can flexibly meet the needs of an organisation using it.

Functional code keeps the code mass manageable
Despite the massive scale of the system, functional code and the expressiveness of the Clojure programming language keep the code mass under control. In addition, comprehensive automatic testing makes continuous delivery possible.

Building the software according to the design principles of Command and Query Responsibility Segregation (CQRS) and Event Sourcing enables the business logic to be built of purely functional code. This is exemplified well by transforming commands into events and combining events. Business logic implemented as purely functional code can be extensively tested and easily modified, if necessary. The functional approach also helps with the automatic production of a REST interface and user interface based on the definitions of commands.

It's great to get to show an example of an extensive system made with Clojure as well as to help others by developing open source components. Unlike many other modern languages, Clojure was originally built for heavy production use and is well suited for programming a system like this.

Markku Rontu, Senior Software Architect, Nitor

A system to meet many needs

The experts at CSC and Nitor together built a system that is adaptable to the needs of different user organisations. New user organisations always bring new needs to a system. REMS was first adopted by the Language Bank of Finland where the system has been in production since 2018. Currently, REMS is utilised by the data permit authority Findata and THL Biobank.

REMS is a good fit for individual researchers who want access to a specific resource as well as service providers that need to be able to manage access to their materials. The customisable, dynamic application process and automation of basic functionalities make managing much easier. REMS is a secure and efficient way of sharing research data that cannot be shared publicly, for example, due to its sensitive nature.

9703

commits by 19 contributors

3426

test assertions

))))]]))))]))

longest chunk of closing parentheses

Agile development enables close cooperation even remotely

Getting started with a proof-of-concept implementation
The suitability of the new system was validated with a proof-of-concept implementation, on top of which new features have been built piece by piece according to the needs of user organisations.

Important architecture decisions of the project are documented in GitHub alongside the source code according to the Architecture Decision Records principles. This way of working could be used in any project, but when working remotely it plays a big role. Recording architecture decisions fosters long-term development, which allows changing any previously made decisions when the reasons for them are no longer valid.

Agile development under any conditions
The success of a project is almost always the result of effective teamwork. Nitor has been the supplier of the REMS system since 2017. From the beginning, the development team has collaborated remotely with CSC developers, which has made the general transition to remote work easier. An open-source project provides good conditions for remote work.