Shifts

ALICE shift operations for global commissioning 2021

  • ALICE will be operated by 4 central shifts during global commissioning 2021 and 2 of 4 will be operated onsite in the ALICE Run Control Center (ARC) at P2 and another 2 will be operated remotely.  The remote operation of ALICE was considered according the current difficulties imposed by the COVID19 and resulting travel restrictions.  
  • ALICE central shifts during global commissioning 2021 will be composed of:
    • SL+ECS/CTP/FLP : onsite 
    • DCS: onsite 
    • EPN/PDP:  remote 
    • QC: remote 
  • ALICE global commissioning schedule will be as follows.
    • 1/7 - 4/9:  central shifts for MA only + oncalls 
    • 5/9 - 10/10: central shifts for MAN + oncalls
    • 11/10 - 21/12: central shifts for MA only or MAN (to be decided later) + oncalls 

 

ALICE shift booking : SLIMOS

 

ALICE shift booking : Procedures 

  • The shift booking will be open from 12th May 14h:00 (CEST).
  • Onsite shift  (SL/ECS and DCS) booking procedures
    1. SLIMOS permission (the one with a 3 year expiration date) in SAMS is mandatory
    2. Sign up for the shift class (https://alice-glance.cern.ch/alice/sams/classes/calendar)
    3. Sign up for the 3 day training shifts
    4. Book the general shifts 
  • Remote shift (QC and EPN/PDP) booking procedure 
    • ​​​​​​RC will pre-book QC and EPN/PDP shift blocks for each institute or country according to their request.
    • Once RC assigns the QC and EPN/PDP shift blocks to the institutes or countries,  the corresponding representatives will be notified and then the individuals will be able to book the assigned blocks. 
    • When the individuals reserve the block, they have to follow: 
      1. SLIMOS permission (the one with a 3 year expiration date) in SAMS is mandatory 
      2. Sign up for both QC and EPN/PDP shift classes (https://alice-glance.cern.ch/alice/sams/classes/calendar)
      3. Sign up for the 3 day training shifts for both QC and EPN/PDP shifts
      4. Book the general shifts 
    • QC and EPN/PDP classes and QC and EPN/PDP training shifts are required for remote central shifts
      • To increase the redundancy of the remote operation,  remote shifters should be familiar with both EPN/PDP and QC shifts. 
      • This is mandatory since, if one remote site "A" is down due to unexpected reasons (power-cut, network outage, etc), the other remote shifter "B" should take over the task of  the site "A". 
      • This implies that you are required to register both QC and EPN/PDP classes and QC and EPN/PDP training shifts before you reserve pre-booked shift block. 

 

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Operations in ALICE

  • Run Coordination

The overall coordination of the operations of the ALICE experiment is under the responsibility of Technical Coordination (TC). The TC appoints a Run Coordinator (RC) and a Deputy Run Coordinator (DRC) who are responsible for the data taking operations of the ALICE detector for a one year running period and represent the experiment at the LHC accelerator control center (CCC).

During physics data taking period, RC is assisted in the daily coordination work by a Run Manager (RM) for the duration of a 2 week period.

Each subsystem nominates a System Run Coordinator (SRC), who is responsible for the detailed organization of the real-time operation of the subsystem and provides the interface between the RC and the project.

The shift crew has the task of operating the experiment for an 8-hour shift period. It is headed by a Shift Leader (SL) who is in charge of implementing the shift plan defined in the run coordination meetings and directing the shift operations.

 

  • Shift overview

There are 4 kinds of shifts in ALICE. 

  1. Central shifts
  2. Central training shifts
  3. Detector shifts
  4. On-Call shifts

  • 1. Central shifts
    • Shift Leader (SL)
      • The SL is responsible for orchestrating the ALICE data taking operations and ensures that the daily program of data taking as approved by the Physics Board (PB) and implemented by the Trigger Coordination (TriCo) and the Run Coordination (RC) is enforced.
    • Experiment Control Shift (ECS/CTP/FLP)
      • The ECS/CTP/FLP shifter is in charge of operating the Data Acquisition (DAQ) including physics and calibration runs, EPN/QC/PDP workflow selection, and partition and trigger configurations and in charge of checking the status of Central Trigger Processor (CTP) and all First Level Processor (FLP) machines including Common Readout Units (CRU) and CR1 networks. 
    • Detector Control Shift (DCS)
      • The DCS shifter is in charge to perform all the actions related to the detector slow control (i.e. changing the physical state of a detector by altering the operational voltages) as well as the interactions with the accelerator systems. The DCS shifter is responsible for reacting to the DCS alarms to maintain ALICEC sate and stable state. 
    • Event Processing Node and Physics Data Processing (EPN/PDP)
      • The EPN/PDP shifter is responsible for monitoring EPN hardware including GPU, DataDistribution software, synchronous and asynchronous reconstruction processes in FLP and EPN, and network from EPN farm to EOS. 
    • Quality Monitoring (QC)
      • The QC shifter has the critical task to monitor the quality of the data flowing into the ALICE permanent storage via a series of histograms  which are updated during the run and signal potential anomalies in the raw data which must be forwarded promptly to the SL. 
  • The central shifts are booked via SAMS. During normal physics data taking period, they strictly follow the MAN scheduling. 
  • Organization of central shifts during global commissioning phase can be different. Shift organization for global commissioning 2021 will be explained later. 

 

  • 2. The Central Training Shifts
    • Central training shifts are mandatory in order to be able to carry out regular shifts. The training shift block runs over three consecutive days and does not follow the MAN schedule. This would mean that you could e.g. have either three consecutive morning or evening training shifts. As a trainee you have the obligation to follow closely the operations performed by your trainer for the first two days. Your trainer has the obligation to provide you with all the required information and to answer your questions. On the third day of your training shifts you are required to swap with your trainer who has to assist while you perform the actual shift as a "shadow" shifter. This model is called TTS (Training, Training, Shadow). Booking a TTS block can be done via SAMS. 

 

  • 3. The Detector Shifts
    • These shifts can only be occupied during commissioning periods or special running periods (typically HI runs). Detector shifts are opened by the Run Coordinator (RC) upon request of the SRC  (System Run Coordinator) who then manages these. In order to be qualified for a detector shift you have to gain expertise on that particular system. The SRC provides ad-hoc training and gives permission according to his judgment of your expertise to take detector shifts for his/her system. Once the shifts are open and you are qualified by your SRC you can book these. Detector shifts are booked via SAMS.

 

  • 4. The On-Call Shifts
    • On-Call shifts are booked by experienced detector shifters. The On-Call qualification for a certain system is given by the corresponding SRC according to his/her judgment. Ad-hoc training sessions can be required to become an On-Call shifter. On-Call shifts are booked via SAMS on a day-by-day basis and last 24h. During this time you are not required to be at P2 however you are required to carry a (CERN) mobile phone so that you can be called at any time of the day in case of problems with the system for which you are the on-call.  As an On-Call shifter you can be asked by the SRC to attend the daily Run Coordination meeting at P2 at 16h30 (either remote or onsite attendance). 

 

  • 5. The Expert Shifts
    • During special running periods (typically HI runs), the Expert Shifts are opened by the the Run Coordinator (RC). The RC or SRC provides the Expert flag in SAMS to you according to your expertise for the central shifts or subsystems. Those who are qualified as the Expert of central shifts and detectors will be able to book the central shifts and oncalls, respectively.