Upgrading VME IOCs to EPICS 7 at SNS
Challenges in upgrading VME IOCs to EPICS 7 at Oak Ridge National Laboratory including hardware and software selection, application development environment, and the transition from VxWorks to EPICS. Learn about the modernization efforts and reasons for moving to EPICS 7.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Experience of upgrading VME IOCs to EPICS 7 at SNS Hao Hao, Johnny Tang, Klemen Vodopivec, Kay Kasemir Oak Ridge National Laboratory September 19, 2024 ORNL is managed by UT-Battelle, LLC for the US Department of Energy
Overview and challenges of SNS VME IOCs VME hardware 2100, 5100, 5500 (Emerson/Artesyn) Most come with custom cards: OMS-58, timing, MPS, LLRF, ... VME software VxWorks 5.5, 6.7, and 6.9 EPICS, 3.14.8.2, 3.14.11, 3.14.12.8 and 7.0.7 VME IOCs > 120 IOCs frontend, LLRF, HPRF, PS, CRYO, timing, diagnostics, RCCS, chopper, dumps, ... Long history raises challenges to maintain and develop software Different ADE versions distributed on multiple servers 2
In the large scope: SNS control modernization Control system modernization at SNS Network, hardware, software, IT infrastructure, development environment alarm handler, OPI Cryo, LLRF, HPRF, Power Supply controls VME IOCs are being migrated to soft or microTCA IOCs Motor controls for cavity frequency tuning: soft IOC, OMS PLC MPS: distributed over dozens of VME IOCs centralized microTCA New LLRF: microTCA Soft IOCs are being rapidly deployed as EPICS 7 3
Application development environment ADE system has been used for almost 20 years 32-bit Redhat Distributed on multiple servers Fragmentation. Each ADE is slightly different. A new ADE system in past few years New hardware Redhat Linux 8 Automated IOC deployment and software version synchronization Minimize modification to the default IOC/module configurations to make things simple Set of new tools: IOC management, building system, ... 4
Why EPICS 7 on VME? EPICS 7 will enable these IOCs to handle more complicated data, expanding it capabilities Less maintenance effort Better support from community Bug fixed Easily fit into the new application development environment Align with soft and microTCA IOCs that are/will be using EPICS 7 It is challenging to maintain multiple versions Not just the EPICS base Build an maintain shared modules It s a chain: hardware, data requirement, development environment, extensions, ... 5
Hardware and software selection Selecting the hardware and software is a multi-dimensional problem VME CPU board: specs, stock, availability, cost. 2100, 5100, 5500 Emerson/Artesyn VxWorks version, 6.7, 6.9 EPICS version, 3.14.11 ---- 7 Development environment, 32-bit, 64-bit Find a workable combination 5500 board (Emerson) + VxWorks 6.7/6.9 + 3.14.12.8/3.15.7/7 5500 board (Emerson) + VxWorks 6.7 + 3.14.11 5500 board (Emerson) + VxWorks 6.9 + 3.14.11 5500 board (Artesyn) + VxWorks 6.9 + 3.14.11 5500 board (Artesyn) + VxWorks 6.9 + 3.14.12.8/3.15.7/7 5500 board (Artesyn) + VxWorks 6.7 + 3.14.11/3.15.7/7 Find an intersection 6
VxWorks 6.9 Bug The bootline string is not properly terminated in VxWorks 6.9.4.10 Upgrade to 6.9.4.12 No official documentation 7
EPICS base and modules Base Add mv5500 support sequencer No change to existing programs, update to 2.1 version connect is used as a keyword, then removed #if EPICS_VERSION == 3 #include<tsDefs.h> #endif Handle deprecated libs and headers Some hacks were made LLRF : the wrapper for old style FUNCPTR based dllEach/execHookAdd/sllEach functions are not compatible. 8
EPICS base and extension modules Stack overflow, signal 11 Each FLNK uses more memory 22 kB stack allocated for VxWorks Long chain of FLNK: returned after the last one. 145 bytes/FLNK for 3.14.12.8 allows 175 FLNKs 304 bytes/FLNK for 7.0.7 allows 75 FLNKs Solution: fanout to more branches, shorten the FLNK chain. limit 7.0.7 3.14.12.8 9
Planning SNS is an user facility, reliability and availability are of the most importance Maintenance day schedule is over-crowded Careful planning Through tests in the lab Decouple the problems Do one thing at a time Chance to go back 10
Progress 24 out of 55 VME LLRF IOCs are upgraded to 7.0.7 3 to 9 IOCs every maintenance day Some IOCs are slightly different Collaborate with RF engineers and operators No news is good news No issue so far. Bootup, CPU, memory, network, NFS. Power supply IOCs will be upgraded in Novermber outage genSub aSub other modules 11
Summary It is demonstrated that the EPICS 7 can run on existing VME IOCs at SNS controls. Selection of VxWorks versions need to be carefully evaluated. EPICS 7 has a good compatibility with old modules. Minor changes are need. Some corner cases have caused problems during compilation. Hacks may be needed. Migrate to microTCA or soft IOCs may be a better idea. 12
Thanks! 13
