Yokogawa Electric Corporation has introduced a reinforcement learning service for edge controllers. Particularly, this autonomous control service for OpreX™ Realtime OS-based Machine Controllers (e-RT3 Plus) utilizes the Factorial Kernel Dynamic Policy Programming (FKDPP) reinforcement learning AI algorithm. In addition, consists of packaged software and an optional consulting service and/or a training program, depending on end user requirements.

Yokogawa targets to release the software globally, while consulting and the training program will be provided first in Japan, then in other markets.

Development Background

Given the complexity of the physical, chemical, and other processes at actual plants, there are still many areas that require the intervention of veteran operators. Equally important, controlling these areas has become difficult and has a direct impact on product quality and profitability.

Conventional control technologies include PID control^*1 and advanced process control (APC)^*2. Thus, achieving complex control using PID control or APC may sometimes require adjustments, which can take a considerable amount of time and effort. Also, some areas of plant operations are not suitable for PID control or APC and must be controlled manually, requiring operators to exercise judgement when entering control inputs. The autonomous control AI is a new technology for such situations that achieves complex control and eliminates the need to rely on manual operations.

Product and Service Features

FKDPP is a new control technology that is different from PID control and APC. Recently, Yokogawa and JSR Corporation’s elastomer business unit (now owned by ENEOS Materials) had successfully concluded a 35-day field test. Particularly, it tapped in which AI was used to autonomously control a facility in a chemical plant that could not be controlled using existing control methods and had necessitated the manual operation of control valves based on the judgements of plant personnel.

A world first^*3, this was accomplished despite the presence of factors such as weather conditions that could have significantly disrupted the control state. With the new service that Yokogawa is announcing today, customers can create AI control models using the FKDPP algorithm and install them on edge controllers. This service has the following features and merits:

Features

1. Thanks to simplification of the AI model creation process, even non-AI experts can create an autonomous control AI model and install it on an e-RT3 edge controller.
2. Retrofit of edge controllers with the installation of the autonomous control AI can be performed while other facilities remain in use.
3. Supports control cycles as short as 0.01 seconds and is optimal for device control applications that require a quick response.

Merits

1. Enables autonomous control where only manual control was possible
   By applying autonomous control AI in areas that are beyond the capabilities of PID control and APC, both autonomy and optimal control can be achieved. It enables stable control that is less susceptible to external disturbances and increases productivity.
2. Suppresses overshoot
   Although this will vary depending on the control targets, FKDPP suppresses overshoot. The reduction of overshoot (a condition where a set value is exceeded) is expected to extend, for example, the lifetime of furnaces and other heating facilities by reducing unnecessary overheating.
3. Significantly reduces settling time
   FKDPP significantly reduces the settling time compared with PID control, saving energy and improving productivity.
4. Ability to achieve the right balance between conflicting requirements
   Although this will depend on the control targets, FKDPP is able to resolve conflicting requirements and, for example, achieve the right balance between the need to reduce energy use while maintaining product quality.

Following demonstrations such as the one described below, Yokogawa is now launching this new autonomous control AI service for use on edge controllers in a wide variety of industries, without necessitating major changes to plant facilities.

Experimental Furnace Control

In comparison to auto-tuned PID control, it was shown that this solution was able to suppress overshoot and shorten settling time by about 65% – auto-tuned PID control took about 30 minutes to settle, while autonomous AI control took about 10 minutes to settle.

To use this system, edge controllers (sold separately), access to the autonomous AI learning service, a software package for the implementation of AI control models on edge controllers, and a license to run AI control models are required. Depending on the application, training programs, related consulting services, engineering services, and more are available to help users get started.

Takamitsu Matsubara, a professor at the Nara Institute of Science and Technology, which jointly developed the FKDPP algorithm with Yokogawa, commented, “It would be fair to say that Yokogawa was able to commercialize reinforcement learning thanks to FKDPP reducing and making the problematic number of learning trials more feasible.

Yokogawa has demonstrated the value and practicality of FKDPP. There appears to be no other example involving the commercialization of reinforcement learning in a control system product for the manufacturing industry. I expect FKDPP will create new value for industries.”

Kenji Hasegawa, a Yokogawa vice president and head of the Yokogawa Products Headquarters, added, “Control is an important part of product quality and customer revenue, but the know-how of veteran operators has been essential. We believe the autonomous control AI (FKDPP) is a disruptive innovation that will address the concerns of customers who have issues with conventional control technologies, and this will do so in a number of new ways.

In the future, we will also open up a new field that will involve the achievement of flexible control based on management strategy. By providing value to our customers, we will contribute to the sustainable development of industry, while minimizing the impact on the environment, and lead IA2IA – the transition from industrial automation to industrial autonomy.”

Notes:

*1 Proportional-Integral-Derivative control, first proposed by Nicolas Minorsky in 1922, is an infrastructure-control technology for processing industries and factory automation that is used to control items such as flow rate, temperature, level, pressure, and ingredients. It implements control aimed at target values based on P, I and D calculations according to deviations between current and set values.

The amount of operation is the sum of the calculated results of P (proportional), I (integral) and D (differential) operations according to the deviation between the current value and the set value, and is controlled toward the “target value”. In some cases, a value might exceed (overshoot) the set value or take time to settle to avoid overshoot.

*2 Advanced Process Control uses a mathematical model to predict process responses and supply set values to the PID control loop in real time in order to improve productivity, quality, and controllability. It is easily applied for control to increase production, reduce labor time, and save energy. APC results in smaller deviations in data, making it possible to more closely approach the limits of operating performance (i.e., optimal performance), but it is not adept at responding to the nonlinear chemical reactions, and changes in machinery.

*3 In a World First, Yokogawa and JSR Use AI to Autonomously Control a Chemical Plant for 35 Consecutive Days – Putting into practical use a next-generation control technology that takes into account quality, yield, energy saving, and sudden disturbances –