"Little Helper" - An Autonomous Industrial Mobile Manipulator Concept

This paper presents the concept “autonomous industrial mobile manipulation” (AIMM) based on the mobile manipulator “Little Helper” – an ongoing research project at Aalborg University, Denmark, concerning the development of an autonomous and flexible manufacturing assistant. The paper focuses on the contextual aspects and the working principles of AIMM. Furthermore, the paper deals with the design principles and overall hardware and software architectures of “Little Helper” from a functional and modular mechatronics point of view, in order to create a generic AIMM platform. The design challenges faced in the project is to integrate commercial off ‐ the ‐ shelf (COTS) and dedicated highly integrated systems into an autonomous mobile manipulator system with the ability to perform diverse tasks in industrial environments. We propose an action based domain specific communication language for AIMM for routine and task definition, in order to lower the entry barriers for the users of the technology. To demonstrate the “Little Helper” concept a full ‐ scale prototype has been built and different application examples carried out. Experiences and knowledge gained from this show promising results regarding industrial integration, exploitation and maturation of the AIMM technology.


Introduction
Today, robots are widely used in industry to perform dumb, dangerous, dull and dirty tasks.Furthermore, robots-based manufacturing increases product quality, improves work conditions, and leads to an optimized use of resources.Therefore, robotics forms an essential part of the manufacturing backbone in Europe (EUROP, 2009).However, the industrial robots of today (Figure 1) are rather inflexible as they are often dedicated and/or fixed.In recent years the inflexibility and inadequacy of industrial robotics has become more and more evident.This is highly related to the globalization of markets, the trade instability, and the explosion of product variety, which are stressing the time to market and the need for increased adjustment and responsiveness.Also the shift in paradigm from mass production to customized production (down to one-of-a-kind) and the resumption of local production in industrialized countries have created industrial needs for agility and flexibility, especially in the field of automation and robotics (Jovan, F., et al., 2003) (Wucherer, K., 2003).Great advances in technology, particularly in processing power and sensoring, provide significant opportunities for increasing the autonomy and flexibility of industrial automation systems, maintaining costs acceptably (Mekid, S. et al., 2007).One way of addressing this is the interaction technology mobile manipulation (Figure 2), which increases the potential of industrial automation.To support the development and to extend the application prospective of industrial robotics, it is rational to combine locomotion capabilities with manipulation abilities, hereby creating mobile manipulators.Task flexibility and robotic mobility are two main advantages that mobile manipulators can bring to manufacturing applications.Compared to traditional industrial robots (Figure 1) it is easier for mobile manipulators to adapt to changing environments and perform a wide variety of manufacturing tasks (Hamner, B. et al., 2009).Another advantage of this type of robot is that the existing industrial environments do not have to be altered or modified as in the case of Automated Guided Vehicles (AGV's), where permanent cable layouts and/or markers are required for navigation (Datta, S. et al., 2008).
The conventional architecture of these types of robots is a robot manipulator mounted upon a mobile platform, extended by a vision and tooling system, respectively (Hentout, A. et al., 2010).The mobility extends the workspace of the robot manipulator, which increments its operational capability and flexibility.In general, mobile manipulators allow the most usual operations of robotics that require both locomotion and manipulation capabilities.Mobile manipulators, either autonomous or tele-operated, are particularly well suited for human-like tasks, and they have applications in many different areas, such as ( This paper presents the underlying concepts, ideas and working principles of autonomous industrial mobile manipulation (AIMM), based on a proof-of-concept philosophy, with the mobile manipulator "Little Helper" (Figure 3) as experimental platform.The focus is on both theoretical and practical aspects.At Aalborg University, Denmark, a research project has been running since 2007 with the aim to develop the state-of-the-art autonomous industrial mobile manipulator "Little Helper".The final goal of this work is to develop an industrially suitable mobile manipulator concept and subsequently test it in practice.In this way fundamental experience with the mobile manipulation technology is obtained.The paper is organized as follows.The next section gives an overview of the general "Little Helper" concept with focus on the contextual aspects and working principles of AIMM.The third section describes the design principles and overall architecture (hardware and software) of "Little Helper" from a functional mechatronics point of view.The fourth section presents a full-scale and working prototype of "Little Helper" divided in hardware, software and safety aspects.Different application examples of the system as a whole (and its functions) are presented in the fifth section.Finally, a conclusion is presented together with outlines on future work.The rationale of "Little Helper" is to seek an optimum between traditional automation and manual labor, with the obvious benefits of a compromise between efficiency and flexibility, as illustrated in Figure 4.

The context of AIMM
"Little Helper" must be able to carry out various applications and tasks in typical semi-structured industrial environments (Figure 5), such as: Process control  Dangerous/inaccessible tasks To achieve this, "Little Helper" must be able to communicate with both the manufacturing equipment (machine-machine) and the operators (human-machine).This is realized by integrating "Little Helper" on the general network of the industrial environment (Figure 6).In this way it is possible to schedule and control globally, as the mobile manipulator becomes an agent and resource on the same level as corresponding manufacturing devices (e.g.stationary robots and production machines).

Working principle
Before autonomous use, "Little Helper" must know the environment, which is achieved through active use of onboard sensors and an a priori layout of the work area (Figure 5).After this the mobile manipulator can navigate to and perform various tasks at the workstations, depending on the current manufacturing needs.A typical operation cycle consists of four steps (Figure 7).The on-board sensors of "Little Helper" are also used to meet the critical safety and robustness requirements.The different systems of the mobile manipulator are decoupled, so when "Little Helper" is driving around the industrial environments, only the mobile platform is active.When using the robot manipulator, the mobile robot is stationary, with active use of the sensors to monitor the workspace, so that reduced operation speed or emergency stop can be executed, if necessary.
To enhance the diversity of operations, ʺLittle Helperʺ is equipped with an automatic tool change system and different tools, thereby realizing dexterous manipulation.
In addition, to increase the degree of autonomy, "Little Helper" is able to charge automatically, when reaching a critical battery level (Bøgh, S. et al., 2008).

Setup phase
A number of steps have to be completed before actual deployment of "Little Helper" in industrial context: 1. Adaption of production environment 2. Software routine learning; production 3. Software routine learning; applications The setup phase contains the configuration of the mobile manipulator -that is the adaptation and learning of the industrial environment and the different applications.All user interaction takes place here, so that "Little Helper" can act fully autonomous during the operation phase.

Adaptation of production environment
The first step is to use the mobility and sensors of the mobile platform to map the surrounding environment into a digital map.To assist the mobile manipulator in achieving acceptable localization tolerances, it is necessary to place reflector marks and calibration targets at workstations.Reflector marks are used for initial coarse localization of the mobile platform relative to the workstations, while calibration targets are used for subsequent fine localization of the robot manipulator, so that industrially acceptable tolerances are realized.

Software routine learning; production
The required characteristics of autonomy and mobility are challenging, because they are conflicting in achieving acceptable localization tolerances.The mobile platform, alone, is not always able to achieve the desired localization.Therefore dedicated calibration can be necessary.This calibration requires use of both the robot manipulator and the vision system (Hvilshøj, M. et al., 2010).Use of calibration ensures unique localization of the robot manipulator, and thereby the entire mobile manipulator, in relation to a workstation.This makes "Little Helper" able to return to configured workstations and carry out the intended tasks and applications.

Software routine learning; applications
Based on the above, the operator is able to define the tasks that "Little Helper" must perform.The mobile manipulator can be configured for two types of tasks: In general, teach-and vision-based configurations allow "Little Helper" to interact autonomously with a semi-structured industrial environment.Furthermore the configurations correspond to different tasks, depending on requirements for; speed, accuracy and safety.

Operation phase
When the setup phase is completed, the configurations of the workstations are saved, resulting in a library of tasks that the mobile manipulator can carry out.The operation phase consists of two strategies, depending on the desired level of autonomy and automation: 1. Machine communication (fully automatic) 2. Predefined work cycle (semi automatic)

Machine communication (fully automatic)
In the machine communication mode, the management of "Little Helper" is carried out by a global control and scheduling device via wireless network, depending on the current manufacturing needs.

Predefined work cycle (semi automatic)
In contrast, in the predefined work cycle mode, an operator locally and chronologically schedules the tasks that "Little Helper" has to carry out.

"Little Helper"; a complex mechatronics system
The development of an autonomous industrial mobile manipulator requires insight in many technology fields and research disciplines, and especially knowledge on how to integrate these.In general, "Little Helper" is a complex and integrated mechatronics system, consisting of the principle subsystems; mobile platform, robot manipulator, vision and tooling.The general mechatronics system model (Figure 8) is composed of input, output, human and environment.
Input and output includes material, energy and signal input/output.Humans can control or influence the mechatronics system, and the mechatronics system is functioning in a certain environment (Isermann, R., 2009).In the "Little Helper" project we are focusing on general application and use of the entire mechatronics system of an autonomous industrial mobile manipulator.

Design considerations and specifications
As "Little Helper" is intended to be a versatile automation solution, it is desirable to maximize the flexibility of the system.However, it is not possible to create a universal autonomous mobile manipulator that can solve all tasks in industrial environments, as the demands differentiate according to the different users.
One way to address this is the use of modularization and standardization, making it possible to achieve a high customization level in a cost effective manner (Harlou, U., 2006) (Pine, B. J., 1993).
Modularity makes it easy to customize the mobile manipulator system by tailoring the module combination or by changing, adding, and/or removing modules of predefined products.Furthermore, modular product architectures are aimed at increasing reuse, reducing development risk and system complexity, and improving upgradability (Sawhney, M. S., 1998).Therefore, it is rational to use a predominantly modular architecture, in order to be able to configure the system for different manufacturing scenarios (e.g. a mobile manipulator with a large payload or a mobile manipulator with welding capabilities), hereby creating a concept platform within the field of autonomous industrial mobile manipulation.
The modular architecture of "Little Helper" is realized by the definitions and methodologies of Ulrich and Eppinger (Ulrich, K. T. et al., 2003), by use of functional elements (individual operations and transformations contributing to the performance) and chunks (building blocks consisting of a collection of physical elements).
Based on the mapping relations between functional elements and chunks, modular product architectures can be realized.Furthermore, "Little Helper" relies on the use of standardized (industrially accepted) and commercial off-the-shelf (COTS) hardware components and software systems, in order to simplify the configuration procedure.

The autonomous industrial mobile manipulator platform
Realization of the "Little Helper" concept relies on architectural principles related to product families and product platforms, respectively.One of the primary goals is to find a generic AIMM platform, based on modularization (functional elements/modules and chunks) and standardization (COTS).The term platform pinpoints the totality of parts of a product that is reused across several products or product families (Harlou, U., 2006).In Figure 9, a generic platform for autonomous industrial mobile manipulators is shown, based on the underlying "Little Helper" concept (section 2).The platform is created from a mechatronics point of view, with inspiration from the five major functional modules.

Software architecture
The software system design for "Little Helper" consists of distributed servers providing services for each mechatronics subsystem available in the GUI (Figure 10).This makes the accessibility generic, which follows the general modular approach (Hvilshøj, M. et al.

2009).
The primary task of the software is to handle the communication between the hardware components.This is an extensive and complex task, because the hardware components use different communication interfaces, e.g.LAN, RS-232, USB and FireWire.Therefore, the software is the central part in the integration of the individual systems and components (Bøgh, S. et al., 2008).
In order to use each subsystem through the GUI, dedicated modules handle the communication between the different layers (Figure 10).That is; the manipulator system is controlled by a server script on the robot controller, the mobile platform system is controlled by a server script on the mobile platform controller, etc.

Overview
In this section "Little Helper" generation 0 is presented.This covers a prototype (hardware, software and safety aspects) of "Little Helper" (Figure 11), in order to test the central parts of the presented AIMM concept (section 2), hereby serving as a technology demonstrator platform.

Hardware
The various hardware systems and components of the "Little Helper" prototype are shown in Figure 12.Each plays an important role in achieving the desired design specifications and general functionalities.The mobile platform system consists of a non-holonomic mobile platform (Neobotix MP-L655) with on-board scanners (laser-range, ultrasonic and motor encoders), used for navigation and safety, when operating in industrial environments.Inside the mobile platform is a battery pack, consisting of eight 12 VDC lead acid batteries, that generates 24 VDC.Furthermore, the mobile platform is equipped with a 2.0 GHz dual core and 2 Gb RAM computer running Windows XP, which handles the computing for the individual systems and components.The manipulator system consists of an industrial six DOF robot manipulator (Adept Viper s650) running on 230 VAC and adjacent components (power/signal and communication module).A DC/AC inverter is used to invert the 24 VDC from the battery package to 230 VAC.
The tooling system consists of a pneumatic tool change unit and two tools (a vacuum device and a parallel gripper) in order to realize dexterous manipulation.In addition, a compressor and air tank is used for air supply.
Finally, the vision system consists of a monochrome CCD FireWire camera (1360 x 1024) and a digital adjustable motorized lens (iris and focus) and bar light system.
The total weight of the mobile manipulator prototype is around 230 kg with a payload of 20 kg.The weight distribution ensures proper traction and stability during operation.The speed of the mobile platform is 1 m/s and the prototype can run continuously for 8 hours.However, it is also able charge automatically during operation.
The "Little Helper" prototype solely consists of COTS components, so each technology entity is well tested and, in general, accepted for industrial use.

Software
The software architecture for the "Little Helper" prototype consists of distributed servers providing services for each hardware subsystem (Figure 13), which the GUI "FlexRob Studio" (Figure 14) interacts and communicates with.The distributed software architecture consists of one client ("FlexRob Studio") and one server for each hardware system carrying out tasks upon request from the client.The client processes the data received and displays it in "FlexRob Studio".In order to achieve the desired functionality and use of "Little Helper", a general language for AIMM has been developed (Table 1).In this way complex industrial tasks can be solved, without the operator worrying about hardware interfaces and low level robot programming.All aspects are integrated in "FlexRob Studio" (Figure 14).
In Table 1 a partial overview of the general command structure is illustrated.All commands are initialized with a three digit unique identifier tag (UIT), followed by hardware specific commands (actions), which is send by TCP/IP to the respective subsystems and/or components.

Safety aspects
In the case of autonomous mobile manipulators in industrial environments proven safety principles are vital.Despite numerous considerations no standard procedures have been provided for the safety conformable design of AIMM.It is preserved that a safety conformable design for AIMM can be achieved.However, experience with the design of autonomous mobile manipulator systems and their operation in industrial environments is crucial for their safety evaluation and clearance by safety agencies.In the prototype design and testing, the safety procedures are realized in a simple manner.The systems of the mobile manipulator are decoupled, so when "Little Helper" is driving around the industrial environments, only the mobile platform is active.When using the robot manipulator, the mobile manipulator system is stationary, with active use of the sensors (laser-range and ultrasonic) to monitor the workspace of the robot manipulator, so that reduced operation speed or emergency stop can be executed, if necessary.

Overview
In this section two application examples of AIMM are presented, which have been carried out by use of the "Little Helper" prototype.In this way it is possible to test the main aspects of the "Little Helper" concept and gain further experience with the AIMM technology.

Figure 2 .
Figure 2. The interaction technology mobile manipulation.Integration of the technology entities; mobile platform, robot manipulator, vision and tooling.

Figure 3 .
Figure 3. "Little Helper" from the Department of Mechanical and Manufacturing Engineering, Aalborg University.

Figure 5 .
Figure 5.A typical industrial environment; the workspace of autonomous industrial mobile manipulators.

Figure 6 .
Figure 6.Agent based industrial network architecture.


Teach-based; covers applications, where the locations of the robot manipulator are referenced to a calibration target or the mobile manipulator itself  Vision-based; covers applications, where the locations of the robot manipulator are controlled by the observations of the vision system

Figure 11 .
Figure 11.Hardware/software overview of the prototype.

Figure 12 .
Figure 12.Hardware architecture of the prototype.
The various hardware components are connected through different types of interfaces.Many of the components use electric and/or pneumatic interfaces, and digital signals for control (Bøgh, S. et al., 2008).

Figure 13 .
Figure 13.Software architecture of the prototype.
However, approaches exist to arrive at a safety conformable solution by considering successful solutions from the overlapping field of service robotics (Haegele, M. et al., 2001):  Mobile robots in human environments can be made safety conformable with limited effort by use of category 4 laser scanners and/or bumpers. Service robots often comply with stringent safety regulations in fully automatic mode by limiting forces (150 N) and velocities (0.25 m/s) induced by its robot manipulator. Controlling the robot manipulator workspace by sensors and predicting human motions, in order to provide collision avoidance. Employing redundancy in safety critical systems (e.g.sensors).

5
Alfaro, C. et al., 2004):  Health-care (professional service) Worldwide, efforts are being made to use autonomous mobile manipulators to relieve human operators of tedious, repetitive, and hazardous tasks.At the moment ), while the integration, use and application have been neglected.This means that few implementations of mobile manipulators, in industrial environments, have been reported -e.g.(Helms, E. et al., 2002), (Stopp, A. et al., 2003), (Katz, D. et al, 2006), (Datta, S. et al, 2008) and (Hentout, A. et al., 2010).Therefore new initiatives are required, in order to realize industrial acceptance and maturation of autonomous mobile manipulators.
(Bøgh, S. et al, 2008)nded to be a flexible and versatile automation solution that is intuitive and simple to use, so it becomes plug-and-produce.In this way the entry barriers for operators are kept lowest possible, so that no technical background or long-term training is needed for the use of the mobile manipulator(Bøgh, S. et al, 2008).In general, "Little Helper" must be able to: