Control of underactuated robots has received significant attention and its application areas comprise several types of industrial and service robotic manipulators. The purpose of research in this area is to design robotic mechanisms that can be controlled despite having a number of actuators that is smaller than their degrees of freedom. This approach can reduce the cost and weight of robots or can provide robotic systems with tolerance to actuators failures. Again the control problem for such robots can be treated with (i) global linearization methods, (ii) approximate linearization approaches and (iii) Lyapunov methods. To achieve model-free control of underactuated manipulators, improved estimation approaches are developed, allowing the real-time identification of their unknown dynamics or kinematics. Moreover, to implement feedback control of underactuated robots through the measurement of a limited number of the robot’s state variables, nonlinear filtering methods of proven convergence are developed. In particular the chapter develops the following topics: (a) Nonlinear optimal control for multi-DOF underactuated overhead cranes, (b) Nonlinear optimal control for ship-mounted cranes (c) Nonlinear optimal control for the rotary (Furuta’s) pendulum, (d) Nonlinear optimal control for the cart and double-pendulum system, and (e) Nonlinear optimal control for a 3-DOF underactuated robotic arm.