Sensors and Actuators for Dexterous Manipulation: Bridging Hardware and Application Needs

High-fidelity visuo-tactile sensing is important for precise robotic manipulation, yet most vision-based tactile sensors rely on opaque coatings that enable tactile sensing but block direct visual observation. We present MuxGel, a spatially multiplexed sensor that captures external visual information and contact-induced tactile signals simultaneously through a single camera. A checkerboard coating interleaves reflective tactile regions with transparent visual windows while preserving the standard GelSight-style form factor. To recover dense signals from these spatially incomplete observations, we develop a physics-based simulation pipeline for large-scale data generation and a dual-modal reconstruction network with sim-to-real adaptation. Experiments on unseen objects demonstrate accurate tactile reconstruction and local visual recovery. Closed-loop robotic grasping further shows that reconstructed vision can support alignment while reconstructed tactile feedback supports contact control. MuxGel therefore provides a plug-and-play approach for simultaneous visuo-tactile sensing by replacing only the gel pad of an existing optical tactile sensor.

Dexterous robot teleoperation and learning from human demonstration require continuous capture of human hand configuration, yet vision-based tracking can degrade under self-occlusion and object occlusion. We present TacPose, which repurposes a triaxial tactile sensor commonly used for robotic contact sensing as a wrist-worn human-state sensor. A single soft 4 × 6 tactile array on the inner wrist captures normal and tangential skin and soft-tissue deformation while leaving the hand uninstrumented. A spatial-temporal network maps short tactile sequences to wrist-relative 3D hand landmarks. In a preliminary five-participant study, normal-force-only sensing achieved mean per-joint position errors of 3.44 mm within-session, 12.27 mm across sessions, and 13.76 mm across users, reaching an error range comparable to prior specialized wrist-worn pose-sensing approaches despite using only a commercial tactile array and simple mounting hardware. Full triaxial sensing slightly improved within-session and cross-session accuracy but reduced cross-user robustness. TacPose demonstrates how tactile hardware already available to robotics researchers can be rapidly repurposed into a compact, soft, and wearable interface for hand-state capture, while identifying mounting repeatability and skin–device coupling as key challenges for deployment.

Surface electromyography (sEMG) can provide muscle-activation information for human demonstration, teleoperation, and assistive robots. However, conventional electrodes depend on a stable skin-electrode contact which often require additional conductive gel prone to drying out. We present a thin, self-adhesive capacitive sEMG (cEMG) interface consisted of liquid-metal electrode fully encapsulated in polydimethylsiloxane (PDMS) and a skin-facing Tween 80 – PDMS adhesive layer. Five participants completed forearm, biceps, and shoulder experiments. The electrodes showed reliable performance under immersed conditions (active/rest ratio: air 15.8 dB; immersed 10.9 dB). An eight-channel shoulder array produced movement-dependent spatial gradients, and biceps movements remained recordable after attaching with 60% strain. These preliminary results show conformal cEMG acquisition across wet, curved and highly deformed conditions.

We present a preload-regulated soft rolling-band gripper for adaptive grasping and in-hand manipulation. Independent finger actuation sets the grasp geometry, while active band-tension modulation adjusts contact preload and holding strength without changing the finger aperture. The same elastic bands serve as rolling contact surfaces for object translation and rotation while maintaining conformal contact. Experiments demonstrate manipulation of diverse rigid, fragile, and elongated objects, and show that increased band preload substantially reduces slip during tight bottle-cap opening.

Quasi-direct-drive (QDD) actuators offer safe, adaptive contact through high force transparency, low reflected inertia, and a reduced sim-to-real gap, but motor torque scaling makes a fully articulated QDD hand heavy, bulky, and prone to heating. We present the Berkeley QUAD (Quasi-direct-drive, Underactuated, Asymmetric Design) Hand, an anthropomorphic hand with four fingers, 11 degrees of freedom, and 8 degrees of actuation, weighing approximately 1 kg. Dexterity is delegated to the radial fingers through individual QDD actuation, while strength and passive adaptability come from the ulnar fingers through an underactuated, compliantly coupled linkage transmission driven by a shared, larger QDD motor.

Soft and rigid robotic hands both provide different advantages; however, combining the two is often challenging due to complex dynamic modeling and control, material limits, and slow actuation. To overcome these challenges, we propose IVSA, an impedance control based variable stiffness actuator, capable of up to a 130× increase in stiffness, and instantaneous stiffness modulation. IVSA works by mechanically transmitting the variable impedance of a single joint-level impedance controlled brushless DC motor through a compliant structure using a tendon, reducing the need for complex modeling or control. To validate and understand the IVSA structure's mechanical properties, we conduct numerical simulations as well as experimental tests to validate these. We also design and test a parallel jaw gripper utilizing four IVSA beams.

Haptic feedback improves the accuracy and stability of teleoperated gripping, where both normal force and tangential torque at the fingertip govern grip control. Rendering such multimodal feedback within a compact, fingertip-scale device, however, remains challenging. We present a Kresling-origami-based haptic device that delivers simultaneous normal, torsional, and vibrotactile feedback from a single compact structure. Kresling dipole, composed of two mirror-symmetric Kresling cells, is driven by a central neodymium magnet and two coaxial coils. Since axial and rotational motions are coupled in the Kresling structure, the structure converts a purely linear electromagnetic force into normal and torsional feedback. The device delivers normal force up to 0.95 N and torque up to 4.53 N·mm with a response time of approximately 8 ms, and could also deliver vibrotactile feedback up to 250 Hz.

Contact-rich manipulation requires object pose before contact and local geometry at contact, typically provided by separate sensors. We present a transparency-switchable visuo-tactile sensor that captures both with one camera. Its photochromic membrane is transparent for proximity imaging and becomes an opaque reflective surface under UV illumination for tactile sensing. A single model predicts depth, surface normals, and planar object pose from the proximity view, opaque view, or their sequential fusion. On 472 held-out real contacts, fusion achieves 0.118 mm depth RMSE and 0.63° rotation error. Proximity-only retains pose accuracy but increases depth error to 0.285 mm, while opaque-only retains contact geometry but increases rotation error to 2.88°. These results show that switchable transparency provides complementary pre-contact pose and contact geometry through one optical path.

Affordable multi-axis force sensors usually measure the deformation of a custom compliant part. Any creep, hysteresis, or thermal sensitivity in that part then becomes a property of the finished sensor. We instead make four commercial load cells carry the full load. Their factory-characterized beams replace the custom compliant part, while one fitted matrix accounts for assembly tolerances, gain differences, and wiring polarity. The resulting three-axis sensor costs under $120 and has a 2.6–3.5 mN noise floor. Off-axis leakage remains below 1 %, and lateral hysteresis remains below 0.5 % of span. After one scale factor, its output agrees with a commercial reference to 1.0–2.5 % of full scale across more than 300,000 in-contact samples. The design suits tools whose contact point is known and makes an accurate three-axis sensor accessible to any research lab with a 3D printer.

The Smart Suction Cup is a flow-based tactile sensing gripper for pick-and-place operations. Prior haptic search methods rely on suction (pull mode), which requires the cup to press against an object and can displace unsecured items. We introduce reverse flow (push mode), in which air is directed outward through the cup face, as a new haptic sensing modality. Push mode remains informative at distances where pull mode yields no signal, correlates with the applied normal force, and locates graspable positions without displacing loose objects — extending the operating envelope of the Smart Suction Cup toward broader real-world grasping applications.

Dexterous manipulation in confined spaces requires local control of hand orientation. Without a wrist, a dexterous hand must obtain this local orientation through coordinated motion of the robot arm, often involving several joints and a more complex end-effector path. We present CRAFTWrist, a concentric 2-DoF wrist extension that mounts between a robot arm and the CRAFT Hand without modifying the hand. Two XC430-T240BB-T servos drive sideways and front-back rotation through short rigid transmissions. Our initial hardware prototype actuates both axes under load, achieving a demonstrated workspace of ±20° in radial–ulnar deviation (left–right) and a front-back range of +80° in flexion and −18° in extension. We characterize the resulting finger-motor loading across several wrist postures and demonstrate how the wrist supplies local orientation in grasping, nail hammering, and blackboard wiping.

Contact-rich dexterous manipulation with multi-finger hands remains an open challenge in robotics because task success depends on multi-point contacts that continuously evolve and are highly sensitive to object geometry, frictional transitions, and slip. Recently, tactile-informed manipulation policies have shown promise. However, most use tactile signals as additional observations rather than modeling contact state or how their action outputs interact with low-level controller dynamics. We present Contact-Grounded Policy (CGP), a visuotactile policy that grounds multi-point contacts by predicting coupled trajectories of actual robot state and tactile feedback, and using a learned contact-consistency mapping to convert these predictions into executable target robot states for a compliance controller. CGP consists of two components: (i) a conditional diffusion model that forecasts future robot state and tactile feedback in a compressed latent space, and (ii) a learned contact-consistency mapping that converts the predicted robot state-tactile pair into executable targets for a compliance controller, enabling it to realize the intended contacts. We evaluate CGP using a physical four-finger Allegro V5 hand with Digit360 fingertip tactile sensors, and a simulated five-finger Tesollo DG-5F hand with dense whole-hand tactile arrays. Across a range of dexterous tasks including in-hand manipulation, delicate grasping, and tool use, CGP outperforms visuomotor and visuotactile diffusion-policy baselines.

Touch sensing is beneficial for solving a wide variety of manipulation tasks. While there exists a wide range of tactile sensors with different properties, exploiting the fusion of multiple heterogeneous tactile sensors to improve manipulation learning remains underexplored. We present Multi-Resolution Tactile Sensing (MiTaS), a representation framework that leverages multiple tactile sensors operating at different temporal resolutions in order to solve complex contact-rich manipulation tasks. We propose a novel architecture using modality-specific convolutional stems and transformer-based fusion that effectively fuses information from an RGB camera stream, a vision-based GelSight Mini sensor and a high-frequency event-based Evetac sensor. This multi-sensor representation then conditions a flow-matching policy for solving downstream tasks. Experimental results across five contact-rich manipulation tasks demonstrate the effectiveness of multi-resolution tactile features in imitation learning. MiTaS achieves an average success rate of 80 %, while vision-only (31 %) and visual-tactile (54 %) baselines cannot solve the task reliably.

We present RevoSim, a scalable and high-fidelity multimodal tactile simulator for the BrainCo Revo3 hand that jointly models 285 piezoresistive taxels and five visuotactile fingertips. RevoSim efficiently generates synchronized pressure, depth, RGB, and marker-displacement observations; on a single RTX 5090, its depth-conditioned marker-displacement configuration supports up to 4,096 parallel environments. We train in-hand manipulation and grasping policies entirely in simulation, establishing a foundation for future sim-to-real transfer.

In-hand manipulation without external sensing is challenging due to uncertainties from finger-object contacts and disturbances by gravity. While reinforcement learning has shown promise in learning complex finger gaiting, existing approaches do not prioritize maintaining well-conditioned grasps for sustained manipulation. We introduce two complementary physics priors for robust in-hand rolling: a global grasp-quality prior derived from classical grasp analysis and a local contact-geometry prior based on fingertip curvature. The grasp-quality prior is used as a dense reward-shaping term that encourages well-distributed contacts with improved worst-case wrench resistance. The contact-geometry prior is expressed in the fingertip geometry that mechanically shapes the contact interface toward task-aligned rolling while reducing off-axis drift. We evaluate the effect of these priors on learning in-hand rotation for a multi-fingered robotic hand manipulating three objects at four palm orientations. Results show significant improvement in rotation efficiency, grasp stability, and disturbance rejection, suggesting that physics priors embedded in both learning and fingertip morphology improve task robustness.

We present Stiffness Copilot, a shared-control method that automatically adjusts robot stiffness during teleoperation: the operator commands pose, and a vision-based policy adjusts robot impedance online to match the contact situation. Appropriate stiffness depends on the contact situation and task phase, so Stiffness Copilot predicts direction-dependent stiffness from wrist-camera images at runtime. We train the policy in two stages. First, in simulation, we use privileged contact and simulator state information to infer stiffness labels for recurring interaction states across demonstrations. Second, we use these labels to supervise a lightweight vision policy, and use a vision foundation model with image augmentation for zero-shot sim-to-real transfer. A human-subject experiment comparing constant high stiffness, constant low stiffness, and Stiffness Copilot indicates that Stiffness Copilot combines the benefits of both baselines.

Dexterous robotic manipulation requires simultaneous consideration of hardware and control algorithms, but these are often developed in isolation. To address the embodiment gap in teleoperation, we present an evaluation framework that integrates three morphologically diverse robotic hands (ORCA, Inspire, LEAP) and two control modalities (direct joint angle and inverse kinematics) on a single robotic arm. Preliminary system integration reveals significant friction between human biomechanics and robot kinematics, such as biologically unnatural IK solutions and limitations of coupled joints. We additionally propose an evaluation methodology using standardized clinical dexterity tests with quantitative metrics.

Teleoperation is increasingly used to collect high-quality human demonstrations for training dexterous manipulation policies, but current systems either rely on visual feedback only or provide force feedback through heavy exoskeletons that constrain natural finger motion. My research addresses this challenge through compact soft actuation for multimodal wearable haptics, including dielectric elastomer actuators that generate localized skin deformation and vibration, and electroosmotic-pump-based actuators capable of larger force feedback. To address the hardware bottleneck of the robotic hand itself, I also explore soft artificial muscles driven by compact electrohydrodynamic fluidic actuation, with a miniature pump generating blocked pressures approaching 500 kPa, toward robotic hands actuated by distributed, intrinsically compliant fluidic muscles inspired by human musculature.

Dexterous robot hands offer rich opportunities for multifunctional manipulation, where a robot must execute multiple skills in sequence while maintaining control over previously grasped objects. Most prior work in dexterous manipulation focuses on single-object, single-skill tasks. In contrast, our insight is that many sequential tasks require resource-aware grasps that conserve fingers for future actions. We study sequential grasp-conditioned dexterous manipulation, where a robot first grasps an object and then performs a second, distinct manipulation subtask while preserving the initial grasp. We introduce HANDFUL, a learning framework that models finger usage as a limited resource and encourages exploration of resource-aware grasps through finger-level contact rewards. These grasps are subsequently selected for downstream tasks via curriculum-based policy learning. We further propose HANDFUL-Bench, a simulation benchmark that introduces sequential dexterous manipulation tasks across multiple second-subtask objectives, including pushing, pulling, and pressing, under a shared grasp-conditioned setup. Extensive simulation results demonstrate that prioritizing resource-aware grasps improves second-subtask success and robustness compared to a baseline that greedily optimizes the initial grasp before attempting the second subtask. We additionally validate our approach on a real dexterous LEAP hand.

Multi-agent manipulation already brings multiple moving cameras into the workspace: every arm carries a wrist camera and changes viewpoint while acting. Yet active perception for manipulation is often framed as requiring a dedicated sensing arm. We introduce Multi-Agent Active Perception (MAAP), where each arm is dual-purpose: it manipulates objects and supplies task-relevant visual observations for the team. To make this practical at the controller level, we propose Role-Aware Imitation Learning (RAIL), an action-chunking policy that infers each arm's manipulation/perception role and conditions action generation on that role through feature-wise modulation and a role-gated action head mixture. Across four simulated collaborative tasks, success rises from 56.5% with a fixed camera to 70.0% with distributed wrist-camera perception, and to 79.2% with MAAP+RAIL. On the most occluded three-arm microwave task, RAIL reaches 82% against 14% for a single active wrist view. A real dual-arm blocked-placement task further shows 14/20 successes with MAAP+RAIL versus 0/20 from a fixed camera.

Haptic devices vary widely in their hardware requirements, and different vibrotactile actuator classes — linear resonant actuators (LRAs) driven by AC signals and eccentric-rotating mass actuators (ERMs) driven by PWM-controlled DC signals — typically need custom, single-purpose control electronics. This lack of flexible actuator interface hardware impedes rapid prototyping and deployment of wearable haptic systems. Building on a prior balance-assistive haptic feedback device, we develop a flexible haptic control platform for multi-actuator wearable systems intended for future open-source release. The primary contribution is a controller architecture that supports both AC- and DC-driven vibrotactile actuators using a common hardware platform.