Watch Me Calibrate My Force-Sensing Shoes!

The force-sensing shoe design is shown in Fig. 1a. The shoe consists of two 3-D printed plates and measures forces using four 1-D load cells. One side of the load cell is cantilevered at the holders of the bottom plate, and its other side connects the corners of the top plate. Slightly different from the first version design in [19], the current design includes a six-by-three array of holes on the top plate, which are used for manual calibration (see III). These holes cover the sensing polygon constructed by the NAO’s factory default FSRs (grey area) [22]. This calibration area, slightly smaller than the shoe’s support polygon, is the allowable CoP region for the robot’s default functionalities for safety concerns.

The shoe measures GRF and CoP. The GRF, $F$, is acquired by summing up all the force output $f_{i}$ (Fig. 1b):

The CoP, C, is obtained by averaging the 2-D position of each load cell in the spatial coordinate, ${\bf t_{i}}$, weighted by their corresponding force outputs, $f_{i}$ (Fig. 1b):

The sensing principle for single support can also be applied to double support by incorporating all eight sensors.

To calibrate each load cell, we first record its no-load voltage $S_{0}$ (Fig. 2a, left) and then record the updated voltage $S_{G}$ after applying a known weight $G$ to the sensor (Fig. 2a, right). Due to the load cell’s linear response between the voltage output and the applied force, the voltage to force ratio is $\sigma=(S_{G}-S_{0})/G$. Then, the force output $F$ corresponding to a loaded voltage output $S$ is given by:

where $a$ and $b$ are the modified scaling factor and offset.

As introduced in [19], our force-sensing shoes possess high accuracy in GRF measurement (Fig. 3a) but low accuracy in CoP measurements (Fig. 3b, left). To improve the CoP measurement accuracy, we apply seven different weights (1, 1.5, 2, 2.5, 3, 3.5, 4kg masses $\approx 18\%\text{, }27\%\text{, }37\%\text{, }46\%\text{, }55\%\text{, }64\%\text{, }73\%$ of the robot’s mass) to the holes on the shoe’s top plate using a 3-D printed weight support (Fig. 2b shows three examples). Then, we utilize nonlinear least squares to minimize the error between measured CoPs and their corresponding ground truths, which are the known 2-D positions of the holes.

Inspired by the force plate calibration method introduced in [23], our manual calibration defines a corrected CoP, $[p_{x},p_{y}]^{T}$, by summing the measured CoP $[p_{0x},p_{0y}]^{T}$ and a correction term $[\Delta x,\Delta y]^{T}$:

The correction terms are formulated as:

which are designed first using a second-order polynomial of the measured CoP, $\mathbf{p_{0}}$, parameterized using $a_{i}$, $b_{i}$ ($i=1\ldots 4$). This part is used to improve the averaged measurement at each calibration hole. The correction terms also includes first-order terms of the force outputs $f_{i}$ parameterized using $m_{i}$ and $n_{i}$ $(i=1\ldots 4$, corresponding to each force output). This second part is for reducing the deviation at the same calibration spot for different applied weights. Eventually, nonlinear least squares is used to minimize the error between the corrected CoP, ${\bf p}$ and the ground truth, ${\bf c}$:

where $k$ is the sample index, $N$ is the sample number, and $\zeta=[a_{1},\ldots,a_{4},m_{1},\ldots,m_{4},$ $b_{1},\ldots,b_{4},n_{1},\ldots,n_{4}]$ in (5) are the optimization variables.

Mean absolute error (MAE) is used to quantify the CoP and GRF accuracy. The MAE of the GRF, $e_{G}$, is given by:

where $f_{z}$ is the measured GRF and $G_{\text{weight}}$ is the calibration weight. The MAE of CoP, $e_{C}$, is defined as:

where ${\bf p}$ and ${\bf c}$ are the measured CoP and the ground truth.

The example of GRF and CoP measurements of the left shoe are shown in Fig. 3. The MAEs of both shoes are listed in Table I. According to the results, the measured GRF (Fig. 3a, blue circles) almost lie perfectly on their corresponding ground truths (black lines) with only around 0.03N MAE (Table I); The corrected CoPs (Fig. 3b, right, colored markers) become much closer to their corresponding ground truth (gray circles) compared with the initial measurements (left, colored markers). The accuracy of the corrected CoPs is improved about five times with only around 1 mm MAE (Table I). The results indicate that our manual calibration enables high CoP measurement accuracy.

When a humanoid robot moves slowly under a quasi-static assumption, its CoP equals the ground projection of its CoM [11]; Its normal GRF is simply the robot’s weight. Ideally, the measured CoP and normal GRF using force sensors should equal their corresponding modeled values. Therefore, our self-calibration method collects modeled CoP, GRF and their corresponding force sensor outputs by moving the robot along planned trajectories in different double support configurations. Then the sensor parameters can be solved by minimizing the error between the sensor measurements and their modeled references using optimization.

In order to initiate the self-calibration, the robot needs to first reach different foot configurations. Two options are considered to sample these configurations: 1. The robot stands with single support and self-calibrate each shoe separately. 2. The robot stands with double support and self-calibrate both shoes together. The former is ideal since it calibrates only four sensors of one shoe, reducing the potential of over-fitting by limiting the dimension of the optimization space. However, this option requires the robot to move its body with only one foot supporting its weight, resulting in excessive torque for smaller-sized humanoid robots with under-powered motors. Practically, our NAO robot platform encountered serious ankle overheating issues standing in single support during our initial testing. Therefore, the double support option is chosen for safety concerns.

To prevent overfitting for calibrating eight sensors in both feet, we collect sensor voltage data covering a sufficient range. Additionally, we ensure a large sample size. Two strategies are implemented to realize the above sampling results. 1. We sample double supports with the position and orientation of one foot sufficiently far from the other (Section V-A). 2. We plan whole-body trajectory for each sampled double support such that the robot’s CoP covers different areas of the sensing polygon (Section V-B).

Each double support configuration is defined using the position and orientation of one shoe relative to another with three parameters: $[\Delta x,\Delta y$,$\Delta\theta]$ (Fig. 4a). We first discretize these parameters in the robot’s factory allowable ranges (blue area) and then randomly sample in the discretized space. For each sampled double support configuration, a collision check is implemented to ensure no collision between the feet. The configuration distance between the feet of each sample must also be greater than a threshold, $d$, designed as:

where $w_{d}$ and $w_{o}$ are the weights of the Cartesian distance and the relative orientation. Once the newly sampled double support passes both checks, it is stored for later execution.

For each sampled double support, a whole-body trajectory is generated using trajectory optimization (Fig. 4b). First, four CoP landmarks are designed at the sensing polygon’s centerlines and are 0.25-length away from the sensing polygon’s front or rear edge ($p_{1}-p_{4}$, $l$ is the foot length). Then a trajectory planner generates a series of joint angle vectors, enabling the modeled CoP (red curve) to move counter-clockwise along the polygon contour constructed by the landmarks (blue curve). Initially, the CoP is driven towards one of the midpoints of the polygon contour ($p_{m}$) that is closest to the robot’s initial CoP (point 0). The robot’s arms are fixed in the trajectory, as they have little impact on CoP.

We adopt the direct collocation technique [24] for trajectory optimization because it effectively solves trajectory including complex path constraints like our case. The optimization is formulated as:

where the system states are a series of joint angle vectors: ${\bf q}=[{\bf q_{1}},\ldots,{\bf q_{N}}]$, and the transition vectors: ${\bf u}=[{\bf u_{1}},\ldots,{\bf u_{N-1}}]$. The transition vectors connect consecutive states along the trajectory (11). The cost function (10) penalizes the distance between the modeled CoP ${{\bf c}[i]}$ and the CoP landmark ${\bf c_{\text{ref}}}$, driving the CoP trajectory towards the landmark. Moreover, the cost (10) minimizes the state transition difference to smooth the trajectory. Several path constraints are imposed: the stability constraint (12) confines the modeled CoP inside the sensing polygon of the double support. The joint limit constraint (13) determines the lower and upper bound of the states. The collision constraints (14) prevent leg collision. 3-D capsules are used to approximate the robot’s link geometry (Fig. 4c). The collision constraints set the minimal distance $d$ of the collision pairs (4c, right) larger than a smaller threshold $d_{\text{min}}$. This minimal distance is calculated using the capsule centerline distance ($R$) to subtract the sum of their radii ($r_{1}+r_{2}$) (Fig. 4c, left). The foot transformation constraint (15) preserves the relative position and orientation between the feet for all the states in the trajectory. Finally, the optimization is solved using a nonlinear programming solver.

In practice, it is challenging for the solver to generate a complete trajectory directly connecting adjacent CoP landmarks in the sampled double supports (Fig. 4b). On the one hand, the convergence of the optimizer is sensitive to the number of states in the trajectory. On the other hand, a feasible trajectory may not exist once the robot’s configuration gets closer to its singularity, which occurs when the CoP is close to its landmark. To ensure our algorithm runs autonomously, the planner is implemented in a model predictive control scheme: the optimizer continuously generates a small portion of the trajectory ahead of time, moving the CoP towards its landmark. Once the CoP is close enough to the landmark, or the CoP trajectory cannot proceed more towards the landmark, a new landmark is updated. The detailed implementation of the planner is presented in Algorithm 1 and the snapshots of the generated whole-body trajectories are shown in Fig. 6 and 7.

To implement the self-calibration algorithm, we sampled five different double supports and then generated their corresponding whole-body trajectories. Among these datasets, three are randomly chosen for training (Fig. 6). The other two (Fig. 7) are used for evaluation.

For the self-calibration to work effectively, two assumptions need to be validated. Assumption 1: The robot’s slow movement can be modeled as a quasi-static system. Assumption 2: The modeled CoP is sufficiently close to the ground truth. To test these two assumptions, we manually calibrated the shoes and use their high-accurate measurements (see Table I) as ground truths to evaluate the modeled results. For Assumption 1, we compared the measured normal GRF using manual calibration and the robot’s weight. For Assumption 2, we compared the modeled CoP and the measured CoP using manual calibration. The results show that the measured GRF (Fig. 5a, colored curves) slightly oscillates around the ground truth (black line) with only around 0.3 N MAE (Table II), indicating that the robot’s slow movement is close enough to quasi-static condition. In addition, the modeled CoP trajectories (Fig. 5b, red curves) are close to the measured references (black curves) with less than 3 mm MAE (Table II), indicating that the modeling is with decent fidelity. The experimental results prove that both assumptions are practically applicable.

Under quasi-static assumption, the modeled CoP equals the projection of the modeled CoM of the robot. With a priori known mass properties of the robot’s body components, the CoM can be modeled by solving whole body forward kinematics using joint angle data obtained by encoders.

The self-calibration utilizes nonlinear least squares to determine the sensor parameters, requiring a reasonable initial guess. Since all load cells are of the same type, we apply the same initial guess $[c_{0},d_{0}]$ (see III-A for load cell modeling) to all the sensors. Assuming the measured GRF and CoP using initial guess approximately equal their modeled references, the following equations hold:

The left side consists of the robot’s weight, $G_{r}$, and the modeled CoP, $[c_{x},c_{y}]^{T}$. The right side consists of the measured GRF and CoP, where $i$ is the sensor index, $S$ and $(t_{x},t_{y})$ are the sensor voltage output and location. We can rearrange the equations to include more training samples:

where $k$ is the sample number. Therefore, the optimal initial guess $(c_{0},d_{0})$ can be solved by least squares regression.