Agentyk Showroom

Fitting benchmark

Question: given a manufacturer CAD whose upholstery is not shaped like the real product and four photos of the real product, does the fit recover the true shape?

Method (worker/showroom/bench/synthetic.py)

For each CC0 model (Poly Haven sofa_02, ArmChair_01, Sofa_01):

Metrics

What silhouettes can and cannot recover: four outlines constrain the outer shape (crowns, arm and back profiles, overall proportions). The 6 mm inset in the middle of panels that face the camera is mostly invisible in silhouettes, so it is expected to stay; the mean error therefore has a floor, while the large errors (p95) are what the fit should remove.

Results

Run of 2026-11-17 on the demo server (CPU, 3 threads; coarse camera search, then 120 registration + 250 fit iterations at 256 px; data/bench/final4).

Upholstery surface error vs ground truth (symmetric point-to-surface, mm)

modelsoft partsCAD as given: mean / p95 / maxafter fitting: mean / p95 / max
sofa_02 (tufted sofa)217.1 / 69.3 / 109.016.5 / 44.3 / 78.1
ArmChair_01115.0 / 53.9 / 155.46.7 / 16.5 / 73.9
Sofa_0139.8 / 32.8 / 96.815.4 / 37.5 / 58.6 (worse, see below)

Silhouette IoU per photo (front / back / left / right) and camera recovery (mean over photos)

modelIoU, registered CADIoU, fittedyaw err (deg)pitch err (deg)distance err (%)pipeline s
sofa_020.930 / 0.828 / 0.910 / 0.9300.955 / 0.937 / 0.927 / 0.9310.93.22.8101
ArmChair_010.857 / 0.799 / 0.860 / 0.8680.941 / 0.921 / 0.930 / 0.9370.81.62.0145
Sofa_010.945 / 0.838 / 0.763 / 0.8860.946 / 0.908 / 0.830 / 0.8940.72.62.5138

Real photos (Amazon Berkeley Objects armchair, 4 frames of a real 360 catalogue spin, no ground truth): after the coarse camera search (found a consistent -30 deg turntable offset and a ~140 mm-equivalent lens) the silhouette IoU is 0.979 / 0.973 / 0.976 / 0.977.

Reading the numbers