Low-cost 3D foot scanner using a mobile app E. Parrilla, A. Ballester, C. Solves, B. Nácher, S.A. Puigcerver, J. Uriel, A. Piérola, J.C. González, S. Alemany

FBS 2015 – Liverpool, 9th July 2015

Introduction • The access to the 3D representation of people’s feet has multiple applications in footwear industry. • There are several barriers that have hindered massive spreading of 3D scanners: price, volume, expertise. • We pose a novel approach: – Estimating the 3D shape of the foot using a mobile phone application – The proposed 3D reconstruction is data-driven since it uses a parameterised shape space created from a feet database.

Introduction Systems to gather the anthropometry of the feet Source: http://brannock.com/

ACTUAL FEET ANTHROPOMETRY •

Traditional mechanical devices (e.g. Brannock) Affordable cost for shops but poor information for footwear customization.

•

3D foot scanners applications

•

Very accurate but still high costly for commercial

Silhouette based 3D scanning around the foot.

These systems use a big set of images

3D DATA-DRIVEN RECONSTRUCTION •

Parametric foot model in the Euclidean space Regression equations relating foot parameters (heights) and x,y,z coordinates (Luximon et al. 2005).

•

Parametric foot model in the Shape space Principal component Analysis (PCA) of a 3D feet database representing human foot shape with 12 components. This approach was used to recover the occluded areas of the foot acquisition with multi-camera images (Mochimaru et al. 2005)

Source: http://3dprint.com/51341/3dshoesapp-foot-scan/

Purpose of the study • To demonstrate the feasibility of reconstructing accurately a 3D foot using only three images taken with a regular smartphone. Foot 2D SHAPE SPACE

HOMOLOGOUS 3D FEET DATABASE

OPTIMISATION PROCESS PCA analysis PC scores Scale Position

3D RECONSTRUCTION

Methodology Shape space • Database of 700 feet scanned using Infoot model from I-Ware Laboratory with five foot landmarks. • Feet were transformed in homologous using these landmarks and a foot template mesh. • Procrustes analysis was used to align the feet. • PCA was applied to get a parameterised foot based on 40 parameters (principal components).

Size

Breadth

Arch

Methodology Obtaining 2D foot outline 1. Obtaining three images: – Lateral, zenithal and medial photographs 2. Image processing: – Segmentation: extracting 3 foot outlines – Calibration: from the four DINA4 sheet corners (f, R, T) (focal, rotation, translation)

f  K =0 0 

0 f 0

P = K (R | T )

w / 2  h/2 1 

Methodology Foot optimisation & reconstruction • The 3D reconstruction departs from the average foot model created from the PCA. • The process consists of minimising the distance from the silhouettes of the PCA model projected into the images planes to the three silhouettes extracted from the images. • Minimisation is approached by iteratively modifying the PCA scores. • At each iteration, the vertices defining the projected silhouettes of the PCA model are computed and distances minimised (Zhu et al. 1997). • After each optimisation, the new set of vertices defining the silhouettes is used in the next iteration. • Convergence is reached usually before the tenth iteration.

Methodology Analysis of system accuracy • Objective: – Accuracy of the system for foot reconstruction – Repetitiveness accuracy (5 repetitions) • The accuracy of the foot scanners has been performed at two different levels: – Accuracy of the 3D geometry. – Accuracy of 7 foot measurements based on IBV’s protocol. FEET USED IN THE ACCURACY STUDY WIDE FOOT

HIGH INSTEP FOOT

LOW INSTEP FOOT

LOW ARCHED FOOT

Four foot patterns representing different morphotypes

Methodology Analysis of system accuracy • Feet obtained from additive manufacturing (SLS technology Formiga P100 from EOS) • Scanning methods: – Capture app – ATOS (Gom) used as the reference scanned models (accuracy 0,1mm)

Results 3D geometry • Global error value: calculating for each node the distance between the surfaces of the app models & the reference 3D models after their alignment. • Iterative Closest Point (ICP) method was used for the alignment. Error (mm)

Global accuracy of the scanner

Average foot

Mean (mm)

1,12

Percentile 50 (mm)

0,94

Percentile 95

2,77

Results 3D geometry • Repetitiveness error value: calculated as the distance between each node and the average position of the node in the 5 repetitions.

Error (mm)

Repetitiveness error

Average foot

Mean (mm)

0,32

Percentile 50 (mm)

0,24

Percentile 95 (mm)

0,89

Results Foot measurement • 7 foot measures automatically calculated through validated algorithms using some landmarks referencing important anatomical points of the foot. Toes section

Ball section T1

T5

1MP

5MP

Instep section

Foot length

Results Foot measurement • Measuring error: average of the measurements on the 20 scanned feet regarding their respective reference model of foot.

Foot length Toes girth Toes width Ball girth Ball width Instep girth Instep height Mean (mm)

242,9

211,8

93,8

228,8

94,8

229,7

62,9

Mean error (mm)

0,8

2,1

1,2

3,4

1,7

3,1

2,7

Error Std. Dv. (mm)

0,5

2,2

0,9

0,9

0,9

1,3

0,9

Mean C.V.

0,3%

1%

1,3%

1,5%

1,8%

1,3%

4,2%

Conclusions

• It has been proved the realistic reconstruction of the foot with an app is feasible. • Despite the accuracy is lower than high end scanners, it can be acceptable for applications such as size allocation and made to measure footwear. • Our solution overcomes the barriers for the massive spreading of the digitisation of feet and its use in ecommerce: price, availability and usability.

Further work • After an validation of usability and operation of the system with more than 300 persons, we are defining a large-scale validation with end-users and real feet to obtain the accuracy and reproducibility of our system in real-use conditions. • Implementation of the system for 3D foot reconstruction in a scanner device. We are working in improving its usability, use, rapidity for its use in shoe retail shops.

References • A. Luximon , R, Goonetilleke, M, Zhang (2005). 3D foot shape generation from 2D information, Ergonomics, 48:6, 625-641. • Wang, J.; Saito, H.; Kimura, M.; Mochimaru, M.; Kanade, T. Shape reconstruction of human foot from multi-camera images based on PCA of human shape database. 3-D Digital Imaging and Modeling, 2005. 3DIM 2005. Fifth International Conference. 13-16 June 2005, • Zhu, C. et al. (1997). ACM T Math Software, 23(4), 550-560.

Thank you for your attention

http://anthropometry.ibv.org [email protected]