Carpenters: 3D geometry

Tangible interfaces for learning 3D geometry

This project is part of the broader DUAL-T project that studies how technology can be used in vocational and education training (VET).

In this specific project, we work with construction carpenter (“charpentier” in French) apprentices. Carpenters build large items, such as roof structures or even entire buildings (see Figure 1). One of the main skills needed by carpenters are spatial skills, which they need to interpret two-dimensional plans into the final three-dimensional constructions and vice-versa.

TapaCarp

Previous research has shown that spatial skills can be improved through drawing exercises. Our hypothesis is that tangible interfaces (Tangible User Interfaces or TUI) can contribute to the effective development of spatial skills.  

TapaCarp is a TUI for the training of spatial skills. It runs on the Tinkerlamp – a novel camera-projector system (see figure 2). The TapaCarp activities include different elements such as wooden blocks, paper cards, drawing tools, and activity booklets (see Figure 3). The video below demonstrates how the interface can be used and the function of each of the components.

Using TapaCarp, apprentices can explore concepts such as orthographic projections, calculation of realistic values, or  cutting a piece of wood. For more detail on the activity and the pedagogical scenario, please refer to the list of publications mentioned in the results section below. 

Large Scale Distribution (work in progress)

Experimental results indicate that TapaCarp can support develop spatial skills. However, TapaCarp requires expensive and uncommon hardware. Large-scale distribution has always been a challenge for tangible interfaces. We are currently working on a solution to make TapaCarp available to large numbers of users. It requires only a simple webcam (see video below). This solution is called ‘eTapaCarp’. 

We conducted an experiment to compare the performance and eye movements of users with TapaCarp and eTapaCarp. Results indicate no loss of performance or speed of interaction with eTapaCarp. This encourages further development of learning activities with eTapaCarp and the integration in classroom settings.

Here are some additional findings from this comparison study:

• In orthogonal projections, the side view is more difficult than the front view or the top view (indicated by the performance measures and the eye tracking data)
• User interaction with the system is more fluid in eTapaCarp (indicated by the performance measures and eye tracking data)
• Symmetric shapes remain in the working memory for a relatively longer period of time due to the fewer details one needs to remember for constructing a mental model (indicated by eye tracking data)
• A high degree of symmetry makes it harder to find reference points in order to map the 2D and 3D representations of the object (indicated by eye tracking data)

General results

We have conducted several studies with carpenter apprentices to assess TapaCarp. Our focus has been on both the usability as well as on measuring the learning outcome. The main result is that TapaCarp can help train spatial skills. Moreover, several aspects of the design can significantly impact the learning outcome and need to be taken into consideration. A summary of the findings is shown below.

• The shape of the block used to manipulate the virtual object is important: when using a token with no resemblance to the object manipulated, the learning is less effective than when using a block with a matching shape to the object manipulated (Nordi’CHI 2012, details). 

• The ease of manipulation that comes with a TUI can lead to too much manipulation and decrease the learning performance. It is therefore important to carefully design activities to discourage manipulation at the expense of reflection (British HCI 2012, details).

• The interface of TapaCarp, spread over several components, can be beneficial for a use in the classroom, because it allows for instance to enforce roles and scaffold the interface by giving parts of the interface to different people at different times (CSCL 2013, details).

• TapaCarp can be used for classroom activity. In particular, TapaCarp was used to simulate all the main steps of carpentry (from plan design to the creation of the final object) in the classroom, as shown in Video 2. The activity was done in pairs and also included a class debriefing. 

• Based on this project and two others, we established general principles for designing augmented reality for the room (Computers and Education, details).

• Our research suggests that eye trackers can be used as a research tool to capture variations among participants using TUIs. Its implementation in TUI research represents a novel contribution and we believe that eye trackers can be adopted as research tool to study the cognitive effects of tangibles (see 2014 article here).

• Participants in the tangible condition have shown gaze values which can be interpreted as indicators of less demanding mental efforts, suggesting some cognitive advantages in using TUIs even when the tangible object and its virtual representation do not share the same geometrical information (2014 study).

• A comparison study using eye trackers suggested no significant differences in performance accuracy and performance speed between TapaCarp and eTapaCarp.

• Findings indicate the presence of differences in user’s gaze patterns in TapaCarp and eTapaCarp, as well as between different views and among objects with different levels of symmetry.

See full publication list here.