Immersive Design Based on Human Motion


This workshop explores scenarios, which take the individual body dimensions and spatial perception of designers and clients as the basis for design. This is achieved through the combination of virtual reality (VR) and Motion Capture (Mocap) technologies. For this purpose, we will use VRoxel, an immersive design environment, using discrete parts in VR, developed by the workshop leaders.

The goal of the workshop is twofold. Firstly, we want to explore design approaches based on small-scale modular systems, suitable for robotic assembly and circular construction. Secondly, we intend to challenge existing normative preconceptions regarding the “human scale” in architecture, basing the design on individual spatial perception and specific human motion, rather than proportional systems like the Modulor or standards books like Neufert’s Bauentwurfslehre.

The workshop is aimed at designers and researchers interested in immersive technology and discrete design. Participants will learn to use immersive design in VRoxel, as well as how to use the Xsens MVN Awinda motion capture system. We will further introduce Rokoko Vision AI motion capture as a cheap alternative to Xsens. Recorded mocap animations can be imported into VRoxel and Grasshopper.

The results of the workshop will be presented in video format. Flythrough animations of VRoxel assemblies are quickly produced and better represent the spatial qualities of the design than renderings. The animations can also be recorded in portrait orientation for social media purposes.

No previous knowledge is needed.


VRoxel is based on discrete concepts of modular assembly, using small-scale modular systems. These have been defined at MIT’s Center for Bits and Atoms (CBA) under the term digital materials (Popescu 2007). This form of additive manufacturing works on a variety of different scales, using mechanical interlocking instead of chemical bonding.
VRoxel can display and edit millions of individual objects in real-time. For editing assemblies, we propose an immersive sculpting methodology in virtual reality based on the following features:

  • 1:1 immersive real-time design in VR
  • Sculpting with 3D brushes based on Boolean logic
  • Recurring spatial patterns arranged in different bonds
  • Patterns allow for localization of individual blocks in assembly
  • Import of context as
    – Point clouds from 3D-scans
    – Meshes through FBX and OBJ formats
    – Motion capture animations of human movements
  • Export to CAD

The Xsens MVN Awinda is a cutting-edge motion capture system that offers accurate and reliable kinematic data, making it a popular choice in fields like biomechanics, sports science, ergonomics, and film making. This system utilises small, wireless inertia-sensors attached to the body, which capture motion data without the need for a camera-based setup. This flexibility allows for motion capture in various environments, both indoors and outdoors. The recorded data can be effortlessly imported into VRoxel. This system has already been successfully tested and evaluated by dMA in research and teaching. As a more affordable alternative, we will introduce AI-based Rokoko Vision software.

Workshop Leaders

Jan Philipp Drude, Leibniz University Hannover

Jan Philipp has been a research assistant at LUH since 2017, where he teaches and conducts research in Prof Mirco Becker’s Department of Digital Methods in Architecture (dMA). His academic interests lie in the real-time simulation of complex systems and in the field of discrete architecture. He combines both topics in his research on generative tools in virtual reality. Jan Philipp taught a workshop at AAG 2018, as well as VR-related workshops at Leibniz University Hannover, TU Darmstadt and the University of Kassel, where he has been teaching since 2021.

Henrik Wiese, Leibniz University Hannover

Henrik has been a researcher at Leibniz University Hannover since 2022, where he is based at the Department of Digital Methods in Architecture (dMA). After studying architecture at LUH, he began working at the start-up Modum-Shoes in 2019, developing technologies for the automated construction of customised shoes for mass-production. His research interests focus on the potential impact of mass customization on the built environment, with an emphasis on the customization towards specific user needs.