ABSTRACT Standard animal behavior paradigms incompletely mimic nature and thus limit our understanding of behavior and brain function. Virtual reality (VR) can help, but it poses challenges.

Typical VR systems require movement restrictions but disrupt sensorimotor experience, causing neuronal and behavioral alterations. We report the development of FreemoVR, a VR system for

freely moving animals. We validate immersive VR for mice, flies, and zebrafish. FreemoVR allows instant, disruption-free environmental reconfigurations and interactions between real

organisms and computer-controlled agents. Using the FreemoVR platform, we established a height-aversion assay in mice and studied visuomotor effects in _Drosophila_ and zebrafish.

unrestrained animals. Access through your institution Buy or subscribe This is a preview of subscription content, access via your institution ACCESS OPTIONS Access through your institution

Access Nature and 54 other Nature Portfolio journals Get Nature+, our best-value online-access subscription $29.99 / 30 days cancel any time Learn more Subscribe to this journal Receive 12

print issues and online access $259.00 per year only $21.58 per issue Learn more Buy this article * Purchase on SpringerLink * Instant access to full article PDF Buy now Prices may be

subject to local taxes which are calculated during checkout ADDITIONAL ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our FAQs * Contact customer support SIMILAR

CONTENT BEING VIEWED BY OTHERS MOCULUS: AN IMMERSIVE VIRTUAL REALITY SYSTEM FOR MICE INCORPORATING STEREO VISION Article Open access 12 December 2024 MOUSEGOGGLES: AN IMMERSIVE VIRTUAL

REALITY HEADSET FOR MOUSE NEUROSCIENCE AND BEHAVIOR Article Open access 12 December 2024 OUVRAI OPENS ACCESS TO REMOTE VIRTUAL REALITY STUDIES OF HUMAN BEHAVIOURAL NEUROSCIENCE Article 26

April 2024 REFERENCES * Aghajan, Z.M. et al. Impaired spatial selectivity and intact phase precession in two-dimensional virtual reality. _Nat. Neurosci._ 18, 121–128 (2015). Article  CAS 

PubMed  Google Scholar  * Chiappe, M.E., Seelig, J.D., Reiser, M.B. & Jayaraman, V. Walking modulates speed sensitivity in _Drosophila_ motion vision. _Curr. Biol._ 20, 1470–1475 (2010).

Article  CAS  PubMed  PubMed Central  Google Scholar  * von Holst, E. & Mittelstaedt, H. Das Reafferenzprinzip - Wechselwirkungen zwischen Zentralnervensystem und Peripherie.

_Naturwissenschaften_ 37, 464–476 (1950). Article  Google Scholar  * Jung, S.N., Borst, A. & Haag, J. Flight activity alters velocity tuning of fly motion-sensitive neurons. _J.

Neurosci._ 31, 9231–9237 (2011). Article  CAS  PubMed  PubMed Central  Google Scholar  * Kim, A.J., Fitzgerald, J.K. & Maimon, G. Cellular evidence for efference copy in _Drosophila_

visuomotor processing. _Nat. Neurosci._ 18, 1247–1255 (2015). Article  CAS  PubMed  PubMed Central  Google Scholar  * Leinweber, M. et al. Two-photon calcium imaging in mice navigating a

virtual reality environment. _J. Vis. Exp._ 20, e50885 (2014). Google Scholar  * Sperry, R.W. Neural basis of the spontaneous optokinetic response produced by visual inversion. _J. Comp.

Physiol. Psychol._ 43, 482–489 (1950). Article  CAS  PubMed  Google Scholar  * Ravassard, P. et al. Multisensory control of hippocampal spatiotemporal selectivity. _Science_ 340, 1342–1346

(2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Acharya, L., Aghajan, Z.M., Vuong, C., Moore, J.J. & Mehta, M.R. Causal influence of visual cues on hippocampal

directional selectivity. _Cell_ 164, 197–207 (2016). Article  CAS  PubMed  Google Scholar  * Harvey, C.D., Collman, F., Dombeck, D.A. & Tank, D.W. Intracellular dynamics of hippocampal

place cells during virtual navigation. _Nature_ 461, 941–946 (2009). Article  CAS  PubMed  PubMed Central  Google Scholar  * Schmidt-Hieber, C. & Häusser, M. Cellular mechanisms of

spatial navigation in the medial entorhinal cortex. _Nat. Neurosci._ 16, 325–331 (2013). Article  CAS  PubMed  Google Scholar  * Aronov, D. & Tank, D.W. Engagement of neural circuits

underlying 2D spatial navigation in a rodent virtual reality system. _Neuron_ 84, 442–456 (2014). Article  CAS  PubMed  PubMed Central  Google Scholar  * Sofroniew, N.J., Cohen, J.D., Lee,

A.K. & Svoboda, K. Natural whisker-guided behavior by head-fixed mice in tactile virtual reality. _J. Neurosci._ 34, 9537–9550 (2014). Article  PubMed  PubMed Central  CAS  Google

Scholar  * Hölscher, C., Schnee, A., Dahmen, H., Setia, L. & Mallot, H.A. Rats are able to navigate in virtual environments. _J. Exp. Biol._ 208, 561–569 (2005). Article  PubMed  Google

Scholar  * Dombeck, D.A., Harvey, C.D., Tian, L., Looger, L.L. & Tank, D.W. Functional imaging of hippocampal place cells at cellular resolution during virtual navigation. _Nat.

Neurosci._ 13, 1433–1440 (2010). Article  CAS  PubMed  PubMed Central  Google Scholar  * Maimon, G., Straw, A.D. & Dickinson, M.H. Active flight increases the gain of visual motion

processing in _Drosophila_. _Nat. Neurosci._ 13, 393–399 (2010). Article  CAS  PubMed  Google Scholar  * Cushman, J.D. et al. Multisensory control of multimodal behavior: do the legs know

what the tongue is doing? _PLoS One_ 8, e80465 (2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Straw, A.D., Branson, K., Neumann, T.R. & Dickinson, M.H. Multi-camera

real-time three-dimensional tracking of multiple flying animals. _J. R. Soc. Interface_ 8, 395–409 (2011). Article  PubMed  Google Scholar  * Fry, S.N., Rohrseitz, N., Straw, A.D. &

Dickinson, M.H. Visual control of flight speed in _Drosophila melanogaster_. _J. Exp. Biol._ 212, 1120–1130 (2009). Article  PubMed  Google Scholar  * Schuster, S., Strauss, R. & Götz,

K.G. Virtual-reality techniques resolve the visual cues used by fruit flies to evaluate object distances. _Curr. Biol._ 12, 1591–1594 (2002). Article  CAS  PubMed  Google Scholar  * Straw,

A.D., Lee, S. & Dickinson, M.H. Visual control of altitude in flying _Drosophila_. _Curr. Biol._ 20, 1550–1556 (2010). Article  CAS  PubMed  Google Scholar  * Stowers, J.R. et al.

Reverse engineering animal vision with virtual reality and genetics. _Computer_ 47, 38–45 (2014). Article  Google Scholar  * Del Grosso, N., Graboski, J., Chen, W., Blanco-Hernández, E.

& Sirota, A. Virtual reality system for freely-moving rodents. Preprint at http://www.biorxiv.org/content/early/2017/07/10/161232 (2017). * Ellard, C.G., Goodale, M.A. & Timney, B.

Distance estimation in the Mongolian gerbil: the role of dynamic depth cues. _Behav. Brain Res._ 14, 29–39 (1984). Article  CAS  PubMed  Google Scholar  * Poggio, T. & Reichardt, W. A

theory of the pattern induced flight orientation of the fly Musca domestica. _Kybernetik_ 12, 185–203 (1973). Article  CAS  PubMed  Google Scholar  * Duistermars, B.J., Care, R.A. &

Frye, M.A. Binocular interactions underlying the classic optomotor responses of flying flies. _Front. Behav. Neurosci._ 6, 6 (2012). Article  PubMed  PubMed Central  Google Scholar  *

Reiser, M.B. & Dickinson, M.H. Visual motion speed determines a behavioral switch from forward flight to expansion avoidance in _Drosophila_. _J. Exp. Biol._ 216, 719–732 (2013). Article

  PubMed  Google Scholar  * Kress, D. & Egelhaaf, M. Head and body stabilization in blowflies walking on differently structured substrates. _J. Exp. Biol._ 215, 1523–1532 (2012). Article

  PubMed  Google Scholar  * Schilstra, C. & Hateren, J.H. Blowfly flight and optic flow. I. Thorax kinematics and flight dynamics. _J. Exp. Biol._ 202, 1481–1490 (1999). PubMed  Google

Scholar  * Lister, J.A., Robertson, C.P., Lepage, T., Johnson, S.L. & Raible, D.W. _nacre_ encodes a zebrafish microphthalmia-related protein that regulates neural-crest-derived pigment

cell fate. _Development_ 126, 3757–3767 (1999). CAS  PubMed  Google Scholar  * Ahrens, M.B. et al. Brain-wide neuronal dynamics during motor adaptation in zebrafish. _Nature_ 485, 471–477

(2012). Article  CAS  PubMed  PubMed Central  Google Scholar  * O'Malley, D.M. et al. Optical physiology and locomotor behaviors of wild-type and _nacre_ zebrafish. _Methods Cell Biol._

76, 261–284 (2004). Article  PubMed  Google Scholar  * Antinucci, P. & Hindges, R. A crystal-clear zebrafish for _in vivo_ imaging. _Sci. Rep._ 6, 29490 (2016). Article  CAS  PubMed 

PubMed Central  Google Scholar  * Lange, M. et al. Inter-individual and inter-strain variations in zebrafish locomotor ontogeny. _PLoS One_ 8, e70172 (2013). Article  CAS  PubMed  PubMed

Central  Google Scholar  * Liu, Y. et al. Statistical analysis of zebrafish locomotor response. _PLoS One_ 10, e0139521 (2015). Article  PubMed  PubMed Central  CAS  Google Scholar  *

Barker, A.J. & Baier, H. Sensorimotor decision making in the zebrafish tectum. _Curr. Biol._ 25, 2804–2814 (2015). Article  CAS  PubMed  Google Scholar  * Thibos, L.N., Still, D.L. &

Bradley, A. Characterization of spatial aliasing and contrast sensitivity in peripheral vision. _Vision Res._ 36, 249–258 (1996). Article  CAS  PubMed  Google Scholar  * Reynolds, C.W.

Flocks, herds and schools: a distributed behavioral model. _Computer Graphics_ 21, 25–34 (1987). Article  Google Scholar  * Couzin, I.D., Krause, J., Franks, N.R. & Levin, S.A. Effective

leadership and decision-making in animal groups on the move. _Nature_ 433, 513–516 (2005). Article  CAS  PubMed  Google Scholar  * Couzin, I.D. et al. Uninformed individuals promote

democratic consensus in animal groups. _Science_ 334, 1578–1580 (2011). Article  CAS  PubMed  Google Scholar  * Ioannou, C.C., Guttal, V. & Couzin, I.D. Predatory fish select for

coordinated collective motion in virtual prey. _Science_ 337, 1212–1215 (2012). Article  CAS  PubMed  Google Scholar  * Naumann, E.A., Kampff, A.R., Prober, D.A., Schier, A.F. & Engert,

F. Monitoring neural activity with bioluminescence during natural behavior. _Nat. Neurosci._ 13, 513–520 (2010). Article  CAS  PubMed  PubMed Central  Google Scholar  * Randlett, O. et al.

Whole-brain activity mapping onto a zebrafish brain atlas. _Nat. Methods_ 12, 1039–1046 (2015). Article  CAS  PubMed  PubMed Central  Google Scholar  * Szuts, T.A. et al. A wireless

multi-channel neural amplifier for freely moving animals. _Nat. Neurosci._ 14, 263–269 (2011). Article  CAS  PubMed  Google Scholar  * Ziv, Y. et al. Long-term dynamics of CA1 hippocampal

place codes. _Nat. Neurosci._ 16, 264–266 (2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Bastien, R. et al. KymoRod: a method for automated kinematic analysis of rod-shaped

plant organs. _Plant J._ 88, 468–475 (2016). Article  CAS  PubMed  Google Scholar  * D'Azzo, J.J. _Linear Control System Analysis and Design: Conventional and Modern_ (McGraw-Hill,

1995). * Fenk, L.M., Poehlmann, A. & Straw, A.D. Asymmetric processing of visual motion for simultaneous object and background responses. _Curr. Biol._ 24, 2913–2919 (2014). Article  CAS

  PubMed  Google Scholar  * Svoboda, T., Martinec, D. & Pajdla, T. A convenient multicamera self-calibration for virtual environments. _PRESENCE Teleoperators Virtual Environ._ 14,

407–422 (2005). Article  Google Scholar  Download references ACKNOWLEDGEMENTS We thank M. Colombini, A. Fuhrmann, L. Fenk, E. Campione, S. Villalba, and the IMP/IMBA Workshop for help

constructing FreemoVR hardware and software. We thank M. Dickinson and T. Klausberger for helpful discussions, V. Böhm for help with experiments, and the MFPL fish facility for fish care.

The manual mouse behavior annotation was performed by the Preclinical Phenotyping Facility at Vienna Biocenter Core Facilities. This work was supported by European Research Council (ERC)

starting grants 281884 to A.D.S., 311701 to W.H., 337011 to K.T.-R.; Wiener Wissenschafts-, Forschungs- und Technologiefonds (WWTF) grant CS2011-029 to A.D.S.; FWF (http://www.fwf.ac.at/)

research project grants P28970 to K.T.-R. and P29077 to K.N.; NSF grants PHY-0848755 to I.D.C., IOS-1355061 to I.D.C., EAGER-IOS-1251585 to I.D.C.; ONR grants N00014-09-1-1074 to I.D.C.,

N00014-14-1-0635 to I.D.C; ARO grants W911NG-11-1-0385 to I.D.C., W911NF-14-1-0431 to I.D.C. A.D.S and W.H. were further supported by the IMP, Boehringer Ingelheim and the Austrian Research

Promotion Agency (FFG). K.T.-R. is supported by grants from the University of Vienna (research platform “Rhythms of Life”). IDC acknowledges further support from the “Struktur- und

Innovationsfonds für die Forschung (SI-BW)” of the State of Baden-Württemberg and from the Max Planck Society. I.D.C. and R.B. gratefully acknowledge fish care and technical support from C.

Bauer, J. Weglarski, A. Bruttel, and G. Mazué. AUTHOR INFORMATION AUTHORS AND AFFILIATIONS * Research Institute of Molecular Pathology, Vienna Biocenter, Vienna, Austria John R Stowers, 

Maximilian Hofbauer, Johannes Griessner, Peter Higgins, Wulf Haubensak & Andrew D Straw * loopbio gmbh, Kritzendorf, Austria John R Stowers & Maximilian Hofbauer * Max F. Perutz

Laboratories, University of Vienna, Vienna, Austria Maximilian Hofbauer, Sarfarazhussain Farooqui, Ruth M Fischer & Kristin Tessmar-Raible * Research Platform “Rhythms of Life,”

University of Vienna, Vienna, Austria Maximilian Hofbauer, Sarfarazhussain Farooqui & Kristin Tessmar-Raible * Department of Collective Behaviour, Max Planck Institute for Ornithology,

Konstanz, Germany Renaud Bastien & Iain D Couzin * Department of Biology, Chair of Biodiversity and Collective Behaviour, University of Konstanz, Konstanz, Germany Renaud Bastien & 

Iain D Couzin * Medizinische Universität Wien, Dept. for Internal Medicine I, Wien, Austria Sarfarazhussain Farooqui & Karin Nowikovsky * Institute of Biology I and Bernstein Center

Freiburg, Faculty of Biology, Albert-Ludwigs-University Freiburg, Freiburg, Germany Andrew D Straw Authors * John R Stowers View author publications You can also search for this author

inPubMed Google Scholar * Maximilian Hofbauer View author publications You can also search for this author inPubMed Google Scholar * Renaud Bastien View author publications You can also

search for this author inPubMed Google Scholar * Johannes Griessner View author publications You can also search for this author inPubMed Google Scholar * Peter Higgins View author

publications You can also search for this author inPubMed Google Scholar * Sarfarazhussain Farooqui View author publications You can also search for this author inPubMed Google Scholar *

Ruth M Fischer View author publications You can also search for this author inPubMed Google Scholar * Karin Nowikovsky View author publications You can also search for this author inPubMed 

Google Scholar * Wulf Haubensak View author publications You can also search for this author inPubMed Google Scholar * Iain D Couzin View author publications You can also search for this

author inPubMed Google Scholar * Kristin Tessmar-Raible View author publications You can also search for this author inPubMed Google Scholar * Andrew D Straw View author publications You can

also search for this author inPubMed Google Scholar CONTRIBUTIONS A.D.S., K.T.-R., W.H., and I.D.C. conceived the projects. J.R.S., M.H., R.M.F., R.B., and A.D.S. developed the hardware and

software and built the apparatus. J.R.S., M.H., R.B., J.G., P.H., S.F., and A.D.S. performed experiments. J.R.S., M.H., R.B., J.G., S.F., W.H., I.D.C., K.T.-R. and A.D.S. performed data

analyses. A.D.S., K.T.-R., I.D.C., J.R.S., M.H., and J.G. wrote the manuscript. A.D.S., K.T.-R., W.H., I.D.C., and K.N. funded the work. J.R.S. and M.H. contributed equally to this work.

J.G., P.H., and S.F. contributed equally to this work. CORRESPONDING AUTHORS Correspondence to Kristin Tessmar-Raible or Andrew D Straw. ETHICS DECLARATIONS COMPETING INTERESTS J.R.S. and

M.H. are executives with loopbio, gmbh, a company offering virtual reality services. The other authors declare no competing financial interests. INTEGRATED SUPPLEMENTARY INFORMATION

SUPPLEMENTARY FIGURE 1 VR ASSAYS FOR FLIES, FISH, AND MICE (a) The ‘Flycave’ assay, a 1m diameter 1m high cylindrical VR arena. Three projectors create a panoramic VR, each projecting

directly onto the surface of the cylinder, and simultaneously via 6 mirrors placed in 3 pairs around the setup. Flies are tracked in 3D from above using custom software and multiple cameras.

(b) The FishVR fishbowl assay, in which fish swim in 9cm deep water in a 32cm diameter hemispherical bowl. A panoramic VR is created by projecting onto the surface of the bowl, via a

mirror, from below. Fish position is tracked in 3D in real time using custom software and multiple cameras. (c) The MouseVR mouse floor assay. A 0.5m diameter elevated circular platform is

placed on top of a 1.9m consumer television. Mouse head position is tracked in real time using custom software and a single camera placed above. SUPPLEMENTARY FIGURE 2 FREEMOVR SOFTWARE AND

SYSTEM ARCHITECTURE (A) The ‘Flycave’ assay, a 1m diameter 1m high cylindrical VR arena. Three projectors create a panoramic VR, each projecting directly onto the surface of the cylinder,

and simultaneously via 6 mirrors placed in 3 pairs around the setup. Flies are tracked in 3D from above using custom software and multiple cameras. (B) The FishVR fishbowl assay, in which

fish swim in 9cm deep water in a 32cm diameter hemispherical bowl. A panoramic VR is created by projecting onto the surface of the bowl, via a mirror, from below. Fish position is tracked in

3D in real time using custom software and multiple cameras. (C) The MouseVR mouse floor assay. A 0.5m diameter elevated circular platform is placed on top of a 1.9m consumer television.

Mouse head position is tracked in real time using custom software and a single camera placed above. SUPPLEMENTARY FIGURE 3 POST AVOIDANCE BEHAVIOR AS A FUNCTION OF LATENCY (A) The FreemoVR

rendering pipeline. From left to right; a virtual world is first rendered as a cube map based on the observer position. Using a model of the display geometry, a geometry texture is rendered.

Based on a calibration of the display elements which maps their pixels to geometry coordinates, a VR is created. (B) The FreemoVR software allows distributed operation across multiple

computers – such as in situations where multiple PCs are required for real-time tracking, or for generating or displaying the stimulus on sufficient displays to suit the experiment. (C)

Schematic describing the information flow between the system components in a FreemoVR assay for a freely moving observer. Dark grey indicates computation occurs in real time, light grey

represents a calibration step specific to the particular display geometry. SUPPLEMENTARY FIGURE 4 _DROSOPHILA_ POST AVOIDANCE BEHAVIOR TRAJECTORIES (A) All trials for post avoidance and

latency measurement analysis. In trials with a real post (RW) N flies = 40, n trials = 240. In all VR trials, N=80, n=1890. In all VR trials, total added latency is indicated in text.

SUPPLEMENTARY FIGURE 5 MOUSEVR ARENA AND ANALYSIS (A) The elements of the mouse VR experiments were located in the laboratory as indicated. The television lies in the corner of the room

surrounded by black walls on two sides, and a black curtain on the other. Mice are introduced from the front side and are shielded from the rest of the lab by a black cardboard divider. (B)

Mouse preference (cumulative) for the shallow side. (C) Cumulative preference of mice for the wall side of the laboratory. (D) Cumulative distance walked by the mice for all trials. Mean ±

s.d. for all plots. N=15 mice for RW trials, N=16 for VR, N=16 for ST. SUPPLEMENTARY FIGURE 6 MOUSEVR HEAD DIP ANALYSIS (A) Occurrence of head dips during the RW, ST and VR trials. Black

bars indicate the time the mouse head was scored to be below the platform; start of bar is head-down, end of bar is head-up. (B) Distribution of the duration of all head dips for all scored

trials. (C) Summary plots per mouse quantifying the number of head dips. Circles indicate each scored mouse. Mann-Whitney test; RW vs VR; p=0.15, RW vs ST; p=0.59, ST vs VR, p=0.43. 9 mice

in each group. (D) Summary plot per mouse quantifying the total head dip duration. Circles indicate each scored mouse. Mann-Whitney test; RW vs VR; p=0.53, RW vs ST; p=0.29, ST vs VR,

p=0.89. 9 mice in each group. (B,C,D) Box plots indicate median, upper- and lower-quartile. Whiskers extend to 1.5 IQRs of the lower and upper quartile, observations outside this range are

indicated with diamonds. (E) Spatial location of each head dip. Arrows indicate dip location; downward arrows indicate head-down phase, upward arrows indicate head-up phase. SUPPLEMENTARY

FIGURE 7 ELICITING PATH FOLLOWING OF FREELY FLYING _DROSOPHILA_ (A) Fly trajectories for path following task with different gain values. Path following is elicited in a gain-dependent

manner. The gain (text in top-left of each panel) describes how strong the virtual world is biased to bring the fly location closer to a target location on the path. As the fly position

approaches the target, it is advanced around the path. (B) Distance from the current fly position to current target position on the path. The accuracy of the path following task can be

quantified by considering the distance from the fly to the target point on the infinity path. When the distance between fly and target is less than 0.1m the target is advanced to the next

point on the path - yielding a theoretical lower bound of 0.1m between the fly and the target in the case of perfect path following. N=50 flies. SUPPLEMENTARY FIGURE 8 FLIGHT PERFORMANCE FOR

GLUED FLY EXPERIMENTS (A) Pearson correlation between angular velocity of stimulus and angular velocity of the fly for free flight path following experiments under different glue

preparations; no glue (N=36 flies), head-free glue (N=39), and head-fixed glue (N=78). Mean and ± 68% c.i. plotted. (B) Distribution of vertical speed for freeflight trails. (C) Distribution

of altitude for freeflight trials. (D) Pearson correlation between angular velocity of stimulus and simulated angular velocity of the fly under head-free (N=6) and head-fixed (N=6) tethered

preparations. (E) Distribution of simulated horizontal speed of tethered trials. (F,G) Distribution of simulated angular velocity for tethered head-free and head-fixed preparations for

different gain couplings between wingbeat amplitude and simulated turning torque. SUPPLEMENTARY FIGURE 9 FISH TRAJECTORIES FOR PATH-FOLLOWING EXPERIMENTS (A-C) All experimental data for AB

and _mitf-_a-/- path following experiments in the A large-dots, B small-dots and C grey conditions. All conditions were tested for all fish. N=56 AB fish, N=62 _mitf-_a-/-. SUPPLEMENTARY

FIGURE 10 SWIMMING BEHAVIOR OF AB AND _MITF-A_-/- DURING PATH EXPERIMENTS (A-C) Large dot condition statistics. (D-F) Small dot condition statistics (G-I) Grey condition statistics.

Distribution of fish forward velocity in all trials A,E,G. during the A large-dots Distribution of control action – the magnitude of the dot velocity shown to the fish in order to elect path

following in all trials B,E,H. Distribution of distance to target – the distance between the target point on the path and the fish position in all trials C,F,I. All conditions were tested

for all fish. N=56 AB fish, N=62 _mitf-_a-/-. SUPPLEMENTARY FIGURE 11 TELEPORTATION AND CHANGING ENVIRONMENTS IN VIRTUAL REALITY (A) Zebrafish could visit a checkerboard scene or (B) a plant

scene by entering a virtual teleportation portal. See also Supplementary Video 12. (C) Each of two portals was coupled to a constant destination per fish but different fish had different

couplings. Upon entering a portal, the portals were rearranged to equalize distance required for subsequent portal entry. Additionally, for the portal coupled to the other scene, the fish

was virtually teleported to that new scene. (D) Decisions, operationally defined as portal entry (vertical marks) and current scene (horizontal line), over time for each fish. (E) Top view

of occupancy. (F) Fraction of all decisions per fish that teleported it to the plant scene (one-sample t-test difference from 0.5, p=0.81). (G) Fraction of all portal entries per fish that

were magenta (one-sample t-test difference from 0.5, p=2.7e-7). (H) Fraction of 30 minutes per fish in which the fish was in the plant scene (one-sample t-test difference from 0.5, p=0.047).

(I) Mean horizontal speed per fish in each coupling condition (two-related-sample t-test, p=0.0411). (D-I) N=12 fish, AB strain. Box plots indicate median, upper- and lower-quartile.

Whiskers extend to 1.5 IQRs of the lower and upper quartile. SUPPLEMENTARY FIGURE 12 ANIMATION OF THE TAIL BEAT OF THE ZEBRAFISH LARVAE To obtain kinematic information during swimming, 3

zebrafish larvae of the control group were recorded in a square arena (20cm width, 1 cm water depth) and filmed with a Basler 2040-um camera at 180 frames per second for a duration of 5

minutes. The curvature, C(s), of the animal is computed along the curvilinear abscissa of the median line, s, from head to tail of the fish. (A) Curvature along the median line of an

individual as a function of time. Three different tail beats are shown for a single animal. The movement is highly stereotyped, and can be described by an inflexible part of the body

(~.0002_m_ from the head) and an oscillation of the body that propagates from the end of the inflexible part to the tail (with temporal frequency, _f_ ~ 20_Hz_ and a velocity of propagation

_c_~.2_m_._s_-1). 0.1s after the beginning of the tail beat, no movement is observed. (B) Movement of the recorded fish (18 frames at 180 fps). (C) Animation of the median line of the fish.

(D) Animation generated of the virtual fish. The rendering of FreemoVR is done at 120fps, so only 12 frames are represented. SUPPLEMENTARY FIGURE 13 BURST AND GLIDE MOVEMENT OF THE ZEBRAFISH

LARVAE IN ABSENCE OF VR STIMULI (A) The velocity, _V__r_, in the direction of the movement is displayed for a single animal as a function of time. Burst and glide events are clearly visible

with phase of high acceleration followed by a decay of in speed resulting from drag. We first identified the position of the local maxima of the velocity (in red). (B) For each individual

of the control group we plotted the distribution of the time between two successive burst and glide events, _t__beat_. (C) The median value for all individuals is given by _t__beat_ ~0.42 ±

0.16s. We selected _t__beat_ = 0.5_s_ for our animation to allow sufficient frames for effective animation. (D) For each individual of the control group we plotted the distribution of the

characteristic time of decay of the velocity, _t__c_. This time has been computed by the fit of an exponential function during the 0.3 s after the detection of each velocity peak. (E) The

median value for all individuals is given by _t__c_ ~0.3 ± 0.2_s_. (F) For each individual of the control group we plotted the distribution of the value of the maximal value of the velocity

for each burst and glide event, _V_0 (in red on a.). (G) The median value for all individuals is _V_0 ~0.9 ± 0.3_m_._s_-1. For simplicity, and due to its extremely short nature, the approach

we use to approximate the burst and glide animation does not account for the first part of acceleration of the burst and glide movement. Furthermore, since the time between two successive

beats is on the higher end of the range, the distance travelled for each event should be longer. The value used for animation were taken slightly higher to account for those discrepancies,

_V_0 ~1.4_m_._s_-1. SUPPLEMENTARY FIGURE 14 INDIVIDUAL FISH DATA FROM SOCIAL FEEDBACK EXPERIMENT Histogram of each individual real fish’s distance from the periphery of the arena, r, as a

function of the strength of the goal-oriented tendency, ω, of the virtual fish. The virtual fish’s internal preferred trajectory was fixed at r = 0.07m (dotted white). These individual fish

data were used (together with control experiments on 15 real fish with no virtual fish) to create panel Fig. 5k. SUPPLEMENTARY FIGURE 15 EMPIRICALLY DERIVED MEASUREMENTS OF GROUP SPLITTING

AND SHARED TRAVEL DIRECTION IN SOCIAL FEEDBACK EXPERIMENT (A) Empirically measured probability of the group (real and virtual zebrafish) splitting as defined by the distance between the two

fish exceeding a threshold value of 0.05 m. (b) Empirically measured probability that the real fish is traveling on the virtual fish’s internal preferred trajectory, r = 0.07 ±0.01m. N=16

fish, same experiment as shown in Fig. 5i-l. SUPPLEMENTARY INFORMATION SUPPLEMENTARY TEXT AND FIGURES Supplementary Figures 1–15 and Supplementary Tables 1–4. LIFE SCIENCES REPORTING SUMMARY

SUPPLEMENTARY SOFTWARE Zip file containing software source code and documentation. SUPPLEMENTARY DATA 1 Metadata associated with figures and videos DEMONSTRATION OF VR FROM THE PERSPECTIVE

OF A FREELY MOVING OBSERVER (LEFT) Video taken from a camera (GoPro) showing the view from the perspective of a freely moving observer. (RIGHT) The colored L-shaped box virtual world

FreemoVR is simulating and the position of the camera in the virtual world (red dot). Once the camera enters the ‘FlyCave’ VR arena its' position is estimated from the tracking software

(right, red dot) and the perspective correct VR is projected onto the walls of the arena. As the camera moves, the projection is updated in real-time to maintain a perspective correct

display. Reproduced with permission from Stowers et al. 2014. DEMONSTRATION OF MULTIPLE-DISPLAY PERSPECTIVE CORRECT VR Related to Video 1. (LEFT) Video taken from above, looking into the

‘FlyCave’ arena, showing the 3D position of the camera (red dot) and the projection onto the arena walls as it moves in space. (RIGHT) The virtual world being simulated, and the estimated

position of the camera in the virtual world (red dot). Reproduced with permission from Stowers et al. 2014. PHOTO REALISTIC AND NATURALISTIC VR FOR FREELY SWIMMING FISH (LEFT) Swimming

behavior of a zebrafish, its position highlighted in red, as it navigates a virtual world. The fish swims in a hemispherical bowl filled with water. (RIGHT) The virtual world being

simulated. The world consists of a cyan sphere and a magenta pyramid in a naturalistic environment. As the fish approaches the pyramid the rendering is updated to display a perspective

correct view of the world. PHOTO REALISTIC AND NATURALISTIC VR FOR FREELY FLYING _DROSOPHILA_ (LEFT) An _Drosophila_ flies inside the cylindrical ‘FlyCave’ VR arena. Its' position is

tracked and highlighted in red. (RIGHT). The virtual world simulated consists of a cyan sphere and a magenta pyramid in a naturalistic environment. As the fly explores the arena the virtual

world is updated in real-time to maintain a perspective correct display for the subject. SIMULATION OF A VIRTUAL POST FOR FREELY FLYING _DROSOPHILA_ (LEFT) A flying _Drosophila_ (position

highlighted in red) interacts with a virtual vertical gray post. (RIGHT) The virtual world being simulated. On the arena walls a checkerboard texture is moved vertically to control the

fly's altitude and to prevent it flying into the walls. INTERACTION OF A _DROSOPHILA_ WITH A REAL POST A flying _Drosophila_ (position highlighted in red) interacts with real post. On

the arena walls a checkerboard texture is moved vertically to control the fly's altitude and to prevent it flying into the walls. SIMULATION OF A VIRTUAL POST FOR FREELY SWIMMING

ZEBRAFISH (LEFT) A juvenile Zebrafish (position highlighted in red) interacts with a virtual post. (RIGHT) The virtual world being simulated contains a black upright post placed at the

center of a sphere covered in a checkerboard pattern. A VIRTUAL ELEVATED MAZE PARADIGM FOR FREELY MOVING MICE An unrestrained mouse explores an elevated platform placed above a

75'' consumer television. FreemoVR simulates a virtual world consisting of two platforms placed virtually 20cm and 40cm below the physical platform. By tracking the mouse head

position, a perspective correct virtual reality can be displayed to the mouse, retaining naturalistic parallax queues and thus the percept of height to the mouse. MOUSE HEAD TRACKING

Illuminated and filmed from above, the software detects the position of the mouse head in real-time (indicated in green) and uses this to create a perspective correct virtual reality. The

detected mouse contour and center are shown in magenta. REMOTE CONTROL FLIES – CONTROLLING THE BEHAVIOR OF FREELY FLYING _DROSOPHILA_ BY EXPLOITING THE OPTOMOTOR RESPONSE (LEFT) A

_Drosophila_ flies in the ‘FlyCave’ VR arena (position highlighted in red). As the fly flies, the virtual world is modified; rotated about its center, eliciting the optomotor response in the

subject and causing it to turn. Doing this continuously causes the fly to follow a path of our design, an infinity-symbol (8) shaped path (RIGHT). ZEBRAFISH SWIMS AMONG A CLOUD OF 3D DOTS

(LEFT) A zebrafish swims (position highlighted in red) among a cloud of dots. (RIGHT) The simulated virtual world containing the 3D cloud of dots. The dots all move with the same velocity.

The velocity of the dots is controlled to cause the fish to swim along an infinity-symbol (∞) shaped path. Dot size is 6.2°, double the size of the “large dot” stimulus in Fig. 4 to increase

visibility in the video recording. ZEBRAFISH IN 2AFC TELEPORTATION EXPERIMENT (LEFT) Wide-angle camera footage of zebrafish swimming in 2AFC teleportation experiment. (RIGHT) The simulated

virtual world containing either a checkerboard floor or virtual plants with a gravel floor. When a fish makes a decision, operationally defined as entering a teleportation portal (black and

white or magenta shape), the fish is virtually teleported to the environment coupled to the portal. Depending on the particular experiment, the specific coupling between portal and

destination varies, but remains fixed for each individual fish. ZEBRAFISH IN 2AFC SWARM TELEPORTATION EXPERIMENT (LEFT) Wide-angle camera footage of zebrafish swimming in 2AFC swarm

experiment. (RIGHT) The simulated virtual world containing the either a swarm of space invaders or a scene without swarm. When a fish makes a decision, operationally defined as entering a

teleportation portal (black and white or magenta shape), the fish is virtually teleported to the environment coupled to the portal. Depending on the particular experiment, the specific

coupling between portal and destination varies, but remains fixed for each individual fish. SOCIAL FEEDBACK EXPERIMENT WITH REAL AND VIRTUAL FISH Camera footage of zebrafish swimming with a

virtual fish. The virtual fish is reacting to the position of the real fish. Here, ω=1, the virtual fish equally balances social and goal-oriented behavior. RIGHTS AND PERMISSIONS Reprints

and permissions ABOUT THIS ARTICLE CITE THIS ARTICLE Stowers, J., Hofbauer, M., Bastien, R. _et al._ Virtual reality for freely moving animals. _Nat Methods_ 14, 995–1002 (2017).

https://doi.org/10.1038/nmeth.4399 Download citation * Received: 08 August 2016 * Accepted: 06 July 2017 * Published: 21 August 2017 * Issue Date: 01 October 2017 * DOI:

https://doi.org/10.1038/nmeth.4399 SHARE THIS ARTICLE Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently

available for this article. Copy to clipboard Provided by the Springer Nature SharedIt content-sharing initiative