Simulation has been used for decades in the military to train everyone from staff officers to pilots.
In the civilian world, a spark of excitement surrounding proclamations of the coming Metaverse has reignited wider interest in Simulation & Virtual Reality. Most recently, Apple’s launch of their Vision Pro, which seeks to bring a version of simulation – Augmented Reality – to the mass market, has brought a real heavyweight to the market.
It is hard to articulate fully the role of simulation in military training. Not only does the term ‘simulation’ cover a range of different technologies, but the possible use cases cover a range of domains and users. Military training and simulation are classified as ‘live’, ‘virtual’ and ‘constructive’.
Live training takes place in a physical environment so is not relevant here, and nor is constructive training, which involves “Simulated people in a simulated environment using simulated equipment with simulated effects” and is often used for operational analysis models and wargaming.
However, virtual training, in the military sense, is concerned with “Real people in a simulated environment using simulated equipment with simulated effects.” So the term covers a range of different types of technologies including what are usually known as Augmented, Mixed, and Virtual reality. It sits across Live and Constructive.
One way of seeing different simulation types in this context is to picture them along a spectrum where virtual reality has the user most fully in the simulation and augmented reality where the physical (live) world and virtual world are blended the most.
Virtual Reality (VR)
This is the domain you will naturally think of when considering virtual technologies; a user conducting all of their training in a computer-game-type simulation. This can either be interfacing through a normal computer screen or with a VR headset such as the Oculus Quest.
What is important is that all the activity the user is experiencing – beyond inputs from computer hardware – is happening inside the simulation. An excellent example of this type of training would be the British Military’s Combined Arms Tactical Trainer, or CATT. The CATT employs hardware which exactly replicates the internals of the British Army’s armoured vehicles. The simulation starts when a user looks through a weapon sight or a window. When they do, what they see is a simulated (virtual) picture. Subsequently, when the Tank(s) take part in a tactical activity, they are all doing so inside the simulation. Other examples of this would be military flight simulators.
VR, therefore, in the context of military training is best executed where the hardware to software interface can be as close to reality as possible, often including vehicles or larger weapons platforms. Precisely replicated controls become highly specific virtual reality input devices. The introduction of less faithful control inputs would train individuals in incorrect drills, which could easily reduce capability rather than enhance it.
Mixed Reality (MR)
With MR, we move further down the virtual spectrum and closer to the physical world. In this type of simulation, the physical world acts as the canvas on which a virtual picture is painted. Imagine a room, where all the walls and any furniture is painted chromakey green. Without a MR headset, such as those developed by Varjo, all a user would see is green. With a head set on, the visual picture is transformed with the colours and textures of whichever training environment is required. A user visually interacts with the space through the virtual world, while physically interacting with the items inside of the room. MR, therefore, provides the opportunity for a single space to be used for multiple different visual scenarios.
This does come with the sacrifice of the potentially boundless space offered by VR as the user is limited by the physical space they are in. However, the advantage is that this type of technology can offer visually varied judgemental training in physically small locations. Military uses can include situations where a user is interacting with a large space but is static themselves (such as training to direct artillery fire).
Augmented Reality (AR)
AR really starts to blend the virtual and physical. Here the physical world still acts as the canvas for a simulation, however it differs from MR in that the full fidelity of the physical world is available. AR overlays the simulation on the real world to allow users to interact with both spaces simultaneously.
Conceptually, this type of technology is likely to be familiar to the user; the Apple Vision Pro mentioned in the introduction being a great example; another is Microsoft’s HoloLens goggles. There is, undoubtably, significant potential in this type of technology and companies such as Red6 are taking some incredible steps to use this technology to train fast jet pilots. In other areas, such as training infantry soldiers, AR development is progressing slower but still offers future potential. It is also with AR that we start to see utility for virtual overlays on operations as well as during training.
The above descriptions are by no means exhaustive. They also overlook where some technologies blur the lines between these descriptions; in a Varjo headset, for instance, anything not in the specified chromakey colour will appear its physical form. That said, we can now move forward to discuss what the roll of virtual technologies is in military training.
Simulated Training
Simulated training has predominantly been adopted to reduce the risks inherent with military training whilst maintaining as many of the relevant features of an activity as possible. Especially in the case of highly complicated platforms, it is much more sensible to have a trainee make mistakes in a simulation than in a real life.
Learning to fly – or not – an F35 in a simulator is much less costly to both the individual and the airframe in a simulation. In addition, successes or mistakes can be reviewed immediately and in detail with a fidelity not possible in Live training. This does, however, require a simulator to be an exact replica of the real thing. This is both in terms of the fidelity of the hardware as well as the performance of a platform in a simulation. The latter is the key difference between military training software and gaming, with military software containing detailed – often sensitive – information on the performance of platforms and weapon systems. Without these two things, simulators and virtual reality are likely to train bad habits rather than good.
At 4GD, we believe the key to successful simulation for dismounted infantry soldiers is in the sensible blend of the live and virtual worlds. This is a particularly difficult problem to achieve when operating outside of the physically controllable environment of a cockpit or tank turret. As such, current successful attempts to bring simulation to dismounts focuses more on wargaming or training the theory of tactical actions and much less on the physical execution.
In this space, simulation still has a role in attacking training problems that are either too difficult, too dangerous, or too costly to achieve normally. However, this should not be through the sacrifice of training fidelity and lies in the pragmatic blend of the virtual and physical worlds. By seeking to transcend the current boundaries of how synthetic and physical can interact, it is possible to bring effective Virtual training out of major weapons platforms and into the domain of the infantry soldier.
