Return to Answer

The canopy's field of view is about half the sphere, while cameras can cover most of the sphere.

The dynamic range of human vision, at 20 stops, is better than the best cameras at 15-16 stops, and LCD drop it further to 10-12 stops (20 for OLED). But this is acceptable in daytime, and will mostly matter at night, where a fighter can rely on IR systems insteads. The night-time performance of IR cameras, even though they lose a lot in resolution, gives synthetic vision a win on this.

For reaction speed, at their open gate resolution, cameras offer a best-case framerate of just 60. I won't copy the full math, but combined with a 60 fps display, this results in a minimum lag of ~40 milliseconds. This sounds acceptable at first. The real pain comes when you consider the motion resolution resulting from this limited framerate.

To explain motion resolution, see how a moving picture looks on a display. This can be better demonstrated with a few illusions. Unless the motion is very slow, your motion resolutio can drop from the 8Kx4K a camera can offer for a still picture down to only 100-600 lines. In practical terms, you're going from discerning objects at 0.4 milliradian all the way to 2-10.

The canopy's field of view is about half the sphere, while cameras can cover most of the sphere. A win for synthetic vision.

The dynamic range of human vision, at 20 stops, is better than the best cameras at 15-16 stops, and much better than LCD at 10-12 stops (20 for OLED). But this is acceptable in daytime, and will mostly matter at night, where a fighter can rely on IR systems instead. The night-time performance of IR cameras, even though they lose a lot in resolution, gives synthetic vision a win on this.

For reaction speed, at their open gate resolution, cameras offer a best-case framerate of just 60. I won't copy the full math, but combined with a 60 fps display, this results in a minimum lag of ~40 milliseconds. This sounds acceptable at first. The real pain comes when you consider the motion resolution resulting from this low framerate.

To explain motion resolution, see how a moving picture looks on a display. This can be better demonstrated with a few illusions. Unless the motion is very slow, your motion resolution can drop from the camera's 8Kx4K to only 100-600 lines depending on the rate. In practical terms, you're going from discerning objects at 0.4 milliradian all the way to 2-10.

Good human vision (healthy young pilot) offers enough acuity to distinguish high-contrast objects down to about 0.1 milliradian in size. The best commercial video camera heads offer up to 6.5K3K or 8K4K resolution. To cover 12.6 steradian with 6 cameras, restricting each camera to a ~2:1 rectangle, somewhat correctable with an anamorphic lens... It's a textbook math problem, but we have the advantage of knowing it's not solved in a mathematically optimal way in real fighters using it.

For a "cube with overlaps" solution, which is viable for a fighter, the FOV of each camera needs to be about 1.6 radian vertically. This results in a best-achievable instantaneous resolution of about 0.5 milliradian for the Arri sensor or 0.4 for the RED. These are commercial cameras that are miles ahead of the F-35's DAS, but indicative of what could be used in a fighter designed today and meant to fly in the 2040s.

At their maximum resolution, all cameras produce significant amounts of noise. So does the human vision, but its noise is filtered out very well by the brain and almost eliminated in the macula. At any rate, camera+eye give more noise than just the eye.

And most importantly, in the last 10 years we've also learned to protect data with authenticated encryption and send it fast over vast distances, so if things like best possible acuity, motion resolution, reaction speed, or tolerance to avionics damage don't matter much, there's no reason for the pilot to be inside the aircraft at all.

Good human vision (healthy young pilot) offers enough acuity to distinguish high-contrast objects down to about 0.1 milliradian in size. The best commercial video camera heads offer 4.5K2K to 8K4K resolution. To cover 12.6 steradian with 6 cameras, restricting each camera to a ~2:1 rectangle, somewhat correctable with an anamorphic lens... It's a textbook math problem, but we have the advantage of knowing it's not solved in a mathematically optimal way in real fighters using it.

For a "cube with overlaps" solution, which is viable for a fighter, the FOV of each camera needs to be about 1.6 radian vertically. This results in a best-achievable instantaneous resolution of 0.5-0.7 milliradian for Arri sensors or 0.4 for the RED (which has other drawbacks). These are commercial cameras that are miles ahead of the F-35's DAS, but indicative of what could be used in a fighter designed today and meant to fly in the 2040s.

This can be improved, 6 cameras are not some hard limit, but you're looking at 100+ to fully match the acuity of the human eye (a more accurate term than resolution, since the eye doesn't perceive the whole FOV simultaneously). At that point the system won't be reasonably compact and easy to add anymore.

At their maximum resolution, all cameras produce significant amounts of noise. So does the human vision, but its noise is filtered out very well by the brain and almost eliminated in the macula. For small megapixel-chasing cameras, the noise is so heavy that their effective resolution is reduced several times. Above we assume professional cameras with full-format sensors. At any rate, camera+eye give more noise than just the eye.

Most importantly, in the last 10 years we've also learned to protect data with authenticated encryption and send it fast over vast distances. So if things like best possible acuity, motion resolution, reaction speed, or tolerance to avionics damage don't matter much, there's no reason for the pilot to be inside the aircraft at all.

At their maximum resolution, all cameras produce significant amounts of noise. So does the human vision, but its noise is filtered out very well by the brain and almost eliminated in the macula.

The dynamic range of human vision, at 20 stops, is better than the best cameras at 15-16 stops. Displays are 10-12 stops for LCD, 20 for OLED, but this is acceptable in daytime. The difference will matter at night, where a fighter just rely on IR systems. All in all, the night-time performance of IR cameras gives synthetic vision a win on this.

For reaction speed, at their open gate resolution, cameras offer a best-case framerate of just 60. I won't replicate the math, but combined with a 60 fps display, this results in a minimum lag of ~40 milliseconds. This sounds acceptable at first. The real pain comes when you consider the motion resolution resulting from this limited framerate.

This won't matter much in level, but the moment a pilot with synthetic vision begins to maneuver, they will suffer a major degradation of their vision. Video gamers compensate by turning instantly and returning to a slow-moving image, but a plane doesn't do things instantly. Whether the ability to see while spinning will make a difference, losing it can affect pilot behavior just to avoid that motion blindness.

At their maximum resolution, all cameras produce significant amounts of noise. So does the human vision, but its noise is filtered out very well by the brain and almost eliminated in the macula. At any rate, camera+eye give more noise than just the eye.

The dynamic range of human vision, at 20 stops, is better than the best cameras at 15-16 stops, and LCD drop it further to 10-12 stops (20 for OLED). But this is acceptable in daytime, and will mostly matter at night, where a fighter can rely on IR systems insteads. The night-time performance of IR cameras, even though they lose a lot in resolution, gives synthetic vision a win on this.

For reaction speed, at their open gate resolution, cameras offer a best-case framerate of just 60. I won't copy the full math, but combined with a 60 fps display, this results in a minimum lag of ~40 milliseconds. This sounds acceptable at first. The real pain comes when you consider the motion resolution resulting from this limited framerate.

This won't matter much in level flight, but the moment a pilot with synthetic vision begins to maneuver, they will suffer major degradation of their vision. Video gamers learn to compensate by turning instantly and returning to a slow-moving image that can be seen at sub-1000 fps, but a plane doesn't do things instantly. Whether the ability to see while spinning will make a difference, losing it can affect pilot behavior just by trying to avoid that motion-induced blindness.