Christopher Cytera, Chief Operating Officer at Spectral Edge discusses how to deliver surveillance colour accuracy detail, capable of improving facial, object and action recognition in even the most challenging lighting conditions.
What is Spectral Edge’s secret of achieving colour accuracy and detail in very low light conditions?
Spectral Edge Fusion achieves colour accuracy thanks to our patented algorithm for image fusion which dynamically weights visual information from visible (RGB) and Near Infrared (NIR) light, pixel-by pixel, before combining the two for best results, all without taking any information away and all in real-time. The detail and texture information comes from fusing the NIR with visible light intelligently.
How did you get into light fusion work originally and what problems did you solve (in other markets/applications)?
Spectral Edge was born out of the University of East Anglia’s Colour Lab. We gained funding to spin out from there and commercialise our expertise by finding applications for RGB and NIR light fusion. After some commercial success in the broadcast television world we decided to focus on the surveillance market as we spotted the need for our technology to support the next generation of video analytics software.
What future challenge(s) does the professional security market still face?
There is a clear need for improvements in people, object and action recognition as well as positive identification in poor lighting conditions. In such conditions, too many false positives are being sent through to control rooms and images are often not of evidential quality in terms of positive identification in a court of law.
There is also a need to lower the cost of high-performance of high quality cameras so they can be installed more widely to prevent crime and send timely, relevant and actionable alerts. Build costs must be reduced without loss of quality, and without opening surveillance systems up to cyber-attacks.
What are the benefits of intelligently-mixing RGB and NIR light in low or mixed lighting conditions in the surveillance market?
Spectral Edge Fusion is capable of making person and object recognition and identification much more accurate because colours are preserved. So, an intruder after dark with a head torch on can still be positively identified using our technology. Facial features and colour of clothing can be gathered even in very low light levels – all from a single camera and lens. This enables high-end day/night camera manufacturers to reduce build costs considerably as they will no longer need several cameras, lenses, IR-cut filters etc, to deliver usable images. Spectral Edge Fusion can also deliver more evidentially-sound images because no RGB or NIR light has been thrown away. Therefore, in low light conditions, no artificial colour will be injected into the image – we only make use of real colour information that is there.
What mistakes do you see in competitor technologies designed for delivering usable images in low light or foggy conditions?
Many day/night cameras today employ more than one sensor (RGB and IR) and IR-cut filters so essentially they switch from RGB to IR as light levels are depleted. So, in RGB mode they discard the NIR information which could work to add texture, depth and detail to the image; whilst in low light/IR mode they fail to collect any visible light at all, even if there is enough to detect colours. Some artificially inject colours into low light images which can create unreal images which sometimes cause video analytics applications to fail. Another option is expensive multi-camera set-ups which make it difficult to cover all sensitive areas effectively.
Is Spectral Edge Fusion technology likely to be embedded into home surveillance systems over time (or is this for high-end airports/transport hubs/perimeter protection of sensitive sites only)?
This will happen over time. We anticipate that Spectral Edge Fusion will be deployed in high-end day/night cameras initially. However, within as little as two years it could be deployed through mid-range cameras and so on through to entry-level, and finally home surveillance cameras. Just like the electric car window, eventually its value is seen throughout the market and is ubiquitously applied.
What type of sensors, lens and IR lighting is required to get the most out of Spectral Edge Fusion?
You need to use sensors with both RGB and IR pixels. The good news is that RGB+IR sensors are fairly widely available now and do not cost any more to manufacture than RGB-only sensors. The lens ideally has a dual band pass (notch) filter instead of an IR-cut filter and is IR-corrected to focus IR in the same manner as the visible light. The only other consideration is selecting the right quality and power of IR illuminator appropriate to the distance it is throwing light over. Remember that at dusk our technology can make use of ambient IR light from the sun.
Why should those specifying new security equipment consider demanding your technology now?
If they are looking for more accurate facial recognition, object or action recognition in poor light conditions then they should look at Spectral Edge Fusion. In addition, if they have failed to secure prosecutions as relevant recorded images have failed to positively identify individuals; or colours in images captured at dusk have been rendered so poorly that the courts have questioned the credibility of the images; then it’s definitely worth exploring the potential of RGB & NIR light fusion.
What sort of feedback are you getting from camera and/or silicon manufacturers right now?
The camera manufacturers are asking for a chip to be made available to them for some of their day/night models. High-end and military grade camera manufacturers have requested Spectral Edge Fusion to be made available on a FPGA chip. We’ve now conducted image quality evaluations for several major chip manufacturers in which they’ve sent us RAW video data and we’ve sent back significantly improved images with more detail and colours rendered. They are now asking for gate counts with a view to adding our technology onto their sensor boards in a highly compact way synthesisable for ASIC.
Why is the technology particularly valuable for some video analytics applications? Which ones are likely to benefit most from using Spectral Edge Fusion?
We think object recognition in poor light, especially establishing precise colour of a car in the dark and facial recognition – positively identifying the driver of that car, would be two clear applications in which our technology would improve quality of that image to a level where it is more usable.
And from a highly topical perspective, we believe Spectral Edge Fusion can support identification of the type of drone which may be hovering over a sensitive site. It’s critical for authorities to know whether a drone which is causing a hazard is military-grade – perhaps capable of disabling a plane or even dropping a payload for example – or a hobbyist’s play thing that would normally pose less of a serious threat.
Which other markets or products are likely to benefit from your technology?
It’s conceivable that our technology could be deployed inside wearable cameras for soldiers in the field or policemen on the streets to support identification of individuals and establishing degree of threat posed by them, all in low or mixed lighting conditions.
