Satellites are open to attack. All communications systems are vulnerable to hackers by definition, and are protected accordingly, with hardware and software. However, because they reside outside the protection of the Earth’s atmosphere, satellites are perhaps exposed to the mightiest and most unpredictable hacker of all: the Sun.
“The Sun’s radiation can affect the data being processed in any system in Space,” explains Rubén Rodríguez Álvarez, a doctoral assistant at ESL, the Embedded Systems Lab of Prof. David Atienza. “The radiation causes bits to flip, so your data is polluted with interference, and you might not even know that this is happening.”
All traditional computing systems use binary logic: bits of binary values of 1 or 0, stored as electrical charges. If any process randomly changes the value of your bits, your data is corrupt. The knee-jerk reaction to protect a system from radiation would be to install reflective shields, but this is exactly the kind of heavy duty solution that satellite engineers are keen to avoid.
“An idea from my colleague, Dr. Miguel Peón-Quirós, seemed to provide a better way: using a software solution that had been pioneered by another researcher in our lab,” explained Rodríguez. “Miguel suggested using Flavio’s technology as an answer to the problem of radiation interference without an increase in overhead, either of hardware or software.”
The Flavio mentioned above is Dr. Flavio Ponzina, recently graduated from ESL, and now a Postdoctoral researcher at the University of California, San Diego. “E2CNN is a technique where we take a neural network, prune it carefully, and then send multiple copies of it down separate channels,” explains Ponzina. “We send the same neural network of data through multiple channels, for the sake of redundancy. If radiation, or any other kind of interference, affects the information in one of these channels, the other ones will be given credibility over the one with the corrupt data.”
6DoF or six degrees of freedom: translations (sliding movements) around the three geometric axes: x, y and z; and rotations around the same three axes
A satellite’s positional data is usually processed as 6DoF. This complex data is compiled from sensor data and stored in a dynamic model: a convolutional neural network, or CNN, developed by the Computer Vision Lab (CVLab) of EPFL for ClearSpace. Video data from the satellite’s optical sensors is also stored in CNNs.
Sending these neural networks of data down multiple separate processor channels means that even if one of these channels is hit by a burst of radiation, the version reported over the other channels will outvote the minority version. Those familiar with Philip K. Dick’s The Minority Report, or the movie based on it, might recognize the concept.
“We use an intelligent method to prune the neural network before we send it down separate channels, to reduce the size of the model,” explains Ponzina. “This means that, even though we send it down multiple channels, there is no increase in processing demands. But there is actually an increase in accuracy of 5%, even without radiation interference! If we simulate radiation interference, accuracy goes up by 9%.”
For some time this technology has been known to ClearSpace, the EPFL spin-off that intends to launch satellites to remove Space debris from Earth’s orbit. ClearSpace-1 will take off some time in the next few years, using the protective system described
Now that Ponzina has left EPFL, it is Rodríguez who is continuing the project, implementing the software on FPGAs – Field Programmable Gate Arrays, a technology uniquely suited to this kind of application. FPGAs are modular chips: processors that can change shape according to the requirements of the application. In this case, they adapt wonderfully well to the sending of neural networks down multiple parallel channels.
Dr. Jacques Viertl of ClearSpace is clear on the benefits of this system: “We asked for an implementation on FPGAs because of the redundancy they offer. Whenever you deal with images there is a trade-off between quality and overhead, but in the case of the visual recognition of targets by ClearSpace-1, the processing of the information will be protected by the multiple channels being used, at no extra cost in processing power.
“The connection between ClearSpace and EPFL is really strong,” explains Viertl. “To us, student projects mean a supply of youthful energy, fresh ideas, and technology transfer.
“It is more often we who are asking for student collaboration, than the other way around. The fact is, it is extremely difficult to find engineers in the domain of FPGAs. We spoke to one student, recently, who told us that we were one of seven companies offering them a placement!”
Prof. David Atienza, director of the Embedded Systems Lab, joined the four scientists mentioned above, in collaboration with other scientists from EPFL and the University of California San Diego, in authoring a paper which was presented at SPAICE 2024, and is now puglished in NASA’s proceedings:
Using ensemble learning to improve radiation tolerance of CNNs in space applications
Flavio Ponzina, Rubén Rodríguez Álvarez, José Miranda Calero, Mathieu Salzmann, Jacques Viertl, Tajana Rosing, Miguel Peón-Quirós, David Atienza
“For ESL-EPFL and EcoCloud this collaboration was very important to make sure the research we do in edge AI and smart embedded systems can be applied to the context of space technology,” explains Atienza. “In particular, ClearSpace had a great understanding of the constraints of the final system, and it helped us tune all the AI/ML algorithms and system performance accordingly to meet such constraints. I also think it is really beneficial for our researchers to see that the research they do can be applied to the real world only if we work with companies like ClearSpace that work on the deployment of the final aerospace systems.”
These are exciting times: science fiction is quickly being outpaced by reality, established technology companies are looking to the youngest engineers for skill and inspiration, and our labs keep finding ways to solve the toughest problems – like how to stop the Sun from flipping our bits.
Text: John Maxwell