Motivation
My previous work has covered Quantum Image Representation, which is the precursor to actual image processing. The premise is that to leverage the power of quantum computation in an image processing task, the first step must involve the encoding of an image into a quantum state. This can be done using a few methods, primarily angle encoding, basis encoding and amplitude encoding. All of the work in this field I have performed thus far uses amplitude encoding. This is the most commonly used method in the field of quantum image processing.
I studied various image encoding algorithms including Flexible Representation of Quantum Images (FRQI) and Novel Enhanced Quantum Representation (NEQR) and several offshoots and variations of these and other well-known methods. I implemented a simple program for FRQI encoding and one of my first experiments. I then moved on two a standard image processing task which was edge detection. I was able to implement a technique presented in an IBM Qiskit Tutorial and was able to realize results on quantum computer simulators, “fake” quantum computers which are quantum computer simulators that are “tuned” to act and respond more closely to a real quantum computer and finally I ran my algorithms against real quantum computers, mainly those offered by IBM’s Q Experience.
For this initial task, I had to process an image in parcels or chunks which were comprised of smaller sub-sections of the image. This was the first pre-processing task. The quantum edge detection algorithm was then executed against each sub-section of the image. After processing, the result per sub-section was saved and when all of the sub-sections were processed, a post-processing task was performed where all of the sub-sections were joined back as one image.
During one of the graduate courses that I took sponsored by QWorld out of Latvia, I was part of a group that attempted to implement a simple QNN using X, Y and Z gates, but we were unable to complete our experiment. Although our attempt was a failure, we were at least able to attempt to go through the motions of encoding, pre-processing, training, testing, validation and deriving results.
For this current project, I implement a simple Quantum Convolutional Neural Network (QCNN) to identify targets in satellite imagery which is lofty goal as most quantum image processing is done at a very low level. At best I would attempt to identify horizontal and vertical lines in a satellite image which is quite a way away from actual target detection but is a step towards this end goal.
However, as I was studying various types of machine learning models, I came across a few references to quantum transformer neural networks and in particular Quantum Vision Transformers. Transformer neural networks used for image processing deal with splitting an image into sub-sections and applying an attention mechanism to each sub-section. This lends well to the fact that some of my initial work dealt with splitting an image into sub-sections as well.
[Image: Bill Gonzalez]
In addition, the process of operating on sub-sections allows for shallow circuits. The depth of a circuit is one key factor when working with today’s Noisy Intermediate-Scale Quantum computers (NISQ).
Methods
In order to continue study in this area I have devised a plan of action:
- Implement a suitable encoding method.
- Devise the transformer circuit.
- Implement the training across each image chunk in a loop
Literature and Code Review
Although I read though many research papers covering quantum convolutional neural networks, I only found a few related to quantum transformer neural networks. The list is short; however, these papers make reference to additional published works. The primary paper I reference is currently under review from what I understand.
I was also able to find an initial GitHub repository that provided an example of a quantum-enhanced transformer neural network. This repository is from 2021.
GitHub – rdisipio/qtransformer: Quantum-enhanced transformer neural network
I also found another useful GitHub repository on Quantum Transformers in use for High Energy Physics at the Large Hadron Collider. This is a much more recent body of work from 2023. This repository primarily operated using the Qulacs quantum simulator.
Although I quickly read through this third reference, I did not focus on it at all. This reference covered on aspect of quantum transformers, primarily Transformer Quantum State (TQS). It was also accompanied by a GitHub repository.