An example answer for the ‘Overcoming uncertainties’ question on the EmpowerRD platform.
How did your project seek to overcome these uncertainties?
Project context:
We’ve created a project example below to help you answer the project questions in alignment with HMRC’s guidance.
The following example answer is for a project that tried to create an imperfect algorithm AI solution for a poker-playing software.
How we tried to overcome the uncertainties we encountered:
– Firstly, we conducted a review of the existing research literature and found no evidence of resource-efficient imperfect information AI algorithms in the public domain. Therefore, we had to develop our own from scratch in C++.
– We explored existing implementations and found their resource lacked efficiency: existing AI software libraries are typically written in Python, which has poor CPU and RAM usage and is therefore not suitable for the level of efficiency we require for our algorithms. Therefore, we had to create all of our software (both the algorithms themselves and the supporting code) from scratch in C++.
💡 As shown above, we have clearly shown 1: what already exists in the public domain, and 2: what the limitations of any existing software applications are. This builds a richer picture of why we couldn’t use standardised approaches to overcome our uncertainties.
– We had to experiment with claimed performance improvements from published AI research papers, to analyse which resources were valid for improving the performance of the end AI agent. Furthermore, we needed to explore which improvements could be reimplemented as part of a resource-efficient algorithm.
💡 Similarly to the previous section, the above details show which steps we completed to try and overcome the outlined uncertainties. This is very important, as we have already explained what uncertainties we encountered; this means that we now have to explain how we did this. This continues in the below sections, where we build upon our findings and methods further.
What we discovered when we tried to overcome these:
– We found that claimed performance improvements from other research papers were not valid, or that it was impossible to re-use them in a resource-efficient way. We also frequently found that code changes we thought would improve efficiency, in theory, did not work in practice.
– This involved a large amount of trial and error. There were no existing solutions we could use – we discovered that we had to create everything from the ground up in C++.
How we tried to work around our previous attempts:
– To tackle this, we trained over 200 AI agents of various complexity over the course of the year in order to test it. It was unknown which efficiency improvements would work and which wouldn’t, as the theory of low-level software optimisation often does not align with the complex reality of computing systems. We wrote the underlying code for our algorithm multiple times during the course of this project.
💡The above details show not only the ‘why’, but also the ‘how’. By using specific metrics, such as 200 AI agents, we are able to show HMRC our workings. Remember, that even if your methods were not successful, it is still important to explain the steps you took and why they did, or didn’t, work.
The outcome of our project:
– The project is ongoing, but we have been successful in meeting the first important milestone. Through testing and coding, we created an AI agent which is 20x more resource efficient than the current state-of-the-art, and has the same performance when applied to the game of two-player poker.
💡 A summary of your project’s outcome can be helpful in showing HMRC whether your product, service or solution was successful in its aims and objectives. A project can be ongoing, so you can also discuss what you were able to achieve during this time, and why your project needs to continue or adapt.
Still not sure?
Ask us a question via the live chat, or get in touch with us at [email protected]
About the author
