Highlights from NeurIPS 2022 and the 2nd Interactive Learning for NLP Workshop – Dr Edwin Simpson

This blog post is written by lecturer in Computer Science, Dr Edwin Simpson

In November I was lucky enough to attend NeurIPS 2022 in person in New Orleans, and take part as a co-organiser of InterNLP, our second interactive learning for NLP workshop. I had many interesting discussions around posters, talks and coffee breaks and took loads of photos of posters. It was hard to write up my highlights and without the post becoming endlessly long, so here is my attempt to pick out a handful of papers that caught my eye and tell you a little bit about how our workshop unfolded.

Main Conference

One topic generating a lot of buzz was in-context learning, where language models learn to perform new tasks without updating their weights from examples given in the model’s input prompt. Models like GPT3 can perform in-context learning from small numbers of examples. Garg et al. presented an interesting paper that triez to understand what classes of functions can be learned in this way [1]. They were able to train Transformers that learn function classes including linear functions and two-layer neural networks.


However, for few-shot learning, in-context learning may not be the best solution: Liu et al. [2] showed that fine-tuning a model by introducing a small number of additional weights can be cheaper and produce more accurate models.



Another interesting NLP paper from Jian, Gao and Vosoughi [3] learns sentence embeddings usingimage and audio data alongside a text training set. The method works by creating pairs of images (or audio) using data augmentation, which are then embedded and fed through a BERT-like transformer to provide additional data for contrastive learning. This is especially useful for low-resource languages and domains, and it is really interesting that we can learn from different modalities without any parallel examples.

Many machine learning researchers are concerned with models that produce well-calibrated probabilities, but what difference does calibration make to end users? Vodrahalli, Gerstenberg and Zou [4] investigated a binary prediction task in which a classifier provides advice ta user, along with its confidence. They found that exaggerating the model’s confidence led the user to perform better. So, the classifier was uncalibrated and had higher training loss but the complete human-AI system was more effective, which shows how important it is for ML researchers to consider real-world use cases for their models.

Sticking with the topic of uncertainty, Bayesian deep learning aims to quantify uncertainty in complex neural network models, but is challenging to apply as it is difficult to specify a suitable prior distribution. Ideally, we’d specify a prior over the functions that the network encodes, rather than over individual network weights. Tran et al. [4] introduce a method for setting functional priors in Bayesian neural networks, by aligning them with Gaussian processes. It will be interesting to try out their approach in some deep learning applications where quantifying uncertainty is important.

At the poster sessions, I also enjoyed learning about the wide range of new benchmarks and datasets that will enable lots of exciting future work. For example, one that relates to my own work that I’d like to make use of is BIGBIO [5], which makes a number of biomedical NLP datasets more accessible and will hopefully to more reproducible results.

Juho Kim, who is associate professor at Korea Advanced Institute of Science and Technology (KAIST), gave a keynote on his vision of Interaction-Centric AI. He called on AI researchers to move beyond data-centric or model-centric research by rethinking the complete AI research process around the user experience of AI. Juho’s talk gave examples of how an interaction-centric approach may affect the way we evaluate models, which cases we focus on when trying to improve accuracy, how to incentivise users to engage with AI, and several other aspects of interaction-centric AI that his lab has been working on. He demonstrated Stylette, a tool that lets you use natural language to change the appearance of a website. The keynote ended with a call to action for AI researchers to rethink performance metrics, the design process and collaboration, particularly with HCI researchers.

Geoff Hinton appeared remotely from home to present the Forward-Forward algorithm, a method for training neural networks without backpropagation that could give insights into how learning in the cortex takes place. His experiments showed some promising early results, and in the Q&A Geoff talked about coding the experiments himself. A preliminary arXiv paper is now out [6].

1. Garg et al., What Can Transformers Learn In-Context? A Case Study of Simple Function Classes, https://arxiv.org/abs/2208.01066

2. Liu et al., Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning, https://arxiv.org/abs/2205.05638

3. Jian, Gao and Vosoughi, Non-Linguistic Supervision for Contrastive Learning of Sentence Embeddings, https://arxiv.org/pdf/2209.09433.pdf

4. Vodrahalli, Gerstenberg and Zou, Uncalibrated Models Can Improve Human-AI Collaboration, https://arxiv.org/abs/2202.05983

5. Fries et al., BigBIO: A Framework for Data-Centric Biomedical Natural Language Processing, https://arxiv.org/abs/2206.15076

6. Hinton, The Forward-Forward Algorithm: Some Preliminary Investigations, https://arxiv.org/abs/2212.13345

InterNLP Workshop

2022 was our second edition of the InterNLP workshop, and we were very happy that the community grew, this year with 20 accepted papers and a chance to meet in person!  Some of the videos are on youtube at https://www.youtube.com/@InterNLP. Others will hopefully be available soon on the NeurIPS archives

The programme was packed with impressive invited talks from Karthik Narasimhan (Princeton), John Langford (Microsoft), Dan Weld (UWashington), Anca Dragan (UCBerkeley) and Aida Nematzadeh (DeepMind). To pick out just a couple, Karthik presented recent work on semantic supervision [1] for few-shot generalization and personalization, which learns from semantic descriptions of classes, providing a way for instruct models through text. Anca Dragan talked about interactive agents that go beyond following instructions about how exactly to perform a task, to inferring the user’s goals, preferences, and constraints. She emphasized that the way people refer to desired actions provides important information about their preferences, and therefore we can infer, from a user’s language, reward functions that reflect their preferences. Aida Nematzadeh compared self-supervised pretraining to language learning in childhood, which involves interacting with other people. Her talk focused on the evaluation of neural representations, and she called for real-world evaluations, strong baselines and probing to provide a much more thorough way of uncovering the strengths and weaknesses of pretrained models.

The contributed talks and posters showcased a wide range of work from human-in-the-loop learning techniques to software libraries and benchmark datasets. For example, PyTAIL [2] is a Python library for active learning that collects new labelling rules and customizes lexicons as well as collecting labels. Mohanty et al. [3] developed the IGLU challenge, in which an agent has to perform tasks by following natural language instructions; their presentation at InterNLP explained how they collected the data. The RL4M library [4] provides a way to optimize language generation models using reinforcement learning, as a way to adapt to human preferences; the paper [4] also presents a benchmark, GRUE, for evaluating RL methods for language generation. Majumder and McAuley [5] investigate the use of explanations to debias NLP models while maintaining a good trade-off between predictive performance and bias mitigation.





At the end of the day, I got to ask a lot of questions to some very smart people during our panel discussion – thanks to John Langford, Karthik Narasimhan, Aida Nematzadeh, and Alane Suhr for taking part, and thanks to the audience for some great interactions too. The wide-ranging discussion touched on the evaluation of interactive systems (how to use static data for evaluation, evaluating how well models adapt to user input), working with researchers and users from other fields, different forms of interaction besides language, and challenges that are specific to interactive NLP.

We plan to be back at a future conference (not sure which one yet!) for the next iteration of InterNLP. Large language models and in-context learning are clearly revolutionizing this space in some ways, but I’m convinced we still have a lot of work to do to design interactive machine learning systems that are accountable, reliable, and require fewer resources.

Thank you to Nguyễn Xuân Khánh for letting us include his InterNLP workshop photos.

1. Aggarwal, Deshpande and Narasimhan, SemSup-XC: Semantic Supervision for Zero and Few-shot Extreme Classification, https://arxiv.org/pdf/2301.11309.pdf

2. Mishra and Diesner, PyTAIL: Interactive and Incremental Learning of NLP Models with Human in the Loop for Online Data, https://internlp.github.io/documents/2022/papers/24.pdf

3. Mohanty et al., Collecting Interactive Multi-modal Datasets for Grounded Language Understanding, https://internlp.github.io/documents/2022/papers/17.pdf

4. Ramamurthy et al., Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization, https://arxiv.org/abs/2210.01241

5. Majumder and McAuley, InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions, https://arxiv.org/abs/2210.07440

BIAS 22 Review Day 1 – Daniel Bennett “Complexity and Embodiment in Human Computer Interaction”

This blog post is written/edited by CDT Students  Amarpal Sahota and Oliver Deane

This was a thought provoking starting point and one that clearly has a large impact on human computer interaction.  Daniel stated that this is a line of research in psychology, cognitive science, and robotics, that has run somewhat parallel to mainstream psychology.

One of the initiators of this was James J Gibson. Gibson and others in the last 70 years did a lot of work on how we use resources outside of just the brain, in our environment and in our bodies, and coordinate all of these together to behave effectively.  Daniel stated that with the lens of embodiment we start focusing on processes, interactions, and relations, and the dynamics that follow and this is primarily a change in how we model things.

Therefore, to summarize one could consider the traditional cognitive model as a linear system. First we sense the world, then we form a representation of that world in our brain. Then the representation gets processed through a bunch of neatly defined modules, updates existing plans and intentions, and results in some representation of an action, which we then carry out. The embodied view is more complex as we are not simply in the world but also a part of it.  The world is changing constantly, and our behaviour and cognition is simply another physical process in this world.

At a high-level embodied approaches consider behaviour in the world as a kind of continual adjustment and adaptation, with most behaviours are grounded in a kind of improvisatory, responsive quality. Daniel shared a good example of this from Lucy Suchman related to canoeing where you may have an idea of your plan as you look down the river ‘I need to stay left there, slow down over there’ but at execution time you have to adapt your plan.

Daniel stated that a lot of work has been done observing a wide range of human behaviours, from technology interaction, to manning air-traffic control centres and crewing ships. In all of these contexts it is argued that our embodied skills – our adaptation and our implicit skills of coordination with the mess of the situation as it plays out – are the most important factor in determining outcomes.

Human Computer Interaction is increasingly focused on complex behaviours. Daniel talked about the idea that we’re going to do more and more in augmented reality and virtual reality. Computing will be integrated to support a wide range of everyday behaviours, which are not conventionally “cognitive” – you’re not sitting and thinking and making only very small movements with your fingers on a keyboard.

Daniel has a particular interest in musical performance and coordination of musicians. His perspective is that musical performance with technology, technology supported sports training and gaming, particularly team multiplayer games, are cases where static models of cognition seem to break down. He believes modelling in terms of processes and synchronization has great power.

Daniel then spoke about how interaction effects are important in Human Computer Interaction. Firstly, giving the example that notifications influence a person to use their phone. Secondly, the more a person uses their phone the more they cause notifications to appear. He posed the interesting question, how does one disentangle this hypothesis to find out the degree to which notifications influence us?

Daniel then spoke about how reciprocal, interaction dominant effects also play a significant role in the organisation of our individual skilled behaviour. He gave us an overview of his own research where he found evidence of interaction dominant coordination processes in a simple skilful game task, where users are asked to control a cursor to herd sheep.

BIAS 22 – Review Day 1 – Dr James Cussens: ‘Algorithms for learning Bayesian networks’

BIAS 22 DAY 1, TALK 1 

This blog post is written by CDT Students  Roussel Desmond Nzoyem, Davide Turco and Mauro Comi 

This Tuesday 06th September 2022 marked the start of the second edition of the Bristol Interactive AI Summer School (BIAS): a unique blend of events (talks, workshops, etc.) focusing on machine learning and other forms of AI explored in the Interactive AI CDT.

Following the tradition, BIAS22 began with a few words of introduction from the CDT director, Professor Peter Flach. He welcomed and warmly thanked the roomful of attendees from academia and industry.

Prof Flach proceeded with a passionate presentation of the range of speakers while giving the audience a brief taste of what to expect during the 3-day long event: talks, workshops, along with a barbecue! He remarked on the variety of Interactive AI ingredients that would be touched: data-driven AI, knowledge-driven AI, human-AI interaction, and Responsible AI.

Prof Flach’s introduction ended with an acknowledgement of the organisers of the event.


Dr James Cussens: ‘Algorithms for learning Bayesian networks’

The first talk of the day was an introduction to Bayesian networks and methods to learn them, given by our very own James Cussens.

Bayesian networks (BN) are directed acyclic graphs, in which each node represents a random variable. The important aspects of these networks, as Dr Cussens highlighted, is that they both define probabilistic distributions and causality relationships: this makes Bayesian networks a popular tool in complex fields such as epidemiology and medical sciences.

Learning BNs is a form of unsupervised learning, based on the assumption that the available data (real or simulated) is generated by an underlying BN. There are multiple reasons for learning a BN from data, such as learning a data-generating probability distribution or learning conditional independence relationships between variables; the talk, however, focused on learning a BN in order to estimate a causal model of the data, which is a task not easy to complete with other machine learning approaches we study and use in the CDT.

A popular algorithm for learning the structure of a BN, the so-called DAG, is constraint-based learning: the basic idea behind this algorithm is to perform statistical tests on data and find a DAG which is consistent with the outcomes of the tests. However, this approach presents some issues: for example, different DAGs could encode the same set of conditional independence relationships.

Dr Cussens then proceeded to introduce DAGgity, a widely used software for creating DAGs and analysing their causal structure. It is important to note that DAGgity does not learn DAGs from data, but allows the researcher to perform interventions and graph surgery. For example, it could allow a clinician to infer a treatment-response causal effects without doing that in practice. The talk also included a small excursus on score-based learning of BNs, which is a Bayesian approach to learning these networks, I.e., it has a prior formulation.

There are many different methods for learning BNs and evaluation is key for choosing the best method. Dr Cussens introduced benchpress, a framework for performing method evaluation and comparison, and showed some results from the benchpress paper, including the evaluation of his own method, GOBNILP (Global Optimum Bayesian Network via Inductive Logic Programming).

We are thankful to James Cussens for opening the BIAS22 with his talk; it was great to get an introduction to these methods that put together many aspects of our CDT, such as causality and graphical models.


BIAS 22 – Review Day 1: Professor James Ladyman: “Attributing cognitive and affective states to AI systems”

This blog post is written by CDT Student Henry Addison

BIAS 22 – Human stupidity about artificial intelligence: My thoughts on Professor James Ladyman’s talk “Attributing cognitive and affective states to AI systems”,  Tuesday 6th September 2022

The speaker arrives, sweaty, excited after his dash across town. The AI never stops working so perhaps he cannot either. He is a doom-monger. How are the lying liars in charge lying to you? When those people-in-charge run large tech companies, how do they take advantage of our failures to both miss and over-attribute cognitive and affective states and agency to AI in order to manipulate us?

AI is a field that seeks to decouple capabilities that in humans are integrated. For humans interacting with them, this can surprise us. In adult humans intelligence is wrapped up with sentience, autonomy is wrapped up with responsibility. Not so the chess robot – very good at chess (which for a human requires intelligence) but not sentient – nor the guided targeting system for which the responsibility of target-picking is left to a human.

Humans are overkeen to believe an AI cares, understands, is autonomous. These words have many meanings to humans, allowing the decoupling of capabilities to confuse us. “Who cares for granny?” This may be a request for the nurse (human or robot) who assists your grandmother in and out of the bath. Or it may be a request of despair by a parent trying to get their children to help prepare a birthday party. If an AI is autonomous, is it a moral agent that is responsible for what it does?

The flip side of the coin are the capabilities that we do not attribute to an AI, perhaps because they are capabilities human do not have. We lose sight of important things. Like how the machines notice and store away far more than we expect and then use these data to serve us ads, recommend us films, deny us loans, guide us to romantic partners, get us hooked on the angry ramblings of narcissists, lock us up.

AI is shit. AI is ruining us. AI is enabling a descent into fascism. But presumably there is enough hope for him to bother building up such a sweat to come and talk to us. We must do more philosophizing so more people can understand AI and avoid unconscious manipulation by it. The business models of data that take advantage of us are our fault, they are human creations not technical necessities but that means we can change them.

Then again how was the speaker lying to me? How is your lying liar of an author lying to you?

AI Worldbuilding Contest – Future Life Institute

This blog post is written by CDT Students Tashi Namgyal and  Vanessa  Hanschke.

Two Interactive AI CDT students were part of a team that won third place in the AI Worldbuilding Contest run by the Future of Life Institute along with their three non-CDT teammates. In this blog post, we would like to tell you more about the competition, its goals and our team’s process of creating the submission.

The Future of Life Institute describe themselves as “an independent non-profit that works to reduce extreme risks from transformative technologies, as well as steer the development and use of these technologies to benefit life”. Besides running contests, their work consists of running grants programs for research projects, educational outreach or engaging in AI policymaking internationally and nationally in the US.

The worldbuilding competition was aimed at creating a discussion around a desirable future, in which Artificial General Intelligence (AI that can complete a wide range of tasks roughly as well as humans) played a major role in shaping the world. The deliverables included a timeline of events until 2045, two “day in the life” short stories, 13 answers to short question prompts and a media piece.

While dystopian or utopian visions of our future are quite commonplace in science fiction, the particular challenge of the competition was to provide an account of the future that was both plausible and hopeful. This formulation raised a lot of questions such as: For whom will the future be hopeful in 2045? How do we resolve or make progress towards existing crises such as climate change that threaten our future? We discussed these questions at length in our meetings before we even got to imagining concrete future worlds.

Our team was composed of different backgrounds and nationalities: we had two IAI CDT PhD students, one civil servant, one Human Computer Interaction researcher and one researcher in Creative Informatics. We were brought together by our shared values, interests, friendship, and our common homes, Bristol and Edinburgh. We tried to exploit these different backgrounds to provide a diverse picture of what the future could look like. We generated future visions for domains that could be influenced by Artificial General Intelligence (AGI), that are often low-tech, but a core part of human society such as art and religion.

To fit the project into our full-time working week, we decided that we would meet weekly during the brainstorming phase to collect ideas and create drafts for stories, events and question prompts on a Miro board. Each week we would also set each other small tasks to build a foundation of our world in 2045, for example everyone had to write a day in the life story for their own life in 2045. We then chose a weekend closer to the deadline, where we had a “Hackathon”-like intense two days to work on more polished versions of all the different parts of the submission. During this weekend we went through each other’s answers, gave each other feedback and made suggestions to make the submission more cohesive. Our team was selected as one of the 20 finalists out of 144 entries and there was a month for the public to give feedback on whether people felt inspired by or would like to live in such worlds, before the final positions were judged by FLI.

Thinking about how AI tools may be used or misused in the future is a core part of the Interactive AI CDT. The first-year taught module on Responsible AI introduces concepts such as fairness, accountability, transparency, privacy and trustworthiness in relation to AI systems. We go through case studies of where these systems have failed in each regard so we can see how ethics, law and regulation apply to our own PhD research, and in turn how our work might impact these things in the future. In the research phase of the programme, the CDT organises further workshops on topics such as Anticipation & Responsible Innovation and Social & Ethical Issues and there are international conferences in this area we can join with our research stipends, such as FAccT.

If you are curious, you can view our full submission here or listen to the podcast, which we submitted as media piece here. In our submission, we really tried to centre humanity’s place in this future. In summary, the world we created was to make you feel the future, really imagine your place in 2045. Current big tech is not addressing the crises of our times including inequality, climate change, war, and pestilence. Our world seeks to imagine a future where human values are still represented – our propensity for cooperation, creativity, and emotion. But we had to include a disclaimer for our world: our solutions are still open to risk of human actors using them for ill purposes. Our solution for regulating AGI was built on it being an expensive technology in the hand of few companies and regulated internationally, but we tried to think beyond the bounds of AGI. We imagine a positive future grounded in a balanced climate, proper political, social and economic solutions to real world problems, and where human dignity is maintained and respected.




Bristol Summer AI day – 30 June 2022

This blog post is written by CDT Students Mauro Comi and Matt Clifford.

For the Bristol summer AI day we were lucky enough to hear from an outstanding group of internationally renowned speakers. The general topic for talks was based around the evaluation of machine learning models. During the day we touched upon a variety of interesting concepts such as: multilabel calibration, visual perception, meta-learning, uncertainty-awareness and the evaluation of calibration. It was an enjoyable and inspiring day and we give a huge thanks to all of the organisers and speakers of the day.

Capability Oriented Evaluation of Models

The day’s talks opened with Prof. José Hernández-Orallo who presented his work around evaluating the capabilities of models rather than their aggregated performance. Capability and performance are two words in machine learning evaluation that are often mistakenly used interchangeably.

Capabilities are a more concrete evaluation of a model which can tell us the prediction of a model’s success on an instance level. This is crucial and reassuring for safety critical applications where knowing the limits of use for a model is essential.

Evaluation of classifier calibration

Prof. Meelis Kull gave a pragmatic demonstration of how the errors of calibration can be determined. After giving us an overview of the possible biases when estimating the calibration error from a given test set, he explained a new paradigm ‘fit-on-the-test’. This approach reduces some biases such as those due to arbitrary binning choices of the probability space.

A Turing Test for Artificial Nets devoted to vision

Jesus’ presented work focused on understanding the visual system. Deep neural networks are the current state of the art in machine vision tasks, taking some degree of inspiration from the human visual system. However, using deep neural networks to understand the visual system is not easy.

Their work proposes an easy-to-use test to determine if human-like behaviour emerges from the network. Which, in principle, is a desirable property of a network that is designed to perform similar tasks that the human brain conducts, such as image segmentation and object classification.

The experiments are a Turing style set of tests that the human visual system is known to pass. They provide a notebook style test bed on GitHub. In theory, if your network that is operating on the visual domain passes the tests then it is regarded as having a competent understanding of the natural visual world.

The evaluation procedures were later explained by Jesus’ PhD students Jorge and Pablo. They take two networks: PerceptNet and a UNet variation, and with them determine the level of similarity to the human visual system. They test known features that occur and that are processed by the human visual system when shown in natural images such as Gabor filter edge outputs and luminosity sensitive scaling. The encoded representations of the images from PerceptNet and UNet are then compared to what is found in the human visual system to illustrate any discrepancies.

This work into evaluating networks’ understanding of natural imaging is useful for justifying decisions such as architecture design and what knowledge a network has learnt.

Uncertainty awareness in machine learning models

Prof. Eyke Hullermeier’s talk expanded on the concept of uncertainty awareness in ML model. ML classifiers tend to be overconfident on their predictions, and this could lead to harmful behaviours, especially in safety-critical contexts. Ideally, we want a ML system to give us an unbiased and statistically reliable estimation of their uncertainty. In simpler words, we want our model to tell us “I am not sure about this”.

When dealing with uncertainty, it is important to distinguish the aleatoric uncertainty, due to stochasticity in the data, from the epistemic one, which is caused by lack of knowledge. However, Prof. Hullermeier explains that it’s often hard to discern the source of uncertainty in real-world scenarios. The conversation moves from a frequentist to a Bayesian perspective of uncertainty, and dives into different levels of probability estimation.

ML for Explainable AI

Explainability is a hot topic in Machine Learning nowadays. Prof. Cèsar Ferri presented his recent work on creating accurate and interpretable explanations using Machine Teaching, an area that looks for the optimal examples that a teacher should use to make a learner capture a concept.

This is an interesting concept for machine learning models where the teacher and student scenario has been flipped. Prof. Ferri showed how this concept was applied to make noisy curves representing battery health monitoring more interpretable. This involves selecting an explanation that balances simplicity and persuasiveness to the user in order to convey the information effectively.

Explainability in meta learning and multilabel calibration

Dr. Telmo SIlva Filho expanded the concept of explainability introduced by Prof. Ferri to the meta-learning setting. The first paper that he described suggested a novel method, Local Performance Regions, to extract rules from a predetermined region in the data space and link them to an expected performance.

He then followed, together with Dr. Hao Sang, with a discussion of multilabel classification and calibration, and how multilabel calibration is often necessary due to the limitation of label-wise classification. The novelty in their approach consists in calibrating a joint label probability with consistent covariance structure.

Final Words

Again, we would like to emphasise our gratitude to all of the speakers and organisers of this event and we look forward to the next interactive AI event!

CDT Research Showcase Day 2 – 31 March 2022

This blog post is written by CDT Student Matt Clifford

The second day of the research showcase focused on the future of interactive AI. This, of course, is a challenging task to predict, so the day was spent highlighting three key areas: AI in green/sustainable technologies, AI in education and AI in creativity.

Addressing each of the three areas, we were given introductory talks from industry/academia.

AI in green/sustainable technologies, Dr. Henk Muller, XMOS

Henk is CTO of Bristol based micro chip designers XMOS. XMOS’s vision is to provide low power solutions that enable AI to be deployed onto edge systems rather than being cloud based.

Edge devices benefit from lower latency and cost as well as facilitating a more private system since all computation is executed locally. However, edge devices have limited power and memory capabilities. This restricts the complexity of models that can be used. Models have to be either reduced in size or precision to conform to the compute requirements. For me, I see this as a positive for model design and implementation. Many machine learning engineers quote Occam’s razor as a philosophical pillar to design. But in practice it is far too tempting to throw power-hungry supercomputer resources at problems where perhaps they aren’t needed.

It’s refreshing to see the type of constraints that XMOS’s chips present us with opening the doors for green and sustainable AI research and innovation in a way that many other hardware manufacturers don’t encourage.

AI in Education, Dr. Niall Twomey, Kidsloop

Niall Twomey, AI in Education talk
Niall Twomey, KidsLoop, giving the AI in Education talk

AI for/in/with education helps teachers by providing the potential for personalised assistants in a classroom environment. They would give aid to students when the teacher’s focus and attention is elsewhere.

The most recent work from kidsloop addresses the needs of neurodivergent students, concentrating on making learning more appropriate to innate ability rather than neurotypical standards. There is potential for the AI in education to reduce biases towards neurotypical students in the education system, with a more dynamic method of teaching that scales well to larger classroom sizes. I think that these prospects are crucial in the battle to reduce stigma and overcome challenges associated with neurodivergent students.

You can find the details of the methods used in their paper: Equitable Ability Estimation in Neurodivergent Student Populations with Zero-Inflated Learner Models, Niall Twomey et al., 2022. https://arxiv.org/abs/2203.10170

It’s worth mentioning that kidsloop will be looking for a research intern soon. So, if you are interested in this exciting area of AI then keep your eyes peeled for the announcements.

AI in Creativity, Prof. Atau Tanaka, University of Bristol

Atau Tanaka, AI and Creativity talk, with Peter Flach leading the Q&A session
Atau Tanaka giving the AI and Creativity talk, with Peter Flach leading the Q&A session

The third and final topic of the day was Ai in a creative environment, specifically for music. Atau showcased an instrument he designed which uses electrical signals produced by the body’s muscles to capture a person’s gesture as the input. He assigns each gesture input to a corresponding sound. From here a regression model is fitted, enabling the interpolation between each gesture. This allows novel sounds to be synthesised with new gestures. The sounds themselves are experimental, dissonant, and distant from the original input sounds, yet Atau seems to have control and intent over the whole process.

The interactive ML training process Atau uses glimpses at the tangibility of ML that we rarely get to experiment with. I would love to see an active learning style component to the learning algorithm that would solidify the human and machine interaction further.

Creativity and technology are intertwined at their core and  I am always excited to see how emerging technologies can influence creativity and how creatives find ways to redefine creativity with technology.

Breakout Groups and Plenary Discussion

Discussion groups
Discussion groups during the Research Showcase

After lunch we split into three groups to share thoughts on our favourite topic area. It was great to share opinions and motivations amongst one another. The overall drive for discussion was to flesh out a rough idea that could be taken forward as a research project with motivations, goals, deliverables etc. A great exercise for us first years to undertake before we enter the research phase of the CDT!

Closing Thoughts

I look forward to having more of these workshop sessions in the future as the restrictions of the covid pandemic ease. I personally find them highly inspirational, and I believe that the upcoming fourth IAI CDT cohort will be able to benefit significantly from having more in person events like these workshops. I think that they will be especially beneficial for exploring, formulating and collaborating on summer project ideas, which is arguably one of the most pivotal aspects of the CDT.

CDT Research Showcase Day 1 – 30 March 2022

Blog post written by CDT Student Oli Deane.

This year’s IAI CDT Research Showcase represented the first real opportunity to bring the entire CDT together in the real world, permitting in-person talks and face-to-face meetings with industry partners.

Student Presentations

Pecha Kucha presentation given by Grant Stevens
Grant Stevens giving his Pecha Kucha talk

The day began with a series of quickfire talks from current CDT students. Presentations had a different feel this year as they followed a Pecha Kucha style; speakers had ~6 minutes to present their research with individual slides automatically progressing after 20 seconds. As a result, listeners received a whistle-stop tour of each project without delving into the nitty gritty details of research methodologies.

Indeed, this quickfire approach highlighted the sheer diversity of projects carried out in the CDT. The presented projects had a bit of everything; from a data set for analyzing great ape behaviors, to classification models that determine dementia progression from time-series data.

It was fascinating to see how students incorporated interactivity into project designs. Grant Stevens, for example, uses active learning and outlier detection methods to classify astronomical phenomena. Tashi Namgyal has developed MIDI-DRAW, an interactive musical platform that permits the curation of short musical samples with user-provided hand-drawn lines and pictures. Meanwhile, Vanessa Hanschke is collaborating with LV to explore how better ethical practices can be incorporated into the data science workflow; for example, her current work explores an ethical ‘Fire-drill’ – a framework of emergency responses to be deployed in response to the identification of problematic features in existing data-sets/procedures. This is, however, just the tip of the research iceberg and I encourage readers to check out all ongoing projects on the IAI CDT website.

Industry Partners

Gustavo Medina Vazquez's presentation, EDF Energy, with Q&A session being led by Peter Flach
Gustavo Medina Vazquez’s EDF Energy presentation with the Q&A session being led by CDT Director Peter Flach

Next, representatives from three of our industry partners presented overviews of their work and their general involvement with the CDT.

First up was Dylan Rees, a Senior Data Engineer at LV. With a data science team stationed in MVB at the University of Bristol, LV are heavily involved with the university’s research. As well as working with Vanessa to develop ethical practices in data science, they run a cross-CDT datathon in which students battle to produce optimal models for predicting fair insurance quotes. Rees emphasized that LV want responsible AI to be at the core of what they do, highlighting how insurance is a key example of how developments in transparent, and interactive, AI are crucial for the successful deployment of AI technologies. Rees closed his talk with a call to action: the LV team are open to, and eager for, any collaboration with UoB students – whether it be to assist with data projects or act as “guinea pigs” for advancing research on responsible AI in industry.

Gustavo Vasquez from EDF Energy then discussed their work in the field and outlined some examples of past collaborations with the CDT. They are exploring how interactive AI methods can assist in the development and maintenance of green practices – for example, one ongoing project uses computer vision to identify faults in wind turbines. EDF previously collaborated with members of the CDT 2019 cohort as they worked on an interactive search-based mini project.

Finally, Dr. Claire Taylor, a representative from QINETIQ, highlighted how interactive approaches are a major focus of much of their research. QINETIC develop AI-driven technologies in a diverse range of sectors: from defense to law enforcement,  aviation to financial services. Dr. Taylor discussed the changing trends in AI, outlining how previously fashionable methods that have lost focus in recent years are making a come-back courtesy of the AI world’s recognition that we need more interpretable, and less compute-intensive, solutions. QINETIQ also sponsor Kevin Flannagan’s (CDT 2020 cohort) PhD project in which he explores the intersection between language and vision, creating models which ground words and sentences within corresponding videos.

Academic Partners and Poster Session

Research Showcase poster session
Research Showcase poster session

To close out the day’s presentations, our academic partners discussed their relevant research. Dr. Oliver Ray first spoke of his work in Inductive Logic Programming before Dr. Paul Marshall gave a perspective from the world of human computer interaction, outlining a collaborative cross-discipline project that developed user-focused technologies for the healthcare sector.

Finally, a poster session rounded off proceedings; a studious buzz filled the conference hall as partners, students and lecturers alike discussed ongoing projects, questioning existing methods and brainstorming potential future directions.

In all, this was a fantastic day of talks, demonstrations, and general AI chat. It was an exciting opportunity to discuss real research with industry partners and I’m sure it has produced fruitful collaborations.

I would like to end this post with a special thank you to Peter Relph and Nikki Horrobin who will be leaving the CDT for bigger and better things. We thank them for their relentless and frankly spectacular efforts in organizing CDT events and responding to students’ concerns and questions. You will both be sorely missed, and we all wish you the very best of luck with your future endeavors!

January Research Skills Event Review: Day 2

This review is written by CDT Student Oliver Deane.

Day 2 of the IAI CDT’s January Research Skills event included a diverse set of talks that introduced valuable strategies for conducting original and impactful research.

Unifiers and Diversifiers

Professor Gavin Brown, a lecturer at the University of Manchester, kicked things off with a captivating talk on a dichotomy of scientific styles: Unifying and Diversifying.

Calling upon a plethora of quotations and concepts from a range of philosophical figures, Prof. Brown contends that most sciences, and indeed scientists, are dominated by one of these styles or the other. He described how a Unifying researcher focuses on general principles, seeking out commonalities between concepts to construct all-encompassing explanations for phenomena, while  a ‘Diversifier’ ventures into the nitty gritty, exploring the details of a task in search of novel solutions for specific problems. Indeed, as Prof. Brown explained, this fascinating dichotomy maintains science in a “dynamic equilibrium”; unifiers construct rounded explanations that are subsequently explored and challenged by diversifying thinkers. In turn, the resulting outcome fuels unifiers’ instinct to adapt initial explanations to account for the new evidence – and round and round we go.

Examples from the field

Prof. Brown proceeded to demonstrate these processes with example class members from the field. He identifies DeepMind founder, Demis Hassabis, as a textbook ‘Unifier’, utilizing a substantial knowledge of the broad research landscape to connect and combine ideas from different disciplines. Contrarily, Yann LeCun, master of the Convolutional Neural Network, falls comfortably into the ‘Diversifier’ category; he has a focused view of the landscape, specializing on a single concept to identify practical, previously unexplored, solutions.

Relevant Research Strategies

We were then encouraged to reflect upon our own research instincts and understand the degree to which we adopt each style. With this in mind, Prof. Brown introduced valuable strategies that permit the identification of novel and worthwhile research avenues. Unifiers can look under the hood of existing solutions, before building bridges across disciplines to identify alternative concepts that can be reconstructed and reapplied for the given problem domain.  Diversifiers on the other hand should adopt a data centric point of view, challenging existing assumptions and, in doing so, altering their mindset to approach tasks from unconventional angles.

This fascinating exploration into the world of Unifiers and Diversifiers offered much food for thought, providing students practical insights that can be applied to our broad research methodologies, as well as our day-to-day studies.

Research Skills in Interactive AI

After a short break, a few familiar faces delved deeper into specific research skills relevant to the three core components of the IAI CDT: Data-driven AI, Knowledge-Driven AI, and Interactive AI.

Data-Driven AI

Professor Peter Flach began his talk by reframing data-driven research as a “design science”; one must analyze a problem, design a solution and build an artefact accordingly. As a result, the emphasis of the research process becomes creativity; researchers should approach problems by identifying novel perspectives and cultivating original solutions – perhaps by challenging some underlying assumptions made by existing methods. Peter proceeded to highlight the importance of the evaluation process in Machine Learning (ML) research, introducing some key Dos and Don’ts to guide scientific practices: DO formulate a hypothesis, DO expect an onerous debugging process, and DO prepare for mixed initial results. DON’T use too many evaluation metrics – select an appropriate metric given a hypothesis and stick with it. AVOID evaluating to favor one method over another to remove bias from the evaluation process; “it  is not the Olympic Games of ML”.

Knowledge-Based AI

Next, Dr. Oliver Ray covered Knowledge -based AI, describing it as the bridge between weak and strong AI. He emphasized that knowledge-based AI is the backbone for building ethical models, permitting interpretability, explainability, and, perhaps most pertinent, interactivity. Oliver framed the talk in the context of the Hypothetico-deductive model, a description of scientific method in which we curate a falsifiable hypothesis before using it to explore why some outcome is not as expected.

Interactive AI

Finally, Dr. Paul Marshall took listeners on a whistle-stop tour of research methods in Interactive AI, focusing on scientific methods adopted by the field of Human-Computer Interaction (HCI). He pointed students towards formal research processes that have had success in HCI. Verplank’s Spiral, for example, takes researchers from ‘Hunch’ to ‘Hack’, guiding a path from idea, through design and prototype, all the way to a well-researched solution or artefact. Such practices are covered in more detail during a core module of the IAI training year: ‘Interactive Design’.

In all, this was a useful and engaging workshop that introduced a diverse set of research practices and perspectives that will prove invaluable tools during the PhD process.

January Research Skills Event Review: Day 1

January Research Skills – Day 1

This review is written by CDT Isabella Degen, @isabelladegen

The first day of the January Research Skills event was about academic web presence. On the agenda were:

  • a presentation on academic blogging and social media by Gavin
  • a group discussion about our experiences
  • a hackathon to extend an authoring platform that makes it easy to publish academic content on the web organised by Benjamin and Tashi

Academic web presence

In his talk, Gavin shared his experience of blogging and tweeting about his research. His web presence is driven by his passion for writing.

I particularly like Gavin’s practice of writing a thread on Twitter for each of his academic papers. I think summarising a complex paper into a few approachable tweets helps to focus on the most important points of the work and provides clarity.


For the hackathon we looked at an authoring platform that can be used to easily publish our work on the Center of Doctoral Training’s website. The aim of the websites is to be a place where people internal and external to the CDT can explore what we all are working on.

The homepage-dev codebase served as starting point. It uses Jekyll as a static site generator. A blog post is written as a markdown file. It can include other online content like PDFs, videos, Jupyter notebooks, Reveal.js presentations, etc. through a Front Matter) template. Uploading the markdown file to an online code repository triggers the publishing workflow.

It only took us a few minutes to get a github.io page started using this setup. We didn’t extend the workflow beyond being able to write our own blogs using what’s already been setup.

At the end we discussed using such a workflow to not repeat the same content for different purposes over and over again. The idea is to apply the software development principle of “DRY” to written content, graphs and presentations. Creating a workflow that keeps all communications about the same research up to date. You can read more about it on: You only Write Thrice.


The event got me thinking about having a web presence dedicated to my research. I’m inspired by sharing clear and concise pieces of my research and how in return this could bring a lot of clarity to my work.

If you are somebody who reads or writes about research on platforms like Twitter, LinkedIn or in your own blog I’d love to hear about your experiences.