Posts by Collection

Uncertainty in GNN Learning Evaluations: The Importance of a Consistent Benchmark for Community Detection Permalink

Accepted by Twelfth International Conference on Complex Networks & Their Applications

Graph Neural Networks (GNNs) have improved unsupervised community detection of clustered nodes due to their ability to encode the dual dimensionality of the connectivity and feature information spaces of graphs. Identifying the latent communities has many practical applications from social networks to genomics. Current benchmarks of real world performance are confusing due to the variety of decisions influencing the evaluation of GNNs at this task. To address this, we propose a framework to establish a common evaluation protocol. We motivate and justify it by demonstrating the differences with and without the protocol. The W Randomness Coefficient is a metric proposed for assessing the consistency of algorithm rankings to quantify the reliability of results under the presence of randomness. We find that by ensuring the same evaluation criteria is followed, there may be significant differences from the reported performance of methods at this task, but a more complete evaluation and comparison of methods is possible.

A Framework for Exploring Federated Community Detection Permalink

Association for the Advancement of Artificial Intelligence (AAAI) 2024 - 4th Workshop on Graphs and more Complex structures for Learning and Reasoning

Federated Learning is machine learning in the context of a network of clients whilst maintaining data residency and/or privacy constraints. Community detection is the unsupervised discovery of clusters of nodes within graph-structured data. The intersection of these two fields uncovers much opportunity, but also challenge. For example, it adds complexity due to missing connectivity information between privately held graphs. In this work, we explore the potential of federated community detection by conducting initial experiments across a range of existing datasets that showcase the gap in performance introduced by the distributed data. We demonstrate that isolated models would benefit from collaboration establishing a framework for investigating challenges within this domain. The intricacies of these research frontiers are discussed alongside proposed solutions to these issues.

Uncertainty in GNN Learning Evaluations: A Comparison Between Measures for Quantifying Randomness in GNN Community Detection Permalink

selected for a possible publication in the special issue of the journal Entropy dedicated to the conference from contribution in COMPLEX NETWORKS 2023

(1) The enhanced capability of Graph Neural Networks (GNNs) in unsupervised community detection of clustered nodes is attributed to their capacity to encode both the connectivity and feature information spaces of graphs. The identification of latent communities holds practical significance in various domains, from social networks to genomics. Current real-world performance benchmarks are perplexing due to the multitude of decisions influencing GNN evaluations for this task. (2) Three metrics are compared to assess the consistency of algorithm rankings in the presence of randomness. The consistency and quality of performance between the results under a hyperparameter optimisation with the default hyperparameters is evaluated. (3) The results compare hyperparameter optimisation with default hyperparameters, revealing a significant performance loss when neglecting hyperparameter investigation. A comparison of metrics indicates that ties in ranks can substantially alter the quantification of randomness. (4) Ensuring adherence to the same evaluation criteria may result in notable differences in the reported performance of methods for this task. The W Randomness coefficient, based on the Wasserstein distance, is identified as providing the most robust assessment of randomness.

1 Obsidian Notetaking Permalink

Obsidian is really cool tool for notetaking. Here’s an Open source plugin that allows you to create branching possibilities in text generation, visualised as a graph, using OpenAI LLM capabilities. Written in TypeScript.

2 WRandAi Permalink

Today I made my first PyPi package. This is for calculating the W Randomness Coefficient of a set of algorithms on a suite of Machine Learning benchmarks. This quantifies the uncertainty of different algorithms’ ranking due to the presence of testing on different random seeds. Just use pip install wrandai.

3 Holistic Spotify Control Permalink

I wanted to play music in my room using the posters of album art. My project uses a camera to detect when I ‘click’ on the album art on my wall using my hand to play the corresponding music on my speakers. This is an ongoing project; the next step is to put the software on a standalone device. - Uses 3D CNNs to detect hand landmark positions on video footage - Recognises simple gestures to control user interface. - Object detection and google vision API to detect album art on wall and search for associated album. - Spotify API to play music on speakers.

4 Poem Generator

This project uses the openai api’s to recognise voice input and convert the audio to text; generate a poem from the trigger words then play the poem in a generated voice. As this uses the research purpose only apis it won’t be released to the public. Nethertheless, this is still pretty cool that it’s really easy to build something like this or I’m one of these 10x coders you hear so much about. Either way, this demonstrates the promise of AI technology in case you hadn’t heard about this little thing called machine learning.

5 Crypto Trader Permalink

I built a cryptocurrency wallet on a Raspberry Pi. I modified a custom reinforcement learning environment to train a deep belief network to trade cryptocurrencies. This is also ongoing; algorithmic trading is hard.

6 Twitter Automation Permalink

This project contains a script for extracting arxiv paper links in twitter likes and extracting them into a reading list notion table.

Introduction to Neural Networks

Published: June 08, 2022

This talk was given as part of the Engineering Mathematics away day at the Oxford Mathematical Institute and won the award for best talk. This talk was in introduction to all things neural networks, including a brief overview of back propagation, autoencoders, recurrent neural networks, transformers and GANs. The slides are available here.

Uncertainty in GNN Learning Evaluations: The Importance of a Consistent Benchmark for Community Detection

Published: November 28, 2023

This talk was selected for Oral Presentation as part of a contribution to Complex Networks 2023. I talked about the need for consistent benchmarking practices in GNN community detection, demonstrated the drop in performance due to not carrying out a hyperparameter optimisation. I proposed a metric for quantifying randomness of the rankings of between different algorithms. The slides are available here.

Introduction to Artificial Intelligence

In the first half of the year, students are introduced to classic artificial intelligence methods followed by an understanding in basic and complex neural network practices. In the later half, students ground this understanding by proposing and completing a machine learning project, where they produce code and an accompanying paper.

Maths and Data Modelling

In this unit, students work on particular mathematical/computational methodologies through grounding in team-based problem-solving applied to real world problems from external stakeholders. Supervision is provided in the form of model feedback, providing guidance to suitable solutions, helping team management and facilitating communication. The deliverable is a substantial technical report for the first two phases and a presentation for the last phase which is proposing and completing a technical startup.