Hany & I have been working for over a year devising a study to examine how well we can estimate height and weight from a single image. We compare state-of-the-art AI, computer vision, and 3D modeling techniques with estimations from experts and non-experts. The results are surprising.
Code: Detecting Deep-Fakes Through Corneal Reflections
Classical computer science techniques for the re-projection of an image onto a non-planar surface have been extended to modeling the geometry of the human eye, as seen in Ko and Nayar (2004). Yet, there are few examples of the application of these methods to the analysis of corneal reflections in deep fake detection. The work of Hu et al. (2021) explored inconsistencies in inter-eye reflections, but focussed solely on specular highlights as opposed to the reflected scene itself. This work aims to build upon these two approaches in order to investigate whether there are discernible differences between the corneal reflections of real vs. deep-fake human portraits.
Findings
The findings from this approach provide several key contributions:
A framework for automatically extracting the corneal surface of a human in a single image and generating iris-based features.
For two publicly-available datasets, we ultimately find that there are no significant discernible differences between real and fake corneal reflections (based on the performance of three classification models).
We hypothesize that this lack of differentiation between real and fake portraits is primarily due to insufficient image quality once cropped (which typically reduces a 1024x1024px photo down to approximately 50x50px). This clearly impacts the pixel representation of the corneal surface and introduces noise that distorts the extracted features. The noisy nature of the dataset is evident from the outputs of Principal Component Analysis, which indicates that 10 components are required to capture just 75% of the variation in the data.
In spite of this, the feature extraction process highlights several features of interest that, qualitatively, can be seen to differentiate between real and fake portraits. This includes the shape of the pupil, which, for the majority of humans, should form a near-perfect circle shape when facing toward the camera. Yet, in several GAN-synthesised images from our dataset, we observe inconsistencies such as straight edges, corners, and even no clear pupil at all.
Additionally, edges observed in the reflections of real corneal surfaces (particularly in outdoor or natural light settings) exhibit discernable properties such as objects or information about the scene. By contrast, several deep-fake corneal reflections were observed to contain noise and abstract lighting, not resembling a coherent scene or object. However, we hypotheise that while evident to a human, the edge count feature was skewed by noise in the low resolution cropped images, thus not providing a strong predictor.
Analysis: https://github.com/sbarrington/corneal-reflections/tree/main/01%20Paper
Public Dataset: https://github.com/sbarrington/cornea-iris-dataset (DOI: 10.5281/zenodo.7396604)
Analysis snapshots:
Paper: The ‘Fungibility’ of Non-Fungible Tokens: A Quantitative Analysis of ERC-721 Metadata
Non-Fungible Tokens (NFTs), digital certificates of ownership for virtual art, have until recently been traded on a highly lucrative and speculative market. Yet, an emergence of misconceptions, along with a sustained market downtime, are calling the value of NFTs into question.
Stanford University CodeX Conference: DAOs & Systems for Resilient Societies
In April, I attended the Stanford University CodeX Blockchain Group's DAOs and Systems for Resilient Societies conference as a collaborator with my colleagues from the Open Earth Foundation (OEF). OEF is a non-profit research organisation whom I have worked with on several research projects, examining the feasibility and security implications of decentralised technologies for climate applications.
Middleware Models for Algorithmic Moderation at Scale
Over the past 2 decades, Digital Platforms (DPs), including Facebook, Google, Amazon and Apple, have risen from insignificant start- ups to a dominating set of firms with a combined 4 trillion dollar market capitalisation1. These now ubiquitous DPs have revolutionised many aspects of everyday life, from working and studying to communicating and dating. In recognition of this unprecedented rise of commercial power, legal and academic scholarship has begun to revisit the concepts of monopolistic market behaviours and the subsequent potential for both economic and political influence.
Metadata and Non-Fungible Token Architecture
Cryptocurrency and Blockchain technologies are fast becoming areas of public interest across a breadth of diverse domains, with novel applications ranging from financial services to state politics. In particular, promising future use cases of these decentralised technologies aim to re-define the concept of value, and potentially enable a new wave of accessible investment opportunities for the general public. One such example of this is through Non-Fungible Tokens (NFTs), which act as digital certificates of ownership for a given piece of digital content. These stamps are stored as tokens on a public Blockchain (such as Ethereum), which uses complex hashing algorithms to ensure that each token is unique, tamper-proof and publicly-visible. This adds value to a range of digital content & artworks through enabling a single source of truth relating to its ownership and historic activity.
The Perception of Threat in Female-Targeted Online Abuse
As a component of the INFO-272 Research course at the School of Information, I lead an independent qualitative research study in order to explore the perception of threat in a range of online abuse situations through the contexts, themes and experience of five self-identifying women. Online abuse is becoming an increasingly prevalent issue in modern day society, with 41% of Americans having experienced online harassment in some capacity in 2021. People who identify as women, in particular, can be subjected to a wide range of abusive behaviour online, with gender-specific experiences cited broadly in recent literature across fields such as blogging, politics and journalism.