Classical computer science techniques for the re-projection of an image onto a non-planar surface have been extended to modeling the geometry of the human eye, as seen in Ko and Nayar (2004). Yet, there are few examples of the application of these methods to the analysis of corneal reflections in deep fake detection. The work of Hu et al. (2021) explored inconsistencies in inter-eye reflections, but focussed solely on specular highlights as opposed to the reflected scene itself. This work aims to build upon these two approaches in order to investigate whether there are discernible differences between the corneal reflections of real vs. deep-fake human portraits.
Findings
The findings from this approach provide several key contributions:
A framework for automatically extracting the corneal surface of a human in a single image and generating iris-based features.
For two publicly-available datasets, we ultimately find that there are no significant discernible differences between real and fake corneal reflections (based on the performance of three classification models).
We hypothesize that this lack of differentiation between real and fake portraits is primarily due to insufficient image quality once cropped (which typically reduces a 1024x1024px photo down to approximately 50x50px). This clearly impacts the pixel representation of the corneal surface and introduces noise that distorts the extracted features. The noisy nature of the dataset is evident from the outputs of Principal Component Analysis, which indicates that 10 components are required to capture just 75% of the variation in the data.
In spite of this, the feature extraction process highlights several features of interest that, qualitatively, can be seen to differentiate between real and fake portraits. This includes the shape of the pupil, which, for the majority of humans, should form a near-perfect circle shape when facing toward the camera. Yet, in several GAN-synthesised images from our dataset, we observe inconsistencies such as straight edges, corners, and even no clear pupil at all.
Additionally, edges observed in the reflections of real corneal surfaces (particularly in outdoor or natural light settings) exhibit discernable properties such as objects or information about the scene. By contrast, several deep-fake corneal reflections were observed to contain noise and abstract lighting, not resembling a coherent scene or object. However, we hypotheise that while evident to a human, the edge count feature was skewed by noise in the low resolution cropped images, thus not providing a strong predictor.
Analysis: https://github.com/sbarrington/corneal-reflections/tree/main/01%20Paper
Public Dataset: https://github.com/sbarrington/cornea-iris-dataset (DOI: 10.5281/zenodo.7396604)
Analysis snapshots:
