My research interests span both fundamental science machine learning (ML) (or as more commonly known as Artificial Intelligence (AI)) and applications of ML/AI in misinformation identification, visual information manipulation, medicine and healthcare, and biometrics. In terms of fundamental science ML/AI, I am particularly interested in developing methods for learning robust representations using deep neural networks. Robustness, in this sense, means invariance to known and unknown confounding and nuisance factors and reducing overfitting effects of small training data. For example, in biometrics and face recognition applications, it is critical to learn robust face/identity representations that are invariant to pose and lighting conditions.
Potential applications for recent ML/AL are massive. I find myself specially passionate about a few application domains that have direct societal impacts. Truth decay and declining democracy prompted me to develop algorithms and systems to combat mis/disinformation, which represents an imminent threat to democracy. I also have significant interest and ongoing and long-term research plans for applications of ML/AI to healthcare and medicine, particularly from observational data and low-cost, every-day sensors, such as wearable devices. Last but not least, I have research interests in applying ML/AI methods to classical and emerging computer vision and biometrics problems, including face recognition and presentation attack detection.
The following are just examples of my research interests. For a complete list of my publications, please refer to my Google Scholar page.
In our NeurIPS 2018 paper “Unsupervised Adversarial Invariance“, we introduced an unsupervised architecture to learn invariant representations that only contain information about the prediction target, without overfitting to confounding or nuisance factor. The architecture learns invariant representations without knowledge and/or labels of any confounding factor. We have, since then, published a series of papers on various problem settings that can be applied in different scenarios and application domains.
Deepfakes have emerged as a serious threat to democracy, society and personal privacies. In our CVPR 2019 paper “Recurrent Convolutional Strategies for Face Manipulation Detection in Videos” we introduced the first known true deepfake detection algorithm that uses temporal information for identifying deepfakes, rather than merely treating a deepfake video as a series of independent images. We have since then developed two more generations of detectors, and will continue investing in this application domain in an effort to combat misinformation.
Similarly, image manipulation both manually with software tools (colloquially known as Photoshopping) and with AI-based tools represents a significant threat to criminal justice, personal privacy, and healthy democracies. We have developed several methods for detecting different types of image manipulations, including our CVPR 2019 paper “ManTra-Net: Manipulation tracing network for detection and localization of image forgeries with anomalous features” in which we introduced the first generalized architecture for detecting various types of manipulations (e.g. copy/move, splicing, etc.) instead of using manipulation-specific architectures.
As I mentioned earlier, I do do have a strong interest and passion for applying ML/AI to medicine and healthcare problems. In our JAMA Open 2020 paper “Assessment of Facial Morphologic Features in Patients With Congenital Adrenal Hyperplasia Using Deep Learning” we introduced the first known phenotypic biomarker for congenital adrenal hyperplaisa (CAH), which was described by the reviewers as a groundbreaking breakthrough, because it opens doors for using facial features, extracted from mobile phone pictures, to assess the severity of CAH in affected children.
Several years ago, I was fortunate to be awarded a large research grant from the Intelligence Advanced Research Activity (IARPA) with the goal of developing novel biometric sensors and algorithms to improve the robustness of face, iris and fingerprint biometric systems against presentations attacks (i.e. spoofing attacks). We developed several generations of a multi-modal biometrics sensor that includes short-wave infrared (SWIR), near infrared (NIR), laser speckle contrast imaging (LSCI), thermal and electro-optical imaging that capture rich data that reflects that physical properties of face, fingerprint and iris spoofing attacks. We have also developed deep learning algorithms for detecting and continuously learning presentation attacks, as well as explaining presentation attacks using natural language.