Research

My research interests include image processing, machine learning, and remote sensing. I solve problems in atmospheric science, multimedia, ehealth, and network security. My research interests in the theme of atmospheric science strongly align with UN Sustainable Development Goals: (a) Goal 7: affordable and clean energy, and (b) Goal 13: climate action.

Our group is currently working on the following research themes. Please navigate into the individual research themes for more details.

Solar Energy

In this project, we use a multi-modal data integration approach, in using various sensors (camera images + weather station recordings) to provide useful insights about solar energy. We use image- and weather-station- data for accurate solar energy estimation and forecasting. This is useful in the field of photovoltaic (PV) generation and integration. [read more]
(From left to right) Utrecht Photovoltaic Outdoor Test facility (UPOT) with photo courtesy of Arjen de Waal, Solar Irradiance fluctuations on the event of an overcast day.

Coastline monitoring

This research aims to improve coastline extraction from satellite images by evaluating different approaches. It considers deep learning methods and edge detection algorithms on spectral bands/indices to automatically extract coastlines and land cover maps. A new open-source dataset of labeled high-resolution satellite images will also be released. This research can have a significant impact on fields such as coastal management and disaster response planning. [read more]

(From left to right) Illustration of a satellite image, along with corresponding binary land/water map and detected coastline.

Affective Computing

This research aims to enhance the interpretation of human emotions using computer vision techniques, with a particular focus on facial expressions. By integrating insights from psychology and technology, the study seeks to develop advanced models that embed emotional attributes into image captions and improve real-time student engagement recognition in educational settings. The ultimate goal is to create more expressive, empathetic, and contextually relevant interactions between humans and artificial systems, enriching applications in healthcare, human-computer interaction, and social robotics.[read more]

We demonstrate the proposed framework for incorporating emotions into the image captioning process. [Das et al. 2023]

Product Placement

This project involves designing an advertisement detection- and integration- system for multimedia videos. It is useful for next-generation online publicity (viz. product placement and embedded marketing), wherein advertisements are seamlessly integrated into the video scenes. We use deep-learning based techniques for determining if a video frame contains an existing advert, and for accurate localization of adverts in the selected video frame. Subsequently, new adverts are seamlessly implanted into the original video, to create a new augmented video. [read more]
(From left to right) Screen grab of original video and augmented video respectively.

Knowledge Graph

In this project, we aim to make knowledge graph technologies more accessible to climate and energy researchers. A large number of today’s climate data centers present their collected data in the form of raw tables (e.g. RDB, CSV, JSON): KNMI Climate Explorer, NOAA datasets. Recently, one of the popular solutions that is greatly explored is employing an ontology or a knowledge graph, that offers the expressivity and flexibility to easily extend to various interoperable domains. [read more]
Our proposed knowledge graph model modelled on NOAA climate data.

Cloud Imaging

In this project, we develop low-cost, high resolution, ground-based sky cameras for imaging the atmosphere. These cameras capture continuous stream of images, which are essential to learn cloud dynamics and understand various atmospheric events. We devise state-of-the-art image segmentation and image classification algorithms that computes automated cloud coverage data, recognizes cloud types and estimates the cloud-base height. [read more]
(From left to right) Sky Camera, Whole Sky Image captured by the camera

Security

Software Defined Networking (SDN) has transformed the manner in which we manage the network, without regard to the underlying network technology. It is now easier and cheaper to scale networking solutions to different devices. However, such networking systems are more prone to security vulnerabilities as compared to traditional systems. In this research theme, we explore machine learning technique to develop security solutions for such networks. [read more]

E-Health

Medicine has traditionally always been an empirical field. With the onset of higher computing power and availability of large-scale dataset, it is now increasingly easier to derive insights of any disease. In this research theme, we have set an ambitious goal to use machine-learning models to characterize the empirical model of medicine. [read more]