What if buildings could understand how comfortable people actually feel—not just what the thermostat says?

Last week, our team travelled to Seville to advance the development of SmartTherm, LIBRA AI Technologies’ project under Challenge 6: Thermal Comfort Visual Recognition System of the COSMIC Support Programme.

Working with the Termotecnia research group at the Universidad de Sevilla, we began preparing the pilot at Cartuja Urban Lab. SmartTherm will combine Edge AI, RGB and infrared cameras, and environmental sensor data to estimate occupants’ thermal comfort in real time.

The goal is to help public buildings respond more intelligently to how people experience a space, while improving energy efficiency and protecting occupants’ privacy by design.

The Challenge

Most public buildings do not actually know whether the people inside them are comfortable.

A typical building management system reads air temperature from a limited number of wall-mounted sensors and maintains a predefined setpoint. It cannot tell how many people are present, what they are wearing, whether they have just walked in from 40 °C heat, or whether afternoon sunlight is affecting one area more than another.

The consequences run both ways. People may feel uncomfortable and leave sooner, reducing dwell time and affecting their experience of the space. At the same time, energy may be spent cooling or heating areas without a clear understanding of where it is needed.

Existing alternatives are difficult to scale. Comfort surveys are periodic, slow and labour-intensive. Wearable physiological sensors can provide useful measurements, but they are intrusive and impractical for people moving through public spaces. Systems that estimate comfort from appearance alone may also lack a physical model of how the human body exchanges heat with its surroundings.

A Different Approach

SmartTherm is designed to estimate thermal comfort without requiring occupants to wear a device, install an application or actively interact with the system.

Instead, it combines anonymous visual observations with environmental sensor measurements.

Each deployment uses two camera modalities: a high-resolution RGB stream and an infrared thermal stream. Camera data is processed locally on NVIDIA Jetson Edge AI hardware at the deployment site. Two complementary inference pipelines analyse the scene in parallel before their outputs are combined into a unified comfort estimate.

Stream 1: Classical Computer Vision

The first pipeline detects and segments occupants within the monitored space.

Face blurring is applied locally immediately after detection and before downstream analysis, storage or transmission. Multi-camera tracking then maps detections from different viewpoints onto a shared floor plane, allowing the system to maintain a temporary anonymous representation of an occupant as they move through the space or pass behind an obstacle.

From the anonymised observations, the pipeline estimates clothing insulation, expressed as a CLO value, and activity level. Infrared imagery registered to the same segmentation masks provides a per-person skin-temperature proxy.

Together, these represent important personal and physiological inputs for thermal-comfort estimation.

Stream 2: Vision-Language Understanding

The second pipeline adds semantic understanding to the classical computer-vision analysis.

A compact, quantised vision-language model analyses the RGB stream and interprets contextual information that traditional vision models may struggle to capture. This includes the type of space, occupancy levels, clothing and behavioural cues such as someone fanning themselves, huddling or removing a jacket.

These behaviours provide additional evidence about how occupants may be experiencing the environment and help cross-check the measurements produced by the first pipeline.

Together, the two streams provide a richer understanding of the conditions affecting occupant comfort.

Grounding AI in Thermal Science

The outputs of both AI pipelines are combined with live environmental measurements from the site’s IoT sensor network. These measurements include air temperature, humidity, wind speed and mean radiant temperature.

The fused information is then processed by COMFA, the COMfort FormulA model.

COMFA estimates a human energy budget by accounting for factors such as metabolic heat production, clothing insulation, radiative and convective heat exchange, and evaporative heat loss. This allows SmartTherm to produce a physically grounded comfort estimate rather than relying on visual interpretation alone.

The system is designed to aggregate these anonymous estimates into a continuously updated comfort map of the monitored space, giving building operators a clearer view of how conditions vary across different areas.

We selected COMFA rather than relying solely on conventional indoor indices such as PMV and PPD. PMV was developed primarily for relatively uniform, steady-state indoor environments. Although it incorporates variables such as air speed and mean radiant temperature, it is less suited to dynamic semi-outdoor settings with direct solar exposure and rapidly changing microclimatic conditions.

The outdoor human-energy-budget approach used by COMFA therefore provides a stronger foundation for the amphitheatres, plazas and semi-outdoor spaces included in the pilot.

Closing the Loop

Understanding comfort is only part of the challenge. The next step is turning that understanding into useful building intelligence.

SmartTherm is designed to communicate with the site’s building or environmental management system, translating comfort estimates into HVAC setpoint recommendations or control commands, depending on the pilot configuration.

Safeguards against rapid cycling will help prevent unstable adjustments, while sensor measurements will be used to verify whether each change produces the intended effect.

Occupants will also be able to submit voluntary and anonymous comfort votes through a lightweight interface. When aggregated feedback shows a trend towards discomfort, it can be used to adjust the control response.

These votes will also provide valuable ground-truth data for evaluating and improving the model throughout the pilot.

Privacy by Design

The use of cameras in public spaces carries a clear responsibility. For SmartTherm, privacy is not an additional compliance step—it is an architectural requirement.

The system has been designed around several core safeguards:

  • Face blurring is performed locally on the Edge AI device immediately after detection and before downstream analysis, storage or transmission.
  • No identifiable face imagery, raw camera footage or individual thermal maps leave the deployment site.
  • Only anonymised aggregate features and statistical summaries are transmitted.
  • Occupant feedback is voluntary, anonymous and stored only in aggregated form.
  • No comfort vote is linked to an identifiable individual.
  • Clear signage informs occupants that cameras are operating as part of the research pilot.
  • Project data will be deleted at the completion of the project in accordance with the approved data-management procedures.

SmartTherm may maintain temporary anonymous track identifiers while occupants move through the monitored area. However, it does not identify individuals or connect their observations across separate visits.

No persistent personal profile is created.

Validating SmartTherm in Seville

SmartTherm is being validated at Cartuja Urban Lab, the Universidad de Sevilla’s smart-city living laboratory in Seville.

The site provides a demanding environment for thermal-comfort estimation, combining Andalusian summer temperatures, semi-outdoor public spaces and continuously changing occupancy patterns.

These conditions make it an appropriate real-world test bed for evaluating whether the system can operate reliably outside controlled laboratory environments.

Over the course of the ten-month project, LIBRA AI Technologies will integrate the complete SmartTherm system, from computer vision and thermal modelling to feedback collection and building-control integration.

The COMFA implementation will be validated with thermal-comfort experts from the Universidad de Sevilla.

The platform has also been designed for portability. A modular processing pipeline and sensor abstraction layer will allow the system to be adapted to airports, stations, transport hubs and other large public buildings without requiring the entire architecture to be redesigned for each deployment.

Looking Ahead

SmartTherm represents more than a new way of managing temperature.

It demonstrates how computer vision, Edge AI, environmental sensing and thermal science can work together to help buildings respond to the people using them, rather than relying only on fixed environmental measurements.

By combining privacy-preserving AI with physically grounded thermal-comfort modelling, LIBRA AI Technologies aims to support a new generation of public buildings that are more human-centric, responsive and energy efficient.