Deep Cognition and Language ResearchNTU Singapore
We study how models connect language with speech, sound, images, video and robot action.
Research themes
- Safety Can agents resist being misled?KAIROS · ECL What would make robots safe to deploy?10 Open Challenges for VLAs
- Trustworthiness When should an agent consult its peers?BaRe-Mem Do agents know what they don't know?WebDetective
- Multimodality Can models reason with what they see?Chain-of-Glimpse Can agents collect and understand evidence?MNIST-PRO
- AI for Science Can agents find diverse research ideas?IDEAgent Do LLMs judge scientific novelty reliably?RQ-Bench
- Efficiency Can online memory remain compact?𝛿-mem What data does a model really need?Learning More by Training Less
- Embodied AI Can we create O(1) efficient memory for VLAs?MemBodied How can we create generalist physical agents?RoboQuest
Safety
Can agents resist being misled?
KAIROS · ECL
What would make robots safe to deploy?
10 Open Challenges for VLAs
Trustworthiness
When should an agent consult its peers?
BaRe-Mem
Do agents know what they don't know?
WebDetective
Multimodality
Can models reason with what they see?
Chain-of-Glimpse
Can agents collect and understand evidence?
MNIST-PRO
AI for Science
Efficiency
Can online memory remain compact?
δ-mem
What data does a model really need?
Learning More by Training Less
Embodied AI
Selected recent research
All researchEfficiencyNeurIPS 2026 · Spotlight
EFLA
EfficiencyNeurIPS 2026
δ-mem
δ-mem writes earlier context into a fixed 8 × 8 associative state. Its readout adjusts attention in a frozen language model through low-rank corrections.
BaRe-Mem keeps a running estimate of how often each advisor agent has been right, on which kinds of question. The agent uses it to decide whose advice to weigh and when to answer alone.
Embodied AIOctober 2026
RoboQuest: agents that search, inspect and test
Introducing RoboQuest, a benchmark of ten mobile manipulation tasks where a robot must act to discover information hidden at the start.
Find targets, turn objects to inspect hidden marks, and test unfamiliar mechanisms. The best of five frontier multimodal agents succeeds in 23% of episodes.
MemBodied: memory for what a robot can no longer see
A robot often has to act on something it can no longer see: where a block started, or the order in which things were placed. MemBodied gives a vision-language-action policy a memory of fixed size. Before each action the policy reads from it; afterwards it writes down what it did and what happened.
On five RMBench tasks it succeeds 50.0% of the time on average. The same π₀ policy without memory succeeds 6.4% of the time.
GE-Act 2.0
GE-Act 2.0 predicts how a scene will change and turns that prediction into robot actions. A single checkpoint was tested on 100 real-robot tasks with no task-specific fine-tuning.
Built with AgiBot. Renhang Liu is DeCLaRe's lead core contributor and Soujanya Poria an academic advisor.
Model and experimentsMultimodalityEfficiency2023–2026
TangoFlux
PromptA basketball bounces rhythmically on a court, shoes squeak against the floor, and a referee’s whistle cuts through the air.
Audio unavailable. Open the audio file or choose another sample.
Mustango
Prompt summaryLead guitar over a steady strummed acoustic accompaniment.
100 BPM · Chords: G7 → F7 → C7 → G7
Audio unavailable. Open the audio file or choose another sample.
JAM
Generated songs; titles identify the lyric references, not the original artists’ recordings.
Lyrics excerptHere I go again, running around on my…
Audio unavailable. Open the audio file or choose another sample.
NORA and NORA-1.5
NORA maps a camera image and an instruction to robot actions. It was trained on 970,000 real-robot demonstrations. NORA-1.5 adds preference training: a world model predicts what each candidate action would do, and candidates are ranked by that prediction and by how far they stray from the demonstrations.
Models and robot trials
A model that refuses harmful requests can lose that habit after fine-tuning. An agent built for one job will often take on requests outside it. CoU, WalledEval and OffTopicEval measure these failures; RESTA restores safety after fine-tuning.
Earlier work
Publications
MultimodalityAffective computing2017–2021
Multimodal representation learning
TFN makes interactions between words, facial expressions and voice explicit. MISA separates what these inputs share from what each contributes on its own; Multimodal-InfoMax trains the combined representation to retain information from all three.
MultimodalityAffective computing2017–2019
Emotion in conversations
A sentence's emotion depends on who says it and what came before. MELD supplies labelled conversations with text, speech and video; DialogueRNN tracks each speaker's changing state, while DialogueGCN models the links between their utterances.
MultimodalityAffective computing2016–2019
Sarcasm detection
Our COLING 2016 work used sentiment, emotion and personality cues to recognise sarcasm in tweets. CASCADE added the discussion and the writer's history. MUStARD brought in voice and facial expression, where the joke can be missed by reading the transcript alone.
Lab notes
All notesOctober 2026 · Peilin Feng
Mathematical Intuition Behind BaRe-Mem
The mathematics behind our Bayesian reliability memory.
September 2026 · Vernon Toh
ScrambleToolBench: the next step is already in the map
An agent discovers what its tools do. Their names change, and it starts searching again—even when its earlier observations point to the next call.
September 2026 · Vernon Toh
MNIST-PRO: seeing the strokes is not enough
When an agent gets a digit wrong, did it miss the evidence or struggle to put the glimpses together? We use MNIST-PRO to tell these failures apart.
Code, models and data
All releasesconv-emotion
CodeReference implementations of emotion-recognition-in-conversation models, including DialogueRNN.
MELD
DatasetEmotion and sentiment labels for 13,708 utterances from multi-party TV dialogue, with text, audio and video.
Tango
ModelLatent-diffusion text-to-audio model that reads the prompt with an instruction-tuned LLM (FLAN-T5).
NORA
Model3B-parameter vision-language-action model trained on 970k real-robot demonstrations. Small enough to run cheaply.
δ-mem
ModelOnline memory for a frozen LLM. Adds 0.12% parameters and steers attention through a low-rank interface.
TangoFlux
ModelText-to-audio with flow matching and CRPO preference training. Makes 30 seconds of 44.1 kHz audio in a few seconds on one A40.
People at DeCLaRe
- Soujanya Poria
- Suorong Yang
- Rishabh Bhardwaj
- Navonil Majumder
- Jingdi Lei
- Keith Kwok
About DeCLaRe
DeCLaRe, short for Deep Cognition and Language Research, was founded by Soujanya Poria at the Singapore University of Technology and Design in 2019 with Navonil Majumder, Devamanyu Hazarika and Deepanway Ghosal. The lab moved to Nanyang Technological University in 2025.
The robot forms 宣 (xuān), “to declare”.