Document detail
ID

oai:arXiv.org:2403.14783

Topic
Computer Science - Computer Vision... Computer Science - Artificial Inte... Computer Science - Computation and... Computer Science - Machine Learnin... Computer Science - Multiagent Syst...
Author
Jiang, Bowen Zhuang, Zhijun Shivakumar, Shreyas S. Roth, Dan Taylor, Camillo J.
Category

Computer Science

Year

2024

listing date

3/27/2024

Keywords
foundation models vqa multi-agent science computer
Metrics

Abstract

This work explores the zero-shot capabilities of foundation models in Visual Question Answering (VQA) tasks.

We propose an adaptive multi-agent system, named Multi-Agent VQA, to overcome the limitations of foundation models in object detection and counting by using specialized agents as tools.

Unlike existing approaches, our study focuses on the system's performance without fine-tuning it on specific VQA datasets, making it more practical and robust in the open world.

We present preliminary experimental results under zero-shot scenarios and highlight some failure cases, offering new directions for future research.

;Comment: A full version of the paper will be released soon.

The codes are available at https://github.com/bowen-upenn/Multi-Agent-VQA

Jiang, Bowen,Zhuang, Zhijun,Shivakumar, Shreyas S.,Roth, Dan,Taylor, Camillo J., 2024, Multi-Agent VQA: Exploring Multi-Agent Foundation Models in Zero-Shot Visual Question Answering

Document

Open

Share

Source

Articles recommended by ES/IODE AI

Systematic druggable genome-wide Mendelian randomization identifies therapeutic targets for lung cancer
agphd1 subtypes replication hykk squamous cell gene carcinoma causal targets mendelian randomization cancer analysis