Abstract
Significant work has been conducted in the domain of food computing, yet
these studies typically focus on single tasks such as t2t (instruction
generation from food titles and ingredients), i2t (recipe generation from food
images), or t2i (food image generation from recipes). None of these approaches
integrate all modalities simultaneously. To address this gap, we introduce a
novel food computing foundation model that achieves true multimodality,
encompassing tasks such as t2t, t2i, i2t, it2t, and t2ti. By leveraging large
language models (LLMs) and pre-trained image encoder and decoder models, our
model can perform a diverse array of food computing-related tasks, including
food understanding, food recognition, recipe generation, and food image
generation. Compared to previous models, our foundation model demonstrates a
significantly broader range of capabilities and exhibits superior performance,
particularly in food image generation and recipe generation tasks. We
open-sourced ChefFusion at GitHub.
Citation
ID:
282973
Ref Key:
chawla2024cheffusion