Allen Institute for AI

Grounded Situation Recognition

Grounded Situation Recognition is the task of identifying the situation observed in the image and also visually ground the identified roles within the corresponding image.

Try it for yourself
1. Upload an Image (or choose one from the examples)
Examples...
Image:
Click to upload your own image
2. Run a model

Joint Situation Localizer

Grounded Situation RecognitionSarah PrattMark YatskarLuca WeihsAli FarhadiAniruddha KembhaviECCV2020

JSL is a method to simultaneously classify a situation and locate objects in that situation. This allows for a role’s noun and grounding to be conditioned on the nouns and groundings of previous roles and the verb. It also allows features to be shared and potential patterns between nouns and positions to be exploited.