Demo Does Poorly at VQA

[edess442]

Hello,

I am interested in using the model to answer questions of the form "Which hand is touching the X". I find the model performs very poorly at this using the demo. I have made sure the checkpoints (Llama2 and checkpoint_55.pth) are being loaded correctly. For example, in the example provided I get the attached output below. Is there anything else to be configured?

[edess442]

Am I interpreting this Table correctly? Is it saying this method contains the correct hand side (at the word-level) in the prediction 46% of the time?