I try to run ./video_chat/demo.ipynb with config_7b.json according to README.md. There are massive incompatibleKeys warnings during load_state_dict of eva_vit_g/blip2_pretrained_flant5xxl and videochat_7b.pth. The answer is also in chaos. I try to run pure text QA without uploading videos and it runs well so the LLM part may be OK.
I don't know which step I didn't do right. Here is the download url i used. Really appreciate your help!
eva_vit_g: https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/eva_vit_g.pth`
blip2_pretrained_flant5xxl: https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/blip2_pretrained_flant5xxl.pth`
videochat_7b: https://drive.google.com/file/d/1C4s65TC5Zr85I8dZmnfrrw6oDAjj1H4P/view?usp=sharing
I try to run ./video_chat/demo.ipynb with config_7b.json according to README.md. There are massive incompatibleKeys warnings during load_state_dict of eva_vit_g/blip2_pretrained_flant5xxl and videochat_7b.pth. The answer is also in chaos. I try to run pure text QA without uploading videos and it runs well so the LLM part may be OK.
I don't know which step I didn't do right. Here is the download url i used. Really appreciate your help!
eva_vit_g: https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/eva_vit_g.pth`
blip2_pretrained_flant5xxl: https://storage.googleapis.com/sfr-vision-language-research/LAVIS/models/BLIP2/blip2_pretrained_flant5xxl.pth`
videochat_7b: https://drive.google.com/file/d/1C4s65TC5Zr85I8dZmnfrrw6oDAjj1H4P/view?usp=sharing