Outline:

1. Multimodal representation learning
   1. expert models
      1. convirt
      2. gloria
      3. biovil
      4. biovil-t
      5. villa
   2. finetuned models
      1. pubmedclip
      2. biomedclip
      3. plip
      4. conch

Multimodal generation

• LLM:
  • llavamed
  • medpalm-m
  • med-flamingo
• expert models (report generation)

Previous reviews/perspectives

https://www.nature.com/articles/s41586-023-05881-4

https://www.nature.com/articles/s41551-022-00914-1

https://www.nature.com/articles/s41591-023-02448-8

https://www.nature.com/articles/s41591-022-01981-2

https://www.nature.com/articles/s41591-021-01614-0

https://www.nature.com/articles/s41746-023-00811-0