Bismuth is actually quite toxic, but it is also poorly absorbed, which means that bismuth compounds are useful for things like gastrointestinal imaging and treating intestinal upset due to food poisoning. read more
Here in the article, for Antimony trioxide is an LD50 given of 5000mg/kg (rat), while 200mg arsenic trioxide is a deadly dose for humans (I read about deaths starting from a dose of 50 to 70mg), Bismuth trioxide on the other hand is far less toxic than antimony trioxide. read more