"Colombia" and "Columbia" aren't really words, they're names. They're both derived from Christopher Columbus's name, but applied in different languages. read more
Columbia is the basically same name, and is named after the same person, but it is an Anglicised spelling of the name, the spelling is derived from a Latin word. As words these do not really have an actual meaning in English, however the etymology of Christopher Columbus' surname is from Latin columbus: a male dove, or cock-pigeon. read more