I have been having incredible dreams about a woman I work with. I've never done anything with a female but I do think she is hot and actually have considered asking her out for drinks after work. Does it mean I am a lesbian?

No, it may mean you have a nifty fantasy imagination. Rest assured what you have happening in dreams is just that dreaming and it does not mean you want to act this out.

However you do mention you would like to ask her out. A tidbit of wisdom, do go forward carefully, as this is a work environment.

One of the best things about being female is we can go back and forth between women or men as partners and it is socially chalked up to experimenting. Perhaps this is so because of the male fantasy of watching two women doing one another so there is more permission to do so.

Versus if a man wants to experiment he isn't experimenting he is considered gay.  Again not necessarily the case but that is what I have seen as social observations.