gender and pov?

I was reading a short story recently and was shocked when the protagonist mentioned a, well, I'll just come out and say it, penis. As in the protag had one of these! I'd thought the protag was female! And, no, it wasn't some Crying Game type of thing. It was in first-person-pov and the name was too unusual to indicate gender.

This prompted me to wonder: How do we know a character's gender? I'm not sure how I ended up thinking the protag was female. The author was female. The writing was thoughtful and verbose. There were several women in the story and the protag never noted their, ah, physical attributes. I'm not sure what I'm going to tell the author. Surely, her character shouldn't succumb to stereotypes...

Actually, one of my favorite authors, Connie Willis, has a gender-indeterminate novel Uncharted Territory. It was a bit challenging and unsettling to read. Apparently, humans like me prefer to know if someone is male or female.

Do you agree? Disagree?