How extraordinarily juvenile. Size may make it a little better, but what's ALWAYS more important is emotion involved and that the guy knows/cares about what you like.
As far as CF guys being small, I could see that as kind of a problem. I myself tend to go for guys 6' or over. But at the same time, I wouldn't NOT go out with a guy just because of his size. If I liked the guy, and he was shorter than me, I wouldn't much care. Certainly not enough to avoid him for it.
But then CF girls have similar little problems with appearance. Our stomachs tend to stick out, even if we've got damn near 0% body fat, and let's face it... no guy wants a fat-looking chick. I'm sure there are other little quirks similar to this for girls, but I'm just not in a thinking mood.
Either way, you get the idea. We all have our problems. I see what you mean about guys having trouble with size, but girls might have trouble with size for quite a different reason.