Reading through thegetting started documentationfor EC2 I find this little gem:

C:\><span class="command"><strong>ec2-describe-images -o self -o amazon | findstr /i windows</strong></span>

grepfor windows. Yay!