Funny I was thinking of posting the same question! Most of the year I too choose smart trousers(mine are grey and brown, I look dreadful in black tho I think it is the best colour if it suits you) I also wear a smart blouse, 3/4 sleeve with lycra (no ironing)
Thats fine most of the year but I was caught out in the recent warm weather. All the skirts I tried looked wrong- too dowdy, too tarty, too dumpy! Blouses outside the skirt looked worse but tucked in looks frumpy or untidy. Also the shoes have to be right.
I usually end up in M & S, poss BHS or M & Co. If I cant find what I like there I go without- not adventurous enough to try elsewhere.