Biennials

I have got wallflowers and sweet williams doing very well at present but they are in my vegetable patch because I do not have the room for them in what I intend to be their final location.  Does a biennial need to remain in place after autumn planting or can I move them next spring?  Will that retard or reduce flowering.  thanks.