Ben May put the Lions ahead after 32 minutes when he controlled a high ball on his chest and beat Marlon Beresford for his third goal in three games.
Luton were level a minute after the interval when Chris Coyne scored after good work by Steve Howard.
Williams then burst between two static defenders late on to prod the ball past Beresford to give Millwall the points.
Luton manager Mike Newell:
"I'm getting used to not winning games that we should away from home. It's more than frustrating and I'm fed up of saying it.
"I didn't feel we competed in the first half, and I told them at the break to go out and compete and I felt we did. But we're not putting sides away when we're away from home.
"If we can sort that out I think next year we will have a real chance of promotion. We're not giving up this year - although obviously this hasn't helped - but we're looking for other teams to slip up now."
Millwall: Marshall, Lawrence, Paul Robinson (Dyer 82), Whitbread, Elliott, Dunne (Williams 72), Cameron (Cogan 72), Craig, Livermore, May, Asaba.
Subs Not Used: Doyle, Braniff.
Booked: May, Paul Robinson, Whitbread.
Goals: May 33, Williams 90.
Luton: Beresford, Foley, Underwood, Heikkinen, Coyne, Edwards, Nicholls, Holmes (Morgan 86), Brkovic (Showunmi 86), Howard, Vine (Feeney 73).
Subs Not Used: Brill, Barnett.
Goals: Coyne 47.
Ref: U Rennie (S Yorkshire).