O'Connor's header failed to reach keeper Brian Jensen and Williams lofted the ball into the net.
A small crowd inside the New Den made for an end-of-season feel and the match lacked any real inspiration.
Michael Duff squandered a chance to equalise for Burnley in the dying seconds when he sent a close range header into the arms of Colin Doyle.
Burnley manager Steve Cotterill:
"I was looking for a performance and I got nothing.
"It took us seven hours to get here on Friday and by the look of us it was
going to take seven hours to get going.
"The players have got two player-of-the-year awards nights this week and they will be embarrassed to walk in after that."
Millwall: Doyle, Phillips, Paul Robinson, Whitbread (Pooley 60), Craig, Cogan, Elliott, Hendry, Williams (Trevor Robinson 90), May, Powel (Fangueiro 81).
Subs Not Used: Weston, Brooks.
Goals: Williams 45.
Burnley: Jensen, Duff, McGreal (Spicer 38), Sinclair, Harley, Mahon, James O'Connor, Hyde, Branch, Ricketts (Elliott 63), Gray.
Subs Not Used: Karbassiyoon, Garreth O'Connor, Lafferty.
Booked: Branch, Spicer.
Ref: C Penton (Sussex).