Rotherham were helped on their way to victory by some poor goalkeeping from Leyton Orient's Glyn Garner.
On his 200th League outing he failed to hold on to Michael Keane's shot and Ian Sharps headed in the opener.
Adam Chambers headed an equaliser, but Garner gifted Will Hoskins a goal before Orient again came from behind when Justin Miller headed in a corner.
The Millers took the lead for a third time, and this time decisively, with a long-range winner from Scott Wiseman.
Leyton Orient: Garner, Lockwood, Saah, Fortune (Barnard 60), Miller, McMahon, Simpson, Chambers, Corden, Connor (Steele 77), Alexander (Ibehre 78).
Subs Not Used: Morris, Keith.
Booked: Fortune, McMahon.
Goals: Chambers 20, Miller 68.
Rotherham: Cutler, Robertson, Sharps, Mills, Worrell, Keane, Williamson, Woods (Wiseman 63), Cochrane, Hibbert (Taylor 83), Hoskins (Bopp 90).
Subs Not Used: Montgomery, Partridge.
Goals: Sharps 20, Hoskins 45, Wiseman 82.
Ref: M Thorpe (Suffolk).