Byfield opened the scoring after four minutes, his effort going in via a wicked deflection off Jermaine Wright.
After Byfield's brilliant chip hit the bar, Roberts doubled the lead before the break with a crisp finish from Patrick Agyemang's low cross.
Michael Ricketts and Julian Joachim went close for Leeds but Danny Pugh's finish from Steve Guppy's cross was too little too late for the away side.
Leeds boss Kevin Blackwell:
"I was disappointed but we know how hard a place Gillingham is to go to and they didn't disappoint us.
"We were on the back foot after a wicked deflection for Byfield's goal and let in a sloppy goal after that.
"But we dominated the game after that and I think if they hadn't got the second goal it would have been a different picture. Overall I'm pleased with how they coped."
Gillingham: Banks, Nosworthy (Hessenthaler 23), Ashby, Hope,
Hills, Smith, Saunders, Spiller, Agyemang (Jarvis 76), Roberts,
Byfield (Sidibe 72).
Subs Not Used: Brown, Leon Johnson.
Goals: Byfield 4, Roberts 45.
Leeds: Sullivan, Kelly, Butler, Duberry, Kilgallon, Richardson,
Walton (Guppy 74), Wright, Pugh, Ricketts (Deane 75), Joachim.
Subs Not Used: Carson, Crainey, Radebe.
Booked: Kilgallon, Pugh.
Goals: Pugh 80.
Att: 10,739
Ref: R Styles (Hampshire).