Seth Johnson scored his third goal in two games for Derby, heading home Lee Holmes' cross from the left.
But Marvin Elliot equalised with a header before Marvin Williams controlled a pass from David Livermore and slipped the ball past Lee Camp.
Holmes had earlier hit the post and forced Colin Doyle into action, while Camp saved from Alan Dunne.
Derby boss Phil Brown:
"We should have been on our way to three points but it is the same old problem again, defending free-kicks and making mistakes.
"We have conceded seven goals over Christmas and three have been from set-pieces and two down to mistakes.
"We can't continue doing it as we can't expect to win games scoring three goals. 1-0 should be enough."
Millwall: Doyle, Dunne, Lawrence, Robinson, Craig, Elliott, Morris (Simpson 68), Livermore, Cogan, May, Braniff (Williams 59).
Subs Not Used: Phillips, Whitbread, Dyer.
Goals: Elliott 73, Williams 78.
Derby: Camp, Edworthy, Nyatanga, Michael Johnson, Jackson, Smith (Barnes 80), Bolder (Peschisolido 80), Idiakez, Seth Johnson, Holmes, Graham.
Subs Not Used: Poole, Doyle, Thirlwell.
Goals: Seth Johnson 60.
Ref: M Atkinson (W Yorkshire).