A knife has been recovered from a drain near the scene of the attack in Cross Flatts Park in Beeston.
Christopher Scholey, 53, was found lying, fatally injured, close to railings near the Dewsbury Road entrance on Monday.
He had suffered several injuries including a stab wound to the heart.
The knife which has been found by the police search team is similar to the six-inch kitchen knife thought to be have been used in the attack.
But West Yorkshire Police say until tests have been completed it is too early to be sure it is the murder weapon.
Mr Scholey was attacked at around 11.30am as he took his usual morning walk.
A man in the park heard an argument and then someone crying out for help
Detective Superintendent Jim Bancroft, who is leading the murder hunt, said: "I believe someone will have seen an argument before the attack or the attack itself and may not have realised its full significance
"Just as Mr Scholey was a regular user of the park, I believe his attackers must also have been regular visitors."
The Barnsley firm of solicitors where Mr Scholey worked for 21 years, until his retirement in March, is offering a reward for information that leads to the conviction of his killers.
Mills, Kemp and Brown have put up what police are describing as a "substantial sum" .