The farther you can get your preliminary grinds down to the better off you’ll be. I will grind down to 600 at least, sometimes 1200. I also try my best to make sure that all of the previous grits scratches are removed.
After that I start hand sanding with 320, up to 1200. The whole hand sanding process takes between 10-15 minutes after that.
Conditioning belts will help with the end surface finish, but they won’t help much getting out the big grinding scratches.