You may already have what you need, but when I built my grinder I first had it setup like you're describing. I made up my own little switch panel with the potentiometer, stop, fwd, rev, etc. I used it for a while like that, but a couple times I'd go to change a belt and forget to zero my pot and shoot a belt off due to different tracking. I started looking at the instructions for the vsd and tried a different control scheme that I kind of like.
My drive can be setup to have ACCEL and DECEL buttons, instead of a pot. You press the FWD button, and then hold down ACCEL to ramp up to the speed you want. When you hit stop, the speed command is automatically zero'd each time, so you always start off slow. You can program the RAMP rate of the accel too, so you can tweak it just like you want it.
I've had it this way for a while now, and I think it'll stay this way. You only have to buy a few momentary push buttons, pretty simple to hook up. Just a thought.