I have a similar problem on my home built grinder. I've found that minute adjustments of the tracking knob is multiplied at least 10x when using a small wheel. Belt speed is another consideration, as the smaller bearings aren't as robust as are found on the 12" wheel. A lite touch is required on my grinder for sure.
Are you saying the tracking adjustment doesn't move the belt on the small wheel. I'll have to look at mine but I am not having any issue with tracking. Hate to hear that. I agree with Axeman above, the adjustment for the smaller wheel is finicky.