You can always ream after drilling.
It's worth noting that twist drills are not super precise. If you have a drill that measures, for example, 0.500", it may drill a little under or over sized. This can be due to a lot of things, like how it's sharpened (uneven flutes can cause an oversize hole), chuck runout, etc. Heck, even things like what kind of cutting oil you use effect bore diameter. If I'm not mistaken, thick cutting oil can cause a slightly oversized hole (compared to something thin like IPA or water-based cutting fluid).
If you're concerned with consistency, I'd pick up some readers. You can get them in pretty much any size imaginable.
Sent from my Champion Forge using Tapatalk