You might see if you can put some epoxy in there and sand it off. The problem is that the pin hole is too big. Even if you go 1/64" over the size of the pin you are going to get a gap. What you need to do is get some metric or wire guage (number) size bits that are just a few thousandths of an inch over the size of the pin stock. A micrometer is very handy for this and I found one with a digital read out in metric, decimal, and fraction measurements. For instance, I use a 6.4mm bit with 1/4" pin stock. I use a 3.2mm for 1/8" stock but I could also use a #30 which would be just a little larger. The reason it's good to mic you pin stock, and possibly your bits, is that they can vary slightly from the nominal size. I have some slightly undersized 1/4" stock that will fit snugly into the pin hole with a 1/4" bit.
Doug