Gluing multi-part handles together all at once is a hassle and prone to screw-ups.
Here's what I do. Stack all the spacers, glue them together with thin CA glue. Then glue them to the end of one of your big handle scales, with excess all around, no need to line anything up. After the glue...