Fortunately you have a road bike for asphalt. I'm often amazed at how my mountain bike can feel so nimble and at home on single track or fire roads and be a complete dog on the road. Thankfully we know enough to use the right tools for the job.

Oh, that is the interesting thing.  I swear my CX bike is sorta doggy on the road, but the INSTANT I hit gravel it's like my mph jump by 1-2.  I don't know how that works, but it's true!  And that bike is aluminum, but is MUCH less buzzy than my Synapse was.  It's a stiffer, heavier frame, too, so I suspect that the geometry and weight of that aluminum are factors.