Bignum arithmetic is challenging to optimize, even just on integers, and it often needs to be re-tuned for each processor generation. I invite you to peruse GMP developers' corner to get an idea of how much work it can be. Pay attention to the last-modification dates on each page; GMP is chronically understaffed.
Having bignums built into the language is a tempting idea, but I'm leery of dumping this much additional work on the stdlib team.