The problem is not physical backing. A hugepage can be up to 2Gb in size, so you’ll have to wait until it fills up completely for a 4k page to get faulted.
So this means you’ll have to stick to 4k pages to map the stack. For a million stacks it’s definitely not free - page table manipulation requires locking (in the kernel) and creates pressure on TLBs.