As far as I know, the description of 'infinitely small' in relation to the Big Bang juts means very small. Otherwise it would mean that there was no matter at all since the smallest one can get in nothing! This is if, of course, the Big Bang was a singularity before expanding and not one of the various other ideas that attempt to explain the exploding start to our universe.
As to infinities themselves, I suppose it depends what one means. After all, the real numbers between 0 and 1 are, in a sense infinite because whatever number I chose - say 0.987645362, adding one extra decimal place and we have another number and so on. I imagine we could, in principle count them but they are effectively and infinity.
Likewise, the universe is expanding and accelerating while doing so. How far can it expand? In what is it expanding? It appears that the universe is expanding time and space as it expands so if it continues to do that is the dimension of the universe so expanding an infinity?
Clearly the use of Infinities in such models as Hilbert's Hotel
would appear to be mind games rather than represent reality (though wouldn't it be fun to stay at such an hotel?) but I can't see how we can rule out infinities in the sense of things like the size of the ever expanding universe.
Of course William Lane Craig like to tell us we can't have infinities of gods so the god he proposes must have always existed (infinite life perhaps) but I'm not sure a philosopher is the best person to ask about such things.