Starter doubling 1st 24 hrs, then little growth

Toast

I'm making a sourdough starter for the first time. I mix the whole wheat flour and water and about 20 hrs in it goes nuts and doubles and when I give it the regular feeding at 24 hrs that doubles too. Then when I go feed it 12 hrs after that it has deflated (which Ive read happens) and it doesn't grow at all the next couple days. Just a few bubbles. It's now day 5 and it has started to grow a little bit. Day 4 was a little, Day 5 is a little more.

Everything seems normal, if not slow, but what was Day 1 all about!? 

What you are experiencing is very normal. There is a quick burst of activity but it's not from the good bacteria and yeasts found in a mature starter. It's the other guys. The gas producing bad bacteria aka leuconostoc activity. All does go quiet after that but that isn't to say stuff isn't happening. The starter is becoming acidic killing off the bad bacteria and making way for the friendly guys - good bacteria and yeasts. These are the ones you're feeding. Once your starter is over that quiet stage and is back to being strong and predictable it's good to go. Sounds like you're over the worst. Keep warm and feed when you see activity. 

With no slat or acid in the mix they are a vigorous bunch.  But they nae acid ans ad time goes on the PpH goes low enough that the bad LAB can't tolerate it and begin to die off which leaves the good LAB a chance to multiply and take over  the culture.  They are more acid resistant but less vigorous.  Same thing is going in with he yeast,  The bad ones that aren't acid resistant are being replaced by those that are.  So by day 4-5 the culture may appear dead since the less vigorous LAB and Yeast have to build up their numbers  But by day 7 all will be well as the culture continues to become mire acidic and the LAB and yeast that live the acid and living together will be established.  They won't be full force powerful ir flavorful but in a month from the start they will be.

You culture is showing the classic signs that is should so well done  SD bread is in your future.