The good news is that
you do not need to take it all out. The bad news is that you do need to take
some out. But that isn’t really all that hard to do.
Depending on the type of tile and the size of the grout lines, here are the
best alternatives:
For standard 4-1/4” tile with a narrow grout line, you can buy a special tool
for scraping out the grout. I have one that I bought long ago, and it still
works fine for relatively soft non-sanded grout on this type of tile. Just be
careful, as some tile is actually quite soft, and can be chipped or scratched
if you get too aggressive.
For grout lines of more than 1/8” in width, the grout should be sanded and will
be harder to remove. You may need to buy, rent or borrow a small grinder or
Dremel type tool. With this you can carefully grind away some of the grout in
the areas needing work.
The main thing you are trying to accomplish is scraping away any soap residue
that is always present in a shower, even one that is kept clean. Once you have
removed enough grout to be sure that you can add more and easily work it in to
the gap, just mix a small amount of the same grout used originally and press it
in. Allow it to set up and wipe away the excess. Allow that to harden and buff
the tile clean. Give that a day to cure and then apply a tile sealer if you
like.