If it sounds fishy to you, it sounds fishy to cosmologists, too. Nobody likes this story very much. Physicists don't like to be too obvious about putting the rabbit in the hat.
There are other possibilities. The obvious one is that maybe we live in a part of the universe where matter predominates, but there are other parts where anti-matter predominates. They just got separated. That's not such a crazy idea, but we run into trouble at the boundaries. Matter will annihilate with the antimatter wherever they mix, turning into two very energetic light particles called gamma rays. According to E=mc2, the gamma ray would have exactly as much energy as the proton has mass, and the gamma ray astronomers have looked for such rays, and we don't find them.
Maybe there are galaxies made of matter and other galaxies made of antimatter and they never touch each other, so they never turn into gamma rays. Actually, to make that work, the galaxies have to be pretty far apart. Even the 'empty' space around the galaxies contain enough protons and electrons that if there were antiprotons and antielectrons, they would find each other and we would see gamma rays.
Here's a long-shot idea that's been bounced around. We don't have a quantum theory of gravity, so we don't know for sure whether the matter galaxies and the antimatter galaxies would attract each other. Maybe they repel each other. Maybe that's why the expansion of the universe is getting faster?
Here's and article and a video by Fraser Cain.





