'make clean' did this.
We're using git clean to perform this action, and git clean takes a -x to clean out the things that are in .gitignore (such as the source files). Thus fedpkg clean has a -x option that will clean the source files.