Agree with Michele completely, definitely mention it to your boss. To me, this sounds like a case of "ignorance/error" when it came to what codes to bill, and not fraud. I am not sure how old these claims are, but it could stand to reason that whoever she was using for billing for whatever reason didn't switch from testing codes to vaccine codes, when the vaccine became available.
I have no idea if testing codes pay more than vaccine codes, but if they do, or vice versa, your boss could either owe money or be owed money from insurances, if these are paid claims.