There are many other cases of legitimate references to /etc/passed in the code (not in include_bytes though). It is the user database on *nix, so if you want to know what local users exist (and don't care about ldap etc) it makes sense to process it.
/etc/shadow, less so. But even so, I'm working on a system config management tool and it will have specific code to deal with generating these files.
Base64 and hex byte strings make perfect sense in test suits, I have written file parsers for file that contained such data. For obvious reasons my test suit included examples of those files.
The problem you'll face with any such scanner is two-fold:
There will be false positives; a program that's meant to use base64 will have base64 encoded strings in the tests, if nowhere else, for example.
A malicious attacker can see what the scanner notices, and ensure that their access to something you shouldn't have access to is not noticed by the scanner. The only way you can avoid this is by ensuring that the scanner and its output are not available to attackers, only to "good" users, but that raises the question of determining who should have access to the scanner output.
Something like cargo-crev is much more likely to succeed in avoiding this sort of malice; you don't know, as the attacker, what my criteria are for a good or bad review, and thus you can't easily evade me. Further, you can't distinguish automation from a human when you get a bad review; maybe someone gave you a bad review because their scanner said "opens /etc/passwd at compile time", but you don't know that.