Solution:
WordPress authentication is a lot more complex than a .php file that handles all this stuff. There’s no one file. Also, changing any core file is highly discouraged as it makes updating WordPress impossible or very difficult.
If you decide on implementing your own plugin that adds Google OAuth login then a good starting point is the codex.
I would recommend against this though. The functionality you are looking after is already implemented by other people in free, open-source plugins. You could also have a look at those for a starting point.