Create a Robot.txt file on WordPress in PHP Script

Solution:

You should be able to use the robots_txt hook in WordPress to add to the robots.txt file.

For example:

function AddToRobotsTxt($robotstext, $public) {
    $robotsrules = "User-agent: SomeBot
                    Allow: /";
    return $robotstext . $robotsrules;
}
add_filter('robots_txt', 'AddToRobotsTxt', 10, 2);

You can also completely replace the robots.txt file with your own dynamically using this hook. You would do something like this in your functions.php file:

function ReplaceRobotsTxt($robotstext, $public) {
    $robotstext = "User-agent: SomeBot
                   Allow: /";
    return $robotstext;
}
add_filter('robots_txt', 'ReplaceRobotsTxt', 10, 2);

Regarding the last two arguments in the add_filter function, the 10 is the $priority argument which defines when your function will run with respect to other functions. The 2 is the $accepted_args parameter which tells WP how many parameters the function you want to add will take.

For more info on the robots_txt hook, check out this link.