Create a custom login functionality in WordPress for API authentication

Solution:

I was able to achieve it by using the plugin JWT Authentication for WP REST API. The plugin supports authentication for WordPress API using username and password. So I used the authenticate filter to use my own custom login function.

function custom_auth_signon($user, $username, $password) {
    global $wpdb;
    $user_details = get_user_by('login', $username);

    $query = $wpdb->prepare("SELECT user_id FROM {$wpdb->prefix}usermeta WHERE user_id = " . $user_details->data->ID . " AND meta_key = 'custom_password' AND meta_value = %s", $password);
    $result = $wpdb->get_results($query);
    if ($result[0]->user_id) {
        return $user_details;
    } else {
        return NULL;
    }

}

add_filter('authenticate', 'custom_auth_signon', 30, 3);