Configuring Dis.co to Work with GCP Clusters

This topic describes how to enable Dis.co to run jobs over your own GCP cluster.

Before you start, ensure that you have enabled role administrator and owner privileges in your GCP account. For more information on how to do so, go to troubleshooting section in the bottom of this document.

In the Dis.co Web view

  1. Log on to Dis.co.

  2. Click your user avatar and select Settings in the list that is displayed.

    The Settings page is displayed.

  3. In the Cloud tab, click Add cloud.

The Add cloud page is displayed.

Note: Dis.co cloud is connected as the default unless another Cloud instance is selected.

  1. Click the Cloud Type list.

  1. Click GCP.

The Cloud details page is displayed.

  1. Provide values for the following fields:

  2. Display name: Name of the cloud instance

  3. ****Region​: Select GCP Region from the list

  4. Project ID: required by Dis.co in order to access instances in your account. The project ID can be obtained as described in Google Api Guide.

  5. Click Next.

The following window is displayed.

In the GCP IAM Dashboard

According to gcp deployment manager, these are 2 prerequisite steps you'll need to adjust on your account, prior to running the script in the cloud shell(next step)

  1. a Google APIs Service Agent - PROJECT_NUMBER@cloudservices.gserviceaccount.com in your account needs to have both role administrator and owner privileges.

  2. in order to have permissions for creating iam role, you need to enable cloud resource managed api.

In the GCP cloud shell

  1. Click on the link which presents in box A

  2. Upload the cloudFormationScript.yaml file which you have downloaded following the instructions in box B to your shell

  3. Copy the onboarding script and run the command in your shell(make sure to change YOUR_PROJECT_ID) gcloud config set project YOUR_PROJECT_ID && gcloud deployment-manager deployments create disco-deployment --config cloudFormationScript.yaml

Please read the following limitations

  • Altering the script configuration(before or after) running it, might cause errors in the onboarding process to disco.

  • Running onboarding on multiple regions, might cause errors in the onboarding process to disco.

In the Dis.co Web view

  1. Return to the window that you left open.

  2. Click Setup.

    Wait up to 30 seconds while Dis.co validates the cloud formation configurations and links your GCP account with Dis.co.

    The following messages might be received:

    cloud_test_success.png

    cloud_test.png

  3. In the event there was a problem connecting, click Back and remedy the problem. Otherwise, click Done. When a 'Success' message is displayed, your GCP cloud is fully connected with Dis.co.

The GCP cloud will be added to your cloud list, as you can see in the following form:

The following options are available:

  • Change name: Edit the name of the cloud instance.

  • Cloud script: Copy the script from the cloud to update your GCP settings and save the changes.

  • Disable: Disable the connection.

  • If the cloud is not connected, click the Reconnect button from the table to set up the connection.

Troubleshooting

Permission error when creating a new deployment

When running the cloud script on the gcp cli, receiving the error: disco-deployment has resource warnings add-iam-policy: {"ResourceType":"gcp-types/cloudresourcemanager-v1:cloudresourcemanager.projects.setIamPolicy","ResourceErrorCode":"403","ResourceErrorMessage":{"code":403,"message":"The caller does not have permission","status":"PERMISSION_DENIED","statusMessage":"Forbidden","requestPath":"https://cloudresourcemanager.googleapis.com/v1/projects/disco-test:setIamPolicy","httpMethod":"POST"}} Actions: make sure that you have enabled role administrator and owner privileges in your GCP account.

  1. edit permissions to this account : YOUR_PROJECT_NUMBER@cloudservices.gserviceaccount.com, and add "role administrator" and "owner" permissions.

  2. You should see the google api client having role administrator and owner privileges

Force delete a deployment

Sometimes when the delete/create process of a deployment doesn't work successfully, we're not able to delete a deployment. There are lots of errors to this scenario, for example: disco-role: {"ResourceType":"gcp-types/iam-v1:projects.roles","ResourceErrorCode":"400","ResourceErrorMessage":{"code":400,"message":"You can't delete role_id (projects/-YOUR_PROJECT_ID/roles/disco.role.1574850580891) as it is already deleted.","status":"FAILED_PRECONDITION","statusMessage":"Bad Request","requestPath":"

We need to force delete the deployment by going to the gcp shell and run the command

gcloud config set project YOUR_PROJECT_ID && gcloud deployment-manager deployments delete DEPLOYMENT_NAME --delete-policy=ABANDON - YOUR_PROJECT_ID is the gcp unique project id. - DEPLOYMENT_NAME is the name of the deployment you'd like to force delete

Upgrading a deployment

If you've ever had a disco-deployment, and you wish to upgrade, since disco-deployment uses the same roleId, re-creating disco-deployment without enabling the roleId, will lead to an error. Follow the steps below to prevent this error.

  1. Delete disco-deployment if not deleted

  2. go to your iam roles : https://console.cloud.google.com/iam-admin/roles?orgonly=true&project=YOUR_PROJECT_ID

    1. Search for disco roles : roles/disco, you're supposed to see 2 roles : roles/disco.role and roles/disco.instance.role 'undelete' them, if they their status is 'Deleted':

  3. Re-create your deployment as described on section `In the GCP cloud shell`