FAQ

Here you'll find best practices and workarounds for various adjacent technologies and use cases.

Logging

Use python's logging tool to get real time information about application state and useful debugging information within disco like this:

import logging
if __name__=="__main__":
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

Write to stdout and stderr like this to get results from your jobs:

print("your stdout message")
print("your stderr message", file=sys.stderr)

In order to print logging output to a stream handler, you can add a pattern like this example to debug your application using stdout provided in the job results in Disco:

import logging
import sys
root = logging.getLogger()
root.setLevel(logging.DEBUG)
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
root.addHandler(handler)

Adding Your Github Project

Please ensure that your ssh-keygen generates a key of type RSA. In macos mojave, the way to ensure your key is of type RSA, run this command (the -m PEM flag forces the key to RSA instead of openSSH):

ssh-keygen -m PEM -t rsa -b 4096 -C "your_email@example.com"

Add your public key to your github account, and your private key in the Git URL setup in Dis.co like this (ensure your paste in the Deploy Key field includes no newlines):

Creating Files in Job Results

To output files other than stdout and stderr as part of job results, you can write them to the run-result folder in the current working directory. You can use a pattern like below:

import os
from os import listdir
# ### Python Packages Installed
try:
from pip._internal.operations import freeze
except ImportError: # pip < 10.0
from pip.operations import freeze
print("\n\nPython Packages Installed: ")
x = freeze.freeze()
for p in x:
print(f"\t{p}")
# ### Input files
print("\nInput Files: ")
for f in listdir('.'):
print(f"\t{f}")
# ### Output files
out_dir = './run-result'
try:
if not os.path.exists(out_dir):
os.makedirs(out_dir)
except OSError as e:
print(f"Error: Unable to create output dir {out_dir}: {e.strerror}")
exit(1)
with open(os.path.join(out_dir, 'SampleOutputFile.txt'), "w+") as f:
print("Here's some sample output", file=f)
f.write("Here's some more sample output\r\n")
# ### END
print("\nProgram Complete")