Quickstart

Overview

Welcome to the Engine Documentation. Engine allows you to easily track and execute machine learning code. In this guide, we will walk you through creating an account and launching your first run.

Create an Account

In order to interact with Engine, create an account and set up your machine.

Install the CLI and Library

Use the following pip install command to install the Engine command line interface and python library. We recommend using a fresh virtual environment.

$ pip install --upgrade -i https://pypi.app.engineml.com/simple engineml-cli engineml

Set up Autocomplete

After you've installed the pip packages, enable autocomplete on the CLI. Autocomplete will allow you to tab-complete engine commands on your terminal.

$ engine autocomplete > ~/.engine/autocomplete
$ echo "source ~/.engine/autocomplete" >> ~/.bashrc
$ . ~/.engine/autocomplete

Add an SSH Key

Engine operates services that communicate over ssh. These services are secured via the use of ssh keys. If you do not have a public/private key pair, generate one:

$ ssh-keygen -t rsa -b 4096 -C "user@mycompany.com"
Generating public/private rsa key pair.
Enter file in which to save the key (~/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
danger

Engine only supports unencrypted private keys. Please do not set a password when creating your key.

Copy the public key contents with your favorite text editor or with pbcopy < ~/.ssh/id_rsa.pub. Paste the contents into the "Add SSH Key" modal on your settings page.

Generate a CLI Key

You will need your CLI key to login in on your terminal. Copy it from your settings page.

copy a CLI key

Log in

Use your new CLI key to log into Engine on your terminal with engine login. If you have not already added an ssh key to your account, you will be prompted to do so during the login process.

$ engine login 'CLI_KEY'
Welcome user@mycompany.com!

Launch a Run

Now that you have initialized your account, let's launch our first run.

Create a Project

Navigate via the Engine dashboard and click on New in the Projects section on the top left corner. Follow the prompt to create a new project called 'quickstart'.

Clone

Engine ML hosts preconfigured examples for training a digit classifier using TensorFlow, Pytorch, and Keras on GitHub.

Clone the repository and use the vision/mnist/get-data.sh script to download the MNIST dataset:

$ git clone git@github.com:EngineML/Examples.git
$ cd Examples/
$ ./vision/mnist/get-data.sh

Configure

Engine uses a run configuration file to know how and where to execute your code. Configuration files are defined in YAML and have different specifications depending on where the code is executed. The most basic run configuration has three required fields: apiVersion, repository and command.

The Engine ML Examples repository comes with run configurations for TensorFlow, Pytorch, and Keras. These configurations exist at vision/mnist/tf/local.yaml, vision/mnist/pytorch/local.yaml, and vision/mnist/tf_keras/local.yaml, respectively.

Choose a machine learning framework and update the respective run configuration so that the repository field points to your newly created project. Note that repository's value must take the form owner/project. Owner is either your username or a team's name and project is your project's name (in this case, username/quickstart).

tip

You can quickly modify the project field in the run configuration with engine run's --override option.

Launch

Engine requires that your code is tracked by Git and that any changes are committed. To expediate your workflow, Engine automatically commits staged and unstaged changes to files that are already in the Git index (i.e. Engine performs the equivalent of git add -u && git commit executed in the root of the repository). Note that files that are not tracked by Git will not be tracked by Engine.

$ pip install -r vision/mnist/tf/requirements.txt
$ engine run vision/mnist/tf/local.yaml

Track Your Progress

View your new run by issuing engine dashboard or visit https://app.engineml.com.

  1. Stream your run's logs with engine tail -f RUN_ID
  2. Download your run's output files with engine get-files RUN_ID PATTERN
  3. Use tags to organize your run with engine tag RUN_ID TAG

Congratulations! You have launched your first Engine run. If you have any questions, reach out on our support Slack group.