Launching a Run

Launch your Run

Runs are launched when you run engine run. Before launching any runs, ensure you have a run configuration.

Engine ML uses git to track source code, so please ensure that you are running engine run from a git repository. If you have any unstaged changes, engine run will automatically commit those for you.

warning

engine run will only track files that are staged and unstaged. Files that are not tracked by git will not be tracked by Engine ML.

View your Run's Progress

You can see your run's progress including logs, system metrics, and output files from the Engine ML Dashboard. Use engine dashboard or visit https://app.engineml.com to view your recent runs. Data generated from runs is synced periodically.

Stop your Run

If your python code completes, exits with an error, or is interrupted with a KeyboardInterrupt, Engine ML will automatically mark the run as finished on the dashboard.

tip

If you use a KeyboardInterrupt to stop a run, or if you lose internet connection during a run, you should run engine sync-cache to ensure that your files and logs are in sync with Engine ML.

Download Checkpoints and Events

When your experiment is complete, you can download your saved model and event files with engine get-files. See the documentation if you want to download individual files.

$ engine get-files lateral-stick '*'
Downloading outputs for run lateral-stick...
Downloading outputs matching "*" for run lateral-stick
Saved to file "lateral-stick-1544679573.tar.gz"

Additional Features

Annotations

The system metrics dashboard is useful for finding bottlenecks in your code. Engine ML makes it easy to correlate system metrics with code using eml.annotate.

Annotations

tip

Metrics annotations are not visible by default. Make sure to check the box at the top of the page to turn them on. Annotation-Box

Common Annotations

Slow operations like image preprocessing
eml.annotate(title='Data', comment='Start Preprocessing', tags=('data'))
data = preprocess_data(data)
eml.annotate(title='Data', comment='Finished Preprocessing', tags=('data'))
The beginning of each epoch
for epoch in range(max_epochs):
for data, label in data_iterator():
train(data, label)
eml.annotate(title='Epoch', comment=epoch)
The start of training and evaluation
eml.annotate(title='Starting training')
for data, label in train_data.iterator():
train(data, label)
eml.annotate(title='Starting evaluation')
for data, label in eval_data.iterator():
evaluate(data, label)

Tags

Keeping track of Run IDs can be tricky. You can organize your runs with tags.

$ engine tag lateral-stick resnet50

To delete tags, use the --delete or -d flag to engine tag.

$ engine tag -d lateral-stick resnet50