Supported models & Tasks

These docs are outdated! Please check out https://docs.titanml.co for the latest information on the TitanML platform. If there's anything that's not covered there, please contact us on our discord.

Supported tasks

• Text classification - sequence_classification or glue.Note that GLUE, the General Language Understanding Evaluation benchmark, is a collection of sequence classification tasks used to evaluate natural language understanding systems - so you'd have to specify which columns in your dataset contain the sequences which are to be classified, as well as how many labels there are. Using glueas the task and the glue task as the dataset is a handy shortcut.

• Question answering - question_answering The most common datasets for question answering are the SQuAD datasets, but TitanML does support others. If you decide to use a different training dataset, you must indicate to Iris whether your dataset contains unanswerable questions (see how to do this here).

• Token classification - token_classification

  TitanML also supports classification tasks involving individual tokens (including Named Entity Recognition). As with sequence classification, you must indicate which columns in your input dataset are to be classified, and how many labelled classes they are to be classified into.

• Causal language modelling - language_modelling

  Causal language modelling is how large language models like GPT-4 and Claude are trained. The model learns to predict the next word (technically, token) given a string of previous words (tokens). TitanML supports language modelling for LLMs like OPT and pythia. Large models will automatically use state of the art parameter efficient training. See below for supported models.

• Conditional language modelling (sequence to sequence) - language_modelling

  TitanML also supports conditional language modelling as a task. Conditional language modelling (also known as sequence to sequence modelling) involves producing output tokens conditioned on both previous tokens, and an additional sequence. Examples include translation, & summarization. Provide language_modelling as the iris task, and the platform will automatically deduce the task type from the model used. See below for supported models.

Supported models

Language modelling

Supported models for language modelling

• t5 family (conditional language modelling) (google/t5-efficient-small-el16 , t5-base, google/flan-t5-base , ...)

• pythia family (language modelling) (EleutherAI/pythia-160m, EleutherAI/pythia-410m, ...)

• OPT family (language modelling) (facebook/opt-125m, facebook/opt-1.3b, facebook/opt-350m, ...)

Sequence classification

Anything supported by huggingface's AutoModelForSequenceClassification can be finetuned on the TitanML platform.

Token classification

Anything supported by huggingface's AutoModelForTokenClassification can be finetuned on the TitanML platform.

Question Answering

Anything supported by huggingface's AutoModelForQuestionAnswering can be finetuned on the TitanML platform.