> For the complete documentation index, see [llms.txt](https://titanml.gitbook.io/iris-documentation/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://titanml.gitbook.io/iris-documentation/getting-started/iris-commands/using-iris-upload.md).

# Using iris upload

{% hint style="danger" %}
These docs are outdated! Please check out <https://docs.titanml.co> for the latest information on the TitanML platform.\
\
If there's anything that's not covered there, please contact us on our [discord](https://discord.com/invite/83RmHTjZgf).
{% endhint %}

*Remember to ensure you have the latest version of iris installed before running any command! You can check this by running `pip install update titan-iris.`*

Although you can upload models and datasets on the fly by using their respective file paths with `iris distil` or `iris finetune`, it may be useful to have access to your models and datasets from other machines. `iris upload` allows models to be uploaded without immediately using them for a job. `iris upload <filepath> <name>` also returns an *artefact ID*, which is a canonical reference to the uploaded artefact (model or a dataset). You can pass this ID anytime as an argument to `iris distil.`

There are a few constraints on the types of models and datasets that can be passed to `iris upload`. These constraints also apply whenever you run an `iris distil` or `iris finetune` job with a local directory.

* Model and dataset paths **must be paths to folders.**
* The dataset folder must contain the following:
  * Dataset files which have already been split into training and validation sets, titled `train.csv` and `val.csv` respectively (only .csv files are accepted!)
  * A common scheme (i.e. the same column names and number of columns) shared by both the training and validation datasets.
  * For sequence classification jobs, a label column titled 'Label' which contains integer values.
* The model folder must contain the following:
  * A `pytorch.bin` model file (of course!)
  * A tokeniser file named `tokenizer_config.json`.
* Model weights must be stored using [HuggingFace Safetensors.](https://huggingface.co/docs/safetensors/index) Either save your model from `transformers` using `safe_serialization=true`or use `iris makesafe` to get a `.safetensors` file.
  * To run `iris makesafe`, simply pass the path to the folder containing the model, e.g. `iris makesafe iris upload /Users/myaccount/All-code/Kaggle/Models/twitter_sc`. This will add a safetensors file to your model directory.

Below are some examples:

### Iris Upload & Distil with sentence classification

```bash
# uploading a dataset
iris upload /Users/myaccount/All-code/Kaggle/Datasets/clothing_data # path to a folder containing train.csv and val.csv files

# plugging in the dataset ID from iris upload and the name of a model from HF Hub
iris distil --model bert-base-uncased --dataset 38f8a758-8cb4-4029-bd30-f440f850d77e --task sequence_classification --name test-floatds --text-fields text --num-labels 1 -s
# 38f8a758-8cb4-4029-bd30-f440f850d77e is an example of a dataset ID generated by iris upload
```

```bash
# uploading a model and a dataset - N.B. the iris upload command must be used once for each model/dataset you want to upload
iris upload /Users/myaccount/All-code/Kaggle/Models/twitter_sc # path to a folder containing a model, including a tokenizer.json file
iris upload /Users/myaccount/All-code/Kaggle/Datasets/twitter_sc # path to a folder containing train.csv and val.csv files
# plugging in the model and dataset IDs output by iris upload
iris distil --model 9b25cca2-f547-46d6-ad35-6e06808383e0 --dataset 2cca2468-46b9-4eb9-afb4-836ff4fe03b6 --task sequence_classification --name test-model+ds --text-fields Sentence  --num-labels 4
```

### Iris Upload & Distil with question-answering

```bash
# uploading a model and a dataset
iris upload /Users/myaccount/All code/Kaggle/custom_squad
iris upload /Users/myaccount/All code/Kaggle/custom_squad_dataset

# plugging in the model and dataset IDs output by iris upload
iris distil --model f29a298f-5f29-4769-a24e-881ad4ac81d4  --dataset 293ae8bc-88c8-443e-80cc-92f4ab96520d --task question_answering --name josh-test-qads -s -hn
```

Remember that you can choose to upload a model, a dataset or both from a local file.&#x20;