How to Upload Files to Colab from Kaggle without Downloading into your Local Drive

This tutorial will guide you on how you can upload files to colab from Kaggle without downloading them into your local drive.

To perform this task you have to use Kaggle’s API.

1. Go to your Kaggle profile and click on your avatar. You would find it at the top right-hand side of your screen.

avatar

2. Click Account

3. Scroll downwards. Under the API option, click on Create New API Token to generate an API token - this is a json file that stores your username and key.

A json file will be downloaded. The file contains your credentials.

4. You may need to install kaggle. Open your colab notebook and type !pip install kaggle. Sometimes, you may get an error when you want to download some files so you may need to use the second command to upgrade.

!pip install kaggle
!pip install --upgrade --force-reinstall --no-deps kaggle

5. Create a folder named kaggle in your root directory and store the credentials with the following code:

!mkdir ~/.kaggle

6. Upload your kaggle credentials into colab.

from google.colab import files
uploaded = files.upload()

7. Next copy the json file credentials into the kaggle folder you created in step 5by copying the paths.

!cp /json_path /kaggle_path

You should have something like this:

!cp /content/kaggle.json ~/.kaggle/kaggle.json

8. Set up your environment so that Kaggle can interact with your operating system.

import os
os.environ["KAGGLE_CONFIG_"] ='json_path'

You should have something like this:

import os
os.environ["KAGGLE_CONFIG_"] ='/content/kaggle.json'

9. Go to kaggle and get the name of the dataset. You can search for a list of all the datasets with that name.

!kaggle datasets list -s "name_of_dataset"

For example, if I want to download the iris dataset from kaggle, I would do this:

!kaggle datasets list -s "iris"

The above gives you a list with references, titles, and other information of datasets that are similar to the name of the dataset you entered.

Copy the ref of your desired dataset then download the datasets by using the following code:

!kaggle datasets download -d "paste-the-copied-ref-here"

Example:

!kaggle datasets download -d "uciml/iris"

Or you can simply write the path of the file provided by kaggle. You should find it under the files’ section on the Data page.

Note: If the dataset is from a competition, you may need to join the competition else you would get a 403 Forbidden error.

10. Now you have a zipped file and you can unzip it using the code below:

import zipfile
!unzip "your_zipped_file_path_here" -d data/

Do leave a comment if you found this interesting. If you have questions, please leave them in the comments’ section.

You can follow me on Twitter at @iamdataqueen_ and LinkedIn Ifeoma Veronica Nwabufo.